Robert Miles discusses recent developments in AI Safety

Maybe gravity is no more fundamental than the force of a stretched elastic band. Maybe gravity is just an entropic byproduct—an emergent effect of the universe’s tendency to disorder.

Abstract:

Researchers have developed artificially intelligent (AI) and expert systems (ES) to assist in the formulation, solution and interpretation of generic mathematical programs (MP). In addition, researchers also have built domain-specific systems either modeled around a mathematical program or which include a mathematical program module. In these systems, the specificity of the domain allows researchers to extend the interpretation or formulation beyond that available from the generic set of assumptions about mathematical programming. Further, researchers have begun to investigate the use of mathematical program formulations of expert systems. The purpose of their research has been to, e.g., understand the complexity of the expert systems and also to examine the feasibility of mathematical programming as an alternative solution methodology for those expert systems. This paper surveys and extends some of that literature that integrates AIlES and MP, and elicits some of the current research issues of concern.

Maybe AI systems would be safer if they avoid gaining too much control over their environment? How might that work?

This video discusses the second stage of the machine learning process: (2) collecting and curating training data to inform the model. There are opportunities to incorporate physics into this stage of the process, such as data augmentation to incorporate known symmetries.

I believe I've found a bug. I intermittently go back multiple pages when pressing back button. It frequently happens on youtube, and occasionally happens elsewhere.

In the picture, find my history where I started with the google search for "the chain" (please forgive me for having history from multiple tabs all mixed up). From here I went back and forth with the expected behavior a couple of times looking for the version I remembered. When I found the correct version, I settled in to listen to it in the background for a few songs minutes until it switched to gypsy which wasn't what I wanted to hear. Naturally enough, I went back once- straight from gypsy to the google search, skipping rhiannon and dreams and the chain. This is reflected on the rightclick drop down from the forward button- observe the three intervening pages which are in the history are not in the forward/back list.

In general, I frequently notice this unexpected and unwanted behavior when I go to youtube. I often go to the homepage, select a video, and then try to back to the homepage only to find it take me back to the blank tab, and when I try to forward it takes me to the video without the intervening homepage.

I seem to recall times when I experienced similar behavior, but cannot put a pin in when or where and cannot at this time, despite trying, reproduce it anywhere but youtube. I haven't tried downloading chrome or shudder using edge, mostly because fuck that, but also slightly because I am very lazy.

I also didn't notice this behavior on my laptop which had win10 and firefox (that is, before I switched to linux as a trial over there, but I digress). I tried to search for this error, but google search is terribly corrupt and difficult to get useful results beyond eg song names. I also searched this forum for "back pages" and "back page" and didn't find anything relevant going back a year.

Do I set my computer on fire for its disobedience, or...?

Abstract:

Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society. In this paper we discuss one such potential impact: the problem of accidents in machine learning systems, defined as unintended and harmful behavior that may emerge from poor design of real-world AI systems. We present a list of five practical research problems related to accident risk, categorized according to whether the problem originates from having the wrong objective function ("avoiding side effects" and "avoiding reward hacking"), an objective function that is too expensive to evaluate frequently ("scalable supervision"), or undesirable behavior during the learning process ("safe exploration" and "distributional shift"). We review previous work in these areas as well as suggesting research directions with a focus on relevance to cutting-edge AI systems. Finally, we consider the high-level question of how to think most productively about the safety of forward-looking applications of AI.

We can expect AI systems to accidentally create serious negative side effects - how can we avoid that? The first of several videos about the paper "Concrete Problems in AI Safety".

A brief overview of the concept of generality in AI systems.

Abstract

For stable and efficient fusion energy production using a tokamak reactor, it is essential to maintain a high-pressure hydrogenic plasma without plasma disruption. Therefore, it is necessary to actively control the tokamak based on the observed plasma state, to manoeuvre high-pressure plasma while avoiding tearing instability, the leading cause of disruptions. This presents an obstacle-avoidance problem for which artificial intelligence based on reinforcement learning has recently shown remarkable performance. However, the obstacle here, the tearing instability, is difficult to forecast and is highly prone to terminating plasma operations, especially in the ITER baseline scenario. Previously, we developed a multimodal dynamic model that estimates the likelihood of future tearing instability based on signals from multiple diagnostics and actuators. Here we harness this dynamic model as a training environment for reinforcement-learning artificial intelligence, facilitating automated instability prevention. We demonstrate artificial intelligence control to lower the possibility of disruptive tearing instabilities in DIII-D, the largest magnetic fusion facility in the United States. The controller maintained the tearing likelihood under a given threshold, even under relatively unfavourable conditions of low safety factor and low torque. In particular, it allowed the plasma to actively track the stable path within the time-varying operational space while maintaining H-mode performance, which was challenging with traditional preprogrammed control. This controller paves the path to developing stable high-performance operational scenarios for future use in ITER.

This video discusses the first stage of the machine learning process [identified in the previous post]: (1) formulating a problem to model. There are lots of opportunities to incorporate physics into this process, and learn new physics by applying ML to the right problem.