AI Governance

Before I summarize these readings, I’ll write down my views so I might reflect on how these readings change them.

In the limit of time, AI Governance is harder than alignment.
Research will proceed, even under the strictest of moretoria.
There is a decent chance superhuman AI could run on current hardware[?], so when that software is found then AGI can only be prevented in cases where all compute is monitored.
If that software doesn’t get found, then you can still survive with course-grained monitoring.

This monitoring will be imperfect. When devices have internet access, it is hard to know which fragments of compute belong to the same “chunk”. Even without internet access, programs can communicate with each other in unorthodox ways. Currently it is very hard however to train a model in a distributed fashion.

As time passes, AGI will become increasingly hard to prevent.
Until that time, governance can stall and can build up a safety-minded culture.

AI Governance: Opportunity and Theory of Impact

Summary of this.

AI governance is a new field and is relatively neglected.

This paper is written in 2020. “Neglected” is an overstatement but right now none of my political parties have a stance on X-risk still.

this piece is primarily aimed at a longtermist perspective

I’ve heard this term come up less and less. Most longtermist cause areas actually have an expected impact within our lifetimes. AIS is no exception.

We see this scramble in contemporary international tax law, competition/antitrust policy, innovation policy, and national security motivated controls on trade and investment.

2 problems

The problem of managing AI competition:
> Problems of building safe superintelligence are made all the more difficult if the researchers, labs, companies, and countries developing advanced AI perceive themselves to be in an intense winner-take-all race with each other, since then each developer will face a strong incentive to “cut corners”

The problem of constitution design:
> A subsequent governance problem concerns how the developer should institutionalize control over and share the bounty from its superintelligence;

3 perspectives

Superintelligence

Ecology
> a diverse, global, ecology of AI systems. Some may be like agents, but others may be more like complex services, systems, or corporations. These systems, individually or in collaboration with humans, could give rise to cognitive capabilities in strategically important tasks that exceed what humans are otherwise capable of

General Purpose Technology, tool AI

risks

Misuse and accident risks are associated with ASI.
> These lenses typically identify the opportunity for safety interventions to be causally proximate to the harm: right before the system is deployed or used there was an opportunity for someone to avert the disaster through better motivation or insight.

Structural risks can be associated with the ecology and GPT perspectives.
> we see that technology can produce social harms, or fail to have its benefits realized, because of a host of structural dynamics

These structural risks might not be existential threats on their own. But they can be “existential risk factors”. They indirectly affect X-risk.

pathways to x-risk

Relatively mundane changes in sensor technology, cyberweapons, and autonomous weapons could increase the risk of nuclear war

Technology can lead to a general turbulence.

The world could become much more unequal, undemocratic, and inhospitable to human labor

the spectre of mass manipulation through psychological profiling as advertised by Cambridge Analytica hovers on the horizon. A decline in the ability of the world’s advanced democracies to deliberate competently would lower the chances that these countries could competently shape the development of advanced AI.