16 Comments

Very useful article. I’d like to see an explainer of what red team testing involves. I’ve read some very brief ones but they really didn’t help for a layman like me. Either way, thanks for the very useful work.

Expand full comment

I think these precautions are unlikely to do anything but slow down the development of AI. The government is not going to have anyone who can judge whether a chatbot is dangerous or not. All the experts are in private companies. The regulators will only be able to say "I don't know what it will do, but it might be bad, so I'm not going to allow it."

Expand full comment

Thanks for the interesting article. It is good that they are dealing with bio-terrorism threats. Inventor of CRISPR-based gene drive proposed other technical solutions (basically using UV lamps in buildings and surveillance of DNA in airplane toilets) for protection: https://80000hours.org/podcast/episodes/kevin-esvelt-stealth-wildfire-pandemics/ I wonder if they are considered in this order.

Expand full comment

This is the new model of government in the USA. Representative Democracy is dead. Now we have Government by Emergency Declaration and the puppet masters holding Biden's strings get to dictate. I mean, do you really think Biden knows what any of this means? What "emergency" is there that allows invoking the DPA or International Emergency Economic Powers Act? You people cheering this on because it's your pet issue are going to rue the day when these executive orders bite you on something you don't like.

Expand full comment

"The order has much less to say about AI systems evading human control by deceiving people. It’s a hard problem. An AI system that’s smart enough to deceive people “in the wild” may also be smart enough to deceive people conducting red-team tests" I found it very hard to get convincing description from AGI safety guys about how it could happen without assumptions about neural networks which reveal lack of understanding of the technology. For example it is not clear why we couldn't just use Yoshua Bengio's AI scientist (https://yoshuabengio.org/2023/05/07/ai-scientists-safe-and-useful-ai/) to evaluate plans of autonomous AI with respect to potential harm for humans

Expand full comment

Here is a Libertarian viewpoint on the issue of AI regulation by the Biden Administration:

-----

The Biden Administration’s Artificial Intelligence Rent-Seeking Play

Paul Mueller

– November 2, 2023

This week President Biden signed a sweeping executive order around the use and development of Artificial Intelligence. While many commentators have praised it for its extensive use of platitudes and ambitious scope, basic economic analysis suggests this policy is business as usual for the Biden administration: usurping authority, brow-beating private-sector companies, slowing innovation, and advancing a divisive progressive agenda in the name of “equity.”

Although the administration claims authority from the Defense Production Act, very little of the executive order is even remotely related to national defense. It uses boilerplate language about “serious risk,” “national economic security,” “national public health,” “ensuring safety,” “ensuring appropriate screening,” and much more.

These aspirations have little connection with what this executive order will do.

...

https://www.aier.org/article/the-biden-administrations-artificial-intelligence-rent-seeking-play/

Expand full comment

Once guardrails are in place, you can drive a lot faster. AI is a tough one, but need to balance solid guardrails for the industry & encouraging entrepreneurial innovation & risk-taking.

Expand full comment

Great read!

Expand full comment

Once AI development moves to quantum computers, all these metrics go out the window.

Expand full comment

Thank you for the article. Personally i believe regulating AI is a good thing, of course getting the balance right is very tricky and will require iterations. It is very hard to figure out what exactly goes on in the hidden layers especially trying to trace a particular prompt or learning path. I think a lot more emphasis and regulation should go into the training data sets. Since we cant control the learning process and, to some extent the output. However we can control the inputs (ie training data) to ensure it avoids biases, inaccuracies, harmful information, etc.

Expand full comment

Sounds like a pretty good start. Definitely way overdue. Thanks for providing the overview. Lots to absorb.

Expand full comment

Thanks for the summary. I have to say I'm not as sanguine as you about locking down the physical world. It seems to me that as technology continues to get cheaper and more democratized, the set of products and activities you'll need to regulate expands uncontrollably. In the limit this leads to something akin to Nick Bostrom's panopticon, in which everyone is under constant video surveillance 24/7 (even in the bedroom and bathroom) to make sure they're not using ultra-miniaturized tools to do something the government deems dangerous.

Expand full comment

Thank you for sharing, Tim! Amazingly insightful analysis. I think regulation is certainly needed. You’ve noted rightly that a regulator(s) might face the risk of being swamped by paperwork. A potential risk is whether the government (any government!) will have the necessary capacity and capability to follow-up on updating and policing regulation. With AI set to grow exponentially, a potential regulator(s) might need to become as large (or larger than) agencies such as the FDA. On the flip side, without sufficient resources, the regulator(s) would lack teeth and we’d possibly be in the same position as having no regulator at all. Would love to hear thoughts on this.

Expand full comment