A super comprehensive post - thanks for writing this up in such a clear manner. Note to self - always try to find a Simpson meme to make your point more understandable!
> But it’s a safe bet that none of those training documents involved a reporter coaxing a chatbot to explore its naughty side.
um, I think it's *extremely likely* that the training corpus includes text like that. It's a pretty common trope about fictional AI, and the training data includes all the fanfiction available on the internet.
Perhaps. But even if true there was a) precious little of it as a percentage in its training data, and so b) it is easy to see how you could get compounding errors off of that baseline.
“If you enjoyed today’s article, please support my work with a paying subscription. That will get you access to the premium Agent Week articles I’ll be publishing later in the week. For this week only I’m offering a 20 percent discount on annual subscriptions. That’s the best price you’ll get for the rest of the year.”
While I understand you have limited direct control of this, I will suggest it anyway.
There is so much very good high quality content available on the Internet for free that while I am perhaps able, I am unwilling to pay $8 or even $5 a month for *any* of it from any single author.
Yours is one of a couple of dozen Substacks where I would consider a paid subscription at $2/month, and would subscribe to immediately for $1/month.
Price controls are bad. Minimum prices are bad. You should lobby Substack to let you control your own pricing. I am confident that folks like you could have 20x-200x more paid subscribers, and earn 2 - 20x more money - and then be able to grow from there - if you were able to charge lower prices.
Hi Andy! I appreciate the feedback, but there are enough people willing to pay $8 per month (or $61/year with this week's sale!) that I'm comfortable with the current pricing. I'm sure I'd get more subscribers if I charged $2/month, but I don't think I'd get anywhere close to four times as many. I'm happy to have you as a free subscriber. Thanks!
Excellent, readable summary! The idea that a constitution can change the entire set of principles used by an LLM is sobering. This seems to get to the crux of whether AI will kill us or not. How can we ensure companies will use human-centric constitutions?
A super comprehensive post - thanks for writing this up in such a clear manner. Note to self - always try to find a Simpson meme to make your point more understandable!
> But it’s a safe bet that none of those training documents involved a reporter coaxing a chatbot to explore its naughty side.
um, I think it's *extremely likely* that the training corpus includes text like that. It's a pretty common trope about fictional AI, and the training data includes all the fanfiction available on the internet.
Perhaps. But even if true there was a) precious little of it as a percentage in its training data, and so b) it is easy to see how you could get compounding errors off of that baseline.
This is such a helpful explainer. Thank you so much!
Thank you for a wonderfully clear explanation of the topics, and of the improvements over time.
“If you enjoyed today’s article, please support my work with a paying subscription. That will get you access to the premium Agent Week articles I’ll be publishing later in the week. For this week only I’m offering a 20 percent discount on annual subscriptions. That’s the best price you’ll get for the rest of the year.”
While I understand you have limited direct control of this, I will suggest it anyway.
There is so much very good high quality content available on the Internet for free that while I am perhaps able, I am unwilling to pay $8 or even $5 a month for *any* of it from any single author.
Yours is one of a couple of dozen Substacks where I would consider a paid subscription at $2/month, and would subscribe to immediately for $1/month.
Price controls are bad. Minimum prices are bad. You should lobby Substack to let you control your own pricing. I am confident that folks like you could have 20x-200x more paid subscribers, and earn 2 - 20x more money - and then be able to grow from there - if you were able to charge lower prices.
Hi Andy! I appreciate the feedback, but there are enough people willing to pay $8 per month (or $61/year with this week's sale!) that I'm comfortable with the current pricing. I'm sure I'd get more subscribers if I charged $2/month, but I don't think I'd get anywhere close to four times as many. I'm happy to have you as a free subscriber. Thanks!
I always enjoy these "minimum of jargon" explainers of yours, and this one didn't disappoint!
Even though I was well-aware of most of the history here, there were several "aha" moments for me too.
The student-teacher analogy for imitation vs. reinforcement learning was also very helpful. Thanks for putting this together!
Excellent, readable summary! The idea that a constitution can change the entire set of principles used by an LLM is sobering. This seems to get to the crux of whether AI will kill us or not. How can we ensure companies will use human-centric constitutions?