5.5 being able to solve the problem is in some ways even more extraordinary, because it means that it can resolve a pretty major math conjecture but can't figure out this mind-blowingly easy task (I guarantee anyone reading this can get it): https://arcprize.org/tasks/cd82
Although I did solve it, I definitely do not consider the current ARC-AGI-3 puzzles anywhere near "mind-blowingly easy" lol. Of course I do struggle with vividly visualizing anything, sense of direction, detailed visual memory, etc., relative to other "intelligence" based tasks, so I suppose it could be some good old human jaggedness showing up
Some of the tasks are pretty tricky, but I definitely found this one the easiest, and it has the fewest number of moves as the human baseline.
What's really wild is that they couldn't even get the first level right (which by design is quite easy). Odds are decent that changes soon, but it shows that there are still some pretty big differences between LLM and human intelligence.
Math is really hard, and humans have pretty good intuition as is. Likely quite some problems will be proved with variations of existing techniques, perhaps in new context, and with sheer thorough exploration of the strategy space.
So, it is quite likely AI will get at least as good as people soon enough. That will likely still leave out a lot of fiendishly hard problems. Will be fun to watch how it plays out.
I'm definitely excited to see how this plays out. I'm also excited to play around with models more with math and see what happens. We'll see if that ends up ever turning into a piece, or if it will always just be a hobby...
It's a defensive view set out by Leif Weatherby…a kind of ever shifting line intended to preserve human competencies from machine ones. It's an interesting book.
Appreciate the write-up, and I thought you did a very good job! (Though even if you didn't, how would I know...?)
The main advantage I suspect humans will retain over AI is... for lack of a better term, people are better thing-wanters. People are voracious and discerning thing-wanters, who are always changing their minds about exactly what things are worth wanting, and are always looking over the horizon for the next thing to want. This gets us in trouble sometimes--like in this case, no mathematician *wanted* to pursue this proof strategy--but in general I think this is an asset.
Modern AIs, on the other hand, more or less want whatever you tell them to want. Right now that's their entire thing: they'll hammer away relentlessly at any dumb thing you come up with. They want to be steered. That's not to say it's *trivial* to steer AI models, but on the whole they seem much more steerable than humans.
This could always change! But I suspect it won't, at least as long building frontier AI remains extremely capital intensive. I can't imagine anyone dumping billions of dollars into making their product *less* compliant and steerable.
Thank you! One of the things that I had to cut for length was a section called "what is the point of math?"
Even if AIs get better at mathematics, and the type of volition you are talking about, there's a sense in which math is about getting humans to understand math. The whole game of proving theorems is rather like the whole game of solving homework problems: the understanding gained and transmitted is the point, rather than the result per se. (There are context where this isn't true).
Thank you for this very clear explanation. I am confused about one thing--I agree that the OpenAI diagram shows the c^2=65 case. However, for a 16x16 grid layout, I think the maximal number of unit distance pairs happens for c^2=25 (976 pairs) not c^2=65 (912 pairs). Am I missing something or did they make the figure incorrectly? Thanks for your work in putting this together.
Hi Chris, thanks for pointing this out! I actually note in a footnote that the maximal number of unit distance pairs happens for c^2=25, so you’re correct about that.
One subtlety here I’m not sure I communicated clearly enough: with something like the grid Erdos constructed, mathematicians really care about the behavior as the number of points expand rather than what is exactly optimal. We already knew that a square grid was not going to be optimal — things which look closer to the screenshot I shared in the second section have more unit distances. So having c^2=25 or c^2=65 isn’t a huge deal in this case: both are illustrating the same main idea.
That being said, OpenAI’s diagram is confusing. I think they thought that c^2=65 would look better, and they might be right tbh.
Thank you, I missed the footnote. Agree that c^2=65 probably is more interesting looking. My interest in this is from an artistic direction (printmaking). You say that we know a square grid is not going to be optimal -- is there a known configuration that beats the square grid approach for a small but not tiny number of points (say 250-1,000 points)? Thanks!
The most important line here is that the model applied existing ideas from several subfields without pioneering a new technique. That makes this a cross-domain recall and persistence result rather than conceptual invention, which is a real but different capability. The human work didn't vanish, it moved up to problem selection and to verifying and extending a proof that still needed cleanup.
Thank you for this brilliant piece! Even though I don't have a background in math, I'm happy to know this monumental breakthrough AI has made.
As other comments have pointed out, to this day, we're still the navigators. AI is more like the untiring employee. But this doesn't mean it can't be innovative. I think what you mentioned in your piece shone light on the cost of math research. AI can be our low-cost, trial and error sandbox. And we can bring back those niche or highly complex methods that once failed to work.
It does make you wonder, once the grunt work is outsourced to AI, will the core value of mathematicians shift more to their intuition and imaginaion? Knowing what to ask and deciding which paths are worth exploring?
Come to Decoding science for answers anchored in science...
The thing everyone would want is God to explain creation. We get the next best thing with God working through Jesus to leave that message. The monks claimed that it was important that we mature enough thanks to the first apostles so that we could implement the second stage of Jesus's plan. That was to release the explanation for creation so that everyone could have a better understanding of God and God's domain. You need to understand everything in our light universe and if you read it in my article: guess what just happened, you'll realize how it directly intimately interacts with God and how God can be all knowing and everywhere. It's a marvelous Revelation but it's also something now that science can verify. Either we are being giving The Theory of Everything or we are not.
Most people do not realize that facts are neutral. We are noticing things and collecting facts about them and want to know what causes all of this which is creation. The causes are what we form theories about or speculate about but we do not need to because we actually have the answers now. This means you are either 100% correct in explaining creation or you are 100% wrong. Science has consistently been 100% wrong and is wandering off into crazy fantasies. This was predicted and is the point. We need this divide intervention. It is the greatest moment in our history. And if you watch the news you'll realize that this gift is coming at exactly the right time.
Yeah, they used my work to for their "discovery". This isnt about some shape and the real shape is a rectangle just so you know, they go eveyone distracted by their garbage abstraction of my work. This is about the elimination of floating point drift. I GUARANTEE IT.
5.5 being able to solve the problem is in some ways even more extraordinary, because it means that it can resolve a pretty major math conjecture but can't figure out this mind-blowingly easy task (I guarantee anyone reading this can get it): https://arcprize.org/tasks/cd82
Here's to jaggedness?
Although I did solve it, I definitely do not consider the current ARC-AGI-3 puzzles anywhere near "mind-blowingly easy" lol. Of course I do struggle with vividly visualizing anything, sense of direction, detailed visual memory, etc., relative to other "intelligence" based tasks, so I suppose it could be some good old human jaggedness showing up
Some of the tasks are pretty tricky, but I definitely found this one the easiest, and it has the fewest number of moves as the human baseline.
What's really wild is that they couldn't even get the first level right (which by design is quite easy). Odds are decent that changes soon, but it shows that there are still some pretty big differences between LLM and human intelligence.
Math is really hard, and humans have pretty good intuition as is. Likely quite some problems will be proved with variations of existing techniques, perhaps in new context, and with sheer thorough exploration of the strategy space.
So, it is quite likely AI will get at least as good as people soon enough. That will likely still leave out a lot of fiendishly hard problems. Will be fun to watch how it plays out.
I'm definitely excited to see how this plays out. I'm also excited to play around with models more with math and see what happens. We'll see if that ends up ever turning into a piece, or if it will always just be a hobby...
Thanks for the fascinating write up. seems like a sophisticated example of remainder humanism.
What do you mean by remainder humanism?
It's a defensive view set out by Leif Weatherby…a kind of ever shifting line intended to preserve human competencies from machine ones. It's an interesting book.
Appreciate the write-up, and I thought you did a very good job! (Though even if you didn't, how would I know...?)
The main advantage I suspect humans will retain over AI is... for lack of a better term, people are better thing-wanters. People are voracious and discerning thing-wanters, who are always changing their minds about exactly what things are worth wanting, and are always looking over the horizon for the next thing to want. This gets us in trouble sometimes--like in this case, no mathematician *wanted* to pursue this proof strategy--but in general I think this is an asset.
Modern AIs, on the other hand, more or less want whatever you tell them to want. Right now that's their entire thing: they'll hammer away relentlessly at any dumb thing you come up with. They want to be steered. That's not to say it's *trivial* to steer AI models, but on the whole they seem much more steerable than humans.
This could always change! But I suspect it won't, at least as long building frontier AI remains extremely capital intensive. I can't imagine anyone dumping billions of dollars into making their product *less* compliant and steerable.
Thank you! One of the things that I had to cut for length was a section called "what is the point of math?"
Even if AIs get better at mathematics, and the type of volition you are talking about, there's a sense in which math is about getting humans to understand math. The whole game of proving theorems is rather like the whole game of solving homework problems: the understanding gained and transmitted is the point, rather than the result per se. (There are context where this isn't true).
So one advantage humans will maintain is that the thing we want in mathematics is fundamentally human, too. That may not pay the bills though, as a really good recent substack post puts it: https://davidbessis.substack.com/p/the-fall-of-the-theorem-economy
Thank you for this very clear explanation. I am confused about one thing--I agree that the OpenAI diagram shows the c^2=65 case. However, for a 16x16 grid layout, I think the maximal number of unit distance pairs happens for c^2=25 (976 pairs) not c^2=65 (912 pairs). Am I missing something or did they make the figure incorrectly? Thanks for your work in putting this together.
Hi Chris, thanks for pointing this out! I actually note in a footnote that the maximal number of unit distance pairs happens for c^2=25, so you’re correct about that.
One subtlety here I’m not sure I communicated clearly enough: with something like the grid Erdos constructed, mathematicians really care about the behavior as the number of points expand rather than what is exactly optimal. We already knew that a square grid was not going to be optimal — things which look closer to the screenshot I shared in the second section have more unit distances. So having c^2=25 or c^2=65 isn’t a huge deal in this case: both are illustrating the same main idea.
That being said, OpenAI’s diagram is confusing. I think they thought that c^2=65 would look better, and they might be right tbh.
Thank you, I missed the footnote. Agree that c^2=65 probably is more interesting looking. My interest in this is from an artistic direction (printmaking). You say that we know a square grid is not going to be optimal -- is there a known configuration that beats the square grid approach for a small but not tiny number of points (say 250-1,000 points)? Thanks!
A very interesting article, thank you.
The most important line here is that the model applied existing ideas from several subfields without pioneering a new technique. That makes this a cross-domain recall and persistence result rather than conceptual invention, which is a real but different capability. The human work didn't vanish, it moved up to problem selection and to verifying and extending a proof that still needed cleanup.
Hooray for those who celebrate. Discrete geometry - how groundbreaking is this in terms of using AGI generally?
Thank you for this brilliant piece! Even though I don't have a background in math, I'm happy to know this monumental breakthrough AI has made.
As other comments have pointed out, to this day, we're still the navigators. AI is more like the untiring employee. But this doesn't mean it can't be innovative. I think what you mentioned in your piece shone light on the cost of math research. AI can be our low-cost, trial and error sandbox. And we can bring back those niche or highly complex methods that once failed to work.
It does make you wonder, once the grunt work is outsourced to AI, will the core value of mathematicians shift more to their intuition and imaginaion? Knowing what to ask and deciding which paths are worth exploring?
Come to Decoding science for answers anchored in science...
The thing everyone would want is God to explain creation. We get the next best thing with God working through Jesus to leave that message. The monks claimed that it was important that we mature enough thanks to the first apostles so that we could implement the second stage of Jesus's plan. That was to release the explanation for creation so that everyone could have a better understanding of God and God's domain. You need to understand everything in our light universe and if you read it in my article: guess what just happened, you'll realize how it directly intimately interacts with God and how God can be all knowing and everywhere. It's a marvelous Revelation but it's also something now that science can verify. Either we are being giving The Theory of Everything or we are not.
Most people do not realize that facts are neutral. We are noticing things and collecting facts about them and want to know what causes all of this which is creation. The causes are what we form theories about or speculate about but we do not need to because we actually have the answers now. This means you are either 100% correct in explaining creation or you are 100% wrong. Science has consistently been 100% wrong and is wandering off into crazy fantasies. This was predicted and is the point. We need this divide intervention. It is the greatest moment in our history. And if you watch the news you'll realize that this gift is coming at exactly the right time.
Yeah, they used my work to for their "discovery". This isnt about some shape and the real shape is a rectangle just so you know, they go eveyone distracted by their garbage abstraction of my work. This is about the elimination of floating point drift. I GUARANTEE IT.