AI Math, Paul Erdos & Attention
Recently like many, I've been spending time on solving Erdos problems - with a caveat. (Shoutout to my extremely talented fellow collaborators, thank you Ysop, Kaoru and Grace)
What I've been hunting for is a general framework for agents to tackle mathematics and open problems
This sounds audacious to start, so I'll explain why I believe that a path to this framework exists.
My rough intuited explanation is the following:
- Math is extremely verifiable.
- Paul Erdos, Terrence Tao, Jacob Tsimerman & the Reimann Hyptohesis.
A famous quote from Paul Erdős is the following about Erdos Problem #77
"Sometime ago, I made the following joke. If an evil spirit would appear and say "unless you give me the value of 𝑅(5) within a year, I will exterminate humanity", then our best bet would be perhaps to get all our computers working on 𝑅(5) and we probably would get its value in a year."
Problem #77 relates to Ramsey Theory, which focuses on the appearance of order in substructures. Roughly translated, this means that there is no true randomness at scale.
Attention is All You Need
The part about this that I find fascinating is that Erdős's punchline is really about attention: given enough of it, even R(5) falls. And attention my friend is exactly what transformers let us manufacture.
This got me thinking about Jacob Tsimerman's most recent interview "The Last Generation of Mathematicians" which was recorded days after winning the fields medal and joining OpenAI's safety team.
In it, what stood out to me was the following:
The number of mathematicians for each given subset of mathematics is extremely sparse (at any given time one may be working on many problems at once)
Tsimerman specifically mentioned that cross-domain, mathematics is extremely broad. If you were to show him a hypothetical proof of Reimann, because it's in his field of study, it may take him a day to digest. However, if you were to show him something outside of his field, it would take him quite literally months to digest.
So quite literally, the entire field of mathematics is bottlenecked by attention!
Agents are quite literally built upon attention blocks (the model) & tools in a harness.
A second point was something I noticed when listening to a number of interviews from Terrence Tao. In his interview with Lex Friedman, Terrence who was describing Gregor Perelman's solution to one of the millennial problems (the Poincare conjecture), he talked about how the solution was found through ingenuity & persistence (it took 7 years of work).
When asked if an AI would be able to solve it, Terrence mentioned that perhaps an agent could find one potential approach, but it would be one out of perhaps 100 approaches.
Tsimerman himself described doing mathematics as having a number of tricks, techniques and approaches & applying those to solving problems.
So my point with this is that perhaps an agent is able to solve the Poincare conjecture, but it would likely need extremely long horizon reasoning & parallelism in attempts (ideally human + agent).
This brings me back to Jacobian. I noticed early on in attempts at conjectures that agents really like python. The problem with this is that python is dynamically typed, making it less verifiable. In many attempts, agents have written python wrong & had to revise it. The second is that mathematics infrastructure is quite lacking (some thing that Terrence Tao has mentioned many times).
A lot of the libraries that agents like to use, Numpy, Sympy are libraries from over 10 years ago. The problem with this is two-fold, there's a lot of gaps in the libraries & the libraries have many bugs that can be decades old in some cases.
Here's one bug in Sympy from 2016 that inverts a sign when doing a certain polynomial computation. And these are just ones that are filed.
The other point is that when a model is tackling a specific problem, it must reinvent the wheel each time to apply the aforementioned techniques.
So Jacobian is exactly that, a growing library of atomic & composable techniques that agents can apply to tackle mathematics.
If you're interested in discussing or collaborating please reachout to my email morluto@hotmail.com or directly contribute to the repo!.
I'll write more about why Jacobian & a bit more into the ideas behind it in my next post:
From Go to Chess