The author argues for evaluating Ph.D. candidates based more on the oral thesis defense than on the actual thesis.
By essentially the same reasoning, I’ve been arguing for prioritizing in-person design/code reviews over code-only async PR comments.
The important thing is to verify that the human has a coherent design in mind and can demonstrate that it got implemented, regardless of who or what was at the keyboard. “I dunno, I guess Claude thought this was a good idea” is not a coherent design.
An oral defense is ultimately a meeting, with the same weaknesses as other types of meetings. In particular, if new information comes up in a meeting, you cannot reasonably expect to get useful responses from the other participants. First impressions maybe, and plausible-sounding bullshit from those prone to generating that. If you want anything more, you need to provide the information in advance or schedule another meeting later.
Oral defenses in academia are largely rituals. If a student fails their defense, it's almost always the supervisor's fault. The supervisor is supposed to be the primary quality control. With their regular meetings with the student over the years, they should be able to tell whether the student has achieved sufficient understanding and contributed enough to graduate. If the supervisor thinks the student is ready to defend, the reasonable expectation is that the student will pass and graduate. The defense is mostly there to let the other examiners validate the supervisor's judgment.
Here's another example of a similar but slightly different failure mode of the live defense: The department where I got my degree had a practice around PhD defense where there would be a prof each representing the four research pillars + the dean. It was well known that certain professors disliked each other enough that they would snipe each other's candidates. You'd just hope not to work for one whose enemies got picked for the panel. If you were a superstar, you'd pass, but if you were an okay candidate or someone strong but with stage fright, you be could toast even if you did good research.
The defense was only briefly about the actual thesis, then switched over to whatever research interest the committee members had, and they'd drill into their pet subjects. This was in physics, hardly politics forward normally, and at a very well respected university.
Person B: “key pieces are delayed due to the developer not understanding all of the LLM doesn’t and implementation.”
Manager: “does it work? What are the risks?”
Person B: “well yes it works for now but we’re accumulating tech debt due to a lack of understanding and potential flaws that haven’t been thought out yet”
Manager: “they want feature X, ship it, we can deal with it later, I don’t care if it’s not coherent as long as it works.”
How many decades at this point has these been a push for functionality over everything at all costs? And you have a mechanical snow plow now. Most businesses don’t care about later risk or any future planning beyond the quarter horizon, they’re not concerned about how it will effect their performance in 3 quarters or lead to instability or issues, those are future problems for a future person and we’re here for money now.
I think upweighting the live components of academia is inevitable in the age of automatically produced writing, but I also find it depressing that people think so little of writing that they imagine it obsolete because of AI.
AI writing is aggressively, aggressively mediocre. It is only good for how cheap it is. If you're indifferent to how much better good human writing is than AI writing, you should not be the one to evaluate human writing.
I do this already if there’s any kind of issue. Just talk the person if they’re next to you or zoom them if not (don’t set up a meeting). A five minute conversation can save hours of back and forth.
As someone who has a degree in math, I still can't help but think mathematicians are getting a little bit of a comeuppance. In a lot of areas of mathematics there had been little effort to make the work understandable and leaves numerous folks who could benefit from the knowledge on the outside looking in. Now AI comes along and do the same to mathematicians. Makes me chuckle a little bit.
It is not lowering the bar to find better ways to demystify and explain things. In fact, I would say those who can explain it well understand it the best. Richard Feynman would be my best example.
Do you have any examples of places you felt like there was a lot of gatekeeping? Perhaps having studied mathematics I am a bit blind to the issue here and would like to learn more.
I agree making simple things sound complicated to appear more impressive is bad but there are limits. Even with Feynman he could only go so far, e.g. his interview about why questions and magnetism.
We have seen the thing in programming. I'm not a gatekeeper, I love seeing more people to code. On the other hand, along the way, we lost the joy of the journey and only fixated at the destination.
Result is more software at lower quality. The reason is statistics. When you increase the population, you increase the population of every kind of programmer, and people who want the result are favored in most competitive sectors because corporations want something somewhat working yesterday.
...and here we are.
Now programmers talking about code quality is stoned en-masse. If it's somewhat working then it's good. Efficiency, resiliency, maintainability and sustainability is an afterthought. Some of my friends who loved debating programming language theory now don't even care about the code. They don't write it, just vibe, and they don't plan to come back to "older, caveman style of development".
Some universities are also adding fuel to the fire: They "prepare students for the job", not teaching the science, but the parts that corporations need for the job only.
Though, hardware was cheap and people were expensive, and now code is cheap and hardware is expensive now. We'll see.
The point I was trying to make was not "enjoying the process for the sake of it", but paying attention and spending effort on the journey created better software at the end.
When you look at older software, most of it was higher quality than the things we have today. A web site contained more information in a more readable way, more features in a smaller footprint. Same for native applications.
Now we slap what we found online together and calling it done. Everything is sparse, takes ages to load, centuries to submit and everything is so disconnected and async that some simple features are straight out impossible.
This is what fixating on the destination brought us. I dare you to download something you purchased 3 months ago via a 4096 character S3 link they have sent you, and double dare you to ask customer support for a new link. I'll bet that with a 80% chance they have no way to verify your serial number, even.
Have people completely forgotten how to read? I'm saying your analogy between software and math is completely, irreconcilably flawed. I don't need to read the rest of your screed because this is a thread about math not software.
Somebody said that lowering the bar is not good. Somebody else asked for examples, and I provided an example. So, if you want to be pedantic, that doesn't track well, because what I answered was not about mathematics.
If you want something about mathematics, computation is mathematics, as software is. So, my example tracks the same way in mathematics.
Finding solutions without understanding its parts or the path is equally detrimental to mathematics as it is detrimental to software.
Maybe you need to read a bit slower and think along the way. Using AI too much blunts critical thinking skills in some, as I read.
What was the end of the proof thing you mathematicians use, was it "Q.E.D."?
> computation is mathematics, as software is. So, my example tracks the same way in mathematics.
brother i already addressed this literally in my first response to you: the reason software went to shit is because it became commodified (ie a thing produced in a factory) not because the bar got lowered. the bar for being able to weave also got lowered hundreds of years but your mom's crocheting doesn't suffer from the same ills as the software industry.
> Using AI too much blunts critical thinking skills in some, as I read.
Half of modern academia, for one. Universities have turned into degree mills with the expectation that >50% of the population requires a college degree, regardless of whether they actually have any interest or need for one beyond doing it because it's a prerequisite for a good career, independent of whether said career actually uses the knowledge in any way.
Not that I actually agree that math was at the right level of gatekeeping. It definitely feels intentionally opaque beyond reason, and I think it's why LLMs are able to cut through the obfuscation and solve problems that maybe wouldn't actually have been considered quite so hard if mathematicians did a better job of making their work accessible.
> We already interview faculty hires; we must now do the same for graduate admissions
They hire PhD students without hearing them give a talk and then doing interviews? In Germany, the applicant gives a talk (30-40 min) to the research group they want to join, usually presenting their master's thesis, engage in discussion, often share lunch with the group, then do 1 on 1s with individual members of the group and a longer one with the PI. Obviously this can vary within Germany too, but I couldn't imagine hiring someone without something like this.
People had the same complaints that the code produced by early coding models was messy, lazy, poorly commented, and so forth. The central complaint was that it was just too difficult for humans to review. The answer is just to improve the models and move on.
Similarly now we're getting AI doing math. The math is a giant vibe-coded ball of wax. So just make the models better at explaining what they're doing to humans - that's the end of it.
Rather than just go on and on about how it's the end of the world if we don't do this, why don't we just do it?
"A computer or monkey could easily start at the axioms of ZFC and iteratively apply deduction rules....simply conjecture all mathematical propositions in alphabetical order...The prospect of automating mathematics by enumerating all conjectures, and all proofs of ZFC, is probably not so disturbing to you."
I thought we were going to get at least some brief comment on Godel here?
Gödel effectively says ZFC must be incomplete, otherwise it would not be sound, but does that stop you from listing all mathematical propositions it can generate in some well-defined order?
I like the quote from Hilbert that was brought up in the article: "we must know, we will know".
With AI, it might be the case that we don't know, we won't know, but the machine does.
The central question, namely whether humans should be in the loop, will be repeated again and again in the years to come for all industries, starting with mathematics.
As with the rest of the domains, AI/LLMs will do syntax and search better than any human.
In code, any developer whose differentiaton was clean code and knowledge of different technologies is now average.
In math, any mathematitian whose differentiation was to manipulate formal systems and know tricks of different domains will be average.
Fortunately, humans do more than syntax and search.
The bad news for developers is that if you know what the output of your program should be (which happens most of the time), almost all of the job is syntax and search to build the code that reproduces the output.
The good news for mathematitians is that for the majority of problems you never know the output, or you just know the output is either "True" or "False". There are some cases where you need something else, for example "a solution that blows up in finite time". For those cases AI will outperform you easily (see new Navier-Stokes solution)
So if as a mathematitian you were doing more than syntax and search, then keep doing that and use AI just for what its best.
Excellent optimistic post in a sea of negativity, and with actual suggestions, too.
After reading, my mental image is this: think of Olympiads in Ancient Greece.
* A weightlifter was only awarded a laureate if he were able to lift a heavy stone (have no idea what they were lifting, for illustrative purposes only :-)
* Along comes Archimedes who invents what we would call an exoskeleton. Now any regular guy can lift twice as much as last year’s athlete.
* What to do? You can cancel the Olympiads, but they are actually useful as training, motivation, etc So now you have to give the prize on other factors, eg how well he can lift, has he opened a gym in the city, etc
BTW, physics and bio are not exempt, so those researchers better read and try to stay ahead.
I don´t think so. For programming agents can run code, check compiler output, etc. For mathematics, it is almost the same once you factor in the usage of lean.
For the reality, you can´t close the loop that fast, or with that precision. You will have to slow down by several orders of magnitude.
I am not a mathematician, but I can’t see how we are going to address the problem which we already see in coding:
Impossibility to independently validate all AI results
And in math it goes even worse. In coding code reviews are typically still the form of action you do within days. In math, historically, the lifecycle of proof is months if not years. Take as an example Millennium problems. They require at least two years of validity after publishing. Two years! In modern times with amount of output AI can produce, it feels like infinity.
We are inches close if not at the moment already when humans can’t reliable validate proofs and mathematics produced by AI. Then next research will be based on this AI-written-no-human-in-the-loop results. And we will end up in just few years in a world where novel and frontier problems will be articulated by AI and proven by AI based on AI results and humans will be incapable of understating the mere nature of the solution.
I don't think that's actually the real problem. Along with the progress in answering mathematical questions, recent progress on AI-powered autoformalisation has been astonishing. All the recent AI discoveries have been accompanied by Lean proofs.
And, yes: that doesn't absolutely guarantee correctness. The Lean kernel has had soundness bugs, and may have some still. But it's pretty strong evidence of correctness nevertheless.
The concern among mathematicians is not mainly that they doubt the correctness of any of these discoveries, but that human understanding may be devalued.
I am not that worried, but rather just observing. Humanity is about to enter the phase when we will be using things based on ideas no human ever properly understands. This thought … disturbing, somehow?
It is perfectly valid counterpoint to say that we already do it. We everyday use myriad of things, tools, and software we have 0 clue how it operates. But for us as humans it was reassuring that we know that at least there are a few other alive humans who know it, who create it and who can explain it.
>Humanity is about to enter the phase when we will be using things based on ideas no human ever properly understands. This thought … disturbing, somehow?
This is just normal though. We were building sophisticated bronze and steel tools long before any complex understanding of metallurgy or chemistry. Medicine is still the wild west.
I first wanted to say fire, even though it's a cliche, but then I thought that in antiquity we used like everything without anything that would qualify today as understanding. Also now we have a lot of stuff that we "know" it works based on complicated numerical simulation.
I think the most "understanding" we ever had was in the 40s-50s designing nuclear bombs with slide rules. It was the culture that produced the idea of psychohistory.
> Medicine is still the wild west.
Reminder that we have no idea how anesthesia works.
But one of the things AI also excels in is summarizing and can do so hierarchically. One of my favorite things to do with a concept I'm new at is "ELI5" then "explain like I'm a high school student" then "explain like a bright undergrad in XYZ" then "explain to a working professional in this domain". It's a lot of steps, but I've found it very effective (for me) to learn with -- and I've done something similar with code and math (although not math proofs -- I'm not a mathematician). But my point is that I think we can use AI to also teach us these proofs they're building in a way that I don't understand today about human proofs.
If we use AI well here we could actually understand math much better than we do now.
I actually think it's the opposite: Lean proofs and autoformalization make it very easy to announce proofs alongside proofs of the correctness of those proofs (Lean certificates). It's not an absolutely fool-proof combination (the Lean kernel could still contain bugs), but it does immediately attach a very substantial degree of credibility to the result.
And that I think is essential to why some of the world's leading mathematicians are taking this so hard. In a world where we "merely" have AI systems capable of superhuman informal reasoning, verification, correctness, and acceptance could still only be conferred or anointed by human mathematicians. But a world that combines superhuman informal reasoning with superhuman autoformalization is a fundamental shakeup in the institutional order.
The recent proof of Fermats Last Theorem is interesting: it is (iirc) 13 million lines of lean code. And type-checking takes 5 hours or so on a pretty beefy machine. I cannot independently verify the proof, and I have to take Anthropics word for it that it actually type-checks.
That seems like a red herring. Have you independently verified the human generated proof of FLT? Surely someone else will try to verify Anthropic's formalization on different hardware. Plus, it seems likely that FLT formalizations will improve / get shorter over time, requiring less compute. And computers (and type-checkers) will continue to get faster over time as well. So maybe in 5 years you could own a computer fast enough to verify a/the proof in say a week, instead of 5 hours.
With formalized math, you only need to validate the problem statement (in theory, in practice agents have already managed to exploit Lean compiler bugs, but the incidence of those should decrease enough to be practically lusable for 'blind' validation of AI proofs in the foreseeable future).
> resulting in the production of an abundance of PDFs. The contents of some of those PDFs may even have important applications.
I hope, from the depths of my soul, that the static typeset report format for transmitting knowledge and understanding will finally die and be laid to rest.
As a researcher in theoretical computer science, I love PDFs more than any other format when it comes to mathematics.
There simply is no contender to LaTeX and PDFs.
Lucky for you, almost all research in math, cs, and physics, are put on arxiv, where you can download the source code (.tex) as well as get an HTML render.
I love how this is (without slighting the problems entailed) coming at it from a perspective of infinitude and abundance (we will always have more problems to solve) - which is the correct framing, particularly when dealing with ideas, or fields in the which ideas are the driver/product/output/material, and ideas themselves, the field itself, are infinite.-
1. It will take much longer to understand the output of the machine that it takes to prompt and create it.
2. The only? best? one? way to /verify/ that you /in fact/ understand the output of the machine is to explain it to someone else.
So there will be a machine generating koans which need to be meditated upon and discussed with human social back-pressure validating understanding. I think this could be much more cooperative and at a minimum this will be a way different math social construct.
The more I see these posts about mathematics institutions reforms and challenges from AI advancements the more it looks like they may need to go through a death. Or to put it another way they may need to start again from first principles.
If math is truly about spreading intuition and understanding then our institutions have dropped the ball decades ago and have not been able to grab hold of it since (if they ever had it to begin with)
Not sure why you're down the bottom when the current top post says pretty much the same thing. I agree that the reevaluation and refocusing that is being forced by AI is one the maths establishment could fruitfully have had a long time ago.
Better plan is to shut down PhD and make students take a oral thesis at bachelor and master level and help them become a productive economic participant as soon as possible.
> I propose the following reconceptualization of the goal of a mathematics PhD: to become a world expert on some interesting, deep topic, and to be able to convey that interest and understanding to others. Part of operationalizing this might be a thesis, but the degree would be awarded primarily on the basis of a rigorous defense, in which the student explains the topic to their examiners until they are satisfied.
I think this is a refreshingly forward looking idea and I agree with it 100%, especially the the "rigorous defense" part. That is a good measure of how well the topic has been researched and understood by the researcher. This is where the humans can be "in the loop".
> How different would this look from current PhDs? I think students would still meet with an advisor, who might suggest a topic. That topic could be explored with AI assistance, or not, but the student would be responsible for understanding it; it might be much more open-ended and larger than the typical PhD is currently.
Interesting point about "more open-ended" and "...larger than the typical PhD". I think the author has a point. Earlier, the bottleneck was the candidate's/researcher's understanding and knowledge. Now with AI tools, it is so much easier to zero in to relevant knowledge, get your questions answered quickly which might lead to understanding more quickly.
For e.g., before the advent of public libraries and printing press, the knowledge was inaccessible and guarded. So that was the bottleneck.
Then books became ubiquitous and the bottleneck to knowledge and understanding was people's motivation AND knowledge of WHAT books and topics to research.
Then came the internet and free PDFs of books and research articles. Now, the bottleneck was still people's motivation and a mild version of what books and topics to research. I say "mild" because one can lookup articles and newsletters, and book reviews and come up with a list of reading.
Now comes AI and it looks like the only bottleneck is people's motivation.
I believe there was also a silent, yet potent, bottleneck all along which is also removed by AI: personal tutor/coach/teacher/professor etc. Let's say if I am reading a textbook on manifolds or some research paper and I have a question about a specific theorem or even a mathematical operator being used. Before AI my only way to get my questions answered was to read more books (PDFs or print), or ask on math exchange or math overflow and wait for someone to answer, or to ask a professor. This could take up to a week.
Now all of that has been cut down to 1 hour or less with an interactive chatting session.
!!!!!
So....the only bottleneck is people's motivation! QED
The idea that most any modern "interesting" aspect of mathematics is going to be understood (or often even explained in enough detail to reveal what is interesting) in an hour is pretty rare. Either the student's aptitude, the tutorial, or the mathematics are unique. There is a reason that these are PhDs and not undergraduate HW sets.
I think we often delude ourselves as to how well we understand problems and their solutions. Some instructors even make you feel that you understand better than you do by pointing to a few approximations or simple solution spaces that obscure the larger complexity. Just looking in wonder at the many categories of three-body solutions (currently on hnews) is enough to remind me of this.
I'm not sure why an advanced degree is necessary for that at all, besides the pride of a vanity title. It's pretty much what Bill Nye does for science.
Another bottleneck is money required to pay corporations who own the technology to "do mathematics". In that future, there will never be another Ramajunan.
> I think so. This machine might produce answers we value, but it would not, in itself, produce human understanding of those answers.
It's nice that the author is optimistic, but won't the AI be best placed to dumb down its increasingly complex proofs into a language us lowly humans can understand? To keep thinking until it can refactor complex proofs into ones from 'the book'?
As they go on to explain, a human understandable proof is different than a human actually understanding the proof. That actual human understanding (like, in a brain of a human) is one of their stated goals.
Producing human understandable proofs is possibly a job best for humans today, but the author appears to agree with you that this is probably fleeting (and argues that even if you disagree, it should probably be treated as if it is fleeting when planning for the future):
> Right now AI systems arguably underperform us at theory-building, asking questions, exposition, … so we could prioritize and reward those skills. I think this is unwise: compare the speed at which the academy adapts to the speed at which model capabilities improve. We need to consider the endgame. If the models remain incapable in some domain, we can adjust later.
Another mathematician waxing poetically about a future that will not come to happen. Call me a pessimist if you'd like, but capital has no incentive to ensure mathematicians maintain their current status in society.
If you're a mathematician you are in the same boat as the software engineer, and the Dodo.
By essentially the same reasoning, I’ve been arguing for prioritizing in-person design/code reviews over code-only async PR comments.
The important thing is to verify that the human has a coherent design in mind and can demonstrate that it got implemented, regardless of who or what was at the keyboard. “I dunno, I guess Claude thought this was a good idea” is not a coherent design.
Oral defenses in academia are largely rituals. If a student fails their defense, it's almost always the supervisor's fault. The supervisor is supposed to be the primary quality control. With their regular meetings with the student over the years, they should be able to tell whether the student has achieved sufficient understanding and contributed enough to graduate. If the supervisor thinks the student is ready to defend, the reasonable expectation is that the student will pass and graduate. The defense is mostly there to let the other examiners validate the supervisor's judgment.
The defense was only briefly about the actual thesis, then switched over to whatever research interest the committee members had, and they'd drill into their pet subjects. This was in physics, hardly politics forward normally, and at a very well respected university.
Manager: “Why isn’t feature X available?”
Person B: “key pieces are delayed due to the developer not understanding all of the LLM doesn’t and implementation.”
Manager: “does it work? What are the risks?”
Person B: “well yes it works for now but we’re accumulating tech debt due to a lack of understanding and potential flaws that haven’t been thought out yet”
Manager: “they want feature X, ship it, we can deal with it later, I don’t care if it’s not coherent as long as it works.”
How many decades at this point has these been a push for functionality over everything at all costs? And you have a mechanical snow plow now. Most businesses don’t care about later risk or any future planning beyond the quarter horizon, they’re not concerned about how it will effect their performance in 3 quarters or lead to instability or issues, those are future problems for a future person and we’re here for money now.
AI writing is aggressively, aggressively mediocre. It is only good for how cheap it is. If you're indifferent to how much better good human writing is than AI writing, you should not be the one to evaluate human writing.
https://www.maths.tcd.ie/pub/Maths/Courseware/ProblemSolving...
I agree making simple things sound complicated to appear more impressive is bad but there are limits. Even with Feynman he could only go so far, e.g. his interview about why questions and magnetism.
Result is more software at lower quality. The reason is statistics. When you increase the population, you increase the population of every kind of programmer, and people who want the result are favored in most competitive sectors because corporations want something somewhat working yesterday.
...and here we are.
Now programmers talking about code quality is stoned en-masse. If it's somewhat working then it's good. Efficiency, resiliency, maintainability and sustainability is an afterthought. Some of my friends who loved debating programming language theory now don't even care about the code. They don't write it, just vibe, and they don't plan to come back to "older, caveman style of development".
Some universities are also adding fuel to the fire: They "prepare students for the job", not teaching the science, but the parts that corporations need for the job only.
Though, hardware was cheap and people were expensive, and now code is cheap and hardware is expensive now. We'll see.
> we lost the joy of the journey and only fixated at the destination.
that's because you can make money with code and so it went the same way as everything else that's capitalized/commodified.
here a better comparison would be art which is generally still pursued on its own merits/pleasures.
When you look at older software, most of it was higher quality than the things we have today. A web site contained more information in a more readable way, more features in a smaller footprint. Same for native applications.
Now we slap what we found online together and calling it done. Everything is sparse, takes ages to load, centuries to submit and everything is so disconnected and async that some simple features are straight out impossible.
This is what fixating on the destination brought us. I dare you to download something you purchased 3 months ago via a 4096 character S3 link they have sent you, and double dare you to ask customer support for a new link. I'll bet that with a 80% chance they have no way to verify your serial number, even.
Somebody said that lowering the bar is not good. Somebody else asked for examples, and I provided an example. So, if you want to be pedantic, that doesn't track well, because what I answered was not about mathematics.
If you want something about mathematics, computation is mathematics, as software is. So, my example tracks the same way in mathematics.
Finding solutions without understanding its parts or the path is equally detrimental to mathematics as it is detrimental to software.
Maybe you need to read a bit slower and think along the way. Using AI too much blunts critical thinking skills in some, as I read.
What was the end of the proof thing you mathematicians use, was it "Q.E.D."?
Q.E.D.
brother i already addressed this literally in my first response to you: the reason software went to shit is because it became commodified (ie a thing produced in a factory) not because the bar got lowered. the bar for being able to weave also got lowered hundreds of years but your mom's crocheting doesn't suffer from the same ills as the software industry.
> Using AI too much blunts critical thinking skills in some, as I read.
irony
What about the crafts? Carpentry, blacksmithing, stonesmithing, etc. are all commodified yet people still pursue these for pleasure.
>this comparison makes little sense.
Stop gatekeeping what people find personally fulfilling.
Not that I actually agree that math was at the right level of gatekeeping. It definitely feels intentionally opaque beyond reason, and I think it's why LLMs are able to cut through the obfuscation and solve problems that maybe wouldn't actually have been considered quite so hard if mathematicians did a better job of making their work accessible.
They hire PhD students without hearing them give a talk and then doing interviews? In Germany, the applicant gives a talk (30-40 min) to the research group they want to join, usually presenting their master's thesis, engage in discussion, often share lunch with the group, then do 1 on 1s with individual members of the group and a longer one with the PI. Obviously this can vary within Germany too, but I couldn't imagine hiring someone without something like this.
People had the same complaints that the code produced by early coding models was messy, lazy, poorly commented, and so forth. The central complaint was that it was just too difficult for humans to review. The answer is just to improve the models and move on.
Similarly now we're getting AI doing math. The math is a giant vibe-coded ball of wax. So just make the models better at explaining what they're doing to humans - that's the end of it.
Rather than just go on and on about how it's the end of the world if we don't do this, why don't we just do it?
I thought we were going to get at least some brief comment on Godel here?
With AI, it might be the case that we don't know, we won't know, but the machine does.
The central question, namely whether humans should be in the loop, will be repeated again and again in the years to come for all industries, starting with mathematics.
In code, any developer whose differentiaton was clean code and knowledge of different technologies is now average.
In math, any mathematitian whose differentiation was to manipulate formal systems and know tricks of different domains will be average.
Fortunately, humans do more than syntax and search.
The bad news for developers is that if you know what the output of your program should be (which happens most of the time), almost all of the job is syntax and search to build the code that reproduces the output.
The good news for mathematitians is that for the majority of problems you never know the output, or you just know the output is either "True" or "False". There are some cases where you need something else, for example "a solution that blows up in finite time". For those cases AI will outperform you easily (see new Navier-Stokes solution)
So if as a mathematitian you were doing more than syntax and search, then keep doing that and use AI just for what its best.
* A weightlifter was only awarded a laureate if he were able to lift a heavy stone (have no idea what they were lifting, for illustrative purposes only :-)
* Along comes Archimedes who invents what we would call an exoskeleton. Now any regular guy can lift twice as much as last year’s athlete.
* What to do? You can cancel the Olympiads, but they are actually useful as training, motivation, etc So now you have to give the prize on other factors, eg how well he can lift, has he opened a gym in the city, etc
BTW, physics and bio are not exempt, so those researchers better read and try to stay ahead.
For the reality, you can´t close the loop that fast, or with that precision. You will have to slow down by several orders of magnitude.
Impossibility to independently validate all AI results
And in math it goes even worse. In coding code reviews are typically still the form of action you do within days. In math, historically, the lifecycle of proof is months if not years. Take as an example Millennium problems. They require at least two years of validity after publishing. Two years! In modern times with amount of output AI can produce, it feels like infinity.
We are inches close if not at the moment already when humans can’t reliable validate proofs and mathematics produced by AI. Then next research will be based on this AI-written-no-human-in-the-loop results. And we will end up in just few years in a world where novel and frontier problems will be articulated by AI and proven by AI based on AI results and humans will be incapable of understating the mere nature of the solution.
And, yes: that doesn't absolutely guarantee correctness. The Lean kernel has had soundness bugs, and may have some still. But it's pretty strong evidence of correctness nevertheless.
The concern among mathematicians is not mainly that they doubt the correctness of any of these discoveries, but that human understanding may be devalued.
It is perfectly valid counterpoint to say that we already do it. We everyday use myriad of things, tools, and software we have 0 clue how it operates. But for us as humans it was reassuring that we know that at least there are a few other alive humans who know it, who create it and who can explain it.
With AI soon that comforting zone will be gone.
This is just normal though. We were building sophisticated bronze and steel tools long before any complex understanding of metallurgy or chemistry. Medicine is still the wild west.
I think the most "understanding" we ever had was in the 40s-50s designing nuclear bombs with slide rules. It was the culture that produced the idea of psychohistory.
> Medicine is still the wild west.
Reminder that we have no idea how anesthesia works.
Do any one person even understand the humble pencil?
https://dn790006.ca.archive.org/0/items/i-pencil-pdf-2019/I%...
If we use AI well here we could actually understand math much better than we do now.
And that I think is essential to why some of the world's leading mathematicians are taking this so hard. In a world where we "merely" have AI systems capable of superhuman informal reasoning, verification, correctness, and acceptance could still only be conferred or anointed by human mathematicians. But a world that combines superhuman informal reasoning with superhuman autoformalization is a fundamental shakeup in the institutional order.
I hope, from the depths of my soul, that the static typeset report format for transmitting knowledge and understanding will finally die and be laid to rest.
There simply is no contender to LaTeX and PDFs.
Lucky for you, almost all research in math, cs, and physics, are put on arxiv, where you can download the source code (.tex) as well as get an HTML render.
https://ciechanow.ski/archives/
...for starters?
PS. The validation problem, being one.-
1. It will take much longer to understand the output of the machine that it takes to prompt and create it. 2. The only? best? one? way to /verify/ that you /in fact/ understand the output of the machine is to explain it to someone else.
So there will be a machine generating koans which need to be meditated upon and discussed with human social back-pressure validating understanding. I think this could be much more cooperative and at a minimum this will be a way different math social construct.
If math is truly about spreading intuition and understanding then our institutions have dropped the ball decades ago and have not been able to grab hold of it since (if they ever had it to begin with)
I think this is a refreshingly forward looking idea and I agree with it 100%, especially the the "rigorous defense" part. That is a good measure of how well the topic has been researched and understood by the researcher. This is where the humans can be "in the loop".
> How different would this look from current PhDs? I think students would still meet with an advisor, who might suggest a topic. That topic could be explored with AI assistance, or not, but the student would be responsible for understanding it; it might be much more open-ended and larger than the typical PhD is currently.
Interesting point about "more open-ended" and "...larger than the typical PhD". I think the author has a point. Earlier, the bottleneck was the candidate's/researcher's understanding and knowledge. Now with AI tools, it is so much easier to zero in to relevant knowledge, get your questions answered quickly which might lead to understanding more quickly.
For e.g., before the advent of public libraries and printing press, the knowledge was inaccessible and guarded. So that was the bottleneck.
Then books became ubiquitous and the bottleneck to knowledge and understanding was people's motivation AND knowledge of WHAT books and topics to research.
Then came the internet and free PDFs of books and research articles. Now, the bottleneck was still people's motivation and a mild version of what books and topics to research. I say "mild" because one can lookup articles and newsletters, and book reviews and come up with a list of reading.
Now comes AI and it looks like the only bottleneck is people's motivation.
I believe there was also a silent, yet potent, bottleneck all along which is also removed by AI: personal tutor/coach/teacher/professor etc. Let's say if I am reading a textbook on manifolds or some research paper and I have a question about a specific theorem or even a mathematical operator being used. Before AI my only way to get my questions answered was to read more books (PDFs or print), or ask on math exchange or math overflow and wait for someone to answer, or to ask a professor. This could take up to a week.
Now all of that has been cut down to 1 hour or less with an interactive chatting session.
!!!!!
So....the only bottleneck is people's motivation! QED
Exciting time!
I think we often delude ourselves as to how well we understand problems and their solutions. Some instructors even make you feel that you understand better than you do by pointing to a few approximations or simple solution spaces that obscure the larger complexity. Just looking in wonder at the many categories of three-body solutions (currently on hnews) is enough to remind me of this.
In my country, that's exactly how it is.
Yes, you need to have a thesis to defend, but ultimately it all comes down to the (oral and live) defense/disputation.
What?
"a rigorous defense, in which the student explains the topic to their examiners until they are satisfied."
is exactly how PhDs were awarded for hundreds of years.
Even my BSc in Applied Physics (1977) had a viva voce that was a substantial fraction of the final exam.
That's a radical departure from "PhD" being a certificate that someone is qualified to produce new research.
What you describe is more like a Masters Degree.
PhD has nothing to do with expertness.
If you have a PhD, you have completed some kind of research training.
That's all there is. Says nothing about knowledge or whether or not you're a genius.
You cannot conclude anything else, and nobody claims that you can.
If someone has a PhD, they have some training in doing research.
Perhaps a parallel to this is what's happening with youth sports in the US. It is becoming increasingly inaccessible.
It's nice that the author is optimistic, but won't the AI be best placed to dumb down its increasingly complex proofs into a language us lowly humans can understand? To keep thinking until it can refactor complex proofs into ones from 'the book'?
Producing human understandable proofs is possibly a job best for humans today, but the author appears to agree with you that this is probably fleeting (and argues that even if you disagree, it should probably be treated as if it is fleeting when planning for the future):
> Right now AI systems arguably underperform us at theory-building, asking questions, exposition, … so we could prioritize and reward those skills. I think this is unwise: compare the speed at which the academy adapts to the speed at which model capabilities improve. We need to consider the endgame. If the models remain incapable in some domain, we can adjust later.
If you're a mathematician you are in the same boat as the software engineer, and the Dodo.
Better learn a trade buddy /s