Sorry, complete tangent, but on the point: "Chimpanzees use tools in the sense of poking sticks into anthills, whereas human beings use tools in the sense of extreme ultra-violet lithography, and presumably intelligence is the thing that makes the difference."
There's a bit of a sleight of hand here; almost every chimp can figure out the stick in anthill trick after even a rough and quick exposure,, but how many humans out of 8 billion figure out Ultra-violet lithography even after being exposed to the idea in some depth? Is *individual* intelligence the real deciding factor here?
I'm reminded of the scene from I, Robot (the movie), where Will Smith asks the robot if it can write a symphony, and the robot replies "Can you?"
I guess the steelman version is whether you compare a median chimpanzee to a median human, or a top percentile chimpanzee to a top percentile human, in both cases the human will be able to demonstrate some form of tool use or problem solving that the chimpanzee cannot.
But to your point, the fact that humans (as a group) outperform chimpanzees (as a group) is probably due to "culture" in addition to individual intelligence. But on the other hand, this "culture" is probably dependent on some base level of intelligence in the individuals.
That line always bugged me a bit. I think that most people could write a symphony, I assume Will Smith's character could. It wouldn't be a good one, but I think he probably could, even if he needed someone to help with the notation.
... Have you written a symphony? I've got more musical knowledge and training than most and I'm confident I couldn't do it in a meaningful sense in a medium term scope. I would need so much help, 'scaffolding', etc that I'm not sure there's a meaningful sense in which I could write a symphony in the next couple of years that wouldn't be either so simple as to not deserve the name or just sounds like noise.
Actually, I realise I'm unclear about the nature of your statement. Are you saying:
1) Symphonies specifically aren't that difficult, such that most people would be able to write a decent one given sustained effort (but there exist other acts, e.g. rocket design, which are not accessible in that way)
Or
2) Symphonies *are* difficult, but humans are just generally very adaptive problem solvers for difficult things in general given the right learning opportunity.
First, your definition would ruffle the feathers of many a self-proclaimed "genius" who purportedly crumbled under the pressure of parent/school and now work as retail clerks. Anyway, I was a terrible under achiever, and should have been held back, failing classes all throughout school and even the crummy regional college I later attended. Fortunately, the whole innate intelligence thing seemed to miss me as I was gearing up for the LSAT. Many people online said that one cannot meaningfully improve his/her own score because it's essentially a bunch of logic problems, with little to now formal "knowledge" tested. I refused to believe this and spent almost two years studying, getting terrible scores over and over again. But something inside me refused to believe that I couldn't get a good score if I didn't study hard enough. It never occurred to me, then, that maybe I just didn't have the stuff. I swear that over the course of studying, something changed. My brain came online or something. For the first time, I learned to problem solve, and like growing pains, it was anguishing initially, but by and by, my scores readily improved until I got a top percentile score. These habits followed me into law school, and allowed me to excel there, paving the way for my top-choice job. Certainly, I'd concede that there's outer limits, but improvement can happen, and our brains are a lot more adaptable than the innate camp would admit. There's no doubt such an "innate" mindset would have mentally crippled me had it taken stock back then.
That was interesting, Thank you. In my previous life I built physics apparatus (for advanced teaching labs.) It was a great gig, and I identified two types of problem solving. The first I call trouble shooting*. This 'thing' (piece of apparatus or maybe a car) worked before, and now it's not working. What is wrong? The second I called debugging (for lack of a better word). You are building something for the first time and it's not working. What's wrong? This second is much harder (in general). You don't know if it's something in the apparatus that's not working right, or if your 'theory' for why it should work has an error in it. The potential problem space for the second is much larger. So I just wanted to add some comments from the trenches to your excellent book review.
1.) Don't jump to step two too soon. There are hundreds of things that could be wrong, picking the 'most likely' one and pursuing that can waste a lot of time and money. Before making a plan get as much data as possible. Learn as much about the problem as you can. That said there is something to be said for trying the 'simplest' solutions first. Is it plugged in? Is there gas in the tank? ... etc.
2.) When stuck explain your problem to a smart audience. I think the 'smart' is important here. You have to think that your audience can help you. You need to explain your problem in depth and with sincerity. I would often use my boss for this task. Or a group of online electronics guys on Usenet. Often laying out the problem for the smart audience would let me see the solution without actually getting any help from the smart person. You are deep into the problem and going back to the beginning is what is needed. "Oh I made this assumption at step 3, and maybe that's not right."
3.) Sleep on it. Let your unconscious help. It's amazing how many solutions come to me either during the morning shower, or commuting to work in the morning. Your brain is doing some mundane task you've done hundreds of time before, which frees up the back of your brain to find the needed connection. I have no idea how this works, but it does!
*There is a corollary to trouble shooting, which I called remote trouble shooting. You are trying to fix a problem over the phone and /or by email... a new set of fun challenges.
Your advice about "design it twice" reminds me of a story about an engineer at Meta that when given a task to be completed in two weeks, would actually design it three times. He would design for a week, throw away his work, design for a few more days, throw away his work, then would concentrate his efforts on the very last day to rebuild it as he sought.
"...intelligence is the ability of an agent to achieve its goals."
I like this one a lot. It fits well with my opinion that intelligence is a category, not an attribute.
Take two people who have different learning styles. The first learns extremely rapidly. It's been said that the average student need to hear concepts 26 times before they understand; this person has grasped them after the third time and is ready to move on. But they are also more forgetful. If they don't use a concept for a few months (or more), it's gone. This isn't a problem for them as they can re-learn something when needed even quicker than the first time they learned it.
The second person needs to hear the concept all 26 times, but once they know it that concept is set. They don't have to re-learn anything; they can just keep building on the concepts they know. Additionally, they're more persistent and less likely to get distracted.
Our current system will say the first person is much more intelligent than the second. A lot of IQ is simply how fast you can learn. But the second person might be better at achieving their long term goals.
That is an interesting observation about the speed of learning. A conclusion I reached from years of observation of business practice was that the kind of people who got promoted to high level were often "intellectual sprinters". They were highly intelligent, able to assess situations very quickly, and come to a decision. All of that was great -- provided the problem was the kind that could be fixed quickly.
There were other people who were "intellectual long-distance runners". They tended not to be the kind of people who got promoted to high level positions because they were more deliberate (slower) about assessing problems. On the other hand, they might continue to grind away at a problem and eventually reach a better solution than the sprinter. The sprinter would of course then adopt this better solution and get the credit!
Maybe some of the smart people who read this Substack have also seen what I'm about to describe and can offer a explanation. It regards Pólya's four essential stages of problem solving: (1) Understanding what it is you’re trying to do; (2) Coming up with a plan; (3) Carrying out the plan; and (4) Reflecting on how it went.
For a few years I was the elected county attorney for a small rural county. I found that the county employees and other elected officials had a violent aversion to all but stage (3) of the four stages. That is, they were unbelievably hostile to any attempt to figure out what needed to be done, to come up with a plan, and to reflect on it afterwards. In fact, I once heard a county commissioner say in a public meeting, "We're going to do something even if it's wrong." I've seen this same aversion in clients in in other areas of legal practice and also in nonlegal endeavors.
Has anyone else seen this? Any idea what's behind it? Or have I just lived an worked in a place that has too much lead in the water?
I have seen this. It comes in 4 flavours I have identified. 1. The people involved do not have enough imagination. Sometimes this is just because there is none there. But often it is because they have had their imagination and initiative punished whenever they used it in the past, so now they reflexively shut it down whenever it might show up. The fix is to get these people to play again. Like a lot of fixes, it is a lot easier said than done.
2. They're frightened. They think they might say something embarassing or foolish and they aren't willing to.
3. They're not interested in solving the problem. They just want to put in the required hours, meetings, what have you so that they can look busy enough, deserving enough, what have you. Meanwhile things continue on the way they always have because the status quo suits them. They didn't get elected to solve problems but to manage them. This is a particular problem with government. You might think the problem that needs solving is "better trash collection" but the problem the others are interested in is "managing the patronage networks including the one where the trash concession is awarded". At this point nobody wants to talk about the problem that they are really solving, which might be of questionable legality .... but of course they don't want to talk about what you see as the problem either. Always look at problems of this sort in terms of "who benefits from having things continue precisely as they are". Sometimes it becomes obvious.
4. They are lazy.
There are certainly other reasons, but those are the ones where I stub my toe most often.
Hmmm. I forgot. In government sometimes the political problem is that if you solved the technical problem the wrong people would get the credit. You, for instance, if they don't like you.
That's very much my experience doing continuous improvement/system implementation in industry as well. People don't like it when the problem with production planning is the general management demanding arbitrary increases in sales that won't happen instead of the MRP system, and the people who don't like it will fire the people who point that out. So the real issue never gets fixed, and no one knows why all those initiatives happen and things remain basically the same.
Politicians and government employees are averse to anything that requires doing something different than what is currently being done, because they desire above all to not be held accountable for negative results. If the status quo existed prior to their involvement, then they can't be blamed for the policy. It's their predecessors fault. If it's their policy, then changing it is an admission that it needs to be changed and that just won't do. Better to claim that "they" (whoever can be blamed, or a nebulous opponent) are sabotaging the current plan and it only needs to be done harder. (This became blatantly obvious during the COVID shutdowns and mask mandates.) All three of the steps they want to avoid are in conflict with this.
There's a standard joke in the UK about how politicians behave: "Something must be done! This is something, therefore it must be done!" It's a product of a media environment which relentlessly punishes "flip-flopping" (otherwise known as "learning from your mistakes").
The other political failure mode is that when they *don't* want to do something they establish a commission to extensively study the thing and provide recommendations for possible actions. By the time the commission reports then the media storm around the thing will have blown over and they'll be free to ignore the recommendations.
"One of my quack beliefs is that both Eudoxus of Cnidus and Archimedes probably figured out at least the basics of calculus ..."
The challenge that both of them would have faced is -- lack of the necessary tools for the job. Mathematician Raul Rojas wrote an interesting book "The Language of Mathematics", unfortunately besmirched by the insistence of the ladies at Princeton University Press on using the ridiculous BCE/CE dating system. Rojas credits Diophantus in the 3rd Century AD with starting the process of developing symbols for mathematical operations. It is hard enough to see how the Romans built their magnificent edifices without positional notation or the number zero -- but how would calculus have been possible without an equals sign?
It is interesting that both Leibnitz and (less successfully) Newton had to expand an already rich set of mathematical symbols in order to develop calculus. Without the basic symbols in place, which developed slowly between the 3rd and 17th Centuries AD, the challenge of developing calculus would probably have thwarted even Leibnitz and Newton.
Archimedes got very close to figuring out the basics of integral calculus, at least the high-school hand-wavey version: https://en.wikipedia.org/wiki/Archimedes_Palimpsest#The_Method_of_Mechanical_Theorems He himself knew that his methods weren't rigorous and treated them as useful heuristics - no shade on him, since it took some of the best mathematicians in Europe a century of work to make Newton and Leibniz's work rigorous!
Outstanding piece on reframing intelligence as measurable impact rather than mystical talent. The toolbox metaphor lands perfectly since most "breakthroughs" in my career came from recognizing a familiar pattern in unfamiliar contexts, not some Eureka flash. What gets overlooked is how much grit compounds with heuristics - having mental moves only helps if people stick around long enough totry them all. I dunno if the ancients were missing training data or just missing the grindset neede to iterate across generations.
Your discussion of the phase 2/phase 3 split reminded me of How Big Things Get Done by Bent Flyvbjerg and Dan Gardner: https://www.amazon.com/How-Big-Things-Get-Done/dp/0593239512. I loved the book, and think it's very relevant to your interests!
"Eventually you learn to notice its spoor, the rank taste in the air, “a problem has passed by this way, moving downwind, two days ago.” One of the many ways school fails us is by actively harming this capacity, it lies and lies to us for decades, teaching us that good problems will be delivered on a silver platter. This is why so many people who do well in school never amount to anything. They never develop a taste for the hunt, never learn that this, actually, is the most important part of the entire site survey: “is this problem worth solving by anybody?”, “am I uniquely well-positioned to solve it?”, “can I amass the resources to solve it?”, “do I have any chance of success?”, “is there some other problem that it is more valuable for me to solve?”"
If you never take the novice hunting and only take them to the range and teach them how to cook venison....
This article delights the system engineer and maker in me. I will need to look into the book...
...and the science fiction enthusiast wonders if there is still a tesseract angle I could turn to grasp the plasma rifle just beyond my perception if I only knew how (Hat tip "The Universe Between", Alan Edward Nourse.)
Is your general relativity caveman allowed to act? Or is he disabled in some way and just has to sit at the back of his cave and watch things? If he's stuck in the cave I don't see how he's supposed to tell the difference between a universe that runs on GR and one that has Newtonian gravity even once he's generated both as hypotheses.
There’s maybe a whole other AI angle to this? I hear a lot about how RL is so much less efficient than pretraining because you do a ton of tokens of rollout and get one bit of information at the end but polya’s post-proof suggestion of re-proving and reflecting on the blind alleys and successes of a proof seems like a way to increase RL information density. Some kind of “post-solution reflection” for RL?
Hmmm I’m not sure it’s the same though. Like my tenuous read on the deepseek strategy is that it’s basically an improved process reward model just with an external LLM2? But LLM2 is still checking correctness of steps it may not know a full proof of the relevant theorem. I’m also really confused why their thing works so well since it seems like they’re just going 2 steps into an infinite “who watches the watchmen” regress (what if LLM3 is wrong?) but that’s not relevant to the main point I think.
The closest implementation of the Polya thing would be like “after you do a proof, keep that proof in context and try to write a more efficient proof, or prove it a bunch of other ways, or see how you can apply that proof to other things, etc”. In contrast, with existing RLVR once you have a correct answer you basically stop
1. would you consider "exploring" to be a valid step 2 of planning?
2. can you say more about pregnant problems? i am worried that haphazard adding of words to the problem definition may direct the solver in a wrong direction. where does this terminology come from?
I am not entirely sure about this article’s statement that Pólya's method is "NOT the same thing as the Socratic method".
"When Antisthenes was asked what profit he had derived from [Socratic] philosophy, he replied: 'The ability to converse with myself.' The intimate connection between dialogue with others and dialogue with oneself is profoundly significant." (Hadot, Philosophy as a Way of Life, p. 91).
Note what Antisthenes is not saying here: the primary benefit of philosophy, for him, is neither ἀπορία in general nor some particular philosophical truth; rather, it is that set of thinking skills by which one may "converse with [oneself]". If I understand correctly, this is precisely what Pólya was trying to teach!
Looks like I have a lot to say that I suspect you're aware of and have read the same studies or summaries than me.
> On the other hand, we (unofficially) treat intelligence as a proxy for moral worth
I'm surprised this isn't a link to Freddie DeBoer's The Cult of Smart book.
> One unusual thing about math is that if you understand your starting point and your ending point sufficiently well, then you’re already done.
Can confirm from experience, but I was always told this holds mostly on the algebraic side of the great algebra/analysis divide (where footnote 7 might apply more). I was mumbling "Grothendieck" at this point, I'm very happy to see him appear later on.
> not an exercise, mind you, but a problem
The only way I can think of pushing back here is to invoke Terry Tao myself: https://terrytao.wordpress.com/career-advice/theres-more-to-mathematics-than-rigour-and-proofs/ it seems to me that getting from stage 1 to 2 is where exercises help you, and as long as that's not viewed as the end goal that's ok. (Very few undergraduates in math these days will come up with a theorem that will bear their name in future, so we have to settle on problems at a difficulty level that a research mathematician would call "exercises".)
Back to dual n-back and Gwern's links to programming: one way to free up working memory is to store things in muscle memory, for example by learning to touch-type properly. That means "grinding" typing exercises, because you're not trying to solve open problems in typing you're trying to take it off the critical path to your math/programming problem in the first place. (I feel like I should have come up with a bad pun about type theory here, but failed. Programmers will know what I mean.)
And, as you say later (paraphrased) small successes -> hope -> endurance -> solve big problem, so as long as your exercises are not too easy, as long as getting them right regularly produces a "Counter Go Up" effect, that's a causation chain even if the exercises are not teaching you the math you need in the end.
> when a student is stuck, the teacher shouldn’t give any hints
At which point the future student will just ask the AI :( It doesn't invalidate the argument that that's one of the best teaching methods, just that ... AI will gain superhuman intelligence by reversing the Flynn effect with its instant answers?
> Archimedes probably figured out at least the basics of calculus but the knowledge died with them.
> learning to recognize not *solutions* you’ve already seen, but *approaches* you’ve already seen
Chi et al., Categorization and representation of physics problems [ed: exercises!] by experts and novices, Cog Sci 5 (1981):
Exactly! Novices tended to sort exercises by surface features ("these ones have a right-angled triangle in it") whereas experts sorted by domain-specific structure ("these ones are conservation-of-energy"). "Does this problem remind you of another one?" pays off when you realise that you're after structure, not presentation.
Sorry, complete tangent, but on the point: "Chimpanzees use tools in the sense of poking sticks into anthills, whereas human beings use tools in the sense of extreme ultra-violet lithography, and presumably intelligence is the thing that makes the difference."
There's a bit of a sleight of hand here; almost every chimp can figure out the stick in anthill trick after even a rough and quick exposure,, but how many humans out of 8 billion figure out Ultra-violet lithography even after being exposed to the idea in some depth? Is *individual* intelligence the real deciding factor here?
I'm reminded of the scene from I, Robot (the movie), where Will Smith asks the robot if it can write a symphony, and the robot replies "Can you?"
I guess the steelman version is whether you compare a median chimpanzee to a median human, or a top percentile chimpanzee to a top percentile human, in both cases the human will be able to demonstrate some form of tool use or problem solving that the chimpanzee cannot.
But to your point, the fact that humans (as a group) outperform chimpanzees (as a group) is probably due to "culture" in addition to individual intelligence. But on the other hand, this "culture" is probably dependent on some base level of intelligence in the individuals.
And I guess it's also worth bringing up that there exists specific intelligence tests where chimpanzees outperform humans: https://www.youtube.com/watch?v=nTgeLEWr614
That line always bugged me a bit. I think that most people could write a symphony, I assume Will Smith's character could. It wouldn't be a good one, but I think he probably could, even if he needed someone to help with the notation.
... Have you written a symphony? I've got more musical knowledge and training than most and I'm confident I couldn't do it in a meaningful sense in a medium term scope. I would need so much help, 'scaffolding', etc that I'm not sure there's a meaningful sense in which I could write a symphony in the next couple of years that wouldn't be either so simple as to not deserve the name or just sounds like noise.
Actually, I realise I'm unclear about the nature of your statement. Are you saying:
1) Symphonies specifically aren't that difficult, such that most people would be able to write a decent one given sustained effort (but there exist other acts, e.g. rocket design, which are not accessible in that way)
Or
2) Symphonies *are* difficult, but humans are just generally very adaptive problem solvers for difficult things in general given the right learning opportunity.
First, your definition would ruffle the feathers of many a self-proclaimed "genius" who purportedly crumbled under the pressure of parent/school and now work as retail clerks. Anyway, I was a terrible under achiever, and should have been held back, failing classes all throughout school and even the crummy regional college I later attended. Fortunately, the whole innate intelligence thing seemed to miss me as I was gearing up for the LSAT. Many people online said that one cannot meaningfully improve his/her own score because it's essentially a bunch of logic problems, with little to now formal "knowledge" tested. I refused to believe this and spent almost two years studying, getting terrible scores over and over again. But something inside me refused to believe that I couldn't get a good score if I didn't study hard enough. It never occurred to me, then, that maybe I just didn't have the stuff. I swear that over the course of studying, something changed. My brain came online or something. For the first time, I learned to problem solve, and like growing pains, it was anguishing initially, but by and by, my scores readily improved until I got a top percentile score. These habits followed me into law school, and allowed me to excel there, paving the way for my top-choice job. Certainly, I'd concede that there's outer limits, but improvement can happen, and our brains are a lot more adaptable than the innate camp would admit. There's no doubt such an "innate" mindset would have mentally crippled me had it taken stock back then.
That was interesting, Thank you. In my previous life I built physics apparatus (for advanced teaching labs.) It was a great gig, and I identified two types of problem solving. The first I call trouble shooting*. This 'thing' (piece of apparatus or maybe a car) worked before, and now it's not working. What is wrong? The second I called debugging (for lack of a better word). You are building something for the first time and it's not working. What's wrong? This second is much harder (in general). You don't know if it's something in the apparatus that's not working right, or if your 'theory' for why it should work has an error in it. The potential problem space for the second is much larger. So I just wanted to add some comments from the trenches to your excellent book review.
1.) Don't jump to step two too soon. There are hundreds of things that could be wrong, picking the 'most likely' one and pursuing that can waste a lot of time and money. Before making a plan get as much data as possible. Learn as much about the problem as you can. That said there is something to be said for trying the 'simplest' solutions first. Is it plugged in? Is there gas in the tank? ... etc.
2.) When stuck explain your problem to a smart audience. I think the 'smart' is important here. You have to think that your audience can help you. You need to explain your problem in depth and with sincerity. I would often use my boss for this task. Or a group of online electronics guys on Usenet. Often laying out the problem for the smart audience would let me see the solution without actually getting any help from the smart person. You are deep into the problem and going back to the beginning is what is needed. "Oh I made this assumption at step 3, and maybe that's not right."
3.) Sleep on it. Let your unconscious help. It's amazing how many solutions come to me either during the morning shower, or commuting to work in the morning. Your brain is doing some mundane task you've done hundreds of time before, which frees up the back of your brain to find the needed connection. I have no idea how this works, but it does!
*There is a corollary to trouble shooting, which I called remote trouble shooting. You are trying to fix a problem over the phone and /or by email... a new set of fun challenges.
Plans are nothing, planning is essential.
Your advice about "design it twice" reminds me of a story about an engineer at Meta that when given a task to be completed in two weeks, would actually design it three times. He would design for a week, throw away his work, design for a few more days, throw away his work, then would concentrate his efforts on the very last day to rebuild it as he sought.
"...intelligence is the ability of an agent to achieve its goals."
I like this one a lot. It fits well with my opinion that intelligence is a category, not an attribute.
Take two people who have different learning styles. The first learns extremely rapidly. It's been said that the average student need to hear concepts 26 times before they understand; this person has grasped them after the third time and is ready to move on. But they are also more forgetful. If they don't use a concept for a few months (or more), it's gone. This isn't a problem for them as they can re-learn something when needed even quicker than the first time they learned it.
The second person needs to hear the concept all 26 times, but once they know it that concept is set. They don't have to re-learn anything; they can just keep building on the concepts they know. Additionally, they're more persistent and less likely to get distracted.
Our current system will say the first person is much more intelligent than the second. A lot of IQ is simply how fast you can learn. But the second person might be better at achieving their long term goals.
That is an interesting observation about the speed of learning. A conclusion I reached from years of observation of business practice was that the kind of people who got promoted to high level were often "intellectual sprinters". They were highly intelligent, able to assess situations very quickly, and come to a decision. All of that was great -- provided the problem was the kind that could be fixed quickly.
There were other people who were "intellectual long-distance runners". They tended not to be the kind of people who got promoted to high level positions because they were more deliberate (slower) about assessing problems. On the other hand, they might continue to grind away at a problem and eventually reach a better solution than the sprinter. The sprinter would of course then adopt this better solution and get the credit!
Maybe some of the smart people who read this Substack have also seen what I'm about to describe and can offer a explanation. It regards Pólya's four essential stages of problem solving: (1) Understanding what it is you’re trying to do; (2) Coming up with a plan; (3) Carrying out the plan; and (4) Reflecting on how it went.
For a few years I was the elected county attorney for a small rural county. I found that the county employees and other elected officials had a violent aversion to all but stage (3) of the four stages. That is, they were unbelievably hostile to any attempt to figure out what needed to be done, to come up with a plan, and to reflect on it afterwards. In fact, I once heard a county commissioner say in a public meeting, "We're going to do something even if it's wrong." I've seen this same aversion in clients in in other areas of legal practice and also in nonlegal endeavors.
Has anyone else seen this? Any idea what's behind it? Or have I just lived an worked in a place that has too much lead in the water?
I have seen this. It comes in 4 flavours I have identified. 1. The people involved do not have enough imagination. Sometimes this is just because there is none there. But often it is because they have had their imagination and initiative punished whenever they used it in the past, so now they reflexively shut it down whenever it might show up. The fix is to get these people to play again. Like a lot of fixes, it is a lot easier said than done.
2. They're frightened. They think they might say something embarassing or foolish and they aren't willing to.
3. They're not interested in solving the problem. They just want to put in the required hours, meetings, what have you so that they can look busy enough, deserving enough, what have you. Meanwhile things continue on the way they always have because the status quo suits them. They didn't get elected to solve problems but to manage them. This is a particular problem with government. You might think the problem that needs solving is "better trash collection" but the problem the others are interested in is "managing the patronage networks including the one where the trash concession is awarded". At this point nobody wants to talk about the problem that they are really solving, which might be of questionable legality .... but of course they don't want to talk about what you see as the problem either. Always look at problems of this sort in terms of "who benefits from having things continue precisely as they are". Sometimes it becomes obvious.
4. They are lazy.
There are certainly other reasons, but those are the ones where I stub my toe most often.
Good luck with all of this.
Hmmm. I forgot. In government sometimes the political problem is that if you solved the technical problem the wrong people would get the credit. You, for instance, if they don't like you.
That's very much my experience doing continuous improvement/system implementation in industry as well. People don't like it when the problem with production planning is the general management demanding arbitrary increases in sales that won't happen instead of the MRP system, and the people who don't like it will fire the people who point that out. So the real issue never gets fixed, and no one knows why all those initiatives happen and things remain basically the same.
Politicians and government employees are averse to anything that requires doing something different than what is currently being done, because they desire above all to not be held accountable for negative results. If the status quo existed prior to their involvement, then they can't be blamed for the policy. It's their predecessors fault. If it's their policy, then changing it is an admission that it needs to be changed and that just won't do. Better to claim that "they" (whoever can be blamed, or a nebulous opponent) are sabotaging the current plan and it only needs to be done harder. (This became blatantly obvious during the COVID shutdowns and mask mandates.) All three of the steps they want to avoid are in conflict with this.
There's a standard joke in the UK about how politicians behave: "Something must be done! This is something, therefore it must be done!" It's a product of a media environment which relentlessly punishes "flip-flopping" (otherwise known as "learning from your mistakes").
The other political failure mode is that when they *don't* want to do something they establish a commission to extensively study the thing and provide recommendations for possible actions. By the time the commission reports then the media storm around the thing will have blown over and they'll be free to ignore the recommendations.
"One of my quack beliefs is that both Eudoxus of Cnidus and Archimedes probably figured out at least the basics of calculus ..."
The challenge that both of them would have faced is -- lack of the necessary tools for the job. Mathematician Raul Rojas wrote an interesting book "The Language of Mathematics", unfortunately besmirched by the insistence of the ladies at Princeton University Press on using the ridiculous BCE/CE dating system. Rojas credits Diophantus in the 3rd Century AD with starting the process of developing symbols for mathematical operations. It is hard enough to see how the Romans built their magnificent edifices without positional notation or the number zero -- but how would calculus have been possible without an equals sign?
It is interesting that both Leibnitz and (less successfully) Newton had to expand an already rich set of mathematical symbols in order to develop calculus. Without the basic symbols in place, which developed slowly between the 3rd and 17th Centuries AD, the challenge of developing calculus would probably have thwarted even Leibnitz and Newton.
Archimedes got very close to figuring out the basics of integral calculus, at least the high-school hand-wavey version: https://en.wikipedia.org/wiki/Archimedes_Palimpsest#The_Method_of_Mechanical_Theorems He himself knew that his methods weren't rigorous and treated them as useful heuristics - no shade on him, since it took some of the best mathematicians in Europe a century of work to make Newton and Leibniz's work rigorous!
In Hebrew the phrase goes שאלת חכם חצי תשובה. A wise man's question is half an answer.
A teacher once explained that it only works if you ask the question intelligently. If the question is ill-formed it won't point to anything useful.
Outstanding piece on reframing intelligence as measurable impact rather than mystical talent. The toolbox metaphor lands perfectly since most "breakthroughs" in my career came from recognizing a familiar pattern in unfamiliar contexts, not some Eureka flash. What gets overlooked is how much grit compounds with heuristics - having mental moves only helps if people stick around long enough totry them all. I dunno if the ancients were missing training data or just missing the grindset neede to iterate across generations.
Your discussion of the phase 2/phase 3 split reminded me of How Big Things Get Done by Bent Flyvbjerg and Dan Gardner: https://www.amazon.com/How-Big-Things-Get-Done/dp/0593239512. I loved the book, and think it's very relevant to your interests!
"Eventually you learn to notice its spoor, the rank taste in the air, “a problem has passed by this way, moving downwind, two days ago.” One of the many ways school fails us is by actively harming this capacity, it lies and lies to us for decades, teaching us that good problems will be delivered on a silver platter. This is why so many people who do well in school never amount to anything. They never develop a taste for the hunt, never learn that this, actually, is the most important part of the entire site survey: “is this problem worth solving by anybody?”, “am I uniquely well-positioned to solve it?”, “can I amass the resources to solve it?”, “do I have any chance of success?”, “is there some other problem that it is more valuable for me to solve?”"
If you never take the novice hunting and only take them to the range and teach them how to cook venison....
This article delights the system engineer and maker in me. I will need to look into the book...
...and the science fiction enthusiast wonders if there is still a tesseract angle I could turn to grasp the plasma rifle just beyond my perception if I only knew how (Hat tip "The Universe Between", Alan Edward Nourse.)
Welcome to the Club of Doom! I sure hope your God exists, because nothing else can save us now....
Something tells me that you'll already have read: https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien-message, but if you haven't, it's excellently scary.
Is your general relativity caveman allowed to act? Or is he disabled in some way and just has to sit at the back of his cave and watch things? If he's stuck in the cave I don't see how he's supposed to tell the difference between a universe that runs on GR and one that has Newtonian gravity even once he's generated both as hypotheses.
There’s maybe a whole other AI angle to this? I hear a lot about how RL is so much less efficient than pretraining because you do a ton of tokens of rollout and get one bit of information at the end but polya’s post-proof suggestion of re-proving and reflecting on the blind alleys and successes of a proof seems like a way to increase RL information density. Some kind of “post-solution reflection” for RL?
I believe something like this was pioneered by DeepSeek and is now standard practice when training frontier models: https://sebastianraschka.com/blog/2025/technical-deepseek.html#5-deepseekmath-v2-with-self-verification-and-self-refinement
Hmmm I’m not sure it’s the same though. Like my tenuous read on the deepseek strategy is that it’s basically an improved process reward model just with an external LLM2? But LLM2 is still checking correctness of steps it may not know a full proof of the relevant theorem. I’m also really confused why their thing works so well since it seems like they’re just going 2 steps into an infinite “who watches the watchmen” regress (what if LLM3 is wrong?) but that’s not relevant to the main point I think.
The closest implementation of the Polya thing would be like “after you do a proof, keep that proof in context and try to write a more efficient proof, or prove it a bunch of other ways, or see how you can apply that proof to other things, etc”. In contrast, with existing RLVR once you have a correct answer you basically stop
Thanks. another great book review.
two questions:
1. would you consider "exploring" to be a valid step 2 of planning?
2. can you say more about pregnant problems? i am worried that haphazard adding of words to the problem definition may direct the solver in a wrong direction. where does this terminology come from?
I am not entirely sure about this article’s statement that Pólya's method is "NOT the same thing as the Socratic method".
"When Antisthenes was asked what profit he had derived from [Socratic] philosophy, he replied: 'The ability to converse with myself.' The intimate connection between dialogue with others and dialogue with oneself is profoundly significant." (Hadot, Philosophy as a Way of Life, p. 91).
Note what Antisthenes is not saying here: the primary benefit of philosophy, for him, is neither ἀπορία in general nor some particular philosophical truth; rather, it is that set of thinking skills by which one may "converse with [oneself]". If I understand correctly, this is precisely what Pólya was trying to teach!
Looks like I have a lot to say that I suspect you're aware of and have read the same studies or summaries than me.
> On the other hand, we (unofficially) treat intelligence as a proxy for moral worth
I'm surprised this isn't a link to Freddie DeBoer's The Cult of Smart book.
> One unusual thing about math is that if you understand your starting point and your ending point sufficiently well, then you’re already done.
Can confirm from experience, but I was always told this holds mostly on the algebraic side of the great algebra/analysis divide (where footnote 7 might apply more). I was mumbling "Grothendieck" at this point, I'm very happy to see him appear later on.
> not an exercise, mind you, but a problem
The only way I can think of pushing back here is to invoke Terry Tao myself: https://terrytao.wordpress.com/career-advice/theres-more-to-mathematics-than-rigour-and-proofs/ it seems to me that getting from stage 1 to 2 is where exercises help you, and as long as that's not viewed as the end goal that's ok. (Very few undergraduates in math these days will come up with a theorem that will bear their name in future, so we have to settle on problems at a difficulty level that a research mathematician would call "exercises".)
Back to dual n-back and Gwern's links to programming: one way to free up working memory is to store things in muscle memory, for example by learning to touch-type properly. That means "grinding" typing exercises, because you're not trying to solve open problems in typing you're trying to take it off the critical path to your math/programming problem in the first place. (I feel like I should have come up with a bad pun about type theory here, but failed. Programmers will know what I mean.)
And, as you say later (paraphrased) small successes -> hope -> endurance -> solve big problem, so as long as your exercises are not too easy, as long as getting them right regularly produces a "Counter Go Up" effect, that's a causation chain even if the exercises are not teaching you the math you need in the end.
> when a student is stuck, the teacher shouldn’t give any hints
At which point the future student will just ask the AI :( It doesn't invalidate the argument that that's one of the best teaching methods, just that ... AI will gain superhuman intelligence by reversing the Flynn effect with its instant answers?
> Archimedes probably figured out at least the basics of calculus but the knowledge died with them.
Traces of his knowledge are still documented in the https://en.wikipedia.org/wiki/Archimedes_Palimpsest - unless that was what you were referring to in the first place? I know it's controversial, but he was at least aware that something vaguely calculus-shaped was needed to make his arguments work so I'm happy to credit him with "the basics". I mean these people were not the brightest ever: https://academia.stackexchange.com/questions/9602/rediscovery-of-calculus-in-1994-what-should-have-happened-to-that-paper so I'm sure Archimedes got further than that.
> learning to recognize not *solutions* you’ve already seen, but *approaches* you’ve already seen
Chi et al., Categorization and representation of physics problems [ed: exercises!] by experts and novices, Cog Sci 5 (1981):
Exactly! Novices tended to sort exercises by surface features ("these ones have a right-angled triangle in it") whereas experts sorted by domain-specific structure ("these ones are conservation-of-energy"). "Does this problem remind you of another one?" pays off when you realise that you're after structure, not presentation.