Interesting, if you happen to come across a specific quote from on of these scientists, I would be interested to read it.

For me the clearest way to see the gap between 'real' reasoning and LLM pre-compiled reasoning is to simply look at the dramatically extended chains of such reasoning.

LLMs tend to fake such reasoning for shorter chains, but fail quickly as the number of steps grow. Humans on the other hand (especially ones with pencil and paper in hand) are able to extend that reasoning with very slow degradation as problem sized grows larger.

Just have humans and LLMs to 30digit by 30 digit long division. That should do the trick. Humans can extend their reasoning indefinitely, they just suffer greater chances of making a mistake, while the LLM quickly gets lost as can just never solve such problems. Thus it seems weird to me that a real scientist would get confused on this point. (I am not doubting you, I am just surprised.)

But for me, while I see LLMs as not really doing reasoning, I feel the steps needed to boost the algs to really do reasoning may not be that massive. So I don't rest with certainty that the field wont develop real type-II reasoning soon. I am suspect when we do develop real type-II reasoning it will be based on LLM-like type one reasoning, just as humans apparently base our type-II on our type-I reasoning.