This coverage of COT in ML is misleadingly anthropomorphic. Have we really lost track of how these things work? Just because we call something "chain of thought" that doesn't make it ACTUAL chain of thought. Anthropic has always done this. #MLsec
And this is a usually excellent reporter falling prey to the nomenclature.
@cigitalgem because “inference-time scaffolding” does not have the same ring to it, but more importantly will not be understood by public at large anymore.
@andrei_chiffa the part that is ridiculous is that the COT (or ITS in your terminology) nonsense is trained in albeit after initial trainig. The machine is doing exactly what it was trained to do which is then hilariously called "reasoning." Color me hugely skeptical that we have magically advanced past auto-associative predictive generation to "understanding."
Anthropic consistently stretches terms implying cognition past the breaking point to fit its narrative.
Normals can understand this.
@cigitalgem COT itself is a problematic term IMHO. Initially it was a way to match generation to high-quality data contexts (textbooks, tutorials, …), but there is no thought, even less à chain in there.
But it looks recently RL got poured over all of it as if COT was actually helping by itself, with close to no discussion of metrics RL is optimizing for.
And tbh with self-proclaimed experts not seeing the difference between fine-tuning and further pre-training, I am doubting about normals.
@andrei_chiffa well put... And in fact, "what we are maximizing for" is turning out to be just as important as we've all been saying for a few decades.