Recent searches

No recent searches

Search options

Only available when logged in.
sigmoid.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A social space for people researching, working with, or just interested in AI!

Administered by:

Server stats:

598
active users

sigmoid.social: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.2

#DL

0 posts0 participants0 posts today
Nate Gaylinn

AI is a hot topic these days, but what does that word even mean? Originally it was about our quest to understand our own minds, but it has come to refer to one technology: deep learning. We often talk about AI as if it were just a human mind in a box, but the reality is quite different, in nuanced ways that AI companies play down. In this month's blog post, I explore how AI relates to human intelligence, what it reproduces, and what it doesn't.

thinkingwithnate.wordpress.com

Thinking with Nate · How is AI like human intelligence?(The image for this post is a photo of a jacquard loom I took in the workshop of Luigi Bevilacqua in Venice. It’s a heavy wooden frame covered in pulleys and string, in the midst of weaving f…
#ai#dl#deeplearning
*
ma𝕏pool

Self-Improving Reasoners.

Both expert human problem solvers and successful language models employ four key cognitive behaviors

1. verification (systematic error-checking),

2. backtracking (abandoning failing approaches),

3. subgoal setting (decomposing problems into manageable steps), and

4. backward chaining (reasoning from desired outcomes to initial inputs).

Some language models naturally exhibits these reasoning behaviors and exhibit substantial gains, while others don't and quickly plateau.

The presence of reasoning behaviors, not the correctness
of answers is the critical factor. Models with incorrect solutions containing proper reasoning patterns achieve comparable performance to those trained on correct solutions.

It seems that the presence of cognitive behaviors enables self-improvement through RL.

Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
arxiv.org/abs/2503.01307

#reinforcementlearning #RL
#AI #DL #LLM

arXiv.orgCognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRsTest-time inference has emerged as a powerful paradigm for enabling language models to ``think'' longer and more carefully about complex challenges, much like skilled human experts. While reinforcement learning (RL) can drive self-improvement in language models on verifiable tasks, some models exhibit substantial gains while others quickly plateau. For instance, we find that Qwen-2.5-3B far exceeds Llama-3.2-3B under identical RL training for the game of Countdown. This discrepancy raises a critical question: what intrinsic properties enable effective self-improvement? We introduce a framework to investigate this question by analyzing four key cognitive behaviors -- verification, backtracking, subgoal setting, and backward chaining -- that both expert human problem solvers and successful language models employ. Our study reveals that Qwen naturally exhibits these reasoning behaviors, whereas Llama initially lacks them. In systematic experimentation with controlled behavioral datasets, we find that priming Llama with examples containing these reasoning behaviors enables substantial improvements during RL, matching or exceeding Qwen's performance. Importantly, the presence of reasoning behaviors, rather than correctness of answers, proves to be the critical factor -- models primed with incorrect solutions containing proper reasoning patterns achieve comparable performance to those trained on correct solutions. Finally, leveraging continued pretraining with OpenWebMath data, filtered to amplify reasoning behaviors, enables the Llama model to match Qwen's self-improvement trajectory. Our findings establish a fundamental relationship between initial reasoning behaviors and the capacity for improvement, explaining why some language models effectively utilize additional computation while others plateau.
Nate Gaylinn

Read any Deep Learning papers that made you do a double take?

Share them here, and we can make a list to blow each other's minds and get closer to actually understanding what the hell is going on. Boosts appreciated! :boost_requested:

We've learned a ton about Deep Learning over the years, but in a fundamental way we still don't get it. There's tons of tricks we use without knowing why, and weird examples that work much better or much worse than you'd expect. We try to probe and visualize what's going on inside the black box, and what we find is often strange and hard to interpret.

I'm in an excellent class right now exploring the "surprises" of deep learning, reading papers like this to build a better understanding. I've shared a few of them here, but now I'm looking for more to share back with the class.

Any suggestions?

#deeplearning#dl#machinelearning
TrendingLive feeds
About