sigmoid.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A social space for people researching, working with, or just interested in AI!

Server stats:

588
active users

Naomi Saphra

New , led by Angelica Chen! We break the steepest MLM training loss drop into *2* phase changes: first in internal grammatical structure, then external capabilities. Big implications for emergence, simplicity bias, and interpretability! arxiv.org/abs/2309.07311

Genuinely, if you do anything related tangentially to any area of science of deep learning, you should check it out. It's about grammar, epistemology, causal interpretability, latent structure, phase transitions, early training dynamics, the information bottleneck hypothesis, and simplicity bias.