sigmoid.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A social space for people researching, working with, or just interested in AI!

Server stats:

588
active users

Two works from
@dieuwke
on stability or consistency of outputs\metrics or if you want (and really, I want) reliability

Datasets for compositional generalizations do not agree with each other. It means that different models are good at different things. But that the metrics don't measure what we thought...

@Adinawilliams
@_dieuwke_

Leshem Choshen

@dieuwke @Adinawilliams How consistent is in context learning?
Across many ICL inputs, they find results vary a lot, ICL training improves model consistency, and bigger models are only more consistent if trained for ICL

No paper yet? couldn't find...