Recent searches

No recent searches

Search options

Only available when logged in.
sigmoid.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A social space for people researching, working with, or just interested in AI!

Administered by:

Server stats:

580
active users

sigmoid.social: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.3

#scientivism

0 posts0 participants0 posts today
Leshem Choshen<p><a href="https://sigmoid.social/tags/LLMs" class="mention hashtag" rel="tag">#<span>LLMs</span></a> &quot;have the potential of playing an important role in [...]opinion formation in online social media&quot;🫡🤖<br />Not surprising. But also not encouraging</p><p>Is it a potential we even want to research? under what terms?<br />Thoughts?<br /><a href="https://arxiv.org/abs/2312.15523" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="">arxiv.org/abs/2312.15523</span><span class="invisible"></span></a></p><p><a href="https://sigmoid.social/tags/NLProc" class="mention hashtag" rel="tag">#<span>NLProc</span></a> <a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> <a href="https://sigmoid.social/tags/ethics" class="mention hashtag" rel="tag">#<span>ethics</span></a> <a href="https://sigmoid.social/tags/ml" class="mention hashtag" rel="tag">#<span>ml</span></a> <a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="tag">#<span>machinelearning</span></a> <a href="https://sigmoid.social/tags/chatgpt" class="mention hashtag" rel="tag">#<span>chatgpt</span></a></p>
Leshem Choshen<p>Solar mixes two base-model copies<br />to create a larger one<br />Then train it a bit more and beat other open models out there.<br />How? and my thoughts 🧵</p><p> (no author with a handle?!)<br /><a href="https://arxiv.org/abs/2312.15166" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="">arxiv.org/abs/2312.15166</span><span class="invisible"></span></a><br /><a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> <a href="https://sigmoid.social/tags/LLM" class="mention hashtag" rel="tag">#<span>LLM</span></a> <a href="https://sigmoid.social/tags/LLMS" class="mention hashtag" rel="tag">#<span>LLMS</span></a> <a href="https://sigmoid.social/tags/pretraining" class="mention hashtag" rel="tag">#<span>pretraining</span></a> <a href="https://sigmoid.social/tags/MIXTRAL" class="mention hashtag" rel="tag">#<span>MIXTRAL</span></a> <a href="https://sigmoid.social/tags/SOLAR" class="mention hashtag" rel="tag">#<span>SOLAR</span></a> <a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="tag">#<span>machinelearning</span></a> <a href="https://sigmoid.social/tags/ml" class="mention hashtag" rel="tag">#<span>ml</span></a> <a href="https://sigmoid.social/tags/NLP" class="mention hashtag" rel="tag">#<span>NLP</span></a></p>
Leshem Choshen<p>So many warn that evaluating with GPT favors GPT </p><p>(or any LLM evaluating itself). </p><p>Now it is also shown </p><p>Science, not just educated guesses </p><p>(Fig: T5, GPT, Bart each prefer their own) <a href="https://arxiv.org/abs/2311.09766" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="">arxiv.org/abs/2311.09766</span><span class="invisible"></span></a> </p><p> <a href="https://sigmoid.social/tags/enough2skim" class="mention hashtag" rel="tag">#<span>enough2skim</span></a> <a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> <a href="https://sigmoid.social/tags/NLP" class="mention hashtag" rel="tag">#<span>NLP</span></a> <a href="https://sigmoid.social/tags/nlproc" class="mention hashtag" rel="tag">#<span>nlproc</span></a> <a href="https://sigmoid.social/tags/GPT" class="mention hashtag" rel="tag">#<span>GPT</span></a> <a href="https://sigmoid.social/tags/LLM" class="mention hashtag" rel="tag">#<span>LLM</span></a> <a href="https://sigmoid.social/tags/eval" class="mention hashtag" rel="tag">#<span>eval</span></a> <a href="https://sigmoid.social/tags/data" class="mention hashtag" rel="tag">#<span>data</span></a></p>
Leshem Choshen<p>You know what?<br />I will stop sharing any LLM &quot;news&quot; if they don&#39;t share with me first (models or code) </p><p><a href="https://sigmoid.social/tags/thereOrIDontCare" class="mention hashtag" rel="tag">#<span>thereOrIDontCare</span></a> <br /><a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> <br /><a href="https://sigmoid.social/tags/uShareFirst" class="mention hashtag" rel="tag">#<span>uShareFirst</span></a></p><p>Thanks delip rao for inspiration<br />And the new vision and language that did open unlike XXXX-e<br /><a href="https://twitter.com/DrJimFan/status/1633179734803890177?t=VaQS4E56eEq55NHKN3y5hg&amp;s=19" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="ellipsis">twitter.com/DrJimFan/status/16</span><span class="invisible">33179734803890177?t=VaQS4E56eEq55NHKN3y5hg&amp;s=19</span></a><br /><a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="tag">#<span>machinelearning</span></a> <a href="https://sigmoid.social/tags/cv" class="mention hashtag" rel="tag">#<span>cv</span></a> <a href="https://sigmoid.social/tags/nlproc" class="mention hashtag" rel="tag">#<span>nlproc</span></a> <a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="tag">#<span>nlp</span></a></p>
Leshem Choshen<p>So often we are reminded that good work goes unnoticed<br />I share others&#39; papers to change that<br />What else could we do?<br />What mechanisms better allow propagation by value rather than by fame?<br />Is there something we can do to make science better?<br /><a href="https://blog.samaltman.com/you-and-your-research" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="ellipsis">blog.samaltman.com/you-and-you</span><span class="invisible">r-research</span></a></p><p><a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> <a href="https://sigmoid.social/tags/ScienceMastodon" class="mention hashtag" rel="tag">#<span>ScienceMastodon</span></a> <a href="https://sigmoid.social/tags/PR" class="mention hashtag" rel="tag">#<span>PR</span></a> <a href="https://sigmoid.social/tags/NLProc" class="mention hashtag" rel="tag">#<span>NLProc</span></a> <a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="tag">#<span>machinelearning</span></a> <a href="https://sigmoid.social/tags/CV" class="mention hashtag" rel="tag">#<span>CV</span></a></p>
Leshem Choshen<p>A surprising take on why we should open LLMs:<br />otherwise empirical research would suffocate and<br />rule-based (nativist) would return</p><p>Not sure I am buying it or even that it is dreadful, but more the reason to share and hear opinions<br /><a href="https://arxiv.org/abs/2301.05272" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="">arxiv.org/abs/2301.05272</span><span class="invisible"></span></a><br />Patrick Perrine<br /><a href="https://sigmoid.social/tags/LLM" class="mention hashtag" rel="tag">#<span>LLM</span></a> <a href="https://sigmoid.social/tags/NLP" class="mention hashtag" rel="tag">#<span>NLP</span></a> <a href="https://sigmoid.social/tags/nlproc" class="mention hashtag" rel="tag">#<span>nlproc</span></a> <a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="tag">#<span>machinelearning</span></a> <a href="https://sigmoid.social/tags/ML" class="mention hashtag" rel="tag">#<span>ML</span></a> <a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a></p>
Leshem Choshen<p>We want to pretrain🤞<br />Instead we finetune🚮😔<br />Could we collaborate?🤗</p><p>ColD Fusion: <br />🔄Recycle finetuning to multitask <br />➡️evolve pretrained models forever</p><p>On 35 datasets <br />+2% improvement over RoBERTa<br />+7% in few shot settings<br />🧵</p><p><a href="https://sigmoid.social/tags/NLProc" class="mention hashtag" rel="tag">#<span>NLProc</span></a> <a href="https://sigmoid.social/tags/MachinLearning" class="mention hashtag" rel="tag">#<span>MachinLearning</span></a> <a href="https://sigmoid.social/tags/NLP" class="mention hashtag" rel="tag">#<span>NLP</span></a> <a href="https://sigmoid.social/tags/ML" class="mention hashtag" rel="tag">#<span>ML</span></a> <a href="https://sigmoid.social/tags/modelRecyclying" class="mention hashtag" rel="tag">#<span>modelRecyclying</span></a> <a href="https://sigmoid.social/tags/collaborativeAI" class="mention hashtag" rel="tag">#<span>collaborativeAI</span></a> <a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> <a href="https://sigmoid.social/tags/pretrain" class="mention hashtag" rel="tag">#<span>pretrain</span></a></p>
Leshem Choshen<p>🔖Reviewing has so many faults📖<br />Finally, there is a dataset of reviews, edits and everything else!</p><p>5 venues 5K papers 11K reviews <br />Enjoy!</p><p><a href="https://arxiv.org/abs/2211.06651" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="">arxiv.org/abs/2211.06651</span><span class="invisible"></span></a><br />Nils Dycke, Ilia Kuznetsov, Iryna Gurevych</p><p><a href="https://sigmoid.social/tags/NLProc" class="mention hashtag" rel="tag">#<span>NLProc</span></a> <a href="https://sigmoid.social/tags/review" class="mention hashtag" rel="tag">#<span>review</span></a> <a href="https://sigmoid.social/tags/CV" class="mention hashtag" rel="tag">#<span>CV</span></a> <a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="tag">#<span>machinelearning</span></a> <a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a></p>
Leshem Choshen<p>Are findings as good as ACL?</p><p>years since the first findings papers were introduced<br />since chris manning &amp; ani nenkova called for a yearly analysis<br />since they were first done </p><p>Who&#39;s game for the yearly analysis?</p><p><a href="https://twitter.com/chrmanning/status/1451261089644089394" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="ellipsis">twitter.com/chrmanning/status/</span><span class="invisible">1451261089644089394</span></a></p><p>For earlier analysis and code (old, not on :mastodondance: , next year links from here?)</p><p><a href="https://twitter.com/gneubig/status/1451317437392113665?s=20&amp;t=B0RYiSShHJQPpITBWbKF8g" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="ellipsis">twitter.com/gneubig/status/145</span><span class="invisible">1317437392113665?s=20&amp;t=B0RYiSShHJQPpITBWbKF8g</span></a><br /><a href="https://twitter.com/ryandcotterell/status/1451551344012181514?s=20&amp;t=bURi5XYeaZS93-GtCWW23Q" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="ellipsis">twitter.com/ryandcotterell/sta</span><span class="invisible">tus/1451551344012181514?s=20&amp;t=bURi5XYeaZS93-GtCWW23Q</span></a><br /><a href="https://twitter.com/wilkeraziz/status/1451896682321485824?s=20&amp;t=bURi5XYeaZS93-GtCWW23Q" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="ellipsis">twitter.com/wilkeraziz/status/</span><span class="invisible">1451896682321485824?s=20&amp;t=bURi5XYeaZS93-GtCWW23Q</span></a><br /><a href="https://twitter.com/sanxing_chen/status/1451325907918934019?s=20&amp;t=cQ8RTuwN1plau9mrLtExXw" target="_blank" rel="nofollow noopener" translate="no"><span class="invisible">https://</span><span class="ellipsis">twitter.com/sanxing_chen/statu</span><span class="invisible">s/1451325907918934019?s=20&amp;t=cQ8RTuwN1plau9mrLtExXw</span></a><br /><a href="https://sigmoid.social/tags/NLProc" class="mention hashtag" rel="tag">#<span>NLProc</span></a> <a href="https://sigmoid.social/tags/findings" class="mention hashtag" rel="tag">#<span>findings</span></a> <a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> <a href="https://sigmoid.social/tags/ACL" class="mention hashtag" rel="tag">#<span>ACL</span></a></p>
Leshem Choshen<p>What do we know about using a fine-tuned model rather than the pretrained<br />They are sometimes much better, but what else?</p><p>A story of great <a href="https://sigmoid.social/tags/scientivism" class="mention hashtag" rel="tag">#<span>scientivism</span></a> hypotheses and their rejections</p><p>The story of a field<br />Survey 🧵</p>
TrendingLive feeds
About