sigmoid.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A social space for people researching, working with, or just interested in AI!

Server stats:

588
active users

Having tested a bunch of #OpenSource #LLM projects, I gotta say that OpenHermes 2.5 is the most helpful out of the ones I can run locally.

I recently wasted a bunch of time getting Phi-2 to do some summarization work, and it just couldn't stay focused for more than a sentence or two.

@mauve I was tinkering with ollama for a bit, but my local hardware just isn't fast enough to make it useful.

@skryking What have you been using to run the models? I find LM Studio really nice for tinkering. lmstudio.ai/

I find Q4 quantized models work pretty well on my steam deck.

LM StudioLM Studio - Discover, download, and run local LLMsRun Llama, Gemma 3, DeepSeek locally on your computer.

@mauve have you heard of a community somewhere (lemmy? matrix?) where we could share our experiences?

We (@codelutin) may want to start to play with things but "it's dangerous to go alone". There is @stablehorde lead by @Db0 which is a great start (but I won't join a Discord 😔)

AI Horde

@lutindiscret @mauve @codelutin @Db0 you can use our lemmy instance as a community. I could also look into setting a matrix bridge to our discord you won't have access to all channels but you'd be able to interact with questions etc