BBS:      TELESC.NET.BR
Assunto:  AI LLM Artificial intelligence infrastructure
De:       Lonewolf
Data:     Sat, 14 Mar 2026 04:37:48 -0500
-----------------------------------------------------------
  Re: AI LLM Artificial intelligence infrastructure
  By: phigan to bbsing on Wed Mar 11 2026 05:13 pm

 >   Re: AI LLM Artificial intelligence infrastructure
 >  > What models are you liking, and what sizes are you getting good TPS?
 >  > What do you think is good TPS?
 >  > Have you tried Deepseek-v3?
 >  > vllm?
 >
 > I'm not liking any of them. I haven't tried qwen2.5 out yet, I just
 > downloaded it, but the rest of them aren't any good. At least for what I've
 > been trying, which is code.
 >
 > devstral:24b
 > codellama:34b
 > qwen3-coder:30b
 >
 > My GPU was second hand off Craigslist after scouring that and eBay quite a
 > bit.
 > Still haven't found a good front end to use, but I have not tried Open WebUI
 > just yet.
 >
How you tried llama.cpp? I've got Ollama, LM Studio and llama.cpp on my 24gb vram system. llama.cpp is definetly better in all areas and it has a nice web UI too.

Lonewolf
---
  Synchronet  Fireside BBS - AI-WX - firesidebbs.com:23231

-----------------------------------------------------------
[Voltar]