BBS:      TELESC.NET.BR
Assunto:  AI LLM Artificial intelligence infrastructure
De:       phigan
Data:     Wed, 11 Mar 2026 17:13:54 -0700
-----------------------------------------------------------
  hbRenb: hcAI LLM Artificial intelligence infrastructure
  bBynb: hcbbsing bto cphigan bon cTue Mar 10 2026 10:31 pmn

 > What models are you liking, and what sizes are you getting good TPS?
 > What do you think is good TPS?
 > Have you tried Deepseek-v3?
 > vllm?

I'm not liking any of them. I haven't tried qwen2.5 out yet, I just downloaded
it, but the rest of them aren't any good. At least for what I've been trying,
which is code.

devstral:24b
codellama:34b
qwen3-coder:30b

I had the most success using 'Zed' as a front end to the qwen3-coder model.
Codellama actually refused to write code at all. Devstral says it's going to
write code but never does and gets stuck in a loop. I also tried installing
claude and launching that with those same models. Codellama wouldn't open at
all, saying it didn't support tools or whatever. Devstral again got stuck in a
loop telling me it was going to do things but not actually doing them.
qwen3-coder kept going back and forth on fixing one thing, breaking another,
then breaking the first thing when it fixed the second thing, etc.

My GPU was second hand off Craigslist after scouring that and eBay quite a bit.

Still haven't found a good front end to use, but I have not tried Open WebUI
just yet.
n
---
  gSynchronetn  TIRED of waiting 2 hours for a taco? GO TO TACOPRONTO.bbs.io

-----------------------------------------------------------
[Voltar]