I usually don’t even understand the lingo they use. “Open-weighted” is the most recent one, then it usually goes down to specific “models” that everybody is supposed to know about.

These are my thoughts (I will stick to the vague “it” for now, but of course therein lies another question: “and how does all this apply to various specialised AIs”):

  • Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?
  • If yes to the previous: the software doesn’t come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?

If what I wrote above is true, what exactly are people arguing when they say it’s still possible to use LLMs ethically or true to FOSS philosophy, because … ???

edit

Thanks to all who answered.

I guess it’s my fault for asking several questions in one, but this thread has attracted exactly the type of people I’m writing about; several even used the term “open-weighted models” without explaining it.

Asking to get arguments explained, I got more arguments instead.

  • WolfLink@sh.itjust.works
    3·
    18 hours ago

    Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still.

    There are small models you can run 100% locally on a mediocre computer or even a phone, and I mean you could be 100% offline and it will still work.

    The better local models require a higher end gaming GPU or an ARM Mac with a decent amount of RAM, but nothing too extreme. You could get a computer to run them for about $2000 to $3000.

    the software doesn’t come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?

    The small local models typically start with a full infustrial-size model and then they “distill” it to smaller models. The concerns over training data are still valid. I’m not sure how much the concerns over power usage are about training vs running the full industrial size models commercially. Running a small local model uses a tiny fraction of the power it takes to run an industrial size model, and the training is a one-time cost (except these companies never stop training because they want to make next year’s model better and faster).

    I do think it’s worth pointing out that running a model locally keeps your conversations with it private. Remember, it can be done 100% offline. So your data won’t be stolen.