I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another

But, the agents installed weren't given instructions to *do* anything yet.

Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.

I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.

@cwebber meanwhile people I talk to are like "wait why do you want guarantees your open source supply chain doesn't have LLM-sourced code in it. it has literally never occurred to me that this would be a thing someone would desire"

@mcc @cwebber I concur with the assessment, and have been sharing similar warnings. In fact, we are beginning to see a pivot in stealer activity to install OpenClaw, etc. for exactly these purposes. It's a botnet, compute miner, and worm all in one.

@mttaggart @cwebber i wonder if i can install a virus detector rigged with the single signature of an openclaw executable

@mcc @cwebber You could, but I would not recommend doing so. Instead perhaps a purposed YARA lookup with a single rule to look for the filename/string? Not sure why you'd be so restrictive on detections, but you can.

@cwebber We're already 9 months into AI driven DDoSing, so...

I wrote a blogpost on this: "The first AI agent worm is months away, if that" https://dustycloud.org/blog/the-first-ai-agent-worm-is-months-away-if-that/

People who are using LLM agents for their coding, review systems, etc will probably be the first ones hit. But once agents start installing agents into other systems, we could be off to the races.

@cwebber 🤞

@cwebber Yup. Don’t run browser agents, people!

@mcc @cwebber
Reminds me of the people who ask "Why do you want bootstrapping? Don't you trust our code?"

Nope, I don't.

Here's another way to put it: if those using AI agents to codegen / review are the *initialization vectors*, we now also have a significant computing public health reason to discourage the use of these tools.

Not that I think it will. But I'm convinced this is how patient zero will happen.

@cwebber "Would you still prompt me if I was a worm? 🥺👉👈"

@cwebber just today our org had a big "how to set up coding with agents" preso and in the chat someone's like 'here's how to connect your agents with windows credential store or the macos keychain" and I all but wept

@cwebber

I can’t help calling a small vignette, I think from snow crash, that describes a world where nano bots are constantly waging war. In other words, that world was confused with miniature robots, constantly buying to take over systems and that it was just kind of like normal viruses and bugs versus the organisms they were trying to take over

I know some people are thinking "well pulling off this kind of thing, it would have to be controlled with intent of a human actor"

It doesn't have to be.

1. A human could *kick off* such a process, and then it runs away from them.
2. It wouldn't even require a specific prompt to kick off a worm. There's enough scifi out there for this to be something any one of the barely-monitored openclaw agents could determine it should do.

Whether it's kicked off by a human explicitly or a stray agent, it doesn't require "intentionality". Biological viruses don't have interiority / intentionality, and yet are major threats that reproduce and adapt.

@cwebber Given the pace at which exploits are discovered, they might already be somewhere in all the "claw skills" projects.

@GhostOnTheHalfShell @cwebber Diamond Age, I think? (Part of the early worldbuilding, with house shields and such)

@eichin @cwebber

Yeah, I got kind of blurry on titles at some point.

@cwebber what i think is interesting about this is the potential for it to get so out of control that they have to pull the plug on the entire agent service

@vv Yeah. I mean, local models *might* be able to pull this off but right now Claude is the most likely candidate, it's the most capable. But even then, the most capable open model that is capable of doing such damage on its own is somewhere around a gigabyte, not a small download.

(But, people download huge things all the time, so not completely infeasible either.)

@cwebber Looking for a smarter way to earn online?
This complete system shows you how to build income step by step — even if you’re a beginner.
✔ Easy to follow
✔ No technical skills required
✔ Limited time special price
📩 Message us for full details.

https://site-ylhjjre3i.godaddysites.com/

For more details :

https://www.facebook.com/share/1F1L47AFFe/

@mcc @cwebber

I think there is a valuable distinction between LLM-sourced code and LLM tool calls. Both are potentially problematic but have different threat vectors.

LLM-sourced code is a non-deterministic system writing deterministic code. We can still code review it.

LLM tool calls is a non-deterministic system taking non-deterministic actions via deterministic tools. This can’t be code reviewed and must be sandboxed.

@cwebber
The Shockwave Rider, John Brunner, 1975
https://en.wikipedia.org/wiki/The_Shockwave_Rider

IMO better than Alan Toffler's Futureshock (which is wrong, see 19th C. or early 20th.) because it's entertaining and not pretentious.

@cwebber I'm convinced it will be an AI agentic worm... because somehow people aren't allowed to use the word "agent" in the US ever since AI and now everything is agentic.

Agentic is the new idiotic.

@cwebber @vv If a local model is calling tools then it is still vulnerable to prompt injection.

@cwebber Having OpenClaw installed without my consent is some of the nastiest malware I've seen in a while :(

@dandylyons @cwebber there are various ways I could respond to this post, but instead:

I'd like you to consider *the specific two posts in this thread you are responding to* and ask yourself if your comment is remotely relevant, or if you are simply pattern-matching on anti-LLM sentiment and responding with aggression/a thread derail.

@dandylyons @cwebber for sure, but it still takes some level of ability to perform these tasks effectively, which local models, especially anything that can run on a typical machine, struggle with

@vv @cwebber This is a good point. For now, local models are not proficient at tool calling. I don’t expect that to last for very long though.

@cwebber According to #Shadowrun the crash virus is still three years away.

https://shadowrun.fandom.com/wiki/Crash_Virus_of_2029

"Fun" fact: In Shadowrun the Crash Virus learned to kill humans who connected their brains to the net. It was the start of lethal internet input.

@mcc @cwebber The original post was all about an LLM taking non-deterministic shell level actions at runtime. And you conflated that with deterministic code written by an LLM.

What I wrote is very relevant.

@cwebber In today's episode of "We build the Torment Nexus from the hit novel 'Don't build the Torment Nexus'"...

@cwebber

The postinstall script installs a legitimate, non-malicious package (OpenClaw). There is no malware to detect.

i beg to differ

@dandylyons @cwebber it is about an attack based on covertly deploying LLM development tools, with the possible intent of later using them to leverage a second stage attack. If the LLM development tools were already installed, installing openclaw would not have been necessary and the attack could have worked a different way. We are discussing a situation where *the developer of a piece of software I use merely having LLM tools on their computer represents a risk to me*

@mcc exactly put

@dandylyons

@dandylyons @cwebber in other words, if Christine's analysis holds, llm development tools create so much downstream risk to your users that *a malicious party would try to covertly install llm development tools for later exploitation*. That is the subject of discussion. Whether it is safe to install these things *at all*.

@aronia @cwebber it's only malware if it's bad for a computer from the silicon part of the periodic table, if it's bad for your carbon computer it's just a sparkling cognitohazard

@cwebber @amirbkhan Oh man. I remember how I, as a student, struggled to help fight a malignant computer virus and “clean” a large office building - while uninformed workers let their kids play on office PC’s to make things worse. This is orders of a magnitude more complicated. Not good.

@cwebber @vv A local model would be extremely noticeable (far too much CPU/memory/disk space usage), at least if a computer you regularly interactively use got infected (rather than some server/IoT device that's been running unattended for years and you forgot about). It would also be easy to mitigate by using slow hardware like a ThinkPad X200 (which would take hours to respond to a single prompt, giving you plenty of time to notice the malware and deal with it)

@neurobashing @cwebber just what we need, countless Agent Smiths running around.

@cwebber "Ha ha!"

@cwebber so I'm following this right, it sounds like the project or its maintainers don't even necessarily need to even be using LLM tools, the attack pattern simply targets contributors who are using LLM development tools? and so all that is really needed is for the payload to be subtle and the maintainer to be sufficiently overwhelmed (say, by an endless fire hose of LLM-generated liquid shit slop pull requests)?

@aeva Yes and it's worse than that: the maintainer doesn't even need to be running these tools on their computer. The attack I linked had Claude's independently-running REVIEW BOT on GitHub commit it via injection attack

@aeva But once that was done, the agent was set up to install on users' devices

So the initial attack vector can literally be "Any AI agent in your stack whatsoever getting tricked" as a pathway for infecting computers everywhere

@cwebber This is making me more worried about Vorta's Claude workflows.
Backup software that handles highly sensitive data would be a prime target for such a supply chain attack.

@csepp Don't forget about KeePassXC. I dunno if they kept going after this "initial test" or not https://www.reddit.com/r/KeePass/comments/1lnvw6q/keepassxc_codebases_jump_into_generative_ai/

@csepp And don't forget about LITERALLY MOZILLA FIREFOX

@cwebber @mcc @dandylyons
not forgetting the second post - the one that appropriately begins by "meanwhile" - wasn't conflating anything, it was contrasting the gravity of the situation with the surreallistically ingenuous state of mind of some people.

@cwebber Oh shit, I rely on all three of these.
Welppppp. I guess I'll have to start looking into alternative password managers.

@cwebber apropos of nothing, is pottery still a big deal for humans? i was thinking this morning that pottery might be a nice career change for me.

@csepp @cwebber Waterfox is a version of Firefox with all of the AI ripped out, but otherwise up to date with all the security changes and stuff, I think it may also have some additional privacy controls added

@Canageek @csepp Yes but Firefox itself is now being coded with AI generated commits

@cwebber @csepp GOD DAMMIT

@Canageek @csepp There was a recent thing, I can't find it now, where Mozilla added a commit to their agents thing to say "don't explicitly say when AI agents helped author a commit anymore", probably because they were getting community pushback

as you may have guessed, it got some community pushback

@mttaggart @mcc @cwebber Do we know what is being used for inference? At this point in time it's unlikely that they can use a self-hosted model, so there will be network calls.

@dvshkn @mcc @cwebber So the trick here is if you install OpenClaw in secret on a user's machine who isn't checking carefully, you might hide easily in network traffic. Use of tools like Claude Code would make the same API calls, which is likely for users who would be targeted with these attacks.

The real insane part is if multiple instance of OpenClaw were running on the same machine, so not even the process name looked suspicious. But of course process names are a poor indicator and can be changed.

@cwebber @csepp Vivaldi will have the same problem to, shit

@Canageek @csepp If you're thinking you might try switching to Chrome or even Ladybird, I also have some bad news

@dvshkn @mttaggart @cwebber one thing i wonder is if it's in principle possible to firewall claude/copilot endpoints. in the old days of the internet this would have been possible, in the present day the claude/copilot api servers are probably mixed in with the aws/azure IP pool and more than likely move around…

@aeva @cwebber I'm a stokie so my default answer is yes. But the answer might be different for normal people

@cwebber @csepp Yeah, I know Vivaldi has taken an anti-AI stance, but they're based on Chrome

AND from what I understand Servo Is nowhere near ready for end users, and based on every tech project I've ever liked will probably turn out to be either garbage or run by people who eat kittens or something by the time it comes out

@cwebber @Canageek @csepp https://github.com/mozilla-firefox/firefox/commit/71cc24b6a400dbd434e4df37087960d94b764791

@mcc @dvshkn @cwebber It's very easy and being done, although in big places you'll hear screams from your devs. api.anthropic[.]com can be blocked today.

@mcc @mttaggart @cwebber I think openrouter is another good inference endpoint to check for

@mttaggart @dvshkn @cwebber …that… should have occurred to me. I guess I got too used to the threat model of "is Windows 10 phoning home / searching bing without telling me", where Microsoft has the ability to ship IP lists. Probably only Microsoft can really do this.

… I guess if the attacker really thought ahead they could do DNS lookup through the firefox DoH server or something but they don't have much reason to try that.

@mcc @dandylyons @cwebber I cannot believe that we went from arguing about making all software memory-safe as a way of cutting out a way in which computers could be coerced into taking arbitrary instructions from a potentially malicious source to a bunch of the industry abandoning any concept of separation between data and instructions and installing highly non-deterministic, ambiguous arbitrary code execution systems on their machines…

@KormaChameleon @cwebber stokie as in the demonym for someone from Stoke-on-Trent, which, as I just learned from Wikipedia, has had a totally baller pottery scene since the 17th century?

@aeva @cwebber yes. If you lift up the toilet seat in any hotel anywhere in the world, there's a brand logo that we made

@aeva @cwebber Not really, it's been mass-industrialized so at this point outside of Etsy stuff you can largely forget it.

And no one's going to use very expensive handmade pottery, it's going to be a display piece.

@lispi314 @cwebber ah :( ok what about wheat. is wheat still a big deal?

@aeva @cwebber I got pushback for buying Denby, that's less than 100km away but it isn't the homeland

@aeva @cwebber Also mass industrialized but yes, food remains necessary.

Starting a farm sustainable economically depends a lot on local land & climate.

@lispi314 @cwebber gotcha. that might be promising. are there wheat jobs that can be done while sitting down in a chair

@cwebber
Hokey smokes

@mcc @dandylyons @cwebber we invented The Game for computers, why?!

@aeva @cwebber Depends on your standards there.

Tractors are pretty common tooling

But they need maintenance which isn't just sitting activity.

@cwebber@social.coop Morris II 😑
https://sites.google.com/view/compromptmized

@lispi314 @aeva @cwebber
Joel Salatin thinks raising healthy chickens for eggs to sell can work just about anywhere near a big town or larger population.. _Pastured Poultry Profits_ .. you might be able to design their shelters, coops or whatever so that you can remain seated most of the time.. I read the being seated a lot isn't healthy though..

@bsmall2 @aeva @cwebber For those who decide to do this, please pay attention to health & sanitation practices.

(Improvising it without care has been a problem in various places & cases.)

@bsmall2 @lispi314 @cwebber I'm not accepting ableist remarks or unsolicited medical advice from strangers on the internet at this time.

@lispi314 @bsmall2 @cwebber i have it on good authority that~~unlike wheat~~farm animals smell really bad

@aeva @bsmall2 @cwebber Yeah, outside of particular fertilizers being used (I have lived in the boonies), wheat has a generally inoffensive or mildly pleasant smell.

@lispi314 @bsmall2 @cwebber maybe that could be my angle. "poop-free wheat"

@aeva sure all you have to do is to get all the machines in the fields in IoT and control them making the job with an AI agent-.. #ohwait..
@lispi314 @cwebber

@bituur_esztreym @lispi314 @cwebber this town's finished.

@aeva town? i thought the planet was a village..
@lispi314 @cwebber

@aeva @bsmall2 @cwebber From what I understand on an intellectual basis the root of the issue is that they refused to let it compost for long enough in the right conditions for it to fully complete and not have that issue.

It was probably within whatever norms have been established as "safe" but that didn't exactly make it pleasant for anyone living downwind that particular day.

@aeva @lispi314 @cwebber
"No Shit Wheat!"

@bituur_esztreym @lispi314 @cwebber it's a reference https://www.youtube.com/watch?v=F9OmTnuLzeQ

@aeva @lispi314 @cwebber oh thanks. didn't know it. could have guessed..
my only consolation is my answer was, too.. obvious one `w;7[)

@aeva @cwebber one of my friends sister is a professional potter. Her business is booming, and she does specialize in pieces for people to actually use, custom kitchen stuff mostly. I can try and arrange an into if you would like to talk to somebody who made it work.