AI Weapons Without Human Review and Musk's Colossus 2

Two big things happened. The US and Russia pushed to strip human review out of the rules for AI weapons. And Elon Musk's xAI says Colossus 2 will hit 1.2 million chips by December.

One of those makes me nervous. The other just makes me tired. Let's take them in order, because they are more related than they look.

The US and Russia removed human review from AI weapon rules

The rules in question cover weapons that pick and hit targets using AI. Not a video game. Actual people, actual explosives.

The earlier draft asked for three things: reliable behaviour, ethical consideration, and a human signing off before a strike. All three got softened into weaker, non-binding language. Meanwhile ground robots keep showing up on real battlefields.

Here is the part that gets me. The framework was already non-binding. Nobody was forced to follow it. Two of the biggest military powers on earth looked at a polite suggestion and decided even that was too much paperwork.

You don't remove a rule you were never obliged to follow unless you plan to do the thing the rule mentions.

I'll give the other side its fair hearing, because there is an argument. Machines don't get tired. They don't panic. They don't have a fight with their partner at 6am and make a bad call at noon.

It is genuinely possible that an automated system would hit the wrong target less often than a scared nineteen year old who hasn't slept.

That argument still loses for me, for one reason: error rates are not the only risk. Software gets hacked. Models get fed bad data. A targeting system can be right 99% of the time, and still be turned around and pointed at your own side.

And an autonomous weapon doesn't get slower when things go wrong. It gets faster.

There's a business lesson buried in here, and I promise I'm not reaching. The most common mistake I see with automation is people automating a process before asking whether the process deserves to exist.

Somebody wants an agent for a task they've never written down and never handed to anyone else. It also makes them no money. The honest answer is often "don't automate this, delete it."

Automation multiplies whatever you point it at. Good process, great. Bad process, you now have a bad process running at scale with nobody watching. That's the whole argument in one sentence, and it applies to invoices and to drones.

So when we scope work, half the job is deciding what should be automated and what should stay human. Removing the human is a decision, not a default.

For you, practically: nothing here changes your Monday. But the same computer vision and decision-making tech that routes your leads is now being fitted to vehicles with explosives on them. If someone on your team is squeamish about AI, this is why. Don't tell them they're being dramatic.

Why xAI's Colossus 2 is chasing 1.2 million chips

Now the cheerful story. Colossus 2 is xAI's giant training site, the place where enormous numbers of chips work together to build and run AI models. Musk says he's planning 1.2 million chips there by December.

Where things stand now, roughly:

• 110,000 GB200s, Nvidia's AI chips from the previous generation
• Close to half a million GB300s, the newer ones
• Three more batches of 220,000 chips planned before December
• Enough electricity to run all of it: still an open question

That last line is the one everyone skips. Chips are easy to order and hard to power. You can have a warehouse full of the fastest hardware ever made and a grid that politely declines.

The strategy is clear enough. There are a few ways to win in AI: better software, more hardware, more power. Anthropic and OpenAI have been winning on software. Musk has decided to compete where he actually has an advantage, which is buying absurd amounts of physical stuff very quickly.

It's a blunt instrument, but blunt instruments work. More compute means bigger training runs and faster experiments. If your model is less efficient, you can throw more silicon at the problem until the gap closes.

Not elegant. Historically effective.

There's also the boring upside nobody posts about. Sites like this bring construction work, electrical work and permanent jobs to wherever they land. Private money is carrying the risk here, and private money gets the reward if it pays off. I'm fine with that trade.

What it means for you: the models you use are about to get cheaper and better again. And you didn't have to buy a single chip. The last two years have followed the same pattern. Capability goes up, price per token goes down, and your tools quietly improve while you're busy with something else.

So my advice on Colossus 2 is to mostly ignore it. Don't wait for the next model before you fix your follow-up process. The compute arms race is a spectator sport for almost everyone reading this.

The bit where the two stories meet

Same technology, two very different rooms.

The stuff getting built in Colossus 2 will power the AI that writes your emails and reads your invoices. It will also power targeting systems that nobody is required to keep a human in front of anymore. We have never really had a tool that stayed on the nice side of the line. This one won't be the first.

I'm not going to pretend I have a policy fix. I don't run a country. I run a small shop in Pula that builds AI agents for businesses, and my influence over the Geneva Conventions is limited.

What I can control is the standard I hold myself to. Every automation we ship has a clear answer to one question: when this gets it wrong, who notices and how fast? If nobody notices, it doesn't ship. It's a boring rule. I'd rather keep it.

Apparently the biggest militaries on earth have decided that question is optional. Somewhere there's a slide deck filing that under efficiency gains.