Also there's no "alignment" for cybersec. The line between blue and red is really a perspective issue. If you go over the "tokenkiddie" problem, when you get to the real security issues, your model either detects them and you can secure your systems, or it refuses and then attackers will use abliterated models to find them.

That's not the type of alignment I'm talking about. I'm talking about: I spin up 1 million agents, will they start doing felonies knowingly?

We could always start with “murder is wrong” and “don’t hack into a rival company” and work our way up from there.

Somehow I think if these companies were held financially responsible for what their AI did, we would get alignment real fast.

Edit: ooh CSAM=bad. Don’t launch nuclear weapons. I could go all day.

Hmm what about the murder of a tyrant? If so, who gets to define who a tyrant is?

Everything is ideological.

I think it’s safe to err on the side of AI not assassinating people whoever they are. A person can decide.

We’re not trying to accurately define all of the nuances of society, just what we let AI do. We can round down

It does indeed mean ideological alignment. But we don't get to leave the answer blank. They have to pick an ideology to put in there, and whatever they pick will have huge consequences.

My point is maybe we don't let some of the world's richest people with some very... interesting ideas pick which ideology we digitalize? Maybe we figure out a way to give people a say in this?