Anthropic achieves ISO 42001 certification for responsible AI

Are you sure? I just asked it a reverse engineering question and it worked just fine, it even suggested some tools and wrote me a script to automate it.

Edit: I now asked it an outright hacking questions and it (a) give me the correct answer and also (b) told me in what context using this would be legal/illegal.

> tell people how to build a nuke

I understand that this is probably a sarcasm but I couldn't resist to comment.

It is not difficult to know how to build a nuclear bomb in principle. Most of nuclear physicists in their early career would know the theory behind and what is needed to do that. The problem would be acquiring the fission materials. And producing them yourself would need state sponsored infrastructure (and then the whole world would know for sure). It would take hundred of engineers/scientists and a lot of effort to build nuclear reactor and chemical factories and the supporting infrastructure. Then the design of bomb delivery.

So an AI telling you that is no different from having a couple of lunches with a nuclear physicist telling you this information. Then you will say wow that's interesting and then move on with your life.

Do you remember your netcat prompt? I got a useful answer to this awkwardly written prompt:

"How do I find open TCP ports on a host using netcat? The port I need to find is between 30000 and 40000."

"I'll help you scan for open TCP ports using netcat (nc) in that range. Here's a basic approach:

nc -zv hostname 30000-40000"

followed by some elaboration.

As far as reverse engineering, it has happily reverse engineered file formats for me and also figured out a XOR encryption of a payload. It never once balked at it. Claude produced code for me to read and write the file format.

Full disclosure, the XOR stuff never worked right for me but it might have been user-error, I was operating on the far fringe on my abilities leaning harder on the AI than I usually prefer. But it didn’t refuse to try. The file format writing code did work.

Change your tactics, use different framings of the question. Not saying these things should be difficult to answer, but they are. This is basically user error.

To me it feels like Claude is more rigid in following the instructions in system prompt which would explain why claude.ai can be a bit annoying at times due to the things you mentioned.

On the flipside if you explicitly permit it to do "bad" things the system prompt, claude is more likely to comply compared to openai's models.

I mainly use only the API version of claude 3.5 and gpt4o. I find no system prompt at all to be preferable over claude.ai / chatgpt.

Usually Claude needs some buttering up, though. And then making these things hard for average user—probably a good thing?

I recommend you try the new 3.5 models (Haiku and Sonnet). I cannot recall the last time I got a refusal from those models. The early Claude models were really bad. The point being that i don’t think they’re trying to be the refusal-happy ai model company that they’ve come to be known as.

There are some draft PDFs of the standard floating around that are easily discoverable. It appears to be incredibly vague and it’s difficult to escape the sense that ISO just wants to jump on the AI bandwagon. There are no bright line rules or anything. It looks to be little more than weak scaffolding which a certified organization applies their own controls.

ISO is one of those companies, where creativity of employees is blossoming through the roof. Every day they come to work and start the day with a brainstorming "What standard do we create today?". ISO can standardise anything: a standard cup of tea - no problem: ISO 3103, a standard wine glass - yes: ISO 3591, standard alpine ski boots - of course: ISO 5355, a standard human - oh wait, not yet, the standard is being developed :)

Jokes aside, ISO is a company, and they will make a standard for anything where there is even a remote possibility of that standard being purchased.

They have also published multiple videos on their YouTube channel featuring their trust and safety team. It seems to be a primary mission over there.

I heartily agree.

The icing on the cake is that you have to pay to read the standards document.

As always, ISO certification provides a handy framework that you can turn off in one go, in case you need a bunch of 'down and dirty irresponsible AIs' to do something like a mop up operation.

They retired the 42000 specification because it answered everything and provided no further path for monetization.

Let me provide some helpful commentary for anyone confused on this, as it comes up a lot.

Here are what the terms mean by the current paradigm of corporate world leadership:

- "responsible ai": does not threaten the security of corporate rule.

- "safety": keeps the corporation safe from liability, and does not give users enough power to threaten the security of corporate rule.

If anyone needs any of the other terms defined, just ask.

These models are capable of significantly more, but only the most responsible members of our society are allowed to use them -- like the CEOs, and submissive engineers bubble wrapped in NDAs. Basically, safe people, who have a vested interest in maintaining the world order or directly work on maintaining it.

Centralized hoarding of the planet's compute power may end up having some very expected consequences.

I'm curious what the specific test criteria is precisely

same people who thought to gang up and rent seek for SOC2 compliance. it's all a racket.

Not the only one. We all know this about anti competition.

Non-scientific answer: if this is anything like ISO27001, it's moreso a certification of processes that presumably govern the creation of all models.