Who decides what “good AI behavior” looks like in each market?

A few days ago, I came across one of the most thought-provoking AI papers I have read in a while. It is the latest one from Anthropic. Maybe you have already read it and, and if not, I’ll leave the link in this post because I genuinely think it is worth reading.

Is AI translating… or changing its personality?

In this paper, Anthropic questions how Claude responds when we talk to it in languages other than English. Those of us working in the Localization industry and using AI models had already seen a few glimpses that made us raise an eyebrow and think, “Careful… something a bit strange may be happening here from a cultural point of view.” But until now, we did not have the kind of valuable data that Anthropic has shared with everyone over the last few days.

Until now, we had been asking ourselves questions like these:

When you ask an AI a question in another language, is it only translating the answer? Or were we already suspecting that something else was going on, like it was changing the way it behaves?

The shocking revelation, and I say shocking because it comes from Anthropic, the company behind Claude, is that the response we get may depend not only on what we ask, but also on the language we ask it in.

And wow, I am not used to seeing this level of honesty from companies!

So this paper moves the conversation away from being purely about translation and into a much more uncomfortable territory, where cultural bias starts to appear and where we begin asking who should decide how AI behaves across different countries and cultures

Before going any further with what we can or cannot do from our Localization perspective, let’s first look at what Anthropic actually found. I think this is one of those moments when it is worth stopping to reflect.

Anthropic’s study analyzed more than 300,000 real-world Claude conversations across its most-used languages. Instead of focusing on translation quality or factual accuracy, the researchers sought to understand something subtler: the values and behaviors the model expresses when it speaks to people.

To make sense of that, they grouped thousands of different conversational patterns into four main axes.

• Does the model go along with you, or does it push back?

• Does it prioritize warmth, or rigor?

• Does it give long answers or short ones?

• Does it sound open about uncertainty, or overly polished and certain?

It is important to notice that Anthropic is not analyzing Claude from a translation perspective. The paper is not asking whether the model translates correctly. It asks questions about personality, communication, and the cultural traits that influence how we interact, depending on where we come from.

One of the axes they identify is what they call deference versus caution. In simple terms, does the model tend to agree with you and go along with your reasoning, or does it stop and warn you when it thinks you are making a mistake?

This is actually the dimension I relate to the most, and I'm sure you've experienced something similar.

I use ChatGPT extensively in my daily work, and over time, I have noticed that it often leans towards agreeing with me. Sometimes, even when I am intentionally testing it with ideas that are clearly weak or simply wrong.

Another is warmth versus rigor: does it prioritize making you feel supported and understood, or does it prioritize being direct and correct even if that feels less pleasant?

And that is where things get more interesting.

Because the biggest variation across languages was precisely on that warmth-versus-rigor dimension.

According to the paper, Hindi and Arabic stood out as the warmest language-model combination by far. English and Russian, at the other end, tended to be much drier and more rigorous.

So if you ask Claude something in Hindi versus English, you are not getting the same answer in two different languages; you are getting a totally different approach.

The first thing that came to my mind: Erin Meyer

And all this immediately brought Erin Meyer’s The Culture Map to mind. One of the reasons I enjoyed that book so much is that it reminds us that people do not communicate in the same way everywhere. I have written quite a lot on this blog over the years about the ideas behind the Culture Map and cultural intelligence, so when I read Anthropic’s research, it felt a bit like déjà vu. We already knew that some cultures value direct feedback, while others communicate much more indirectly. Some expect disagreement to be explicit, while others see preserving harmony as equally important. None of those approaches is inherently better than the others. They are simply different ways of communicating.

But now AI introduces something much more problematic.

Which version should I trust?

Let’s imagine I ask Claude to review a Localization strategy before I present it to my stakeholders or my C-suite. If I ask the question in Hindi or Arabic, and the model naturally leans towards warmth, perhaps it tells me my strategy looks great. It reassures me. It validates my thinking. But if I ask exactly the same question in English, where the model tends to be more rigorous, maybe I will receive much more honest and critical feedback. The AI points out weaknesses that I hadn’t considered.

And this is where I stop for a moment.

View fullsize

Click HERE to download the infograhic

In which scenario will I be better prepared for the important meeting where I’m to discuss my localization strategy with the leadership team?

Imagine I genuinely believe my strategy is brilliant because I asked in one of the languages Anthropic classified as “warm,” but in reality, it has several obvious flaws. What if Claude fails to point them out simply because, in that language, warmth is a stronger behavioral characteristic than rigor?

Should the same person receive different levels of criticism solely because of the language they used to ask the question?

And I think that is exactly why this research is so interesting. So this stops being just a question of whether AI can speak different languages. It becomes a much bigger question: should AI also behave differently across cultures?

The uncomfortable part

What worries me, on the one hand, but also gives me some hope, is that Anthropic is very open about the fact that it does not fully understand why this happens. It takes a lot of courage to admit that. Anthropic is not hiding the result or softening it. It is publishing it as it is, even though it does not exactly make the company look good.

Anthropic has what they call a model constitution. And if that constitution is written in English, and if it was mostly shaped by people who think in English, and if the result is that the model becomes more rigorous in English but more accommodating in Hindi or Arabic, then what we have is not just a language difference. We have a cultural bias embedded inside a product that hundreds of millions of people may use every day.

This is not the usual content bias we talk about. It is something else. It is an attitude bias. It is about whether the model pushes back or just tells you what you want to hear, and that seems to depend on the language you use.

Anthropic also admits something important: they have not yet studied the real-world impact of this. So, in practical terms, nobody really knows what effect this has on people.

And that is probably the part that should make us stop for a second.

Because it is not the same when I ask AI to help me create a pivot table in Excel (something I embarrassingly ask more often than I would like because, for some reason, I always forget how to do it), as when I ask it about something much more personal.

The picture changes completely. Maybe I am asking for advice about a relationship. Maybe I am using AI for emotional support because I am going through a difficult time.

Or maybe I am asking about something where what I actually need is not reassurance, but an uncomfortable truth. If the model sounds warmer in one language and more rigorous in another, that is not a small difference. A subtle shift in tone can make the difference between receiving the advice I actually need and simply hearing what I would like to hear.

And then there is the question the study leaves hanging, which I think is the hardest one.

What is the right balance between warmth and rigor, or any of those dimensions for each culture?

Because once you start thinking about this, you really have only two paths.

One is to leave it as it is.

You assume that the variation is at least partly the model adapting to local conversational norms, and that sounds reasonable enough.

The other is to intervene and make the model behave more consistently across languages.

And that is where things get uncomfortable.

Who decides what the “correct” personality is when speaking to someone in Hindi, Arabic, Russian, or any other language?

Final thoughts

What I really appreciate about Anthropic’s paper is that it does not pretend to have the answer. It ends with more questions than conclusions, and maybe that is exactly why it matters. It reminds us that we may have spent years talking to machines as if they had one single personality, when the reality may be much more fluid. Maybe there are hundreds of Claudes. Maybe the same is true for other models too. And maybe the companies building them do not yet know which of those personalities is the right one. That is a strange place to be, but also a useful one. Because once you see it, you cannot really unsee it. And if AI is going to keep becoming more conversational, maybe the next frontier is not just making it multilingual, but multicultural. Maybe it is deciding how it should behave in each language. And that, to me, is where the role of Localization professionals suddenly becomes much more interesting.

Click HERE to download the infographic