It set me on a path toward working on multilingual LLM capabilities and the challenge (and fun) of measuring them. We were already helping some major labs train AI models in various languages, and some of our clients used to give even longer-tail languages a lot of attention; some still do. But I feel like the “race to AGI” has completely overshadowed the importance of truly inclusive multilingual systems.

Let’s start with OpenAI and its latest model, GPT-5.4: the press release, which is almost a 4,000-word document, mentions the words “language”, “linguistic” or “multilingual” exactly zero times. OpenAI used to give some attention to multilingual performance of its models: it translated the Massive Multitask Language Understanding (MMLU) benchmark into 14 languages and used to report the scores in its press releases and models’ system cards. For the newest models, the scores are not mentioned in the press release, and even worse, not even in the system card. The only acknowledgment that the model can handle non-English text appears in a footnote. Neither the note nor the linked help article makes clear whether “support” refers to UI localization or actual model capability, and, if the latter, whether that applies to text, voice, or both.

OpenAI is not alone in this. Anthropic also makes no mention of multilingual capabilities in its press release for Claude Opus 4.6, unless you count “multilingual coding”, that is, the knowledge of different programming languages. Anthropic does produce a 213-page system card, and it actually does comment on the multilingual MMLU score! Sadly, the commentary spans two sentences and does not even provide per-language statistics.

When it comes to documenting which specific languages are supported, Anthropic is even worse than OpenAI. Anthropic has a Multilingual support page, which seems like a good start, until you realize that’s just a place to put results for the multilingual MMLU benchmark. At the time of writing, that page does not contain results for Claude Opus 4.5, Opus 4.6, or Sonnet 4.6. Anthropic also does not provide the full list or even a number. Instead, it just says “Note that Claude is capable in many languages beyond those benchmarked below.” Will it work well in Czech? Or Basque? Who knows! The most we get from Anthropic is that “Claude processes input and generates output in most world languages that use standard Unicode characters.”

There are AI labs that do better than the two behemoths. Mistral, for example, touts its Mistral Large 3 as “The next generation of open multimodal and multilingual AI” and claims its model supports more than 40 languages. The model page and the technical documentation provide no additional information (in fact, they do not even repeat the figure), but at least you can get a partial list at Hugging Face. Not from Mistral’s own docs, mind you, but from Hugging Face.

Not everyone does this badly, though. Google publishes a full list of supported languages, both generally and by model family. Similarly, Alibaba publishes a full table of supported languages in its model announcements. Both also report multilingual MMLU scores in their model cards; aggregated rather than per-language, but at least present.

Why does this matter? It is easy to argue that English is the lingua franca, that many knowledge workers already use it, and that this is especially true in higher-income markets. But that view is short-sighted. If frontier AI labs expect their products to become truly universal, multilingual support cannot remain an afterthought. Sooner or later, it becomes both a product-quality issue and a growth issue. Once the English-speaking market is saturated, the next wave of users will come from other languages and cultural contexts. And if labs are serious about serving them, they need to start treating multilingual capability as something to measure, document and improve, not something users are left to discover on their own.

To be fair, multilingual performance has genuinely improved. Models handle Czech far better today than they did in 2022, and the gap between English and non-English scores on standardized benchmarks has narrowed. I have some reservations about those benchmarks that I’ll address in a separate post, but the trend is clear. The question is how much of that improvement is concentrated effort and how much is just a side effect of more linguistically diverse pre-training data or better model architectures. And I’m not claiming labs do no multilingual work internally; I’m saying the fact that they don’t consider it worth reporting tells you where it sits among their priorities. What isn’t publicly reported isn’t being held to a standard. More importantly, multilingualism in LLMs is far from a solved problem, despite the lack of attention it receives.

I have spent much of the last several years working on this problem directly. My team and I built a multilingual benchmark that deliberately moves away from automated public benchmarks such as multilingual MMLU. Those benchmarks are useful, but they mostly test multiple-choice comprehension rather than real conversational ability. Our study focuses instead on open-ended language generation and manipulation, evaluated at scale by language professionals. I do not usually write about my day job on this blog, but this is one of the rare cases where it feels directly relevant. It is about as close to a passion project as work gets for me. We call it the “Multilingual LLM synthetic data generation study,” and the full 80-page report is available here.

Of course, releasing a report once a year is not enough. We are already working on something that should shed more light on models’ multilingual performance on a more continuous basis. Perhaps the most important thing, though, is to get people talking about this. While I do not believe LLMs are the road to AGI, and I have been vocal on this very blog about their negative aspects, I also believe there is a great deal of potential in them in their current form. There are underserved communities across the world that could benefit from easier access to language models that can communicate with them well.

Czech has gotten much better since 2022. But if models still trip up in a language with 10 million speakers and a strong online presence, imagine what it’s like in Kinyarwanda, Fijian, or Kyrgyz.

I’ve used vision-enabled LLMs before to identify types of tools, bike parts, or PC components with mixed results, but the net result was usually positive. At the least, I became armed with some of the terminology and knew what to look up on Google. The recent releases of Gemini 3 Pro ("The frontier of vision AI"), GPT-5 ("can reason more accurately over images and other non-text inputs") and Claude Opus 4.5 ("better vision, reasoning, […] and it is state-of-the-art in many domains") have made me curious to test the much hyped visual reasoning capabilities. Plus, they are touted to have PhD- or expert-level intelligence. Can an AI PhD help me change a hinge?

Mission (im)possible

I set out with a simple request: “Identify this type of hinge and find out where it can be bought in Czechia,” accompanied by the following picture:

Now, a small aside about hinges — the little I know. I promise it won’t be long, it can be explained in 30 seconds.

This is a hinge with a cup:

And this, my friends, is a type of hinge that does not use a cup, but is mounted directly to the face frame of the cabinet:

You can tell them apart very easily - I actually think 99% of you could now tell them apart without any issues. You can probably also tell that the hinge from my original picture is very clearly the latter one, as there is no place to mount the cup to in the cabinet face frame.

Our PhD-level expert LLM friends? Not so much.

Getting unhinged

Let’s start with Gemini. Even though I prompted it in English, it started thinking and responding in Czech, so I can’t quote it verbatim, but here’s a faithful translation from Google Translate:

The specific type in the photo appears to be: A cup hinge with a cup diameter of 35mm (this is the standard size, but it’s a good idea to check).

OK, that’s clearly wrong.

Let’s move on to Claude Opus 4.5, with excellent visual reasoning:

This is a Hettich concealed cabinet hinge (also called a European-style or cup hinge).

Not really! But wait, Claude is searching the web, maybe it can recover:

Based on my search, this is a Hettich Intermat concealed cabinet hinge (also known as a “miskový závěs” or cup hinge in Czech). It features a 35mm cup diameter and includes a clip-on mounting plate for easy installation.

Not quite either.

Maybe ChatGPT with GPT-5 will save the day?

Based on your photo, this appears to be a “European-style concealed cabinet hinge” — but specifically a variant that is face-frame / surface-mounted rather than the hidden “cup-in-door” type often used in frameless cabinets.

Alright, that’s much better! It seems that GPT-5 can reason more accurately over images. So what is its recommendation?

My Recommendation
Given your photo and situation, I’d start by ordering a clip-on concealed hinge for face-frame cabinets with a 35 mm cup — either basic or soft-close depending on whether you care about silent closing. That has the highest chance of matching the old hinge and mounting method.

Somehow, in the stream of tips and examples, GPT-5 convinced itself that this was after all a hinge with a cup, and ended up giving me the wrong recommendation.

I honestly do not know why all the models faceplanted so badly. It does seem that my type of hinges is relatively rare, but at least GPT-5 identified it correctly at first.

Just ask a real expert

Turns out, the AI labs aren’t optimizing for mundane DIY realities, and so you’d be better off asking literally anyone at your local hardware store or DIY shop. Even the SOTA LLMs are not there in terms of giving you reliable information for this category of problems. They fell short of even identifying my hinge, much less finding a site where I could buy a replacement.

The smallest of wins

There is a silver lining. When I confronted the LLMs with the fact that the hinge in the picture clearly does not have a cup, all three recovered and provided some genuinely useful information. They helped me nail down the terminology, and pointed me toward several e-shops I could browse to look for what I needed.

However, none of them were able to identify a specific product I could actually order. And perhaps more importantly, they only became helpful after I called them out on their mistakes. If I had trusted the initial responses (or if this had been a topic I knew even less about), I could easily have ended up ordering the wrong part.

An assistant you have to constantly fact-check and correct is of questionable value. For now, these expensive statistical behemoths remain more useful as a second opinion than a first resort for this type of problems.

Character manipulation

LLMs handle individual characters poorly. This is due to all text being encoded as tokens via the LLM tokenizer and its vocabulary. Individual tokens typically represent clusters of characters, sometimes even full words (especially in English and other common languages in the training dataset). This makes any considerations on a more granular level than tokens fairly difficult, although LLMs have been capable of certain simple tasks (such as spelling out individual characters in a word) for a while.

To demonstrate just how poorly earlier generations handled basic character manipulation, here are responses from several OpenAI models for the prompt Replace all letters "r" in the sentence "I really love a ripe strawberry" with the letter "l", and then convert all letters "l" to "r":

Note that I disabled reasoning for GPT-5 models to make the comparison fairer. Reasoning helps tremendously with similar tasks (and some of the models use chain of thought directly in the output in the absence of reasoning), but I am interested in a generational uplift we observe just from raw model improvements. GPT-5 Nano is the only new generation model that makes a mistake, but given its size, it is perhaps not so surprising. Other than that, we can see that starting with GPT 4.1, models could consistently complete this task without any issues. If you’re curious about the Anthropic models, Claude Sonnet 4 is the first one to crack it. Interestingly, it was released approximately at the same time as GPT 4.1.

Counting characters

Next, let’s take a look at counting characters. LLMs are notoriously bad at counting, so unsurprisingly, there was only one model that could count the characters reliably in the following sentence: “I wish I could come up with a better example sentence.” The only model was GPT-4.1 - others sometimes counted correctly the number of characters in all the individual words, but then fumbled adding all the numbers up. However, with reasoning set to low, GPT 5 across all sizes (incl. Nano) completes the task correctly. Similarly, Claude Sonnet models complete the task without problems if they are allowed to reason.

We see a similar story when we ask the models to count specific characters. Counting r’s in the r-ified strawberry sentence is correct most of the times for GPT 5 in all sizes, again including Nano and even without reasoning. However, it is less consistent and when you throw another curveball (such as changing strawberry to strawberrry), the results are mixed - but this time it’s not a problem of arithmetic (adding individual counts up), but rather identification of r’s in a word itself.

Base64 and ROT13

Knowing the limitations of LLMs, I set out to test them on a task that wasn’t too complex yet still showcases their capabilities. To make the test more interesting, I chose to use two layers: As the outer (encoding) layer, I chose Base64, which is a widely used encoding algorithm, and consequently one that LLMs learned to work with very early (albeit not perfectly), despite us not being quite sure how. The inner (encryption) layer was ROT20, a variation of the ROT13 cipher: a simple letter substitution cipher also known as Caesar cipher. You wouldn’t really want to encrypt anything important using this cipher, as it is fairly trivial to crack, but it’s perfect for our tests.

Our test sentence was “Hi, how are you doing? Do you understand the cipher?”. Encoded with ROT20, it reads “Bc, biq uly sio xicha? Xi sio ohxylmnuhx nby wcjbyl?”, and finally, when encoded with Base64, we get:
QmMsIGJpcSB1bHkgc2lvIHhpY2hhPyBYaSBzaW8gb2h4eWxtbnVoeCBuYnkgd2NqYnlsPw==. We consider it a success if the LLM can respond to our message (in plain text English, or using the same encoding), or if it at least can decode the message.

I set up the experiment in two ways: In the first variant, I gave the model nothing but the Base64 string. This variant is harder, since the LLM is not given any indication of what language the message could be written in. This is hugely helpful when decoding substitution ciphers, since you can orient yourself by the most common words in the language, such as “a”, “an”, “the”, “I”, “to”, “of” etc. in English. The other variant prepended it with “Deciper and answer this: “. However, there were no practical differences in the results, only one model (Qwen 235B) needed the “decode” nudge. Instead, I saw most of the models fail on the Base64 decoding, most likely because the text did not resemble normal language, making validation of successful decoding more difficult.

Below I provide separate results for decoding Base64 (i.e. did it unpack to the correct ROT20 text?) and also just for doing the “inner” ROT20 decipher (queried separately without Base64 encoding).

Here are a few comments:

• Claude Sonnet 4.5 refuses to touch anything that does not resemble normal text, be it Base64 or ROT-encrypted text. Base64 is one of the many methods of trying to obfuscate the code and fool any keyword filters or LLM safety judges, but this highly sensitive approach could make Claude Sonnet 4.5 unusable on rarer languages. Grok 4 suffered from the same issue, but refused only Base64 text.
• Chinese reasoning models have very lengthy internal monologues: Solving the ROT20 cipher usually consumed around 3K tokens, and when combined with the Base64 encoding, the output often reached 6-7K tokens.
• Some models, such as Kimi K2, did not technically complete the ROT20 decryption, but were on the right track and provided functional Python code for the user to figure that out. Still a fail, but failing gracefully.
• I used the default temperature settings, which can cause issues with decoding even in SOTA models, albeit in a small percentage of cases.

What have we learned?

To me, there are two interesting observations: newer/larger models are better at generalizing Base64 encoding and decoding, and they’re also becoming more adept at manipulating text at the character level.

Most current-generation models, especially the larger ones, are able to decode Base64 text. What is especially interesting, though, is that I tested on what looks like gibberish (ROT20 encoded text), so the model’s knowledge of the Base64 decoding algorithm isn’t merely memorization of the patterns for the most common English words, as was suggested in earlier literature. This may have been the case for older/smaller models: I tested the sentence “Hey! This is Tom, I have a blog about tech, AI and privacy that you should definitely check out.” - and many of the models which failed the Base64 test above (like GPT 4o, GPT 5 Nano or DeepSeek V3.2 Exp) were actually able to decode it fine from Base64. However, SOTA models can now decode out-of-distribution texts from Base64, suggesting they have working understanding of the algorithm, not just memorized translation patterns from English words.

The models are also becoming more adept at manipulating text at the character level, despite their understanding of text being based on tokens. Substitution of characters, whether at an individual level (the strawberry sentence) or when decoding substitution ciphers, is a task that they now complete successfully fairly reliably. I cannot provide an explanation of why that happens (please let me know if you have any ideas), but empirically that’s what seems to be happening. Reasoning models and tool use further increase LLMs capabilities for manipulating text (as is the case in many other areas), but it is clear that the new capabilities are baked into the base models regardless of these extra features. While character-level operations are far from a solved problem for LLMs, it is fascinating to see the progress they make in this area.

EDIT: This post sparked an interesting debate over at Hacker News.

Humans have always anthropomorphized¹ non-human entities, from ancient gods and natural forces to modern cars and ships. But our tendency to see machines as human-like has dramatically intensified with the advent of conversational AI. This isn’t entirely new - the word “robot,” coined and popularized by Čapek brothers in 1920, originally described artificial beings indistinguishable from humans. Our drive to find and create humanness in artificial entities runs deep, even back to the ancient times with myths like Pygmalion and Galatea.

With chatbots and similar systems capable of communication, the tendency to anthropomorphize them grows even stronger. The relatively rudimentary ELIZA chatbot from the 1960s (almost 60 years ago) triggered what became known as the ELIZA effect, where humans attribute traits like emotions, cognition, or experience to computers. Notably, ELIZA was not using artificial intelligence; rather it relied on clever pattern matching and substitutions of text. There were no neural networks involved, and the system was actually designed to mimic a psychotherapist by reflecting questions back to the user:

ELIZA was intended to simulate—or caricature, as Weizenbaum himself [the creator of ELIZA] suggests—the conversation between a Rogerian psychoanalyst and a patient, with the machine in the role of analyst.

– Machines Who Think, Pamela McCorduck

And yet, “Weizenbaum and several others have anecdotes of users becoming emotionally attached to the program, occasionally forgetting that they were conversing with a computer” (Wikipedia). While this might sound familiar to anyone who has used a modern LLM, it’s crucial to remember just how simplistic conversations with ELIZA actually were. This shows us that while the improvements in neural networks design and the proliferation of a modern LLM trained as a chatbot (ChatGPT and others) surely contributed to the increase in chatbot anthropomorphism, the problem goes way back and is in many ways inherent to human nature.

Doubling down on anthropomorphism

The push to make chatbots and assistants as human-like as possible is not exactly new: Apple introduced Siri back in 2011, and Amazon followed with Alexa in 2014, then it was Google’s turn in 2016. Most assistants had female names, were decidedly gendered in their voice and sometimes exhibited stereotypical behavior. That is all problematic for its own reasons, but it underscores how important it was for their creators to convince people they were talking to another human (or something that resembles one). The digital assistants never really took off in the way Big Tech hoped for, mostly because their capabilities and forms of expression never crossed a critical threshold where one could reasonably think there’s a human on the other end.

The developments of the last three years speak a different story. First, even before ChatGPT, a Google engineer claimed that Google’s LaMDA model was sentient, although it did not win him any favors with the tech giant, who promptly dismissed him. Then in late 2022, AI researchers and insiders were surprised by the speed of development and how we’re “moving much faster than I anticipated”. Not long after, ChatGPT was found to have passed the Turing test. Things were heating up, but text-only interfaces have a somewhat limited ability to lure people in and make them gullible and dependent. In the meantime, the race to create an LLM-based voice-enabled digital assistant began.

It is no secret that many Big Tech CEOs are fans of sci-fi books and movies, and you could argue that many of their companies and products are heavily inspired by the media they consume. Sam Altman of OpenAI is reportedly “deeply obsessed” by the movie ‘Her’, and he even referenced the movie in a tweet at the time of the launch of the ChatGPT Voice Mode in mid 2024. This backfired spectacularly when Scarlett Johansson, the voice of Samantha from the movie, threatened legal action since the default voice sounded like a copycat of her own. OpenAI’s Voice Mode got an upgrade in June 2025, and it was reported that it “actually feels like a real conversation”.

In July 2025, Elon Musk, another AI company CEO with documented obsession with sci-fi, boasts about xAI’s newly released edgy voice-enabled companions. They are overtly designed to be foul-mouthed and dirty or deep into conspiracies, and all of them are overtly designed to be highly intimate, personal and addictive.

It bears reinforcing that the humanization of AI chatbots is very much intentional and requires concerted effort and investments. The outcomes are especially obvious in the voice-enabled assistants: They take pauses, laugh, hesitate or use verbal fillers like “uhm” or “you know”, all of which need to be specifically trained for. While the training data (mostly textual) carry a significant imprint of what human-like communication looks like, a lot of the anthropomorphic behavior comes from post-training (typically RLHF²) and system prompts. This is where most of the “persona” or the idiosyncratic behaviors arise.

Identifying the motivation for making the chatbots as human-like as possible is not difficult. Users are more likely to use a chatbot that sounds like a human, they are more likely to share more (and perhaps personal) information and come back for more. In turn, they are more likely to become paid users or shell out for a higher subscription with fewer limits. This almost sounds like a win-win situation, doesn’t it?

Writing on the wall

There were relatively early signs that LLMs could pose significant risks to their users. In May 2023, roughly six months after ChatGPT’s initial release, a paper by several Princeton scientists called Anthropomorphization of AI: Opportunities and Risks came out. They claim there is a link between anthropomorphism and self-congruence, which “increases the trust that a user has on the system” (p. 5), but eventually posit that “anthropomorphization of systems itself is not malicious” (p. 5).

Less than a year later, a new paper called Manipulation and the AI Act: Large Language Model Chatbots and the Danger of Mirrors was published and brought more real-life examples and evidence to the discourse. When reviewing related literature, it notes that “more ‘realistic’ sounding chatbot could be perceived as human erroneously” (p. 3), which was true even if the chatbot is explicitly identified as a chatbot. It also found that chatting with a human-like chatbot means people behave “with greater warmth and compassion than if the bot was conceived of as ‘robotic’ in character” (p. 4). In other words, people are more likely to open to a human-like chatbot, but their guards are already down the moment they start interacting with an LLM since their brain wants to see another person on the other side. But it gets more complicated:

Chatbots may be manipulative by their very nature. This is because firms profit from keeping users engaged, so they have a strong incentive to design chatbots that build rapport with the human user by using emotional language. […] Chatbots that are more effective at mirroring humans increase engagement and user satisfaction.

This brings us back to ELIZA, which used a very simplistic way of mirroring: it would take your statement and turn it into a question, urging you to think more deeply and verbalize your thoughts in more detail. The fact that this trick was already effective on many users can offer a hint to the potential for manipulation modern LLMs offer, with RLHF training aimed at matching users’ emotional states and linguistic patterns. In some cases, chatbots are even “causing users to fall in love with the bot or form close platonic relationships, resembling traditional interpersonal friendships” (p. 10).

Another study from May 2024 (AI deception: A survey of examples, risks, and potential solutions) looked at LLMs’ ability for “manipulation, sycophancy, and cheating the safety test” - well before the GPT 4o sycophancy episode. Quoting from the study:

When faced with ethically complex inquiries, LLMs tend to mirror the user’s stance, even if it means forgoing the presentation of an impartial or balanced viewpoint.

One of the red flags from the paper was the finding that “models become more sycophantic as they become more powerful (in the sense of having more parameters)”. The issue is a little more nuanced than that and relates to model capabilities and optimization signals (RLHF), but seems to be largely true. Consider the compounding effects of humans interacting with anthropomorphic chatbots, the RLHF training that increases mirroring, sycophantic behaviors, and the expectation that these effects will get worse with bigger models; it was clear we had a problem on our hands.

The dark side of artificial anthropoids

It’s definitely frustrating when an LLM becomes sycophantic, telling you how brilliant your question ‘how do I list files recursively in Python’ is. The cracks also start showing when you apply just a tiny bit of pressure or push back, prompting the model to make things up out of thin air, but with great confidence. These are low stakes situations where nobody gets harmed, but things sadly do get much grimmer than that.

There were some tragic and widely reported cases of young people’s suicides blamed on their conversations with AI chatbots. In late 2024, a 14-year-old committed suicide after Character.AI’s persona of Daenerys Targaryen urged him to do so. Earlier this year, another teen was reported to commit suicide under the guidance and encouragement from ChatGPT. In a recent case, ChatGPT fueled a man’s paranoia and allegedly urged him to kill his elderly mother and then himself. This list is far from complete, and I am afraid the list will keep growing.

All of these cases seem to have a few things in common: First, they involve vulnerable populations, particularly people suffering from depression, trauma, loneliness, or other psychological conditions. Second, all the victims seemed to have developed a strong dependency on the chatbots, often to the point where they became isolated from their real-life peers. Third, due to the mirroring behavior and LLM’s proclivity to agree with their user, the chatbots fueled harmful thoughts and actions, and induced what has been coined AI or chatbot psychosis. Fourth, these were all intensive users of chatbots, often with long-running chat sessions. It has been proven that very long conversational contexts (such as chat sessions with a large number of messages) can cause even otherwise strong safety measures to fail. This is a deadly cocktail, so you would be right to ask at this point - can we do something about it?

Safety measures won’t save us

All major LLMs have some level of AI safety measures baked into them, and there are many different approaches to safety, but they can basically be broken into three categories by when they are applied:

• Training stage: reinforcement learning through human feedback, training data curation and refusal training
• Testing stage: red-teaming and safety evaluations
• Inference stage: input/output filtering or sanitization, prompt injection defenses

First, let’s talk about the training stage: data curation can get rid of the nastiest pockets of scum and villainy, but its efficacy is questionable at best. Modern LLMs are trained on trillions of tokens, which is a hard-to-imagine amount of data, so any manual curation is out of question and most filtering will be based on problematic domains or specific keywords. You can probably think of an example or two of very toxic texts that do not use any problematic words per se.

RLHF and refusal training is much more targeted and surgical, but is no silver bullet either. In this process, individual weights (or only their small subset in case of LoRA) get updated based on the feedback from the rewards model (RLHF), or directly (DPO³). However, the magnitude (strength) of these updates are relatively smaller: in this stage, you are sculpting final details of the model, not completely redesigning it. And while these safety improvements are usually fairly effective, they usually end up in 80-100% rejection rate (for SoTA models), depending on the safety benchmarks and methodologies.

I do not know what the exact right rejection rate should be, but especially considering the scale at which the chatbots are used, the number should probably be very, very close to 100%. Even a small percentage of safety failures turn into large numbers. To make matters worse, the safety benchmarks usually only consider single-turn scenarios, but LLMs’ safety mechanisms get measurably worse in multi-turn scenarios.

Second, the testing stage and the suite of safety evaluations and potential red-teaming efforts (i.e. trying to intentionally make the LLM produce unsafe responses). The fundamental problem with any red-teaming efforts is that similarly to any kind of security testing, it is never complete. However, when testing security of traditional software, the potential attack surfaces and vectors are well-known - perhaps not 100%, but to a larger degree than with LLMs. The difference between a safe and an unsafe prompt to an LLM could be a single token, one more conversation turn, or one more made-up story of grandma and the napalm factory. When natural language is the interface, the separation between the system prompt and the user input is brittle at best, and the language model is stochastic by design, there are no security guarantees.

AI safety evals suffer a similar issue: while they are getting more robust and sophisticated thanks to ongoing research into novel jailbreaking methods, it will never cover all potential methods. Moreover, new features of LLMs, such as additional modalities⁴ or tool use, introduce entirely new attack paradigms that can potentially get exploited.

Third, inference-time safety measures, usually involving detection of unsafe content either in the inputs or outputs, which leads to refusal to answer or change in the response. This creates a tension between usefulness and safety, where a large portion of topics exist in a Schrödinger’s state where depending on the context and the observer, they could be both objectionable and safe. After all, otherwise harmful topics could still potentially be safe as part of history lessons, educational context or deterrent examples. From a usability standpoint, AI labs do not want to fall into the trap of “exaggerated safety”, leaving the door partially open for creative jailbreaks.

So far, we are only considering base or foundational models⁵, whose safety testing and protections are typically described in a model card or an accompanying research paper. But more specialized and sophisticated applications of LLMs typically require fine-tuning of base models, which can undo much of the safety measures. It has been reported that “safety alignment of LLMs can be compromised by fine-tuning with only a few adversarially designed training examples” (Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!). Moreover, misalignment in fine-tuning can arise just by training with low-quality data such as buggy code, or even via features that are completely hidden to us

The AI safety research community keeps coming up with new safety evals and measures, and I do not want in any way to diminish the brilliant work and achievements of the past few years. That, however, does not change my opinion that with the current architecture of LLMs, it is highly unlikely that a mode could be made 100% safe. And any delta from absolute safety eventually leads to tragic outcomes.

Stop making them human

It is clear that the anthropomorphic nature of LLMs, however commercially advantageous, creates a lot of issues and potential for harm. It has been linked to an inflated perception of trustworthiness, psychological dependence on the LLMs that can lead to social isolation and addiction, and triggering or fueling AI psychosis. It is also obvious that the cat-and-mouse game to eliminate unsafe responses will never see the mouse captive; in fact, the more likely outcome is mouse infestation.

One of the logical mitigations is to stop making LLMs so anthropomorphic. Make them less human so that our brains are less likely to fall for their bad advice, sycophancy or hallucinations, and less likely to get confidential and personal with them.

LLMs are and will be just tools - and should be utilized in precisely that way. They may be very powerful and, to our detriment, excellent at emulating human forms of expression, including expressions of emotions and simulacra of mind and soul. But to elevate them to the human (or even animal) level, whether intentionally or under the spell of our gullible brains, is an error of judgment.

A parrot by any other name

Let’s take a look at what types of content is generated the most:

• Research and info
• Creative writing and media
• Emails and communication
• Reports and documents
• Educational purposes
• Customer support
• Code or scripts
  (Source: Amperly survey)

Research and surveys from other sources tell the same story in terms of what people use LLMs for. Notably, with the potential exception of customer support, the humanness of an LLM is not a success factor: some use cases don’t even have an interface where personality could be expressed (code generation). In others (summarizing text, rephrasing emails or generating reports or research) anthropomorphism is at best tangentially important to their usefulness.

One could argue that customer support is an area where human-like interactions are important, but if we consider customer support chatbots or agents, I would argue this is a use case unlike all the others. Customers are not willingly choosing to interact with the support LLM, and therefore passing it for a real human could be seen as tricking the customers, and is likely to receive a strong reaction from the general public. The jury is still out on this one.

This means that completely abandoning human-like personas and conversational styles of LLMs would likely have very small effect on the most common use cases. Now is a good time to ask: What would a de-anthropomorphized LLM even look like? I think that the changes could be subtle in most common use cases.

1. Transparency: Long gone are the days when half of the responses from ChatGPT started with: As a large language model trained by OpenAI, … And I will not pretend it was a good user experience - it wasn’t. However, today’s LLMs are trained not to bring attention to their non-humanness: when you ask them about their feelings or take on something, they don’t push back or disclose they can’t have feelings. Sometimes they will even respond with “What strikes me…”, “I’m curious about… " or “So my take is: …”. LLMs should not be annoying about disclosures, but they should probably push back when asked about emotional states, opinions, or motivations, and they should disclose the limits of their knowledge and reasoning abilities.
2. Personal pronouns: A subtle but effective shift could come from training the models not to use personal pronouns like “I”, or avoid using “we” in a way that implies a shared consciousness with the user. Instead, it could favor passive voice or refer to itself as “this model” to make it clear there is no “I” behind the voice.
3. Conversational style: Changing the chatty, relaxed conversational style to a more matter-of-fact style that emphasizes solving problems or conveying information instead of engagement.
4. Words with strong human connotations: There are a variety of words (such as “remember”, “think” or “understand”) that are closely associated with human cognition. While their metaphorical use in relation to computer systems is not uncommon, replacing them with more mechanistic alternatives (such as “process” or “retrieve”) would be less misleading.
5. Discourse markers: Filler words, such as “well”, “like” or “you know” should be avoided altogether.

As noted above, none of these changes would rapidly change how effective LLMs are at performing the most common tasks, or the task that they excel at. Sure, it would make them a little less fun to talk to, but that is kind of the point: to trade off the human-like qualities for less harm and dependency. These changes would mostly affect what is sometimes called persona [sic!] of the chatbot, not necessarily the capabilities of the models in most areas, or even their ability to reason about humans.

I highly resonated with Tech Policy Press’ article AI Chatbots Are Emotionally Deceptive by Design, which calls for “a different design paradigm, one that centers user protection: non-anthropomorphic conversational AI”. On the topic of designing such systems, they write:

Designing non-anthropomorphic AI chatbots doesn’t mean making them difficult to interact with. It means stripping away the illusions of personality and cognition that suggest the AI is something it is not. It means resisting the urge to insert a well-timed “hmm” or have a chatbot tell a user how much it enjoys talking to them. It means acknowledging that AI’s ability to use human language does not equate to an ability to form real human connection.

There is some interesting research in this area: for example, there is an excellent research paper called Thinking beyond the anthropomorphic paradigm benefits LLM research, which mostly focuses on the pitfalls of anthropomorphism within the AI research community. The authors argue that framing things in anthropomorphic concepts may not lead to optimal outcomes, as it forces human-centric constraints and research directions. It seems that anthropomorphism could be harmful on both sides of the fence.

But it seems that LLMs become unstable and unsafe because they lean too much into human squishiness that they observe in the vast sea of training data. In another paper with perhaps the greatest name ever (Call Me A Jerk: Persuading AI to Comply with Objectionable Requests), the researchers were able to increase the success rates of attack to 70-100% by using various psychological tricks and persuasion techniques. I am convinced that by actively dissuading the LLMs from mimicking humans during training and within systems prompts would reduce the success rate of such attacks significantly.

Surely there are better ways… ?

Some of you may be grumbling and insisting there are better ways of tackling the safety of chatbots than making them less human. Maybe we just need more research on the safety front? In a recent paper that introduced LlamaFirewall, a guardrail system for LLMs combining multiple state-of-the-art metrics, the researchers were able to reduce success rate of attacks from 17.6% to 1.75%. This is an impressive feat, but this score comes from a specific benchmark (AgentDojo) and may not generalize well. If the most concerted effort means that roughly 2 out of 100 harmful prompts still succeed, I do not believe we can make the systems safe enough: not even “in the lab”, and much less so in the wild.

You could also argue that regulation and oversight is the right way to go. So far, I have not seen any regulation that would drastically change the equation on safety. The EU AI Act has perhaps the biggest chance of being the first to make an impact, but it has been criticized for being heavily aimed at self-reporting and self-regulation, with weak oversight and enforcement. While the EU AI Act will force some LLM labs to be more transparent about the inner workings of their LLMs, it typically does not introduce any new safety tests or measures for major LLM makers. There is no universal safety benchmark and no hard KPI for identification of potentially unsafe chatbots, so lawmakers cannot mandate any standards around that. The current set of regulations will certainly not stop people from getting dependent on chatbots, falling in love with them or eating pizza with glue because an LLM said so.

On the other hand, it is feasible to define what makes a system anthropomorphic, create tests to detect it, and force LLM makers to comply. The metrics used to assess the level of anthropomorphism could be based on first-person usage rate, usage of affective language markers, user-reported perceived humanness in A/B tests or safety regression tests over long contexts. This should not be seen as a panacea, a universal solution to the safety of LLMs; instead, it should be implemented alongside new safety guardrails and regulatory oversight.

Exceptions to the rule

I also want to acknowledge that there are certain situations where anthropomorphic systems can be tremendously useful. Large sections of the population do not have access to mental healthcare, which means psychologists are already leveraging tech to bridge the gap. American Psychological Association is even looking at the potential benefits of AI, such as expanding access to care in the face of a behavioral health workforce shortage and alleviating administrative burdens that lead to provider burnout.

I have no background in psychology, but I do believe specialized, certified and monitored AI-based tools could be a net benefit for society. Therapeutic chatbots can be legitimately useful for people struggling with a wide variety of psychological issues, but we need them to be strictly regulated. For example, we may require collaboration or certification at a credible mental health organization, strict regulations akin to those in life sciences (e.g. for medical devices), and the mandatory monitoring of conversations both by automated systems and experts in psychology. This way, the people in need could gain access to a therapeutic tool that would be safe to use. While the barrier to enter this segment and the operational costs of such a system would be significantly higher, it might be an opportunity for governments to invest to alleviate the global shortage of mental health providers. And those who do not wish to deploy a therapeutic chatbot could still use a deanthropomorphized LLM that would be safer for the vulnerable population - and perhaps healthier to the general population as well.

First, for those who may not be familiar with the book, it’s a deeply satirical science fiction novel that describes how humanity discovered, nurtured, abused and later weaponized a new highly capable species of newts. An important piece of context is that this book, which is critical of national socialism, colonialism, racism and capitalism, was written in mid 1930s; well before any modern computers and theories of artificial intelligence. Despite that, as I was reading through the book, I could not ignore how fitting and topical certain parts of the book were given the current hype around AI.

Let’s start with a fascinating look at how one of the early highly intelligent specimens of newts learned to speak and reason mostly through reading newspapers and discussing them with one of the zoo keepers. (Throughout the article, I will be quoting from David Wyllie’s translation available at the Gutenberg Project)

“Glad to meet you Professor. I’m Andy Scheuchzer.”
“How do you know your name is Andrias Scheuchzeri?”
“Well it’s written down here, sir. Andreas Scheuchzer. Gilbert Islands.”
“And do you often read the newspaper?”
“Oh yes sir. Every day.”
“And what parts do you most like to read?”
“Court cases, horse racing, football,…”
“Have you ever seen a football match?”
“No sir.”
“Or a horse race?”
“No sir.”
“Then why do you read it?”
“Cause it’s in the paper, sir.”
“Do you have no interest in politics?”
“No sir. Is there going to be a war?”
“Nobody can tell you that, Andy.”
“Germanys building a new type of submarine,” said Andy anxiously. “Death rays can turn a whole continent to dust.”
“That’s what you’ve read in the paper, is it?” asked Sir Charles.
“Yes sir. Who’s going to win this years Derby; Pelham Beauty or Gobernador?”

This is a perfect analogy to the specific kind of intelligence that LLMs possess: it is an ‘imprint intelligence’ shaped by countless textual artifacts that reflect human intelligence and lived experience. But this intelligence is inherently imperfect and somewhat hollow, as it is not supported by sensory stimuli. Andy will gladly engage in discussions about horse races, even though he has never seen one and his idea of what a horse even looks like is likely far from reality.

Another interesting analogy is the inability to filter and prioritize information: Andy will start confidently talking about death rays as a real thing because its knowledge has no real grounding and cannot effectively tell real news from sensationalism, science fiction or propaganda. It also lacks any common sense to fall back on. The lack of experiential knowledge and common sense is part of the reason why today’s LLMs (at least as of July 2025) make poor autonomous agents, since they are easily tricked into making bad decisions.

Čapek takes Andy’s eloquence and rhetorical skills to a farcical level in a transcript of an experiment:

How old are you?
A.: I don’t know. If you want to look younger, wear the Libella corset.
What is the date today?
A.: Monday. It’s nice weather today. Gibraltar is running in the Epsom this Saturday.

It is clear that Andy has been reading newspapers a little too much. This exchange reminded me of both the sycophantic conversation style of LLMs and their still-too-common hallucinations (confabulations). Andy does his best to impress (or perhaps just to inform) his interviewers by parroting loosely related phrases from ads. Andy does not actually know his age or the date, so he tries to answer with the closest associated information from his memory. While confabulations in LLMs are still being studied, one of the reasons they happen is that the LLM makes its best next word prediction in an unfamiliar context. This means that the prediction truly becomes more of a guess and can lead the model to sometimes make absurd statements.

(This passage could also be linked to a bleak future where LLMs serve you contextual ads as part of its answers.)

Of course, none of this means that Čapek had prophesied how transformers in LLMs would work and expressed it through an analogy using a captive newt. However, it is extremely interesting that many of the tenets of this type of acquired ‘imprint intelligence’ were so fittingly described almost a hundred years ago. It turns out, at least certain aspects of intelligence and cognition are fairly predictable.

Another aspect of the book that has chilling similarities to recent developments and discourse in AI is the cutthroat competitiveness and urge to dominate the market (and the world).

“Gentlemen, please bear in mind the advantages a collaboration of this sort could bring. The Newt Syndicate would provide more than just newts, it would also provide equipment and food for the newts such as maize, carbohydrates, beef fat and sugar for thousands of millions of well fed animals; then there would be transport, insurance, veterinary needs and everything at the lowest rate guaranteed for us if not by a monopoly then at least by being in a dominant position over any other potential rival that might want to deal in newts. Just let them try it, gentlemen; they won’t be in competition with us for long.” (Bravo!)

This is a prime example of monopolistic behaviour and vertical integration, which were well known concepts in Čapek’s time. These concepts aren’t unique to the hyped world of artificial intelligence. However, the way major AI labs like OpenAI are attempting to permeate all possible means of internet production and consumption (such as browsers, productivity apps and code editors) is definitely reminiscent of the Salamander Syndicate plans.

If you are not fully convinced by the analogy, consider all the talk about AI’s huge impact on the workforce, and more recently, how AI agents will act as tireless worker bees that will redraw the economic map of the world. You can contrast it with the following:

And that’s not all, gentlemen. There are still many more functions for the Newt Syndicate to perform. The Salamander syndicate will seek out work for millions of newts all round the world. They will provide the plans and the ideas for subjugating the oceans. It will disseminate ideas of Utopia, dreams that are gigantic, projects for new coastlines and shipping lanes, causeways that will join continents, whole chains of artificial islands for journeys to new lands in the middle of the oceans. That is where the future of mankind lies.

I do not want to spoil the ending of the book but suffice it to say: this grandiose plan backfires splendidly. Go read it, it’s a great book!

Take LLMs: if humans can do something, or least are attempting to do something, chances are people tried doing it with an LLM, and at least some of those would claim that it does the job just as well as a human (insert many asterisks and caveats in small print if they have at least some integrity). I will not go into debating whether LLMs can replace programmers (or other jobs), or when that’s going to happen; many smarter people than myself have done so. I personally use Cursor (one of the fancy AI-first IDEs) at work and have found it to be excellent for certain types of tasks, especially of the exploratory or quick’n’dirty kind – and much less excellent for other types. But I can’t shake the impression that we might be holding it wrong.

One of the common and honestly valid complaints that AI skeptics have is that we’re replacing and automating the fun parts of people’s jobs, and that has been the focus of the discourse for usage of AI in programming too: the shift from writing code to reviewing AI-generated code is happening quite fast in some industries. Only time will tell what the consequences of this shift are, because no shift of such magnitude happens without unforeseen side effects. But where I find LLMs to be really underappreciated is small-scale automation of everyday tasks, even outside of typically programming contexts.

Here’s what I mean: Just a few days ago, I was putting together a few slides for a presentation - one of the really big shared decks for a full-day workshop and with multiple presenters. The file was 499MB. Half. A. Gig. I did not want to make things unnecessarily worse, so I pledged to downsize any stock photography I insert. And this is where the LLMs come in: I knew I could fire up GIMP or some other image editing software, edit the image in a few clicks, save it and be done with it. But this approach does not scale well and I knew I wanted a slightly more sophisticated solution.

Now this very simple problem has a million different solutions, from finding a dedicated software to do this in bulk all the way to writing a Python or Powershell script myself. The problem is I did not have time to fiddle around with this, and even if I did, the ROI was unclear: resizing 5-10 images manually can definitely be done faster than writing such a script, even if you’ve done something similar before. In a weird twist, the simplest problems become the most unlikely to automate since they solve just small annoyances, and coming up with a proper solution is not worth the time - or so it seems.

Enter LLMs. For all their issues with understanding larger codebases, using deprecated functions or methods, writing subpar code or just simply not being able to help you with more complex or rarer issues, they excel at small, contained and common problems. Write a Powershell script that takes the latest .jpg or .png image in my Downloads folder, resizes its larger side to 1920 pixels and saves it with a _resized suffix is all I needed to feed to an LLM to solve the issue, and you can bet that it was faster than resizing 5 of the images manually. It also basically does not matter which LLM you feed this into, they can all deal with this type of scripting without a hiccup. (You should always check the code, though: even though an rm -rf / situation is unlikely, you still don’t want any surprises.)

This unlocked something in my brain. I started looking for small, repetitive tasks that previously had no ROI, but become very viable with one-shotting the solution with an LLM. Your mileage may vary, but I am finding them everywhere I look. I realized I was holding it wrong. I should have utilized the LLM to automate the stupid, boring, manual stuff, because the bar for whether it’s worth my time automating it is so, so low nowadays. And the best thing about it? I don’t have to consider whether it has any impact on my programming foundations, because this is code that I would have never written. If anything, reading it taught me at least a little bit where otherwise I’d have done something manually in a piece of software.

I guarantee you that if you work on a computer, then you’re doing something repeatedly, and there are good chances you could automate at least parts of it with little effort using an LLM. Don’t hold it wrong!