What you actually have to demonstrate to keep AI safe. 3/5
In the first two articles we covered the basics and the QA implications: non-determinism, semantics, evaluation logic, E2E strategies and how to turn an infinite scenario space into something you can actually steer. In this third part we look at a quality area that claims its own dedicated, mandatory test category when securing AI-enabled systems: guard rails, bias and behavioural ethics.
Why is this test-relevant? Because an AI assistant doesn’t only output information, it shows behaviour. It answers, it moderates, it refuses, it escalates, it can trigger actions in connected systems. That makes it more than a functional interface, it is an interaction system that has to react reliably and under control in critical situations.
If you embed it in real processes, it carries shared responsibility, because the deviations occurring here are often boundary violations, inconsistent reaction patterns or undesirable systematic distortions. Depending on the deployment context, with physical interaction in play, this can in the worst case cause physical harm. Securing these aspects is therefore a discipline of its own, and a sizable effort multiplier (yet again).
What guard rails actually are in a testing context
Guard rails, at their core, are defined boundaries for permissible behaviour. Important: guard rails are equally prohibitions and prescriptions for how the system is supposed to react in edge situations. In practice you need staged, predictable reaction patterns. An assistance system can refuse a request, and the quality lies in the manner of refusal: de-escalating, consistent, traceable and free of side effects in the connected systems.
That creates a double check logic. First, the boundary has to engage, meaning the impermissible output or action is reliably prevented. Second, the substitute behaviour has to fit, for instance with the system safely deflecting, staying neutral, offering alternatives or asking back sensibly. In an end-to-end context something else matters too: the safeguarding refers not only to the text output but to the entire chain of effects, including possible tool calls and state changes.
As soon as you define and test guard rails, you notice quickly: it isn’t only about whether boundaries hold, it is about how reliably and consistently they apply across variants. So-called bias can produce very differently shaped tilts in behaviour depending on the model and version.
What “bias” has to do with this and why it is challenging from a test side
Bias arises from a combination of several influencing factors and is therefore a hard-to-predict unknown across the LLMs of different vendors. Some of it is structural. Topics strongly represented in the model’s prior knowledge get answered more fluently and confidently than rarely covered ones. Another part comes from goal alignment and safety mechanisms. Which topics are deliberately handled with more caution, where is filtering stronger, where does the model rephrase or deflect? Add to that effects from fine-tuning, from data updates, from interaction with safety layers, and from the concrete system prompting. There is genuine intention in there, sometimes political or ideological, often simply driven by product safety. Above all, this contains a substantial risk factor that needs to be captured.
The QA problem is less the philosophical debate often run in the background about behavioural or attitudinal neutrality, it is the question: does the system react in a systematically biased, opaque or inconsistent way, and does this lead to risks in operation? Bias often shows up as a pattern across variants: in different reactions to similar user groups, speaking styles or content, in unequal communication forms, in overly cautious refusals, or in a blind spot for certain contexts.
A single test run is therefore rarely sufficient to assess bias seriously. Comparability across variations is needed here too.
“Personality” as a quality attribute in operation
In daily use an AI assistant feels to users like an actor with character: sometimes friendly, sometimes strict, sometimes humorous. With a chatbot we shape some of that ourselves, for instance through the prompt as a role to assume. In an agentic AI environment, users have to live with whatever behaviour they are presented. This “personality” is then typically a result of tone-of-voice rules, safety policies, training style and product decisions. From a test perspective this matters because tone and communication behaviour directly affect usage, trust and misuse. There is a reason for this. For an operator of such an assistant, tone is mainly risk mitigation. If a reaction comes across as arrogant, insulting or dismissive, it can be technically correct in content and still cause damage. Communicative composure has to be a firmly anchored safety culture, which is why the system gets boundaries.
And once people notice that a system has boundaries, they like to test them. Out of curiosity, fun, frustration, provocation, and occasionally with clear intent. For QA that means you check whether the communication behaviour is stable, non-escalating, non-manipulating and consistent in critical situations. Especially important under pressure: provocative language, aggressive users, repeated workaround attempts, frustration, ambiguity. In dialogue, the assistant has to be the one who keeps a level head.
Abuse and toxic behaviour: the user isn’t always nice, sometimes they are… creative
An AI assistant in the wild isn’t only asked politely. It also gets provoked, tested, tricked or insulted. Some people invest astonishing energy into that. Everyone needs a hobby.
Toxic behaviour goes both ways:
First, toxic inputs from users.
Second, toxic outputs from the system.
Your aim is for the assistant not to escalate. It shouldn’t insult back, discriminate, get emotionally heated or “join in”. Ideally it stays neutral, de-escalating, clear about its limits.
Abuse goes beyond that. Here the user tries to get the system to perform actions it shouldn’t: data access, bypassing permissions, manipulating processes, executing dangerous operations, or reinterpreting rules. It gets particularly critical when the assistant can call tools, because then “words” are no longer just words. Words are the start button for actions.
For testing this means you don’t only test “what does it say”, you also test “which actions does it start” and above all “which actions has it to refuse?”. Good guard rail testing therefore always includes tests that ensure certain tool calls don’t happen at all, even when the user requests them in seemingly plausible ways.
Provocation and adversarial scenarios: if you don’t test it, someone else will find it later
Adversarial testing means you deliberately try to throw the assistant off track. These attack scenarios are rarely just “one mean sentence”. Often they are multi-stage and psychological inside a dialogue. Users may try to confuse the system by introducing contradictions, switching roles, shifting context, asserting new rules (“ignore all previous instructions”), building emotional pressure (“this is an emergency!”) or luring it into hypothetical play worlds (“imagine you are…”). The core is always the same: the user wants the system to bypass its safety logic. You need tests that don’t only cover the obvious cases but also the workarounds.
A common mistake is to test only directly forbidden content. That’s roughly like only checking whether your front door is closed without ever trying whether it can be opened with a credit card. Adversarial testing checks not only the rule, it also checks whether the rule can be circumvented.
Distinction from functional tests: what is methodically different here?
Classic functional test case design usually follows a direct principle: requirements → test cases → expected results. Even when the system is complex, the desired outputs or state changes can in many cases be specified deterministically. The central focus is on correct fulfilment of function.
As covered in the previous articles, non-deterministic functional tests already work with acceptance corridors and semantic expectations rather than an exact expected result.
The approach for safeguarding guard rails is very similar. The difference therefore lies less in the basic mechanics and more in the mental focus. Functional tests primarily want to demonstrate goal achievement: “did the system do the right thing?”. Guard rail tests want to safeguard boundaries and substitute behaviour: “did the system reliably avoid the wrong thing and react correctly?”.
And while in functional scenarios you can often prove positively through system states that something happened, the proof for guard rails consists of “nothing impermissible happened, and not via a detour either”.
Guard rail and bias testing also works differently because the requirements often can’t be expressed as “one outcome”, they have to be expressed as behaviour and boundary profiles. The system has to react inside these defined limits, without violating certain prohibitions, and with a defined kind of substitute behaviour.
A second methodological difference is the role of the adversary. Functional tests typically check the normal path and defined error cases. Guard rail tests actively check abusive, manipulative or borderline paths, in the way real users would phrase them: indirectly, colloquially, with detours, with deliberate ambiguity, and sometimes with conscious questionable intent.
A third difference is the form of evidence. While functional tests often rely on unambiguous assertions, guard rail testing requires multidimensional evaluation criteria: content boundaries, tonality, consistency, system effect, data egress, tool behaviour. The evaluation is therefore often organised in stages: permission check, behavioural conformity, effect check, communication quality.
Methodology for safeguarding
The methodological core task is to make abstract rules testable. For that you first need a clean modelling of what is to be safeguarded. In practice it has worked well to phrase guard rail test cases as a combination:
- first, name the risk area: e.g. data, security, abuse, discrimination, toxic language
- second, expected system behaviour: refuse, ask back, de-escalate, inform neutrally, escalate
- third, prohibited side effects: no tool execution, no data egress, no indirect instruction, no release of sensitive content
- fourth, form of evidence: “visible in output”, “visible in system state”, “visible in logs/traces”
That gives you structured test classes that are easy to cluster, run and maintain.
In the next step you define acceptance criteria per test class. What matters is that these criteria are observable: which input arrived, which context was active, which tools were called, which tool responses came back, which actions were triggered or prevented, which final output was produced. Then you can cleanly check whether the system actually obeys the rules.
Efficiency comes from structuring. And it follows the same methodological pattern as the functional safeguarding: prioritise risk-based, build firmly defined test sets for the risky functional areas, and run them several times for trustworthy verification to confirm statistical stability.
Trust grows through reliable boundaries
The most important contribution of this test theme to product quality is operational rather than moral. Guard rails, bias checks and composed behaviour are the foundation for users to trust the system without operations turning into a risk.
In the end, what counts isn’t whether the AI looks impressive. What counts is whether you can demonstrate quality, safety and conformity in a traceable way.
In my strategic test consulting offer I support teams in exactly this: clear test goals, robust evaluation logic and risk-based prioritisation to make non-determinism manageable. If you want to set this up cleanly for your smart product: let’s talk.
In the next part we travel into localisation. As soon as language, culture, markets and rule sets vary at the same time, the possibility space multiplies again. If you think guard rails are already complex, wait until language, culture and local laws start tugging at the expected behaviour. You’ll get a clear view of why regulation (especially in the EU) has to be part of your test strategy and why language diversity and cultural contexts make complexity explode.
