You ship a chatbot on a Friday. By Monday someone has convinced it to write a phishing email, and a screenshot is doing numbers on social media. This is the fear that sells guardrails, and it is a real fear. What people rarely tell you is that guardrails are less a wall and more a series of speed bumps, and that a determined person in a car will clear all of them. The goal is not perfection. The goal is to make the bad outcome rare, boring, and logged.
The four layers that actually do something
Think of a guardrail system as filters wrapped around the model, not as changes to the model itself. There is what goes in, and there is what comes out, and you can inspect both.
Input filtering catches trouble before the model ever sees it. This is where prompt-injection detection lives, where you strip or flag attempts to smuggle instructions into user text, and where you block obvious abuse. Tools like LLM Guard run scanners for injection patterns, personal data, and banned topics. They add maybe 10 to 50 milliseconds and can run in parallel with the main call, so latency is rarely the reason to skip them.
Output filtering reads the model's answer before your user does. A moderation model scores the text, and anything over a threshold gets blocked or rewritten. OpenAI's approach here is almost quaint in its simplicity: you describe the content domain, give grading criteria, and get back a score from one to five, blocking anything at three or higher. Llama Guard, which is a fine-tuned Llama model, frames the whole thing as an instruction-following task across a taxonomy of unsafe categories: violent crime, self-harm, weapons, and so on. It works because language models are genuinely good at following instructions, which is the same reason they are so easy to trick.
System prompts are the cheapest guardrail and the most oversold. You tell the model who it is and what it will not do. This shapes default behavior well and stops nothing determined. Treat the system prompt as tone and policy, not as security.
Allow and deny lists are the least glamorous and often the most reliable. A deny list of exact strings, regexes, or topics will never be clever, but it also will never be talked out of its job by a clever user. If your product must never output a competitor's name or a specific slur, a hard string match beats a probabilistic classifier every time.
Where each one breaks
Every layer has a failure mode, and knowing them is the whole job.
- System prompts leak and get overridden. "Ignore previous instructions" is a cliche because it kept working.
- Moderation classifiers miss novel phrasings and flag harmless ones. They were trained on yesterday's attacks.
- Deny lists are brittle. Users route around them with spacing, synonyms, or another language.
- Input filters cannot see intent that is spread across a long, innocent-looking conversation.
The pattern underneath all of these: any guardrail built out of a language model can be attacked with language, and any guardrail built out of fixed rules can be stepped around by changing the words. You do not get to pick a layer that has no weakness. You get to stack layers so that a single trick has to beat several different mechanisms at once.
What a sane setup looks like
Do not reach for the heavyweight toolkit on day one. Start with a moderation call on both input and output, a short and specific system prompt, and a deny list for the handful of things that are truly non-negotiable for your business. Log every block with the input that triggered it. Those logs are the actual product here, because they show you the attacks you did not imagine, and next month's deny list writes itself from them.
Reserve the programmable frameworks, NeMo Guardrails and its relatives, for when you have real conversational flows to constrain: topic steering, tool-call gating, structured dialogue where you need the bot to refuse to leave a lane. They are powerful and they are also a lot of configuration to maintain, so earn your way up to them.
The uncomfortable truth is that a public embarrassment is usually a monitoring failure, not a filtering failure. The teams that get burned are not the ones without guardrails. They are the ones who set up guardrails, saw the demo work, and never looked at the logs again. A speed bump you are watching is worth more than a wall you have forgotten about.