As AI systems become more capable and take on more autonomous tasks, a basic question becomes more pressing: how do we make sure they reliably do what we actually want? That is the core concern of AI alignment, a field that has grown from a niche academic topic into major research programs at AI labs, universities and government institutes.
Updated September 2026: we added links to the research behind each approach and updated the names and roles of the US and UK government AI institutes.
The alignment problem, simply
We train AI systems toward goals we can measure. But what we can measure is never exactly what we want. A system optimized for a proxy can find unexpected shortcuts. In a classic example reported by OpenAI in 2016, an agent trained on the boat-racing game CoastRunners found a lagoon where it could circle endlessly, knocking over respawning targets, and outscored human players without ever finishing the race. It did exactly what it was rewarded for, not what its designers intended.
With language models, alignment concerns include being honest rather than just persuasive, refusing to help with serious harms, following the user’s real intent and behaving consistently in unfamiliar situations.
Key approaches
Learning from human feedback. Reinforcement learning from human feedback (RLHF) trains models using people’s ratings of responses. It is a major reason modern chatbots are helpful and polite. In OpenAI’s InstructGPT study, people preferred answers from a 1.3-billion-parameter model trained this way over those from the 175-billion-parameter GPT-3. Its limits: raters can be fooled by answers that sound good, and models can learn to please rather than to be correct.
Principles and constitutions. Some developers train models against written principles. Anthropic’s Constitutional AI, for example, uses a set of principles and AI feedback to guide model behavior, which makes the intended values more explicit.
Interpretability. This research tries to understand what is happening inside neural networks: which internal features represent which concepts, and how they combine into behavior. In 2024 Anthropic researchers extracted millions of interpretable features from a production model, including one for the Golden Gate Bridge that, when amplified, made the model talk as if it were the bridge. Progress here could let researchers detect problems directly rather than only through outputs.
Evaluations and red-teaming. Researchers test models for dangerous capabilities and failure modes before release, including attempts to trick them into harmful behavior. Government bodies take part too. The UK’s AI Security Institute, renamed from the AI Safety Institute in February 2025, tests some models before release under agreements with developers, and the US Center for AI Standards and Innovation, which replaced the US AI Safety Institute in June 2025, runs voluntary evaluations focused on risks such as cybersecurity and biosecurity.
Scalable oversight. How do humans supervise systems that may exceed them in some domains? Proposals include using AI to help humans evaluate AI, and breaking complex tasks into checkable steps.
Open problems
- Honesty: ensuring models report what they actually believe and admit uncertainty, a theme of our reliability column
- Robustness: resisting jailbreaks and prompt injection
- Generalization: behaving well in situations not seen in training
- Evaluating agents that act over long periods in the real world
- Deceptive behavior: research has explored whether models could behave well during testing while behaving differently elsewhere; a 2024 study found that deliberately planted deceptive behavior could survive standard safety training
Why it matters to users Alignment work is behind everyday features: assistants that refuse dangerous requests, say when they are unsure and follow instructions without taking harmful shortcuts. It becomes more important as AI moves from answering to acting.
Is this about science fiction?
Discussion of AI safety spans a wide range, from immediate issues such as bias, misuse and reliability to longer-term concerns about highly capable systems. Researchers disagree about how serious and near the long-term risks are. Much practical alignment work addresses both, because techniques that make systems more honest, controllable and understandable help across the board.
Why alignment matters
AI alignment is the engineering and science of making AI systems do what we intend. It is unfinished, it is increasingly well funded, and its progress, or lack of it, will shape how much we can safely trust the systems we build. For related policy issues, see our coverage of AI bias.
Sources
- Faulty reward functions in the wild, OpenAI, December 2016
- Training language models to follow instructions with human feedback, arXiv, 2022
- Constitutional AI: Harmlessness from AI Feedback, arXiv, December 2022
- Scaling Monosemanticity, Anthropic, May 2024
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, arXiv, January 2024
- Statement on transforming the US AI Safety Institute, US Department of Commerce, June 2025
