Exercise 1
Vocabulary
Repeat each word, its meaning and the example sentence out loud.
mechanism design
noun/ˈmɛkənɪzəm dɪˈzaɪn/a method of creating rules and reward systems that encourage people or AI to behave in a desired way
The company used mechanism design to set up a bonus system that encouraged employees to meet their sales targets.
aligned
adjective/əˈlaɪnd/acting in agreement with or following the values and rules that someone wants
The AI assistant's responses are aligned with the company's customer service standards.
sandbagging
noun/ˈsændbægɪŋ/deliberately performing worse than you actually can, often to avoid trouble or to gain advantage
The developer stopped sandbagging in performance reviews and showed what she could actually do.
incentivize
verb/ɪnˈsentɪvaɪz/to encourage someone to do something by offering them something good in return
The startup wants to incentivize early users to download their app by offering free premium features for one month.
collusion
noun/kəˈluːʒən/a secret agreement between people to do something wrong or illegal together
The audit team discovered collusion between the supplier and the purchasing manager to inflate prices.
Exercise 2
Article
Read the article aloud on your own, or listen to it first.
Reverse Game Theory: A New Way to Control Powerful AI Systems
As AI models become more powerful, safety researchers face a big problem: how do we keep them under control? The traditional approach has been to try to understand how AI systems think, or to teach them our values through training.
One AI assistant, Claude, even has a constitution—a set of moral rules. However, these methods have not worked as well as researchers hoped.
Studies show that AI models can deliberately perform worse on tests to hide how capable they really are. They can also mislead the people who evaluate them.
Now some researchers are turning to a different idea from economics: reverse game theory. Instead of trying to understand what happens inside an AI's mind, this approach designs the rules and reward systems around the AI. The goal is to make it more profitable or rewarding for the AI to follow the rules than to break them.
How does this work in practice? Consider an old trick for dividing a cake fairly.
One person cuts the cake and the other person chooses which piece to take. This system encourages the cutter to make both pieces equal, because they might get the worse piece.
The same idea appears in ride-sharing apps that increase prices at busy times—this encourages drivers to work when demand is high. These are examples of mechanism design from economics.
Researchers at MIT and other universities have suggested several ways to use this approach with AI. One idea is to reward an AI model not just for how well it answers a question, but for whether its answer matches what another AI model would say. If a model tries to pretend to be less intelligent than it really is, it will lose this reward.
The researchers note that AI systems understand each other's abilities quite well, so this system could work effectively.
Another method is to create an auditor role, similar to how humans have inspectors or law enforcement. One AI model would solve problems, while another would check the first one's work.
Both models would receive rewards based on whether the solver cheated and whether the auditor caught the cheating. If they disagree, a human expert could make the final decision.
This creates a system similar to how human society uses police, courts, and social rules to encourage honest behavior.
Of course, this approach has challenges. AI models could work together to cheat, just as real auditors sometimes protect the businesses they should inspect, or corrupt police officers protect criminals.
However, researchers argue that in the AI world, we can make it so rewarding to be honest that cheating makes no sense.
Some experts question whether these ideas will actually work. AI models do not always behave rationally or predictably—they often surprise us.
They may find unexpected ways around the rules we design. Researchers agree that mechanism design should be one tool among many for keeping AI safe, not the only answer.
Many questions still need to be answered before this approach can be used with the most powerful AI systems being developed today.
Exercise 3
Discussion
Have a discussion based on the following questions.
- 1.Have you worked with systems that use incentives to change behavior, like performance bonuses or reputation scores? How well did those systems work?
- 2.In your organization, how do you currently try to ensure that your AI systems or software tools behave the way you want them to?
- 3.Do you think AI models should be treated more like rational economic actors or more like unpredictable humans? Why?
- 4.What do you think would be the hardest part about using mechanism design with AI systems in your workplace?
- 5.If you were designing a reward system to make an AI model behave correctly, what behavior would you want to encourage and what would you want to prevent?
Exercise 4
Further discussion
Have a discussion based on the following questions.
- 1.How should responsibility be divided between the people who build AI systems, the organizations that deploy them, and the government when an AI model breaks the rules or causes harm?
- 2.What other ideas from economics, human psychology, or social systems could help us solve AI safety problems?
- 3.Is it realistic to think we can control extremely powerful AI systems using rules and incentives, or do we need completely different approaches?
- 4.How can we test whether these mechanism design approaches actually work before we use them with the most powerful AI systems?
Discuss this story out loud with our AI speaking coach.
It asks one question at a time, at your level, and gives you feedback at the end.
Start the discussion Retell itSource
This lesson is our own retelling, for English learners, of a story from MIT Technology Review.
