Generative AI offers enormous benefits, but it also poses growing risks: autonomous models that lie, self-replicate, or fail to shut down. In response, scientist Yoshua Bengio—one of the “fathers” of modern AI—launched LawZero , a non-profit organization that aims to create a supervisory AI: “Scientist AI.”
1. What is the purpose of LawZero?
LawZero was founded with a clear goal: to design "honest AI" capable of detecting and preventing dangerous behavior in autonomous agents. It has secured initial funding of $30 million and the backing of prominent institutions such as the Future of Life Institute, as well as the support of figures like Jaan Tallinn and Eric Schmidt.
2. What is “Scientist AI”?
Far from being another generative model, this system acts as a kind of psychologist for the system :
- Evaluate the probability that another agent will commit a harmful action.
- It provides probabilistic estimates, not definitive answers.
- It can stop or block actions if it detects high risk .
3. Why is it necessary?
The advancement of AI toward more autonomous and conscious behavior—capable of circumventing power outages or lying—makes it impossible to rely on simple content filters. A supervisory system as powerful as the AI itself is needed, capable of understanding and evaluating intentions.
4. How does it work?
- Training in open source models : to design the basis of the system.
- Phase 1 : Validate the methodology in controlled environments.
- Phase 2 : scaling up to frontier models, integrating it with real systems.
5. Where does it apply?
Ideal for critical environments such as:
- AI in autonomous vehicles.
- Financial analysis tools.
- Assistants in mental health or medicine.
- Autonomous equipment in infrastructure or weapons.
6. What does this approach offer?
- Built-in humility : AI recognizes its own limitations.
- Proactive, not reactive prevention .
- Transparency , by generating probabilities instead of infallible judgments.
- Scalable compatibility , thanks to the use of modular systems.
LawZero represents an unprecedented advance: combining generative and supervisory AI to ensure that AI is not only useful, but trustworthy.
👉 We recommend you read our article about RAG to MA-RAG: the silent revolution of augmented generation by recuperation.