{"id":17700,"date":"2026-02-03T23:18:23","date_gmt":"2026-02-04T07:18:23","guid":{"rendered":"https:\/\/worldcampaign.net\/?p=17700"},"modified":"2026-02-11T00:29:22","modified_gmt":"2026-02-11T08:29:22","slug":"yoshua-bengio-turing-award-winner-there-is-empirical-evidence-of-ai-acting-against-our-instructions-el-pais","status":"publish","type":"post","link":"https:\/\/worldcampaign.net\/?p=17700","title":{"rendered":"&#8220;Yoshua Bengio, Turing Award winner: \u2018There is empirical evidence of AI acting against our instructions\u2019&#8221;, El Pais"},"content":{"rendered":"\n<p>The scientist, a pioneer in neural networks, warns that \u2018the capabilities of artificial intelligence continue to advance at a rate which seems faster than risk management practices\u2019<\/p>\n\n\n\n<p><a href=\"https:\/\/english.elpais.com\/author\/andrea-rizzi\/#?rel=author_top\">ANDREA RIZZI<\/a><\/p>\n\n\n\n<p>Madrid &#8211;&nbsp;<a href=\"https:\/\/english.elpais.com\/archive\/2026-02-03\/\"><abbr title=\"February\">FEB<\/abbr>&nbsp;03, 2026<\/a><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/imagenes.elpais.com\/resizer\/v2\/YXB2YBNSNNBVNOBWKGB2Q6U6TA.jpg?auth=7ca49545177d757ec2f7a71cbf4ae638f5748e1f8eed9050a09aceb8651327cb&amp;width=414\" alt=\"Yoshua Bengio\"\/><figcaption class=\"wp-element-caption\">Yoshua Bengio in Montreal in 2023.&nbsp;NASUNA STUART (NEW YORK TIMES\/CONTACTOPHOTO)<\/figcaption><\/figure>\n\n\n\n<p>Yoshua Bengio, 64, is one of the pioneers in the development of deep learning, the most promising artificial intelligence&nbsp;<a href=\"https:\/\/english.elpais.com\/technology\/2025-11-21\/why-ai-cannot-create-new-scientific-knowledge.html\">(AI) technique<\/a>&nbsp;today. His research in this field was recognized with the prestigious Turing Award \u2014 often described as the Nobel Prize of computer science \u2014 which he received in 2018 along with Yann LeCun and Geoffrey Hinton. A Canadian national and professor at the Universit\u00e9 de Montreal, Bengio is world-renowned for the extraordinary level of scientific rigor in his research and is now a leading advocate for raising awareness of the risks involved in the development of AI. He chairs the&nbsp;<a href=\"https:\/\/internationalaisafetyreport.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">International AI Safety Report<\/a>, an annual study that seeks to compile scientific evidence on emerging AI risks to support decisions for managing them, and whose second edition is published this Tuesday.<\/p>\n\n\n\n<p>In an interview with this newspaper \u2014 arranged on the occasion of the World Ecomic Forum in Davos, in which he participated, but conducted via videoconference on January 30 due to scheduling conflicts \u2014 Bengio warned that there is \u201cempirical evidence and laboratory incidents where AI is acting against our instructions\u201d and that \u201c<a href=\"https:\/\/english.elpais.com\/technology\/2026-01-31\/how-does-artificial-intelligence-think-the-big-surprise-is-that-it-intuits.html\">the capabilities of AI continue to advance<\/a>at a rate which seems faster than risk management practices,\u201d a concept that includes misuse of systems \u2014 fraud, mind manipulation \u2014; dysfunction \u2014 errors, loss of control \u2014; or systemic consequences, such as the impact on the labor market or on cognitive processes.<\/p>\n\n\n\n<p><strong>Question.<\/strong>&nbsp;What are the most worrying signs you identify when analyzing the evolution of current models and agents?<\/p>\n\n\n\n<p><strong>Answer.<\/strong>&nbsp;There are two elements in the report that I think reflect some of the most serious concerns many scientists have. Two things are happening at the same time. One is the continued advances in AI capabilities, more specifically in reasoning, that includes being able to strategize in order to achieve goals, which you can think&nbsp;<a href=\"https:\/\/english.elpais.com\/technology\/2024-12-12\/ai-is-a-real-pipe-dream-it-can-make-the-world-radically-better.html\">is a good thing<\/a>. That is, AIs are getting smarter. But at the same time, we\u2019re seeing empirical evidence and laboratory incidents where AIs are acting against our instructions and sometimes with an apparent drive to preserve themselves and being willing to be deceptive in order to avoid our oversight, in order to avoid being replaced by a new version and things like that. So, these two things together are problematic.&nbsp;<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;This points to the risk of&nbsp;<a href=\"https:\/\/english.elpais.com\/technology\/2025-12-17\/mustafa-suleyman-controlling-ai-is-the-challenge-of-our-time.html\">a loss-of-control scenario<\/a>. How would you characterize it? How serious is that risk?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;The most important thing to understand is that it\u2019s an emerging risk. It hasn\u2019t happened. And it is one for which the probability that it will happen for real is very hard to estimate, and scientists in AI don\u2019t agree about its probability. But what we can all agree on is that if it does happen,&nbsp;<a href=\"https:\/\/english.elpais.com\/technology\/2025-12-15\/shoshana-zuboff-philosopher-ai-is-surveillance-capitalism-continuing-to-evolve-and-expand.html\">it could be catastrophic<\/a>. It could be terrible. That\u2019s why it deserves attention. What the report really focuses on is that we have those early signs, which still require improved human methodology and systematic conclusions. But these early signs, because of the severity of those risks, do mandate increased monitoring and research to understand why it\u2019s happening and how it can be mitigated.<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;Perhaps one of the most worrying risks is the misuse of these systems to try to manipulate people\u2019s minds and, therefore, manipulate the proper development of democratic life. What is your opinion, and where do we stand on this issue?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;What the report does is it documents the studies that are showing various aspects of this risk, which is already happening to some degree, but there\u2019s also possibility that it could get worse. Let me elaborate on this. First, it\u2019s very obvious that quality of the content that is generated by AI is getting more and more difficult to distinguish from real content, because the AI is getting better, whether it is images, texts, voice or video. Another area of concern is personalized persuasion and manipulation. There\u2019s been a number of studies that show, in laboratory settings, that the frontier models \u2014 the most powerful ones \u2014 are at least as good as humans at persuasion. Persuasion means somebody will actually change their mind on a topic after several rounds of dialogue. There\u2019s a real risk of a very large-scale&nbsp;<a href=\"https:\/\/english.elpais.com\/technology\/2025-12-04\/programs-like-chatgpt-can-change-the-opinion-of-one-in-four-voters.html#?rel=mas\">influence on public opinion<\/a>, because such bots could be multiplied, and you can have millions of these things. And this calls for some kind of intervention in order to mitigate those risks.<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;These machines can create a pattern where people give up the effort of solving problems, which is one of the foundations of intelligence and its development. How concerned are you about the effect on human cognitive abilities?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;That is a difficult question. I don\u2019t think that we have very clear answers from science at this point. I suspect that it\u2019s going to depend on context. It might be bad for some people and for some types of users or some type of relationship between the person and the AI, and it might actually be good for some others. So, the good example would be to think of the AI as an assistant. I use AI as a research assistant, and the assistant is not perfect. They can make mistakes, but they have much more time than you do, so they can read the literature and give you reports about relevant information that can be useful for you. Of course, this could become something unhealthy if the decisions are really taken by the AI, and you\u2019re almost a passive actor in the decision-making process. So, that\u2019s where it becomes dangerous for a number of reasons. What is kind of decline if we rely completely too much on the AI, and especially for children. The other issue is connected to the loss of control problem.<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;How?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;if we rely a lot on AIs that are not really trustworthy because they have their own intentions, that\u2019s also dangerous. Another misaligned behavior that is creating problems is sycophancy: the AI is not telling you the truth, but what you want to hear. And that is already causing mental health problems, because people can have delusions, and the AI will go along with those delusions. It\u2019s a little bit like in social media: you\u2019re going to get your crazy ideas reinforced, the bubble effect. It\u2019s something similar with AI, where the AI will go along with your crazy ideas or even amplify them in the feedback loop. That has given rise, at least anecdotally and via lawsuits, to episodes of psychosis and even tragic incidents where people harm themselves, even kill themselves.<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;I would now like to ask you about your assessment of the work being done by large companies that develop these models and agents in relation to security measures. How do you assess that work?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;If we look at the change from a year ago, there has been a lot of progress. Many more of the leading companies are implementing some form of explicit risk management. Several are working with the&nbsp;<a href=\"https:\/\/www.frontiermodelforum.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">Frontier Model Forum<\/a>&nbsp;to synchronize their risk management practices. I believe there are currently about 12 companies that have published their risk management processes. In addition, there\u2019s been advances on the legislation side in Europe, as you know, but also in the United States and in China. We\u2019re seeing a gradual convergence of these different safety requirements and public protection risk management processes, which is a good thing. The downside is that the capabilities of AI continues to advance at a rate which seems to be faster than the risk management practice.<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;You mentioned regulation by public actors, such as states or the European Union. In Europe, we often hear the argument that there is over-regulation, and that this stifles innovation, while the United States and China are moving faster. How would you describe this situation and its consequences?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;As chair of the report, I must point out that the mandate is very explicit that we\u2019re not making policy recommendations. We focus on providing scientific evidence that can inform policy debate. But, if I change hats and speak as an independent university professor and as someone who was involved in the European effort, I can say that the requirements of the EU AI Act\u2019s Code of Practice only apply to the largest models. In Europe, very few companies exceed that threshold. These requirements do not place an undue burden on companies of that size, and most large U.S. companies already accept them because, to a large extent, that is what they were already doing. They do, however, introduce a degree of transparency towards governments and thus the public, which many leading companies consider a good thing. In my view, the real reasons why Europe has been lagging has nothing to do with those regulations and has to do with lack of large-scale investment, being willing to take risks, and other factors that have to do with the structure of the European economy. Europe needs to wake up to these realities and invest massively in order to be, as Canadian&nbsp;<a href=\"https:\/\/english.elpais.com\/international\/2026-01-30\/mark-carney-trumpisms-unlikely-foe.html\">Prime Minister Mark Carney said in his Davos speech<\/a>, at the table rather than on the menu.<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;Perhaps it would be ideal to have something similar to what already exists in the nuclear sector, with the Non-Proliferation Treaty and the International Atomic Energy Agency. But that doesn\u2019t seem viable right now for AI. So, what can be done?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;Yes. The problem is that the main perception of many countries and companies is that they are in a competition with other companies and other countries. And that isn\u2019t a good starting point for any kind of international coordination treaty and so on. What the report hopefully helps to do is to establish the facts, the scientific evidence for countries to realize that they also have a lot to lose in common. Catastrophic misuse or loss of control would be bad whether you\u2019re Chinese or American. None of us should accept those risks. It\u2019s just that unfortunately the scientific understanding and the geopolitical understanding of the magnitude of those risks seems to be still weak. What I\u2019m hoping, as we move forward with scientific evidence about these and other risks, is that governments will see it is in their self-interest to seek international coordination to mitigate the risks, similar to what happened, as you said, with nuclear weapons.<\/p>\n\n\n\n<p><strong>Q.<\/strong>&nbsp;At the beginning of the&nbsp;<em>Divine Comedy<\/em>, Dante Alighieri encounters a she-wolf representing lust for power and a lion representing pride. In the work, he argues that the she-wolf was the worst beast of all. In the context of AI, is it the wolf we should fear more, or the lion? Greed or arrogance?<\/p>\n\n\n\n<p><strong>A.<\/strong>&nbsp;I would add that there is scientific evidence for the things you\u2019re talking about, not just poetry; scientists have also studied psychology and social psychology. And we tend to easily blind ourselves to reality. This is called motivated cognition. In other words, we will not even have the thoughts that would allow us to see reality as it is. Instead, consciously or unconsciously, we can be driven by greed and hubris. And it\u2019s not necessarily because the person has bad intentions. In fact, they may sincerely think they are doing the right thing. But it\u2019s not necessarily rational, nor is it even grounded in the values \u200b\u200bthe person professes. That includes all of us, myself included. It\u2019s human nature.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The scientist, a pioneer in neural networks, warns that \u2018the capabilities of artificial intelligence continue to advance at a rate which seems faster than risk management practices\u2019 ANDREA RIZZI Madrid &#8211;&nbsp;FEB&nbsp;03, 2026 Yoshua Bengio, 64, is one of the pioneers in the development of deep learning, the most promising artificial intelligence&nbsp;(AI) technique&nbsp;today. His research in [&hellip;]<\/p>\n","protected":false},"author":1001004,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[53],"tags":[],"_links":{"self":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts\/17700"}],"collection":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/users\/1001004"}],"replies":[{"embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=17700"}],"version-history":[{"count":3,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts\/17700\/revisions"}],"predecessor-version":[{"id":17736,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts\/17700\/revisions\/17736"}],"wp:attachment":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=17700"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=17700"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=17700"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}