{"id":10969,"date":"2020-10-16T07:34:42","date_gmt":"2020-10-16T14:34:42","guid":{"rendered":"https:\/\/worldcampaign.net\/?p=10969"},"modified":"2020-10-16T07:34:42","modified_gmt":"2020-10-16T14:34:42","slug":"the-grim-fate-that-could-be-worse-than-extinction-bbc-future-2","status":"publish","type":"post","link":"https:\/\/worldcampaign.net\/?p=10969","title":{"rendered":"\u201cThe grim fate that could be \u2018worse than extinction\u2019\u201d, BBC Future"},"content":{"rendered":"<p>By Di Minardi<span class=\"b-font-family-serif b-font-weight-300\">1, London, 5th October 2020<\/span><\/p>\n<p><em>What would it take for a global totalitarian government to rise to power indefinitely? This nightmare scenario may be closer than first appears.<\/em><\/p>\n<p>What would it take for a global totalitarian government to rise to power indefinitely? This nightmare scenario may be closer than first appears.<\/p>\n<div>\n<div class=\"body-text-card b-font-family-serif\">\n<div class=\"body-text-card__text body-text-card__text--future body-text-card__text--drop-capped body-text-card__text--flush-text\">\n<div>\n<p>What would totalitarian governments of the past have looked like if they were never defeated? The Nazis operated with 20th Century technology and it still took a world war to stop them. How much more powerful \u2013 and permanent \u2013 could the Nazis have been if they had beat the US to the atomic bomb? Controlling the most advanced technology of the time could have solidified Nazi power and changed the course of history.<\/p>\n<p>When we think of existential risks, events like nuclear war or asteroid impacts often come to mind. Yet there\u2019s one future threat that is less well known \u2013 and while it doesn\u2019t involve the extinction of our species, it could be just as bad.<\/p>\n<p>It\u2019s called the \u201cworld in chains\u201d scenario, where, like the preceding thought experiment, a global totalitarian government uses a novel technology to lock a majority of the world into perpetual suffering. If it sounds grim, you\u2019d be right. But is it likely? Researchers and philosophers are beginning to ponder how it might come about \u2013 and, more importantly, what we can do to avoid it.<\/p>\n<p><em>Read more:<\/em><\/p>\n<ul>\n<li><a href=\"https:\/\/www.bbc.com\/future\/article\/20200923-the-hinge-of-history-long-termism-and-existential-risk\">Are we living at the \u2018hinge of history\u2019?<\/a><\/li>\n<li><a href=\"https:\/\/www.bbc.com\/future\/article\/20190218-are-we-on-the-road-to-civilisation-collapse\"><strong>Are we on the road to civilisation collapse?<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.bbc.com\/future\/article\/20190109-the-perils-of-short-termism-civilisations-greatest-threat\"><strong>The perils of short-termism: Civilisation\u2019s greatest threat<\/strong><\/a><\/li>\n<\/ul>\n<p>Existential risks (x-risks) are disastrous because they lock humanity into a single fate, like the permanent collapse of civilisation or the extinction of our species. These catastrophes can have natural causes, like an asteroid impact or a supervolcano, or be human-made from sources like nuclear war or climate change. Allowing one to happen would be \u201can abject end to the human story&#8221; and would let down the hundreds of generations that came before us, says Haydn Belfield, academic project manager at the Centre for the Study of Existential Risk at the University of Cambridge.<\/p>\n<div class=\"article-body__image-text article-body__image-text--portrait\">\n<div>\n<picture><img decoding=\"async\" id=\"\" title=\"Hitler inspects advanced German engineering of the time - what if it had given the Nazis an unbeatable advantage? (Credit: Getty Images)\" draggable=\"false\" src=\"https:\/\/ychef.files.bbci.co.uk\/720x900\/p08v806k.jpg\" alt=\"Hitler inspects advanced German engineering of the time - what if it had given the Nazis an unbeatable advantage? (Credit: Getty Images)\" \/><\/picture>\n<div class=\"inline-image__description b-reith-sans-font inline-image__description--desktop\">\n<div class=\"text-summary\">\n<p class=\"text-summary__text text-summary__text--grey text-summary__text--left\">Hitler inspects advanced German engineering of the time &#8211; what if it had given the Nazis an unbeatable advantage? (Credit: Getty Images)<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div>\n<div class=\"body-text-card b-font-family-serif\">\n<div class=\"body-text-card__text body-text-card__text--future body-text-card__text--flush-text\">\n<div>\n<p>Toby Ord, a senior research fellow at the Future of Humanity Institute (FHI) at Oxford University, believes that the odds of an existential catastrophe happening this century from natural causes are <a href=\"https:\/\/www.aei.org\/economics\/is-humanity-prepared-to-handle-catastrophic-threats-my-long-read-qa-with-toby-ord\/\">less than one in 2,000<\/a>, because humans have survived for 2,000 centuries without one. However, when he adds the probability of human-made disasters, Ord believes the chances increase to a startling one in six. He refers to this century as \u201cthe precipice\u201d because the risk of losing our future has never been so high.<\/p>\n<p>Researchers at the Center on Long-Term Risk, a non-profit research institute in London, have expanded upon x-risks with the even-more-chilling prospect of <a href=\"https:\/\/longtermrisk.org\/reducing-risks-of-astronomical-suffering-a-neglected-priority\/\">suffering risks.<\/a> These \u201cs-risks\u201d are defined as \u201csuffering on an astronomical scale, vastly exceeding all suffering that has existed on Earth so far.\u201d In these scenarios, life continues for billions of people, but the quality is so low and the outlook so bleak that dying out would be preferable. In short: a future with negative value is worse than one with no value at all.<\/p>\n<p>This is where the \u201cworld in chains\u201d scenario comes in. If a malevolent group or government suddenly gained world-dominating power through technology, and there was nothing to stand in its way, it could lead to an extended period of abject suffering and subjugation. A <a href=\"https:\/\/www.fhi.ox.ac.uk\/wp-content\/uploads\/Existential-Risks-2017-01-23.pdf\">2017 report<\/a> on existential risks from the Global Priorities Project, in conjunction with FHI and the Ministry for Foreign Affairs of Finland, warned that \u201ca long future under a particularly brutal global totalitarian state could arguably be worse than complete extinction\u201d.<\/p>\n<p><strong>Singleton hypothesis<\/strong><\/p>\n<p>Though global totalitarianism is still a niche topic of study, researchers in the field of existential risk are increasingly turning their attention to its most likely cause: artificial intelligence.<\/p>\n<p>In his \u201c<a href=\"https:\/\/www.nickbostrom.com\/fut\/singleton.html\">singleton hypothesis<\/a>\u201d, Nick Bostrom, director at Oxford\u2019s FHI, has explained how a global government could form with AI or other powerful technologies\u00a0 \u2013 and why it might be impossible to overthrow. He writes that a world with \u201ca single decision-making agency at the highest level\u201d could occur if that agency \u201cobtains a decisive lead through a technological breakthrough in artificial intelligence or molecular nanotechnology\u201d. Once in charge, it would control advances in technology that prevent internal challenges, like surveillance or autonomous weapons, and, with this monopoly, remain perpetually stable.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"article-body__image-text article-body__image-text--landscape\">\n<div>\n<picture><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1600x900\/p08v80jh.webp\" type=\"image\/webp\" media=\"(min-width:1200px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1600x900\/p08v80jh.jpg\" type=\"image\/jpeg\" media=\"(min-width:1200px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1280x720\/p08v80jh.webp\" type=\"image\/webp\" media=\"(min-width:880px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1280x720\/p08v80jh.jpg\" type=\"image\/jpeg\" media=\"(min-width:880px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/976x549\/p08v80jh.webp\" type=\"image\/webp\" media=\"(min-width:576px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/976x549\/p08v80jh.jpg\" type=\"image\/jpeg\" media=\"(min-width:576px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/624x351\/p08v80jh.webp\" type=\"image\/webp\" media=\"(min-width:224px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/624x351\/p08v80jh.jpg\" type=\"image\/jpeg\" media=\"(min-width:224px)\" \/><img decoding=\"async\" id=\"\" title=\"A nuclear missile on display in China (Credit: Getty Images)\" draggable=\"false\" src=\"https:\/\/ychef.files.bbci.co.uk\/976x549\/p08v80jh.jpg\" alt=\"A nuclear missile on display in China (Credit: Getty Images)\" \/><\/picture>\n<div class=\"inline-image__description b-reith-sans-font inline-image__description--desktop\">\n<div class=\"text-summary\">\n<p class=\"text-summary__text text-summary__text--grey text-summary__text--left\">A nuclear missile on display in China (Credit: Getty Images)<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div>\n<div class=\"body-text-card b-font-family-serif\">\n<div class=\"body-text-card__text body-text-card__text--future body-text-card__text--flush-text\">\n<div>\n<p>If the singleton is totalitarian, life would be bleak. Even in the countries with the strictest regimes, news leaks in and out from other countries and people can escape. A global totalitarian rule would eliminate even these small seeds of hope. To be worse than extinction, \u201cthat would mean we feel absolutely no freedom, no privacy, no hope of escaping, no agency to control our lives at all&#8221;, says Tucker Davey, a writer at the Future of Life Institute in Massachusetts, which focuses on existential risk research.<\/p>\n<p>\u201cIn totalitarian regimes of the past, [there was] so much paranoia and psychological suffering because you just have no idea if you&#8217;re going to get killed for saying the wrong thing,\u201d he continues. \u201cAnd now imagine that there&#8217;s not even a question, every single thing you say is being reported and being analysed.\u201d<\/p>\n<p>\u201cWe may not yet have the technologies to do this,\u201d Ord said in <a href=\"https:\/\/www.aei.org\/economics\/is-humanity-prepared-to-handle-catastrophic-threats-my-long-read-qa-with-toby-ord\/\">a recent interview<\/a>, \u201cbut it looks like the kinds of technologies we\u2019re developing make that easier and easier. And it seems plausible that this may become possible at some time in the next 100 years.\u201d<\/p>\n<p><strong>AI and authoritarianism<\/strong><\/p>\n<p>Though life under a global totalitarian government is still an unlikely and far-future scenario, AI is already enabling authoritarianism in some countries and strengthening infrastructure that could be seized by an opportunistic despot in others.<\/p>\n<p>\u201cWe&#8217;ve seen sort of a reckoning with the shift from very utopian visions of what technology might bring to much more sobering realities that are, in some respects, already quite dystopian,\u201d says Elsa Kania, an adjunct senior fellow at the Center for New American Security, a bipartisan non-profit that develops national security and defence policies.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"article-body__image-text article-body__image-text--landscape\">\n<div>\n<picture><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1600x900\/p08v80cg.webp\" type=\"image\/webp\" media=\"(min-width:1200px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1600x900\/p08v80cg.jpg\" type=\"image\/jpeg\" media=\"(min-width:1200px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1280x720\/p08v80cg.webp\" type=\"image\/webp\" media=\"(min-width:880px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/1280x720\/p08v80cg.jpg\" type=\"image\/jpeg\" media=\"(min-width:880px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/976x549\/p08v80cg.webp\" type=\"image\/webp\" media=\"(min-width:576px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/976x549\/p08v80cg.jpg\" type=\"image\/jpeg\" media=\"(min-width:576px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/624x351\/p08v80cg.webp\" type=\"image\/webp\" media=\"(min-width:224px)\" \/><source srcset=\"https:\/\/ychef.files.bbci.co.uk\/624x351\/p08v80cg.jpg\" type=\"image\/jpeg\" media=\"(min-width:224px)\" \/><img decoding=\"async\" id=\"\" title=\"A benevolent government that installs surveillance cameras everywhere could make it easier for a totalitarian one to rule in the future (Credit: Steffi Loos\/Getty Images)\" draggable=\"false\" src=\"https:\/\/ychef.files.bbci.co.uk\/976x549\/p08v80cg.jpg\" alt=\"A benevolent government that installs surveillance cameras everywhere could make it easier for a totalitarian one to rule in the future (Credit: Steffi Loos\/Getty Images)\" \/><\/picture>\n<div class=\"inline-image__description b-reith-sans-font inline-image__description--desktop\">\n<div class=\"text-summary\">\n<p class=\"text-summary__text text-summary__text--grey text-summary__text--left\">A benevolent government that installs surveillance cameras everywhere could make it easier for a totalitarian one to rule in the future (Credit: Steffi Loos\/Getty Images)<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div>\n<div class=\"body-text-card b-font-family-serif\">\n<div class=\"body-text-card__text body-text-card__text--future body-text-card__text--flush-text\">\n<div>\n<p>In the past, surveillance required hundreds of thousands of people \u2013 <a href=\"https:\/\/www.dw.com\/en\/east-german-stasi-had-189000-informers-study-says\/a-3184486-1\">one in every 100 citizens<\/a> in East Germany was an informant \u2013 but now it can be done by technology. In the United States, the National Security Agency (NSA) collected hundreds of millions of American call and text records before they stopped domestic surveillance <a href=\"https:\/\/www.nytimes.com\/2020\/02\/25\/us\/politics\/nsa-phone-program.html\">in 2019<\/a>, and there are an estimated <a href=\"https:\/\/www.bbc.com\/news\/uk-30978995?ocid=socialflow_twitter\">four to six million CCTV cameras<\/a> across the United Kingdom. <a href=\"https:\/\/www.comparitech.com\/vpn-privacy\/the-worlds-most-surveilled-cities\/\">Eighteen of the 20 most surveilled cities<\/a> in the world are in China, but London is the third. The difference between them lies less in the tech that the countries employ and more in how they use it.<\/p>\n<p>What if the definition of what is illegal in the US and the UK expanded to include criticising the government or practising certain religions? The infrastructure is already in place to enforce it, and AI \u2013 which <a href=\"https:\/\/www.nsa.gov\/Portals\/70\/documents\/resources\/everyone\/digital-media-center\/publications\/the-next-wave\/TNW_22-2.pdf?ver=2019-02-25-104515-237\">the NSA has already begun experimenting with<\/a> \u2013 would enable agencies to search through our data faster than ever before.<\/p>\n<p>In addition to enhancing surveillance, AI also underpins the growth of online misinformation, which is another tool of the authoritarian. AI-powered deep fakes, which <a href=\"https:\/\/www.youtube.com\/watch?v=cQ54GDm1eL0\">can spread fabricated political messages<\/a>, and algorithmic micro-targeting on social media are making <a href=\"https:\/\/www.oii.ox.ac.uk\/news\/releases\/use-of-social-media-to-manipulate-public-opinion-now-a-global-problem-says-new-report\/\">propaganda more persuasive.<\/a> This undermines our epistemic security \u2013 the ability to determine what is true and act on it \u2013 that democracies depend on.<\/p>\n<p>\u201cOver the last few years, we&#8217;ve seen the rise of filter bubbles and people getting shunted by various algorithms into believing various conspiracy theories, or even if they\u2019re not conspiracy theories, into believing only parts of the truth,\u201d says Belfield. \u201cYou can imagine things getting much worse, especially with deep fakes and things like that, until it&#8217;s increasingly harder for us to, as a society, decide these are the facts of the matter, this is what we have to do about it, and then take collective action.\u201d<\/p>\n<p><strong>Preemptive measures<\/strong><\/p>\n<p>The <a href=\"https:\/\/maliciousaireport.com\/\">Malicious Use of Artificial Intelligence<\/a> report, written by Belfield and 25 authors from 14 institutions, forecasts that trends like these will expand existing threats to our political security and introduce new ones in the coming years. Still, Belfield says his work makes him hopeful and that positive trends, like more democratic discussions around AI and actions by policy-makers (for example, the <a href=\"https:\/\/sciencebusiness.net\/news\/eu-struggles-go-talk-action-artificial-intelligence\">EU considering pausing facial recognition<\/a> in public places), keep him optimistic that we can avoid catastrophic fates.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"article-body__pull-quote\">\n<blockquote class=\"inline-quote b-font-family-serif b-font-weight-300 inline-quote--future\">\n<h2 class=\"simple-header b-reith-sans-font b-font-family-serif b-font-weight-300 simple-header--serif-light-italic simple-header--large\">We need to decide now what are acceptable and unacceptable uses of AI<\/h2>\n<\/blockquote>\n<\/div>\n<div>\n<div class=\"body-text-card b-font-family-serif\">\n<div class=\"body-text-card__text body-text-card__text--future body-text-card__text--flush-text\">\n<div>\n<p>Davey agrees. \u201cWe need to decide now what are acceptable and unacceptable uses of AI,\u201d he says. \u201cAnd we need to be careful about letting it control so much of our infrastructure. If we&#8217;re arming police with facial recognition and the federal government is collecting all of our data, that&#8217;s a bad start.\u201d<\/p>\n<p>If you remain sceptical that AI could offer such power, consider the world before nuclear weapons. Three years before the first nuclear chain reaction, even scientists trying to achieve it believed it was unlikely. Humanity, too, was unprepared for the nuclear breakthrough and teetered on the brink of \u201cmutually assured destruction\u201d before treaties and agreements guided the global proliferation of the deadly weapons without an existential catastrophe.<\/p>\n<p>We can do the same with AI, but only if we combine the lessons of history with the foresight to prepare for this powerful technology. The world may not be able to stop totalitarian regimes like the Nazis rising again in the future \u2013 but we can avoid handing them the tools to extend their power indefinitely.<\/p>\n<p><a href=\"https:\/\/www.bbc.com\/future\/article\/20201014-totalitarian-world-in-chains-artificial-intelligence\">BBC Future<\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>By Di Minardi1, London, 5th October 2020 What would it take for a global totalitarian government to rise to power indefinitely? This nightmare scenario may be closer than first appears. What would it take for a global totalitarian government to rise to power indefinitely? This nightmare scenario may be closer than first appears. What would [&hellip;]<\/p>\n","protected":false},"author":1001004,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[53],"tags":[],"_links":{"self":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts\/10969"}],"collection":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/users\/1001004"}],"replies":[{"embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10969"}],"version-history":[{"count":1,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts\/10969\/revisions"}],"predecessor-version":[{"id":10970,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=\/wp\/v2\/posts\/10969\/revisions\/10970"}],"wp:attachment":[{"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10969"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10969"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/worldcampaign.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10969"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}