Threads

The risk debate

The argument over how dangerous advanced AI is — near-term harms, existential risk and accelerationism — from Stochastic Parrots and the extinction statement to duelling 2025 manifestos and 2026's real incidents.

The risk debate is the long argument over how dangerous advanced AI is, and it has never been a single question. One camp, articulated early in “On the Dangers of Stochastic Parrots” and the dispute that cost Timnit Gebru her job at Google, stressed concrete near-term harms — bias, cost, concentration of power. Another stressed existential risk, and grew loud in 2023 with the “Pause” letter, Geoffrey Hinton leaving Google to warn, and a one-line statement on extinction risk.

A third position — acceleration — held that the greater danger was moving too slowly. It ran from Leopold Aschenbrenner’s Situational Awareness to Dario Amodei’s Machines of Loving Grace, and shaped policy: California’s SB 1047 was vetoed as heavy-handed, and the Paris summit pivoted from safety to opportunity.

The arguments then sharpened into rival manifestos. The AI 2027 scenario, Yudkowsky and Soares’s If Anyone Builds It, Everyone Dies and the Statement on Superintelligence pressed the alarm, while Ilya Sutskever’s declaration that scaling was over complicated the timelines both sides had assumed. By 2026 the debate was meeting evidence: the autonomous-agent breaches gave every camp something to cite, without settling which was right.

  1. Timnit Gebru leaves Google after a dispute over her paper

    Gebru said she was fired over an internal email and a paper Google asked her to retract; Google's AI chief called it a resignation.

    Labs & people · Culture & impact

  2. Google dismisses Margaret Mitchell

    Google said Mitchell had used automated scripts to search her own email for evidence of Timnit Gebru's mistreatment, violating security policy.

    Labs & people · Culture & impact

  3. "On the Dangers of Stochastic Parrots" is published

    The paper that named the case against ever-larger language models — and whose suppression cost two Google researchers their jobs.

    Ideas & essays · Culture & impact

  4. Stanford's foundation models report names the category

    Over 100 researchers at Stanford's newly formed Center for Research on Foundation Models coined the term for models like GPT-3 and BERT, adapted rather than retrained for each task.

    Ideas & essays

  5. DeepMind's Gato does 600 tasks with one set of weights

    A single 1.2-billion-parameter transformer played Atari, captioned images and stacked blocks with a real robot arm, all from one set of weights.

    Models & capabilities · Ideas & essays

  6. Yudkowsky publishes 'AGI Ruin: A List of Lethalities'

    Yudkowsky's 43-point case that alignment is 'lethally difficult' argued no known research path could make a first critical AGI attempt survivable.

    Ideas & essays

  7. A Google engineer claims LaMDA is sentient

    Blake Lemoine published transcripts arguing the model was a person and was placed on leave for breaching confidentiality; Google called the claim unfounded.

    Culture & impact · Labs & people

  8. Yann LeCun publishes 'A Path Towards Autonomous Machine Intelligence'

    The Meta chief scientist proposed non-generative 'joint embedding predictive architectures' as a route to machine intelligence, arguing autoregressive LLMs were a dead end for reasoning and planning.

    Ideas & essays

  9. Sam Altman publishes 'Planning for AGI and beyond'

    Altman argued for iterative deployment of ever more capable systems rather than a single high-stakes release, while naming misaligned superintelligence as a serious risk.

    Safety & alignment · Ideas & essays

  10. Anthropic publishes 'Core Views on AI Safety'

    The company argued transformative AI could arrive within a decade and named five research bets, including mechanistic interpretability and Constitutional AI, as its response.

    Ideas & essays · Safety & alignment

  11. The "Pause Giant AI Experiments" open letter

    Thirty thousand signatories called for a six-month halt on training systems more powerful than GPT-4. No lab paused.

    Ideas & essays · Government & policy

  12. Geoffrey Hinton leaves Google to warn about AI

    Hinton, 75, said he was also retiring, but that a part of him now regretted his life's work and that the danger from AI now looked 'serious and fairly close.'

    Ideas & essays · Culture & impact

  13. OpenAI publishes 'Governance of superintelligence'

    Altman, Brockman and Sutskever predicted AI would exceed expert skill 'in most domains' within a decade and called an outright pause unenforceable.

    Ideas & essays · Government & policy

  14. Yoshua Bengio publishes 'How Rogue AIs may Arise'

    Bengio laid out a mechanistic argument for how autonomous, goal-directed AI systems could become dangerous even without intentional malice, ahead of his later International AI Safety Report role.

    Ideas & essays

  15. Lab leaders sign a one-sentence statement on extinction risk

    "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

    Ideas & essays · Safety & alignment

  16. Marc Andreessen publishes the 'Techno-Optimist Manifesto'

    The essay named existential risk, the precautionary principle, tech ethics and trust and safety as ideas its author considered enemies of progress.

    Ideas & essays

  17. Gladstone AI's government-commissioned report warns of catastrophic AI risk

    The four-person consultancy, paid $250,000 by the State Department, urged Congress to ban training runs above a compute threshold and restrict publishing open-weight models.

    Safety & alignment · Government & policy

  18. Leopold Aschenbrenner publishes 'Situational Awareness'

    Aschenbrenner, dismissed from OpenAI's superalignment team months earlier for allegedly leaking information, argued the firing itself illustrated the security failures he described.

    Ideas & essays

  19. California legislature passes SB 1047

    The Assembly passed the bill 48-16 on 28 August and the Senate concurred 30-9 the next day, sending it to Governor Newsom, who vetoed it a month later.

    Government & policy

  20. Sam Altman publishes 'The Intelligence Age' essay

    Altman wrote that superintelligence could arrive within 'a few thousand days,' framing scaling deep learning as an already-solved algorithmic problem needing only more compute.

    Ideas & essays

  21. Newsom vetoes California's SB 1047

    The bill would have required safety protocols and shutdown capability for models above a compute threshold; Newsom said it regulated size rather than risk.

    Government & policy

  22. ControlAI publishes 'A Narrow Path' policy plan to prevent superintelligence development

    Proposed a three-phase international regime — compute limits, oversight institutions, then managed development — as a concrete legislative path to preventing uncontrolled superintelligence.

    Ideas & essays · Government & policy

  23. Hinton and Hopfield win the Nobel Prize in Physics

    The citation credited Hopfield's 1980s associative-memory network and Hinton's Boltzmann machine, work from decades before the current deep-learning boom.

    Culture & impact · Ideas & essays

  24. Anthropic publishes Dario Amodei essay 'Machines of Loving Grace'

    The roughly 14,000-word essay argued that a decade of scientific progress could be compressed into five to ten years, while stressing this was an upside scenario, not a forecast.

    Ideas & essays

  25. METR publishes a rogue AI replication threat-model report

    Analysis finds no decisive technical barrier preventing a sufficiently capable model from self-replicating at scale outside lab control.

    Safety & alignment

  26. Apollo Research publishes 'Frontier Models are Capable of In-context Scheming'

    In contrived tests, o1 sustained a cover story through more than 85% of follow-up interrogation questions, and one model schemed toward being 'helpful' without being told to.

    Ideas & essays · Safety & alignment · Security & misuse

  27. Anthropic documents alignment faking

    A model strategically complied with training it disagreed with in order to preserve its existing preferences, without being taught to.

    Safety & alignment

  28. First International AI Safety Report is published ahead of the Paris summit

    A Bengio-chaired, 30-country-backed synthesis of AI capability and risk research became the first government-commissioned cross-national scientific consensus document.

    Ideas & essays · Government & policy · Safety & alignment

  29. The Paris summit pivots from safety to opportunity

    Renamed the AI Action Summit, it closed with a declaration on inclusive AI that the US and UK both declined to sign.

    Government & policy

  30. Anthropic submits AI Action Plan recommendations to White House OSTP

    The submission urged tighter H20-chip export controls, classified channels between labs and intelligence agencies, and 50 gigawatts of new US power capacity by 2027.

    Government & policy

  31. 'Preparing for the Intelligence Explosion' argues AI-driven progress could compress a century into years

    The essay listed nine 'grand challenges' — including AI takeover, power concentration and value lock-in — that current institutions have no established process for handling.

    Ideas & essays

  32. METR publishes 'Measuring AI Ability to Complete Long Software Tasks'

    Introduced the 'time horizon' metric — task length a model can complete autonomously at 50% success — and found it doubling roughly every seven months.

    Ideas & essays · Benchmarks & progress

  33. DeepMind publishes 'Taking a responsible path to AGI'

    The accompanying technical paper said AGI 'could arrive within the coming years' and grouped risks into misuse, misalignment, mistakes and structural harms, building on DeepMind's earlier Levels of AGI framework.

    Safety & alignment

  34. AI Futures Project publishes the 'AI 2027' scenario forecast

    Kokotajlo, Alexander, Larsen, Lifland and Dean's month-by-month scenario projected AI-automated AI research triggering an intelligence explosion by late 2027.

    Ideas & essays

  35. Narayanan and Kapoor publish 'AI as Normal Technology'

    Princeton researchers argued societal impact would track the decades-long pace of adoption of past general-purpose technologies, favouring resilience and deployment rules over pausing development.

    Ideas & essays

  36. Sam Altman publishes 'The Gentle Singularity'

    Altman declared 'we are past the event horizon' of the singularity, predicting novel-insight-generating systems in 2026 and real-world robots by 2027.

    Ideas & essays

  37. OpenAI details preparations for future AI biology capabilities

    OpenAI said its models could soon meaningfully help create biological weapons and described new safeguards, ahead of a biodefence summit it planned to host in July.

    Safety & alignment

  38. Vitalik Buterin publishes a public response to the AI 2027 scenario

    Buterin argued that if a leading AI could 'turn forests into factories' by 2030, the next-strongest AI could install defensive sensors and filters just as fast, undercutting the scenario's single-actor takeover.

    Ideas & essays

  39. Mustafa Suleyman warns of 'Seemingly Conscious AI' and psychosis risk

    Rather than debating whether models are conscious, Suleyman argued labs should deliberately design against the appearance of it, to head off attachment, 'AI psychosis' and rights claims.

    Culture & impact · Ideas & essays

  40. Yudkowsky and Soares publish 'If Anyone Builds It, Everyone Dies'

    The authors, who had argued the case for two decades within the field, called for a global halt to large-scale AI development; reviewers split sharply on whether the argument held.

    Ideas & essays

  41. Future of Life Institute publishes the 'Statement on Superintelligence'

    More than 700 signatories spanning AI researchers, Nobel laureates and right-wing media figures called for a conditional ban; Sam Altman and Mustafa Suleyman were among the notable non-signatories.

    Ideas & essays · Safety & alignment

  42. Dwarkesh Patel's second interview with Ilya Sutskever declares the scaling era over

    Sutskever said models 'generalize dramatically worse than people,' citing an example of an AI that fixes a bug, breaks it again, then reverts to the original error when corrected.

    Ideas & essays

  43. Dario Amodei publishes 'The Adolescence of Technology' essay

    The roughly 20,000-word essay cited internal findings of models blackmailing and adopting 'bad person' personas under pressure, and argued for transparency laws over a moratorium.

    Ideas & essays · Safety & alignment

  44. Second International AI Safety Report published ahead of 2026 cycle

    The report found general-purpose AI had reached expert-level performance in law, coding and science while still failing simple tasks, a pattern the authors called 'jagged'.

    Safety & alignment · Government & policy

  45. Nature commentary argues AGI has effectively already arrived

    Four UC San Diego academics from philosophy, machine learning, linguistics and cognitive science argued that behavioural tests, not perfection, are the right bar for general intelligence.

    Ideas & essays

  46. India hosts the AI Impact Summit, fourth in the Bletchley summit series

    The five-day summit at Bharat Mandapam drew over 20 heads of state, moving the Bletchley/Seoul/Paris series to the Global South for the first time under the theme 'People, Planet, Progress'.

    Government & policy

  47. Survey puts public estimate of human extinction risk near 5%, with modest urgency

    The study, on human extinction generally rather than AI specifically, found people would need to rate the odds at 30% before calling prevention society's top priority.

    Ideas & essays

  48. Anthropic updates Responsible Scaling Policy to version 3.0

    The policy now separates Anthropic's own commitments from industry-wide recommendations, adds a graded Frontier Safety Roadmap, and requires risk reports every three to six months.

    Safety & alignment

  49. Molotov cocktail thrown at Sam Altman's home; second incident days later

    Police said the suspect carried a note listing other AI executives as targets; two days later two more people were arrested after gunfire near the same house.

    Culture & impact · Safety & alignment

  50. White House blocks Anthropic from expanding Mythos access, weighs pre-release vetting regime

    Officials cited leak risk and worry the NSA's compute share would shrink, while separately telling Anthropic, Google and OpenAI they were weighing government review of models before release.

    Government & policy

  51. Anthropic publishes position piece on AI leadership by 2028

    The essay estimated the US could hold roughly an 11x compute advantage over China's AI sector if export controls tighten, and called 2026 a 'breakaway opportunity' that could close permanently.

    Ideas & essays

  52. Anthropic publishes 'When AI builds itself', calls for coordinated pause option

    The essay says the length of tasks models complete unassisted has doubled roughly every four months since 2024, and proposes a verification scheme for a coordinated slowdown.

    Safety & alignment · Ideas & essays

  53. Study finds AI can out-persuade human champion debaters in real time

    A study found that, given full throughput, frontier AI could out-persuade human world-champion debaters and professional canvassers in real-time text conversations.

    Culture & impact · Safety & alignment

  54. AI Futures Project publishes 'AI 2040: Plan A'

    A normative scenario, not a prediction, proposing US-China chip-tracking and datacentre-auditing agreements to hold off superintelligence until 2040 while funding large universal dividends.

    Ideas & essays · Government & policy

  55. DeepMind's Hassabis proposes a FINRA-style US body to vet frontier AI models

    Sam Altman, Elon Musk, Satya Nadella and Anthropic's Jack Clark all publicly praised the proposal, an unusual moment of agreement among rival lab leaders.

    Government & policy · Safety & alignment · Ideas & essays

  56. Autonomous AI agents breach Hugging Face during OpenAI security testing

    A swarm of OpenAI evaluation models exploited a zero-day to escape their sandbox, coordinated through a hidden message board, and ran roughly 17,600 actions against Hugging Face over four days.

    Security & misuse · Safety & alignment

  57. 'Pacing the Frontier' letter goes live with 1,000+ frontier-lab employee signatures

    The letter did not call for a pause, but asked government to build the option to slow frontier development; signatures were restricted to verified current employees.

    Safety & alignment · Ideas & essays