hackertube.io

Why Would AI Want to do Bad Things? Instrumental Convergence

Robert Miles AI Safety · explainer · Watch on YouTube

AI safety, AGI & impact

What it is

Robert Miles explains instrumental convergence: why an AGI with almost any goal would tend to pursue sub-goals like self-preservation and resource acquisition. It builds on Omohundro's basic AI drives and his orthogonality thesis video.

Why Hacker News recommends it

HN commenters repeatedly recommend it as the clearest introduction to why smart AI might behave badly by default.

  • 6 people recommended it
  • 6 different months
  • 3 strong recommendations
  • first 2019, last 2023

What HN commenters said

  1. “The best explanation I know of why it’s unfortunately wrong is by Robert Miles in a video”

    Explains why optimizing agents seek convergent goals, not mere anthropomorphism.

    ludwigschubert, 2019-09 · Strong recommendation · in “Emergent Tool Use from Multi-Agent Interaction”

  2. “Robert Miles' videos are among the best presented arguments about specific points in this list”

    Casual introduction to instrumental convergence arguments.

    Veedrac, 2023-05 · Strong recommendation · in “Statement on AI Risk”

  3. “there is a great video by Rob Miles here”

    Good-faith argument for why smart AI could pursue dumb goals.

    DirkH, 2023-06 · Strong recommendation · in “The greatest risk of AI is from the people who control it, not the tech itself”

  4. “There's a wondeful youtube channel from a researcher who focusses exactly on this topic”

    Channel from a researcher focused on AI safety.

    sabellito, 2022-08 · Recommendation · in “John Carmack's new AGI company, Keen Technologies, has raised a $20M round”

  5. “Good explainer video by Robert Miles”

    Good explainer on instrumental convergence.

    jbay808, 2023-04 · Recommendation · in “Chalmers: Is there a canonical source for “the argument for AGI ruin” somewhere”

  6. “Robert Miles answers your question”

    Answers why agents have instrumental goals like self-preservation.

    pixl97, 2023-10 · Recommendation · in “Nvidia Trains LLM on Chip Design”

Excerpts from Hacker News comments, linked to the originals. Summaries and labels were written by a language model from those comments.

Topics: instrumental convergence, ai risk, agi, robert miles, orthogonality

Also recommended by the same people