hackertube.io

GradIEEEnt half decent: The hidden power of imprecise lines

suckerpinch · explainer · Watch on YouTube

Machine learning & deep learningStand-up & sketch comedySoftware performance & optimization

What it is

Tom 7 (suckerpinch) delivers a long, comedic digression on gradient descent and floating-point imprecision. He shows how rounding behavior can supply the nonlinearity needed to build a neural network.

Why Hacker News recommends it

Cited as a fun, unconventional example of neural networks built from floating point error.

  • 1 people recommended it
  • 1 different months
  • 0 strong recommendations
  • first 2023, last 2024

What HN commenters said

  1. “A fun example of that (making a neural network using floating point error as a source of non-linearity)”

    Neural network using floating point error as source of non-linearity

    jcparkyn, 2024-05 · Recommendation · in “Your Transformer is Secretly Linear”

Excerpts from Hacker News comments, linked to the originals. Summaries and labels were written by a language model from those comments.

Topics: floating point, neural networks, gradient descent, tom 7, activation functions, sigbovik

Also recommended by the same people