OpenAI publishes multimodal neurons research

OpenAI’s interpretability team, publishing jointly with the journal Distill, reported that individual neurons inside CLIP — the image-text model OpenAI had released two months earlier — responded to the same concept regardless of whether it was presented as a photograph, a sketch, or the word itself. Their best-known example, a “Spider-Man” neuron, fired for photographs of real spiders, comic-book drawings of the character, and the rendered text “spider.”

The work extended earlier circuits-style analysis of vision networks, previously released through OpenAI Microscope, to a model trained by matching images to their captions rather than to fixed class labels. The researchers drew an explicit parallel to neuroscience findings of multimodal neurons in the human brain that respond to a concept across sight, sound and language.

The paper’s central risk finding was what the authors called “typographic attacks”: because a neuron for a concept could be triggered by the word for that concept, writing a label — for instance “iPod” — on a sticker and pasting it over an apple was enough to make CLIP classify the fruit as an iPod. The demonstration mattered less for the trick itself than for what it implied about the robustness of vision-language models trained on image-caption pairs scraped from the web: text glued onto an object could override what the object actually was, a weakness that later resurfaced as a concern for models built on CLIP-style encoders, including OpenAI’s own DALL·E and a generation of vision-language systems that followed it.