At Brains in Silicon, Stanford’s neuromorphic computing lab, I work in neuroscience theory, hoping to find inspiration for the hardware and algorithms people to work with. Recently, I’ve been messing with place fields, grid cells, and BTSP (behavioral time scale plasticity, a novel proposal for a learning mechanism that occurs on slower time-scales), mainly to see how animals localize themselves in unfamiliar environments. In my research, I’ve found that, while the brain can guess where the animal is located based on internal estimates of motion (like an inertial measurement unit), this estimate drifts significantly without external stimuli. This coincides with observations I’ve had while building Corvus, where I’ve learned that it is impossible to build a system that is able to navigate perfectly with no external input, even with an inertial navigation system. This brings me to an insight: all forms of intelligence, no matter how sophisticated, rely on input from the environment for feedback on actions, including metacognitive updates. Why? Because intelligence is all about how well an agent is able to do in its environment. No agent can internally verify the success of an action at 100% accuracy. Perfect simulation and pre-verification of action outcomes is physically intractable due to the inherent stochasticity and chaos present in real-world systems, including quantum uncertainty. To speed up the rate of updates, agents can build latent representations (internal models) of the world to simulate actions and feedback, but these representations are approximate, and from what I’ve learned with Corvus and neuroscience, all systems that are not 100% accurate have drift that grows exponentially over large time-scales. External input is required to recalibrate these internal representations. Thus, even a self-improving superintelligence will depend upon external input for feedback. So, what would set apart this superintelligence from other, normal intelligences? One possible axis would be the magnitude of environmental input required per intelligence update, where the intelligence update could be across various tasks such as math, maze solving, language, etc. I would say that, generally, the more intelligent an agent is, the closer to 0 it is on this axis across many different tasks. If there were a self-improving agent, it would attempt to get closer to 0 on this axis across many tasks (note: if my theory about drift is true, it is impossible to have a perfect singularity for intelligence, since 0 cannot be reached, although something very close to one might be possible). How would it get close to 0? Like I hinted at earlier, by improving its latent representations, it can get close to 0. Then, a well-designed, self-improving superintelligent agent would be very good at creating its own latent representations of the world. This reframes superintelligence not as the ability to recursively rewrite itself in isolation, but as the capacity to build highly accurate internal representations of the world that minimize, but never eliminate, the need for feedback. Given sufficient compute, these internal representations can be updated and queried at high speed, shifting the bottleneck from processing time to feedback efficiency. Intelligence is thus better understood not as raw computational power, but as the ability to form accurate models with minimal reliance on external correction: a property of a feedback-efficient loop between agent and world.