At Rise of AI 2016 in Berlin, Anish Mohammed questions whether human thought is really 'authentic' compared to machine cognition, and traces a possible path toward brain-computer interfaces, whole-brain emulation and eventual superintelligence. He argues the human brain, using 8 to 20% of the body's energy, is already extended by devices like the iPhone, a primitive brain-computer interface limited by the bandwidth of eyes and fingers, and describes retinal implants as an early, already-existing way of feeding electronic signal directly into the optic nerve. He cites a science study showing human memories are unreliable yet highly valued as authentic, and argues privacy of thought has never been debated in EU policy the way data privacy has. Moving to governance, he uses the DAO hack as a cautionary example for any future system of emulated minds ('EMs'), and explains trusted platform modules and zero-knowledge proofs, specifically direct anonymous attestation, as tools that could let entities prove properties about themselves without revealing information. He closes on ethics for machines, the trolley problem for self-driving cars, and an extended audience Q&A.
Key takeaways
- Mohammed argues the iPhone already works as a primitive, low-bandwidth brain-computer interface that has not changed since 2007.
- He cites a study finding human memories are fragile and often false, yet people still value them as authentic.
- He warns that privacy of thought itself has never been debated in EU policy, unlike data privacy under GDPR.
- He uses the real DAO hack as a cautionary example for any future governance system built for emulated minds.
- He explains direct anonymous attestation and zero-knowledge proofs as ways to prove a fact without revealing how or what.
Detailed notes
The brain as a bottleneck and the iPhone as brain interface
Mohammed opens by questioning why humans consider their thoughts more 'authentic' than a machine's, and notes the brain consumes 8 to 20% of the body's total energy. He argues the iPhone already functions as a primitive brain-computer interface: before smartphones, people memorized directions, but now the phone is 'an extension of your brain', constrained by the low bandwidth of using eyes and fingers to interact with it. He says this interface has not changed even as phone functionality has grown enormously since 2007, and argues this fixed human interface will become the bottleneck for further human evolution unless people find a way to send signals directly in and out of the brain.
Brain implants, scanning and emulation
He describes retinal implants as an existing technology that feeds electronic signal into the optic nerve, which he calls a high-bandwidth bus into the visual cortex, arguing this already amounts to 'hacking your brain' to put data in, even though no equivalent method exists yet to get data out. He frames the next logical step as scanning a brain, understanding its connections, and creating an emulation, referencing Robin Hanson's work on emulated minds ('EMs') as looking at this from a societal rather than technological perspective, and states this approach seems computationally more attainable than some proposed alternatives on the path to a singularity.
Memory, authenticity and the missing debate on thought privacy
Mohammed cites a recent study, which he says was published in Science, showing human memories are fragile, biased and often unrelated to what actually happened, yet people still value the authenticity of their own memories highly even after being shown they are false; he notes court testimony relies heavily on such unreliable recollection. He then turns to privacy, noting his own background in privacy and security and Germany's relatively strong data protection culture ahead of GDPR, but argues that no EU Parliament debate has yet addressed privacy of thoughts themselves. He warns that once the skull is opened for two-way brain-computer communication, anyone with access to the connection could in principle know what a person is thinking, and that current cryptographic protections such as homomorphic operations remain largely theoretical.
From barter to blockchain: governing emulated minds
He sketches a history of transaction mechanisms from direct barter, to professional arbitration by lawyers, to centralized and then decentralized arbitration, arguing that if emulated minds ('EMs') exist and hold economic value, they will need governance mechanisms to protect them from each other and from other agents. He introduces public blockchain and smart contracts as one candidate mechanism, describing a spectrum from automation at the edges with humans at the center, to automation at the center with humans at the edges, the latter being a distributed autonomous organization (DAO). He references the real DAO hack as a cautionary example, noting researchers had warned against publishing the system beforehand, and uses it to motivate a more careful, trust-hardware-based approach.
Trusted platform modules and zero-knowledge proofs
Mohammed explains the Trusted Platform Module (TPM) as hardware that can hold keys and provide a baseline of verifiable trust, comparing it to a padlock on a house door. He highlights direct anonymous attestation as its most interesting capability: proving that an action was performed correctly without revealing how, illustrated with an Alice-and-Eve cave example and a zero-knowledge protocol for proving possession of a disease without revealing anything else to onlookers. He extends this to a phone verifying it is talking to the correct brain, and to emulated minds validating each other without knowing whether the other party is malicious, contrasting this zero-knowledge approach with symmetric-key schemes where anyone holding the key can open the box and see everything inside.
Ethics for machines, the trolley problem, and Q&A highlights
Turning to a side project done with friends, Mohammed asks how ethical and non-ethical behavior could be judged between emulated minds when no shared human understanding exists, proposing either a third-party entity or a cohort-based mechanism to make that judgment, alongside assurances of privacy and authenticity. He shows a simplified self-driving-car decision table (object in lane, erratic driver) as a toy example of translating human driving judgment into machine-readable rules. Asked whether he would buy a self-driving car that values another life over his own, he admits that despite considering himself an effective altruist, he is largely selfish and would not, and argues self-driving cars will only sell if buyers are convinced their own life is the priority. Discussing liability, he cites a past fatal radiotherapy overdose case that led to a lawsuit as a precedent suggesting legal test cases, not just philosophy, will shape how autonomous-vehicle ethics gets decided. Asked how to formally verify ethical decisions at scale, he says his team modeled decisions as a graph with high-probability traversal paths, but acknowledges the approach breaks down entirely once the state space becomes very large.
Quotes
Humans seem to value authenticity of their thought and authenticity of their memory way above everything else, even when it's proven to them that it is not really true.
Anish Mohammed
This will be the bottleneck for human evolution as such.
Anish Mohammed
Even though I consider myself to be an effective altruist, to a large degree, I'm selfish. I'm unlikely to do that.
Anish Mohammed
Questions answered in this talk
Is the iPhone a form of brain-computer interface?
According to Anish Mohammed at Rise of AI 2016, yes: the phone functions as a primitive brain-computer interface, an extension of the brain limited by the low bandwidth of using eyes and fingers, and its interface has stayed fixed even as its capabilities have grown enormously.
Are human memories reliable?
According to Anish Mohammed at Rise of AI 2016, a study he cites as published in Science found human memories are fragile, biased and often disconnected from what really happened, yet people continue to value their authenticity even when shown they are false.
What is direct anonymous attestation and why does it matter for privacy?
According to Anish Mohammed at Rise of AI 2016, direct anonymous attestation, enabled by trusted platform module hardware, lets one party prove it can do something without revealing how it did it, similar to a zero-knowledge proof, which he argues will be needed for entities like emulated minds to validate each other without exposing private information.