Learning the Language of the Brain’s Cell Types

This essay is part of The Launch Sequence, a collection of concrete, ambitious ideas to prepare the world for advanced AI. These projects need people to build them. Get in touch.

---

Summary

Conversing with an AI chatbot can feel uncannily similar to talking with a human. But the resemblance is only skin-deep: the two systems are built very differently. Artificial neural networks are constructed from many copies of the same basic “neural” units, which become specialized only through training. Real brains, by contrast, are composed of a wide range of cell types connected into intricate circuits. The properties of these cell types, shaped over hundreds of millions of years of evolution, underpin our ability to perceive, reason about, and interact with the world.

Cell types have been cataloged extensively in fixed brain tissue, but they are difficult to identify in recordings from living brains. Without a link between identity and neural activity, we can see neither how cell types coordinate their electrical signals to drive behavior, nor how they break down in disease.

To overcome this gap, we propose building an “ImageNet for cell types”: a large-scale, standardized reference dataset linking specific neuron types to their activity patterns. A modest set of examples has demonstrated that distinct cell types produce characteristic electrical signatures. With a larger dataset, we could train AI systems to classify neurons based on electrical activity recorded in intact brains. As a result, neural recordings — including those from living humans — would become dramatically more interpretable and useful, accelerating the path to biologically realistic models of the brain and revealing therapeutic strategies for disorders in which the activity of particular cell types goes awry.

Motivation

When you read an AI-generated summary of your search query, or a large language model (LLM) solves a math problem for you, you’re engaging with artificial systems that emulate some of the abilities of the human brain. But under the hood, these systems look vastly different from biological brains. LLMs are composed of many repetitions of a small number of computational building blocks that acquire specializations through extensive training. Real brains, on the other hand, are built from a diverse array of intermingled cell types that develop to perform particular jobs. For example, a brain region called the substantia nigra contains cell types that release dopamine, a built-in teaching signal that shapes (among other things) how we learn from experience. Rods and cones, specialized cell types in the eye, contain proteins that convert light into electrical signals, giving us the ability to see. Evolution has spent hundreds of millions of years refining the features and proportions of each cell type, yielding a system whose capabilities depend on the precisely tuned interactions of its components.

Studying the diversity of cells that make up the brain has important implications for health, as many disorders stem from the dysfunction of specific cell types. When those dopamine neurons in the substantia nigra start breaking down, it leads to Parkinson’s disease. Degeneration of the eye’s rods and cones, in turn, results in retinitis pigmentosa, a major cause of inherited blindness. Other conditions, such as depression, schizophrenia, and autism, have more complex origins that have not yet been attributed to a single cell type. But we will ultimately need to understand their effects at the level of different types of neurons, because cell type identity links neural activity to the biological machinery that interventions can actually target: receptors for drugs to bind to, genes that can be edited, or pathways that can be recruited by stimulation.

Advanced molecular technologies, such as RNA sequencing and multiplexed RNA imaging, have already given us detailed catalogs of nearly all the brain’s cell types. But these catalogs are based on isolated neurons or thin brain sections. To discover how cell types work together to perform useful functions, we need to measure their activity in living brains in the context of behavior.

The most direct way to measure neural activity is by inserting electrodes to detect the electrical “spikes” that constitute the brain’s internal language. But there is a big gap: electrical recordings can tell us what neurons are saying, but they do not reveal who is speaking. Without knowing which cell type generated a given spike, we cannot predict how that spike will affect the surrounding circuit. Will it trigger the release of a neurotransmitter that excites nearby neurons, or a different neurotransmitter that inhibits them? Will the spike propagate only locally within a given region, or target different regions across the brain? In most of the electrical recordings that neuroscientists make today, these questions are impossible to answer, because we are missing a broadly applicable, scalable way to connect spiking activity with the cell type identities of the neurons producing it.

Solution

If we want to understand how activity in neural circuits generates our behaviors, and to repair these circuits when they break down, we need to know which cell types exist, what each one does, how they interact, and how to identify them as non-invasively as possible in the brains of living humans. We can do this by developing a classifier that can identify cell types from the activity we pick up in electrical recordings.

The first step toward building a general-purpose cell type classifier is creating an “ImageNet for cell types”: a large-scale, open collection of neural recordings with labeled cell type identities.

ImageNet illustrates how a massive database with carefully curated ground truth labels can accelerate an entire field. Before ImageNet, image classifiers relied on hand-picked heuristics, guidelines such as “three dark circles arranged in a triangle = dog” that were translated into machine language. After ImageNet, convolutional neural networks learned to identify images across thousands of categories of objects and scenes by training on an enormous set of labeled examples.

Right now, electrical recordings of brain activity are in the pre-ImageNet era: we can identify a limited number of cell types based on simple rules, such as “narrow spike shape in the neocortex = parvalbumin-positive inhibitory cell.” But this isn’t close to comprehensive; clues to a neuron’s cell type may be distributed across many additional features contained in the timing of its spikes and their relationship to the spikes of surrounding neurons.

Thankfully, interpreting complex features to extract cell type identity is an ideal application for modern deep learning systems, which excel at extracting signals distributed across many dimensions. Smaller-scale labeled datasets from the Allen Institute (where the author is based), the Buzsáki Lab, and a coalition of labs studying the cerebellum have already demonstrated that these more complex features hold valuable information about cell identity. But the present scale of the data is not sufficient or diverse enough to train a general-purpose cell type classifier. To extend this approach across more cell types in more brain regions, we need a much larger and more comprehensive database of labeled examples that connect spikes to their underlying cell types.

How will this database be generated? Because access to human neurons is ethically and technically challenging, most of the data will need to be collected in model organisms. Fortunately, cell types in mouse, macaque, and human brains show remarkably similar genetic profiles, shapes, and connectivity patterns. If evolution has also retained similar functions for these cell types, their activity patterns may be similar as well. If so, we can train classifiers on ground truth data from other species and apply them to human recordings.

One way to add cell type labels to spiking activity is “optotagging,” which relies on optogenetics, the 2026 Nobel Prize-winning technique, to make selected populations of neurons express light-sensitive proteins. Neurons from the targeted cell types can then be “tagged” in electrical recordings by shining brief pulses of light on the tissue. Light directly evokes spikes only in these genetically modified neurons, thereby establishing their cell type identity. (Use of these optogenetic tools is currently not possible in the living human brain.) AI models can then learn to associate that identity with the neuron’s electrical signature outside the tagging pulses. Once these models have been trained, they can be applied to any recording, even if no optotagging was performed.

Two advances have recently made optotagging more scalable. Neuropixels Opto is a recording device co-developed by the author that integrates electrodes and light emitters onto a shank thinner than a human hair. This makes it straightforward to read out spikes while delivering the light needed for optotagging. Enhancer AAVs are engineered adeno-associated viruses that use genetic switches to ensure that proteins of interest are only produced by certain cell populations, expanding the range of cell types that can be genetically targeted. Many enhancer AAVs retain their targeting specificity across species. Mouse recordings could therefore form the bulk of the required optotagging dataset, with a smaller collection from macaques serving to test which cell-type-specific features of spiking activity are conserved in primates.

This project would evolve over time across many dimensions: the methodologies used¹, the cell types that can be targeted, and the community that forms around the data and models to provide feedback and apply the products to their research. As the database grows, the classifiers can be retrained and compared on shared benchmarks, much as image classifiers were developed through ImageNet.

New methods and scientific organization give us reason to be optimistic: in a few short years, we believe that we can develop classifiers trained mostly on mice and translate them to identify cell types correctly in monkeys, thereby bridging roughly 90 million years of evolutionary divergence. This would make the much shorter step from monkeys to humans even more plausible. Before classifiers are used on patient data, however, they would need to be subjected to a series of checks. These include testing on neurons from surgically removed tissue whose electrical properties and gene expression have both been measured, and checking whether classifiers assign labels in proportions consistent with human cell type catalogs. Classifiers that pass these tests could then reveal which cell types change their activity in disease, and how those cells respond to drugs, stimulation, and other treatments. The labeled recordings would in turn sharpen simulations of human brain circuits, helping researchers decide which interventions are most worth testing.

Why we need to fund this resource now

The burden of neurological and neuropsychiatric disease is enormous and ever-growing. One report estimated that nine common neurological conditions cost the United States over $1 trillion annually (in 2026 dollars), in terms of expenses associated with disability and lost productivity. Therefore, even modest improvements in prevention and treatment would have substantial value. Past experience has shown that identifying the cell types at the root of certain diseases can reveal concrete paths toward treatment. The realization that Parkinson’s disease was caused by the loss of dopamine neurons led to the discovery that levodopa, a molecular precursor to dopamine, could relieve many of its worst symptoms. Experimental treatments for retinitis pigmentosa use optogenetics to introduce light-sensing abilities into a different cell type (retinal ganglion cells) that, unlike rods and cones, has not degraded.

A validated cell type classifier would give us clues about how cell types are affected by many, many more conditions. This is especially important for conditions where no obvious cell-type-specific origins are known (e.g., depression, schizophrenia, and autism). Being able to identify which cell populations change their activity in disease, how those changes disrupt interactions between brain regions, and whether interventions restore more typical activity patterns would point toward new avenues for treatment.

The potential benefits of this resource stretch well beyond the realm of presently diagnosable disease. More accurate brain simulations depend critically on recordings with labeled cell types. An electrical engineer trying to simulate a circuit board may start with a parts list and a wiring diagram but will ultimately need to switch it on and watch how those parts behave. The same applies to simulations of neural circuits, which could help us unlock new understandings of the brain and mind. The MICrONS project showed the value of combining anatomical and functional measurements in individual brains: researchers recorded the activity of tens of thousands of neurons in a cubic millimeter of mouse visual cortex, then reconstructed the connections between them. Because connectomics efforts on this scale are presently much too slow and costly to repeat across the whole brain, cell type classifiers offer a shortcut. Knowing a recorded neuron’s cell type tells us what its connections and intrinsic properties are likely to be. This could provide a key link between ordinary electrical recordings and existing anatomical and connectomic maps. Simulations which are then built on those links could help us predict, in silico, what happens when specific cell populations are activated or suppressed, letting researchers virtually screen hypotheses about candidate interventions before testing the most promising ones in the lab.

Our proposed resource could also inform AI alignment. The Launch Sequence essay “Mapping the Brain for Alignment” proposes that surveying the connections of mammalian dopamine circuits could help build AI models that are better at balancing immediate rewards with long-term goals, and ultimately cooperating with humans. A cell type classifier would make it easier to identify dopamine neurons in electrical recordings (including in humans), and such recordings would provide a necessary functional complement to large-scale anatomical maps.

How to get it built

An initial target for this program would be to record spikes from tens of thousands of identified neurons spanning dozens of cell types across multiple brain regions, behavioral conditions, and species. This kind of dataset could train an initial set of classifiers and kickstart a great deal of work in the field. We estimate this will cost around $20 million over five years. About $15 million could fund a team of 13 people dedicated to mouse data collection and classifier development at an institution such as the Allen Institute, where animal facilities and experimental capabilities are already established. Macaque work could be carried out in two partner labs, at a cost of about $500,000 per lab per year. Data collection protocols and quality control criteria could be published openly, so labs outside the core consortium could contribute data from new cell populations, species, and recording conditions, and benefit from the ongoing data and methods releases. Like ImageNet, this ongoing release structure would give many teams the platform they need to develop cell type classifiers, compare performance against models released by other labs, and build on one another’s progress to help advance understanding across the entire field.²

Projects like this rarely fit conventional funding channels, which is why existing brain-mapping, connectomics, and methodologically entrepreneurial projects tend to take place outside of traditional labs in new kinds of scientific arrangements like special consortia or Focused Research Organizations. Funding from government agencies or philanthropies often requires a focus on a particular disease, even when a fundamental advance in the form of a dataset, a tool, or a platform would accelerate work across many domains. Moreover, a typical grant to an individual lab usually doesn’t offer support for a dedicated team of scientists and engineers building up an organization to run the same protocols across hundreds of recordings, applying the same quality standards to every neuron, and releasing data on a predictable schedule. Traditional grants tend to require work toward a single discovery rather than the release of data that will enable many others. As a result, projects such as this often turn to private philanthropy or to special funding mechanisms to fill the gap by underwriting an initial phase with clear milestones, then expanding support as the resource demonstrates its value to the field at large.

Complementary efforts

ImageNet’s impact depended on complementary advances in hardware, algorithms, and research infrastructure. A similar network of related investments could help the initial phase of this project go faster and broaden its benefits to the community:

Hardware

Devices that integrate optical stimulation and electrical readout. Neuropixels Opto is an example of a device that simplifies optotagging at scale. Similar devices with even more recording sites, a smaller form factor, or flexible shanks would facilitate optotagging across a wider range of conditions and species.

Flexible, high-density devices for human brain recordings. Devices with Neuropixels-like recording site density optimized for long-term implantation would expand the range of clinical settings in which spiking activity can be collected.

Reagents

Enhancer AAVs for cell-type-specific gene expression. Validated viral tools would extend targeted labeling to species where transgenic animals are unavailable or impractical. Expanding their coverage to more cell types — and carefully measuring the specificity of their expression — would increase the diversity and reliability of cell type labels in the reference dataset.

Faster, brighter voltage indicators. Voltage indicators provide a complementary route to measuring the electrical activity of genetically identified neurons. This technique is capable of sampling from more cell types in parallel than optotagging, potentially making it easier to record from populations that may be difficult to isolate with implanted electrodes.

Software

Foundation models for neural activity. Models pretrained on large collections of unlabeled recordings may be better at extracting cell-type-specific electrophysiological signatures than models trained from scratch. Any labeled reference dataset would also contain many unlabeled neurons, which could contribute to the training of foundation models.

Connectome-constrained brain simulations. Simulations are the definitive test of whether we truly understand how cell types and their interactions give rise to behavior. A database of spiking activity with cell type labels would provide the data needed to fit and evaluate these simulations. In turn, these simulations would help identify gaps in the ground truth datasets, providing important information for guiding future data collection.

Infrastructure for sharing and processing large datasets. More widespread adoption of common data formats, standardized metadata, scalable processing pipelines, and accessible data repositories would make it easier to combine recordings across labs. Standards for tracking data provenance would support reproducible benchmarks of classifier performance, regardless of who developed the classifiers.

Acknowledgments

The author would like to thank Dan Turner-Evans, Joseph Fridman, Karel Svoboda, Chand Chandrasekaran, Edward Chang, Brett Mensch, and Patrick Mineault for their feedback throughout the ideation and writing process, and Beez Africa for designing the interactive figures.

Further resources

• Lein et al. (2007) “Genome-wide atlas of gene expression in the adult mouse brain.” A successful example of philanthropic investment creating an open scientific reference resource with long-term, field-wide impact.
• Beau et al. (2025) “A deep learning strategy to identify cell types across species from high-density extracellular recordings.” Demonstrates the feasibility of a cross-species AI cell type classifier in one brain region, the cerebellum.
• Bakken et al. (2021) “Comparative cellular analysis of motor cortex in human, marmoset and mouse.” Shows that many cortical cell type families are conserved across species, providing evidence in favor of mouse-to-primate translation of cell-type-specific electrophysiological signatures.
• Lakunina et al. (2026) “Neuropixels Opto: Combining high-resolution electrophysiology and optogenetics.” Introduces new hardware that combines large-scale neural recording with dual-color light emission; devices like Neuropixels Opto greatly improve the economics of building the proposed dataset.
• Patrick Mineault (2026) “Cell types: encoding the brain’s BIOS” and “Cell types: from genes to circuits.” Two recent essays about the importance of cell types in biological neural networks.