This mural artwork is the first digital reconstruction of human grey matter of the temporal lobe made with a large-scale and curvature-dependent neuronal circuit builder that I engineered. White matter is below these mountains of cells. Colors reveal structural properties of the circuit and cell types. Full mural artwork size: 48000*27000px, 4m by 2.2m at 300dpi.
5% of neurons are visualised from a circuit of 50 million cells. The system looks so complex that we can hardly distinguish spines – at a very low density per cell here due to memory limit. Cell nuclei are visible but there are no other organelles here. Neither glia nor vasculature are in this model.
Detail of the mural artwork with excitatory and inhibitory neurons across six layers. Some nuclei are visible.
Paul-Löbe-Haus, Bundestag, Berlin where my artworks were presented. Picture by Ansgar Koreng.
Mural artwork based on MRI data and on The Virtual Brain. Here, I wanted to give digital neuroscience a face, an ultra-realistic one not only to show the biological source of this image but also to give us – the audience – a way in this fascinating microscopic world. The dramatic lighting and the gaze of this woman make a canvas for us to question and confront modern neuroscience with what we are as individuals, with the perception of ourselves as macro organisms.
In this illustration, I reveal multimodal data of TVB with functional pathways in the human brain as well as mathematical research on the representation of brain activity that goes beyond EEG signals that people are used to see. My deliberate choice of having a woman subject here is a hope that this image will contribute in a very modest way to inspire more women to study and do neuroscience research but more than that, to lead future research.
Scientific mural artworks
Several mural artworks were commissionned by the Human Brain Project and were revealed at Bundestag German parliament, Paul-Löbe-Haus, Berlin on the 27th of November until December 19th, 2019. After that, the exhibition will move to the state parliament of Northrhine-Westfalia in Düsseldorf from January 21st to 31st.
Scientific data kindly provided by CNCR Amsterdam, Szeged University, Institut de Neurosciences de Marseilles, Charité University Berlin and Human Brain Project.
For the making of the neuronal circuit visualization, I reviewed scientific literature and curated data and I developed a first version of a new Python tool to reconstruct and visualize detailed and large-scale circuits of cells.
For the production of the murals, I deployed my custom modification of Blender 2.81 for large-scale graph visualization with Docker onto a Cray supercomputer at CSCS Lugano, I automatically distributed the computing of these artworks in smaller chunks and I composited the result afterwards.
Special thanks to
Human Brain Project
The first digital circuit of neurons, astrocytes and blood vessels made at Blue Brain Project.
This illustration is the result of an on-going long term collaboration with scientist E. Zisis. What you see here is the tip of the iceberg as a lot of my visualisation and technical research went into debugging the model with images until I could design the final illustration, a usual workflow in digital illustration for science.
In the brain, neurons are supported and regulated by other types of cells known as the glia. Among the glia, astrocytes are territorial cells – shown here in blue – that, among other complex interactions with their surroundings, take nutrients from the blood vessels in red and distribute them to neurons in yellow. But astrocytes do more than that. For instance, they support a part of the synaptic activity within their domain. What you see in this illustration of neocortical grey matter is roughly 1.5mm tall and 0.5mm wide.
Looking at digital simulations of neuronal activity, scientists found connectivity patterns: neurons seem to have a preference to connect with a chosen set of neurons and the resulting connectivity forms patterns like the "rich club" in the middle of this illustration or the feed-forward vertical pattern. The emergence of a communication pattern among cells reveals a functional structure within what appears to us as a chaotic forest of cells.
Nature Neuroscience TOC, July 17
Blender, Sverchok, Cycles.
Cellular variety in a 1mm^3 neocortical brain tissue
This artwork illustrates data obtained with electron microscopes of a small brain tissue.
A variety of cells are visible here, from microglia in blue/turquoise to neurons in purple, astrocytes in orange-yellow with a perivascular process and also myelinated axons in dark blue and blood vessels, here not reconstructed as endothelial cells, which shows the limits of the reconstruction.
The 3D models except red blood cells were automatically reconstructed from EM stacks by scientist Corrado Cali.
Digital 3D reconstruction of capillaries in the neocortex
Using field-based meshing – aka "metaballs" in Blender, I developed a way to automatically reconstruct 3D surfaces for large datasets. This method builds the mesh by blocks so it can build any dataset.
The graph skeleton was calculated by E. Zisis and the raw data was provided by Bruno Weber et al. ETHZ.
2nd place award, professional illustrators category
AEIMS 18 medical illustration congress
Eleftherios Zisis et al.
Particle simulation for an astrocyte
On top of automatically creating the 3D models for this illustration, I proposed a way to illustrate perivascular end feet – the parts that wrap around red tubes – with particle simulations that extend and flow over the capillaries.
This shows how astrocytes interact structurally with blood vessels. The figure below illustrates in more details what happens at a molecular level.
Scientific poster, SfN 2017
Le grand Atlas du cerveau, 2018
Energy supply between cells in the brain
Illustration about noradrenergic modulation of energy supply in the neuro-glial-vascular ensemble or in less barbarious terms, how molecules flow and interact between blood vessels, astrocytes and neurons.
A major step after reconstructing vasculature is to have synthesized astrocytes filling a good part of the volume occupied by blood vessels and neurons. Here neurons are not shown. The astrocytes in blue grow special branches in white that are about to wrap nearby capillaries in red to get nutrients and distribute them to neurons, among many other roles theses cells perform.
This inspired BBP visualisation team to explore similar techniques for large-scale visualization.
Meshless objects: 260 million splines
Surface meshes: 12 million polygons
Synaptome of a layer 5 pyramidal neuron
A close-up view of the 4138 local connections – input spines and output boutons – that this neuron forms with other cells (invisible here) in a neocortical circuit. Given points on the surface of the branches and end points, I developed a way to represent spines in a realistic way, based on scientific papers documenting neuron spine types and sizes.
2nd place award, professional illustrators category
AEIMS 18 medical illustration congress
Custom Python library for meshing and visualization, Blender, Cycles.
Eilif Muller et al.
Cerebral Cortex journal cover
From a collaboration with post-doc L. Kanari, I recreated a digital neuronal circuit given cellular densities for the illustration of the scientists' publication on the automatic classification of neuronal cells. When the artwork was submitted to Cerebral Cortex journal, it was chosen as the cover for its April 2019 issue. After that, it has been published in many science news websites such as medicalxpress.com, sciencedaily.com and eurekalert.org.
Topology, a mathematical tool for interpreting brain activity
Illustration concept of algebraic topology coupled with neuronal simulation. Using simulation data on the left hand, I revealed the other "multidimensional side" in an orthogonal projection. Black and white here is a visual code I chose for domains that are beyond our sight. When many neurons are connected together at a given time, they form simplices of high dimensional orders and even "cavities", such as the one illustrated here.
This publication generated a small buzz in the scientific community with over 90’000 views in two weeks and a lot of questions asked to BBP.
Synapses evolve over time. While connections are formed and strengthened, others dissolve.
This illustration was requested to show research on neuroplasticity which got a grant of 100M core hours offered by ALCF (Argonne, USA) to Blue Brain in order to run simulations to study plasticity in the neocortex.
Somatosensory cortex simulation
This is one of the largest simulations of the neocortex that BBP produced in mid 2017. The simulation is 87 Terrabyte large for a mere 217’000 neurons. A collaboration with engineer Cyrille Favreau.
Guiseppe Chindemi et al.
Cell cores densities
Illustration of neocortical densities of neurons (red), astrocytes (green) and blood vessels (blue). Intricate details appear close-up. The denser layer 4 is clearly visible in this illustration.
Optomechanics illustrations below done for the Laboratory of Photonics and Quantum Measurements, EPFL.
Close-up on tiophenol concentration in a 7Å gap
Published in Nature Nanotechnology
Cover of The Analytical Scientist
Molecular cavity optomechanics
Nano particles of gold are positioned to make a nano gap of about 7Å. When this gap is lit with a precise wavelength, tiophenol molecules react, "store" photons and scatter them soon after [Raman scattering]. Philippe Roelli et al described the inner workings of this phenomenon which resulted in a lot of attention in the scientific community and this will pave the way for new quantum technology.
I strive to make meaningful and aesthetic visuals for each and every project. I engineer tools to process the data produced by the hard work of scientists and I use such tools as an artist to reveal the complexity of research and its meaning in close collaboration with scientists.
These artworks are produced for scientific publications, public outreach and exhibitions. Making such visualizations often helps to find flaws in digital models and to improve them, which in turn provide new data for visualization. All works here are based on scientific digital data.
Blender & Cycles are key tools for what I do. In the context of large-scale visualization of spline primitives, I use my modification of Cycles renderer. In the case of interactive and VR projects, I use Unity and Unreal Engine. I usually engineer solutions to reveal scientific content in a coherent visual experience.