Non-trivial music instrument at Boch Digital Vorspiel, 2 February 2024. Copyright: Maxime Le Calvé
Workshop and Participatory Concert
The NTMI (NonTrivial Musical Instrument) embodies the idea of making complex sound worlds playable by intuition. It enables children and laypersons to explore many sound synthesis processes playfully and allows expert musicians to realize their sonic imagination by adapting/expanding its open-source architecture. In the workshop on April 18th, organized by MoA member Maxime Le Calvé and conducted together with Alberto de Campo, Professor for Generative Art / Computational Art (Berlin University of the Arts), sound researcher Nico Daleman, and the Speculative Realities Lab, the team explained and demonstrated the central concepts, then dedicated time to hands-on playing, experimenting, and improvising, first on individual setups, and finally as multiple sources of influence on a single NTMI environment.
more
Site-Writing Technologies of Capture. Copyright: Maxime Le Calvé
Two-Day Workshop and Installation on Mixed Reality Crafting at Kunstgewerbemuseum Berlin
In December 2023, the Kunstgewerbemuseum Berlin will host the workshop »Site-Writing with Technologies of Capture« in partnership with the ExC »Matters of Activity«. This workshop, curated by Maxime Le Calvé, is a collaboration between the Speculative Realities Lab (MoA, Charité), Shauna Janssen, a visiting scholar from Concordia University in performance studies and site-specific practices, and the Berlin-based artist duo MELT. The dual focus of this two-day workshop is an interdisciplinary exploration of the notions of border and dis/orientation, and the first presentation of Subcortical Matters, an immersive brain data visualization platform developed by the SpecLab.
more
Schematic illustration of constraints for making neural networks more similar to real brains: The models can be used as tools to predict and/or explain behavior and brain activity patterns. The brain area and connectivity diagram on the right has been kindly provided by Rosario Tomasello and the local circuit diagram in the middle is reprinted, with permission, from Schmidt et al. 2018.
Friedemann Pulvermüller and his Team Publish Study on Causal Effects of Language on Thought
The influence of language on human thinking could be stronger than previously assumed. This is the result of a new study by Professor Friedemann Pulvermüller and his team from the Brain Language Laboratory at Freie Universität Berlin. In this study, the researchers examined the modeling of human concept formation and the impact of language mechanisms on the emergence of concepts. The results were recently published in the journal »Progress in Neurobiology«.
more
Brain Networks and Plasticity
Hugues Duffault Holds Lecture at Charité
Accompanying the »Brain Roads« Workshop during Berlin Science Week, Hugues Duffau shared and discussed his groundbreaking insights into the mapping of different brain functions and his revolutionary work in awake brain surgery, which has not only changed surgical practice but also increased our understanding of the intricate processes of the human brain.
more
Brainroads. Copyright: Lene Zech
Exploring Neuroplasticity through Art and Science
Like the sculptor with marble, neurosurgeons dialogue with the brain they operate on. Mapping and representing these ›brain roads‹ can be achieved through various imaging methods, which help neurosurgeons extend their knowledge of the cerebral anatomy. The French Professor in neuro-oncology Hugues Duffau is famous for operating on low-grade gliomas by performing awake surgery with very little technology. His methodology includes a complex mix of techniques, disciplines and experiments based on the concept of neuroplasticity and a limited use of medical imaging in the operating room. The workshop, organized by Patricia Ribault, will explore these »Brain Roads« during Berlin Science Week on November 2nd from 10.00 am– 5.00 pm.
more
Image of the cortical ends of the white matter of the human brain, created in the »Matters of Activity« Cluster of Excellence. Copyright: MoA/Lucius Fekonja
In this prognostic study of 112 patients and 70 patients with glioblastoma, published in JAMA Neurology, a newly developed white matter tract density index (TDI) was significantly associated with the overall survival of patients with glioblastoma. The TDI had a stronger correlation and survival prediction than other commonly used prognostic factors. The TDI appears to be a reliable predictor of patient outcomes; given these findings, this index may be considered in future clinical trials and potentially incorporated into clinical practice.
more
Copyright: Nadezhda Moryak
Presentation at the Long Night of the Sciences 2023
Are your senses up for a challenge? Dive into our interactive experiment and put your memory to the test as you enter Virtual Reality to explore digital objects using both vision and touch. Will you be able to recognize objects while racing against time? Researchers from the »Virtual Dissection« team of the Cluster of Excellence »Matters of Activity« invite you to unlock the secrets of working memory performance while delving into how our brains deal with sensory information. Join us at the Long Night of the Sciences on June 17th at Zuse-Institute!
more
Brain-constrained Neural Modeling Explains Fast Mapping of Words to Meaning
New Publication by Cluster Members Rosario Tomasello and Friedemann Pulvermüller
Although teaching animals a few meaningful signs is usually time-consuming, children acquire words easily after only a few exposures, a phenomenon termed »fast mapping«. Meanwhile, most neural network learning algorithms fail to achieve reliable information storage quickly, raising the question of whether a mechanistic explanation of fast mapping is possible. Here, we applied brain-constrained neural models mimicking fronto-temporal-occipital regions to simulate key features of semantic associative learning.
more
Exemplary visualization of the arcuate fasciculus. Copyright: Reisch, Böttcher, Tuncer, Schneider, Vajkoczy, Picht and Fekonja
Results of the Study Now Published on Pubmed
In this study, to which MoA Members Lucius Fekonja and Thomas Picht contributed to, the authors looked at a new way to use transcranial magnetic stimulation (TMS, a brain stimulation method) to map language in the brain. The new method uses information about the brain's white matter pathways to target specific areas of the brain that are important for language. The study compared this new method to a more general method that targets a wider area of the brain. We found that the new method was more effective in disrupting language abilities and provided more specific information about the brain regions involved in language. This new method could be useful in future studies of the brain's language network.
more
Sketching brains. Copyright: Maxime Le Calvé
An Ethnographic Participant Exhibition at the Neurosurgery Department of Charité in Berlin
»Sketching Brains« is a collaborative exhibition-as-research project which intends to trigger new conversations around the act of sketching in and about neurosurgical practice. Based on a series of fieldwork within the Neurosurgical Department at Charité-Universitätsmedizin Berlin by graphic ethnographer Maxime Le Calvé and on the research of neurosurgeon and cultural scientist Anna L. Roethe, this initiative takes the form of conversations based on graphic-based stories, presented and continued both onsite and online. In German and English, the texts are comment-enabled to bring the community to react and question the description. The ›participant exhibition‹ becomes a vibrant collaborative forum, choreographed by the ethnographer, to understand more profoundly the implications of neurosurgery in people’s lives, on both sides of the scalpel.
more
Tagesspiegel Special Neurology: Copyright: Tagesspiegel
Cutting Portrayed in Tagesspiegel Special Neurology
An article written by Thomas Picht, Patricia Ribault, Jürgen P. Rabe and John Nyakatura has been published in the Tagesspiegel Special on Neurology. »Cutting, separating and exposing are ancient human cultural techniques. The Cluster of Excellence project »Cutting« is investigating them on the basis of tumors in the brain and of fossils in rock. The results could change the future of neurosurgery.« Read the full article and find out more about how »Cutting« and the »Virtual Sensing Knife« can change the future of medicine!
more
Distribution of the patients’ lesions. Copyright: Lucius Fekonja
New Findings by MoA Researcher Lucius Fekonja and Colleagues Published in »Communications Biology«
How do tumors in the motor system affect the structural connectome? Tumors and their location distinctly alter both local and global brain connectivity within the ipsilesional hemisphere of glioma patients. This study, led by Cluster Member Lucius Fekonja, links the complex relationships between function and the underlying matter, the brain's white matter, and demonstrates how tumor activity affects the cerebral network using graph analysis and network-based statistics. The work was done in the framework of »Cutting«/»Adaptive Digital Twin« and initiates its new Cluster phase by enriching our understanding of structural characteristics of active materials, their functional and scientific implications in clinical and translational medicine and neuroscience. The publication in »Communications Biology« further highlights the interdisciplinary aspect of the study.
more
Copyright: The Author(s) 2022. Published by Oxford University Press
New Investigations on Speech Act Understanding by MoA Members Published in Cerebral Cortex
During conversations, speech prosody provides important clues about the speaker’s communicative intentions. In many languages, a rising vocal pitch at the end of a sentence typically expresses a question function, whereas a falling pitch suggests a statement. In their latest publication entitled »Instantaneous Neural Processing of Communicative Functions Conveyed by Speech Prosody«, Rosario Tomasello, Luigi Grisoni, Isabella Boux, Daniela Sammler, and Friedemann Pulvermüller investigated the neurophysiological basis of intonation and speech act understanding with high-density electroencephalography (EEG) to determine whether prosodic features are reflected at the neurophysiological level.
more
Kanzi, a bonobo (Pan paniscus), performing a match-to-sample task at his home Ape Initiative
in Des Moines, Iowa, USA. Print-screen from: Margiotoudi K et al., Proc. R. Soc. B, February 2022 (https://doi.org/10.1098/rspb.2021.1717) [see online video materials].
MoA Member Friedemann Pulvermüller Involved in Significant International Study
Is the ability to perceive connections between words that »sound round« and things that »look round« specific to humans? Or can other animals, including our closest living relatives, the great apes, also infer that a meaningless speech sound is ›sharp‹ or ›round‹ and refers to a curved or spiky shape? An international and multidisciplinary team of researchers, including MoA member Friedemann Pulvermüller, has now been able to answer this question using a new experiment with a language-competent bonobo. The results of the collaborative study by multidisciplinary researchers in Europe and the United States, led by Konstantina Margiotoudi, was published in the Proceedings of the Royal Society B on February 2nd, 2022.
more
Image Podcast Optimal neuro|spine. Copyright: 2022 Optimal neuro|spine
Cluster Professor Interviewed by US-American Neurosurgeon and Podcaster Max Boakye
In episode 15 of the podcast »Optimal neuro|spine«, Cluster Professor for Digital Neurosurgery Thomas Picht was interviewed by US-American colleague Max Boakye. Together they take a deep dive into everything that digital neurosurgery entails, exciting technological advancements in the field, and the future ramifications of the digitalization of medicine.
more
Scaling Nature (1): Wrinkles«: Liquid Lacquer. Copyright: weißensee school of art and design berlin / Luis Magg | Bacterial cellulose curtain. Copyright: Bastian Beyer and Iva Rešetar
Neuroscientist Timo Torsten Schmidt about the Central Question of How Mental Representations are Realized by our Brains
At the last talk of our lecture series »Talking Matters« this year, on Tuesday, December 7th, 2021, at 4.15 pm we were happy to have neuroscientist Timo Torsten Schmidt from Freie Universität as our guest. In his talk, he provided insights for us into the current research on tactile mental representations.
more
Poster Online Lecture Series »Talking Matters«, Copyright: Lucius Fekonja, Bastian Beyer, Iva Rešetar, adapted by NODE Berlin
Alexander von Humboldt Professor of Robotics Oliver Brock About the Fine-Tuned Interplay Between Brain, Body, and Environment
On Tuesday, 23rd November 2021, Alexander von Humboldt Professor of Robotics and Director of the Cluster of Excellence »Science of Intelligence« Oliver Brock was a guest at our lecture series »Talking Matters«. In his talk entitled »Intelligence Without a Brain – Soft Robotics«, he argued that intelligent behavior requires the fine-tuned interplay between brain, body, and environment, a central aspect that AI research has stubbornly ignored for the last 60 years.
more
Lesion overlay map of brain tumor patients, Copyright: Lucius Fekonja
Latest Research Results by MoA Members Published in the Renowned Journal »Cortex«
MoA researchers have succeeded in determining more precisely the relation between certain brain networks and language function. The interdisciplinary team investigated language dysfunction on a voxel level using the voxel-lesion-symptom-mapping method in glioma patients. Among the contributors are MoA members Lucius Fekonja (Scientific Visualization) and Thomas Picht (Digital Neurosurgery) from the Image Guidance Lab at Charité Berlin and Friedemann Pulvermüller (Neuroscience, Linguistics, and Psychology) and Felix Dreyer (Linguistics and Brain Research) from the Brain Language Laboratory at Freie Universität Berlin.
more
Fantastic Voyage 2021. Copyright: Congress of Neurological Surgeons 2021
Article by Members of the MoA »Cutting« Project Published in Journal »Neurosurgery«
In their article in the prominent neurosurgical journal, researchers from the MoA »Cutting« project describe how future tools will enable practitioners to treat brain diseases with unmatched accuracy by integrating analog and digital tools in new hybrid devices and workflows.
more
Towards a Material Basis for Symbols
Cluster Members and Researchers at University of Plymouth Use Novel Network Models for Understanding the Human Ability to Manipulate Symbols and Language
The paper »Biological Constraints on Neural Network Models of Cognitive Function« is one of the first key outputs from a recently initiated Advanced Grant funded by the European Research Council called »Material Constraints Enabling Human Cognition«.
In this project and in the Cluster, Friedemann Pulvermüller and his team are now systematically approaching material-based answers to questions such as the following: How can humans learn a vocabulary of 10,000s of words whereas our closest relatives are normally stuck with 10s? How is it possible that little children quickly interlink signs with meanings, upon only one experience in the extreme, although our closest relatives have great difficulty building such links and neural networks require excessive time for learning them? By which mechanisms can we build abstract concepts and what contribution (if any) makes language to this process (… and many others)?
more
Poster Online Lecture Series »Talking Matters«, Copyright: Lucius Fekonja, Bastian Beyer, Iva Rešetar, adapted by NODE Berlin
Online Lecture Series of the Cluster of Excellence »Matters of Activity«
On Tuesday, May 18th, 2021, »Matters of Activity« launched the online lecture series »Talking Matters«, in which external speakers from various disciplines were invited by the six cluster projects to provide insights into their research, which is related to central issues of »Matters of Activity«. In addition to researchers and students from various disciplines, the lecture series was open to anyone interested in our research.
more
Horst Bredekamp Discusses the Provocation of Thinking »Image Forms« with Neuroscientist David Poeppel
On April 19th 2021, the anthology »Bilder als Denkformen« was presented, which opened up recent positions on image science that are dedicated to the interplay of understanding and visualization. In the subsequent panel discussion, Horst Bredekamp and David Poeppel took up central problems of the publication and trace the provocation of thinking »image forms«.
more
Material Constraints Enabling Human Cognition »MatCo«
New Research Project at Freie Universität Berlin led by Cluster Member Friedemann Pulvermüller
How is it that humans effortlessly learn a vocabulary of tens of thousands of words and symbols, whereas their closest relatives, the great apes, only manage about 100 characters? How can young children associate so many symbols with meanings without special instruction, and then, after only brief learning, use them to express their desires, feelings, opinions, and fears?
Clearly, these abilities must somehow be related to our brains - or, more specifically, to the differences that exist between our brains and the brains of other species. The »MatCo« project, funded by the European Research Council, is looking at precisely these material bases and mechanisms.
more
Connectome construction of a patient with a tumor in the speech-eloquent area of the brain.
The connection strengths of the white matter are shown as edges with a heat map (dark = weak connection, light = strong connection). The cortical areas of the brain and their size are illustrated as nodes in the form of spheres. Additionally, three orange TMS points are shown (TMS = Transcranial Magnetic Stimulation). In this case, the patient’s speech network could be non-invasively disturbed in these areas and mapped accordingly.
The visualization of the white matter is based on tractography. The different directions of the white matter can be measured and displayed by means of diffusion-weighted magnetic resonance imaging. Copyright: Lucius Fekonja & Image Guidance Lab, Charité – Universitätsmedizin Berlin
Creation of Personalized Models of the Brain of Tumor Patients
The field of connectomics aims to comprehensively describe the physical and functional coupling among the neural elements of the brain. Creating personalized models of the brain of tumor patients helps to better understand the impact of a brain tumor on the cerebral network and plasticity and functional reserve capacity. Similarly, it helps to improve effects of neurosurgery on the connectome and identify key nodes and edges, i.e., potential high-risk areas for surgery, reveal the network basis of language function in relation to tumors, and predict the course of recovery.
more
Funding by European Research Council
Cluster Member Prof. Dr. Dr. Pulvermüller Received Funding for the Project »Material Constraints Enabling Human Cognition«
The European Research Council is the premiere European funding organisation for excellent frontier research. A total of 1881 proposals were submitted, of which 35 ERC Advanced Grants went to Germany. The project »Material Constraints Enabling Human Cognition« will be funded with 2.5 million euros and will use novel insights from human neurobiology translated into mathematically exact computational models to find new answers to long-standing questions in cognitive science, linguistics and philosophy.
more
Connectome. Rendering of a group connectome based on 20 subjects. Anatomical fibers that constitute the white matter architecture of the human brain are visualized color-coded by traversing direction (xyz-directions mapping to rgb colors respectively). Visualization of fibers was done using TrackVis software (www.trackvis.org). Data established in a scientific publication that can be found here: https://www.ncbi.nlm.nih.gov/pubmed/24099851. Andreas Horn, 2015. This file is licensed under the Creative Commons Attribution 4.0 International license.
Exploring the White Matter of the Human Brain
»Brain Roads« is an interdisciplinary French-German project for visualization and interactive exploration of the brain. It brings together graphic & digital designers from ESAD de Reims, engineers from Télécom SudParis, and researchers in Neurosurgery and Social Sciences from the Cluster »Matters of Activity« and Charité Universitätsmedizin Berlin. Its ambition is to develop visual representations and interactive tools for exploring the white matter of the human brain.
more
Lecture Series »Nature of Deception – Art of Illusion«
A current theory describes the human brain as a »prediction machine«: It posits that our brain is constantly constructing hypotheses of what will happen next. When there is sensory input, it is quickly matched or corrected/updated by comparing it with already existing hypotheses. This process increases the efficiency of our brain – that is, our data processing and reaction abilities.
more
Adaptive Digital Twin
The »Adaptive Digital Twin« project focuses on neuroscientific questions and predictive models concerning patients with brain tumors from different disciplinary perspectives. Subject-specific adaptive digital twin models are concepts that allow us to evaluate and simulate the effects of pathologies or tumor resectioning on the anatomy, function and connectivity of the central nervous system.
more
Sensing Knife
Achieving a minimal level of lesion is often essential when cutting and manipulating materials. For this, we foresee an immense advance with the invention of cutting tools able to detect and distinguish materials based on their properties at the relevant scales.
more
Mapping of shear wave speed as surrogate of stiffness in abdominal organs based on tomoelastography compared with conventional imaging such as T2-weighted magnetic resonance imaging (MRI) and T1-weighted MRI and computed tomography.
https://doi.org/10.1038/s42254-022-00543-2. Copyright: Ingolf Sack
The effects of brain tumors on brain function are difficult to predict due to the complexity of individual brain networks and the brain's potential for functional reorganization. Invasive and noninvasive brain imaging techniques can help to better understand the effects of brain tumors on brain networks and improve treatment efficacy and levels of control.
more
From Brain Matter to Cognitive Function. Copyright: Rosario Tomasello
Understanding the mechanisms by which the structure and function of neuronal populations bring about specific cognitive functions is the aim of this project. Using models of specific parts of the brain, the team aims to engineer and imitate motor function, symbol production and understanding using precise mathematical models of the human cortex.
more
Brain Roads
Brain Roads is an interdisciplinary French-German project for the visualization and interactive exploration of the brain. It brings together designers, engineers and researchers in neurosurgery and humanities. Its ambition is to develop visual representations and interactive tools for exploring white matter. We wish to bridge the analog and the digital in a cooperation regime between the specific knowledge of some outstanding neurosurgeons, which we will try to translate into visual forms, and new representations of brain models, which allow us to explore dense, digitalized matter, and to visualize its changes of state, but also its zones of uncertainty.
more
The Speculative Realities Lab. Copyright: Maxime Le Calvé
The SpecLab is a space connecting neurosurgery, anthropology, and the immersive arts. It contributes to expanding their respective disciplinary equipment – conceptual, material, and digital – in order to animate brain-related knowledge for navigation, prediction, and inspiration. We want to make neurosurgery appear strange again in order to unsettle certain ›status quo‹ of the clinical practice.
more
Dissect event, Tieranatomisches Theater, 02.11.22, Cluster of Excellence »Matters of Activity« | Joffrey Becker, Maaike Bleeker, Samuel Bianchini, Horst Bredekamp, Johann Habakuk Israel, Thomas Picht, Margrit Shildrick, Patricia Ribault. Copyright: Samuel Bianchini
»Behavioral Matter« proposes a new approach to processes of giving form – ›Gestaltung‹ – based on a living, semi-living and non-living meshwork of things, and demonstrated by innovative formats to perform and disseminate the research.
more