Author Archives: Carla Pegoraro

Music based brain-computer interfaces – an interview with Stefan Ehrlich and Kat Agres

Posted on by
Reply

Music can evoke strong emotions and affect human behaviour. We process music via a series of complex cognitive operations. Consequently, it can be a window to understanding higher brain functions, as well as being used as a diagnostic and therapeutic tool. So how can we understand the way music evokes emotions and effectively use this in healthcare technologies?

Recently PLOS ONE launched a collection on “Affective Computing and Human-Computer Interactions” and we discuss with Stefan Ehrlich from the Technische Universität München and Kat Agres from the National University of Singapore their paper on a music-based brain-computer interface for emotion mediation.

PLOS – In your paper “A closed-loop, music-based brain-computer interface for emotion mediation” you present a Brain-Computer Interface (BCI) pilot study that uses an automatic music generation system to both affect users’ emotional states and allows them to mediate the music via their emotions. What would you say are the key points of your work?

Stefan Ehrlich – Our work focuses on the integration of music with healthcare technology to mediate and reinforce listeners’ emotional states. The key point we see is in providing a novel automatic music generation system that allows a listener to continuously interact with it via an “emotion display”. The system translates the listener’s brain activity, corresponding to a specific emotional state, into a musical representation that seamlessly and continuously adapts to the listener’s current emotional state. Whilst the user listens, they are made aware of their current emotional state by the type of generated music, and the feedback allows them to mediate or to regain control over the emotional state. Many of the neurofeedback applications that have been already proposed often only have one-dimensional feedback provided to the to the subject. For instance, a levitating ball is displayed on the screen, and the subject is asked to control it up or down. The advantage of using music is that it’s possible to map a relatively complex signal, in this case brain activity, in a multi-dimensional manner to a cohesive, seemingly only one- dimensional feedback. It’s possible to embed different information in a single cohesive BCI feedback by using the different features of music, such as rhythm, tempo, the roughness of the rhythm or the harmonic structure.

PLOS – Were there any particular health care applications that you had in mind when designing this pilot study?

Kat Agres – I tend to think of music as being a sort of Swiss army knife where there are lots of features that can come in handy, depending on the scenario or the clinical population. For example, it’s social, it’s engaging, it often evokes personal memories, and it often lends itself to rhythmic entrainment. It’s these properties or features of music that lend itself particularly well to health care applications. Our main focus is on mental health and emotional wellbeing, and teaching people how to control their own emotions. And I think that’s the really interesting part about this study, that the music is a sonification of the listener’s emotional state, as measured via their EEG. It is meant to influence their emotional state, and helps teach the listener how to mediate their emotional states as they interact with the music system. This sonification can show the listener both what’s happening emotionally but it also allows them to mediate the sound of the music by affecting their own emotional state. The music is being created in real time based on the brain activity. We’ve recently been awarded a fairly large grant in Singapore to develop a holistic BCI system that we’re actually calling a Brain-Computer-Brain Interface. The project will cover different aspects, e.g., motor skills, cognition and emotion. We’ve already started developing the 2.0 version of the automatic generation system, and we are about to validate it with a listening study with both healthy adults and depressed patients. Once all these validation steps have been completed and we can effectively say that the system is flexible enough to induce different emotion states in a depressed population, we will be applying this to stroke patients who are battling depression.

PLOS – What do you think the main differences will be in the ability of depressed and healthy populations to affect emotions with this system?

Kat Agres – The number one reason people listen to music is to enhance or modify their emotion state or their mood. There is very significant literature now supporting the use of music for various mental health scenarios and for people who are struggling with various mental health conditions. I think that music is particularly well positioned to help people when other things are not helping them. The first group of depressed patients that we will be testing our system on is made up of many young people who actually think of their identity in part in terms of their music. Based on the literature and unique affordances of music, I think that we have a decent shot at reaching these individuals and helping them figure out how to gain better control of their motion states. In our pilot study, some individuals really got the hang of it and some had a harder time figuring out how to use the system. I think we’ll find the same thing in this population of depressed patients. I’m cautiously optimistic that this system will be effective for this population.

Stefan Ehrlich – When using the system, different psychiatric and neurological populations will probably elicit different patterns of interaction. These will lead to the next steps in understanding how to modify the system in order to better help the patients. At the moment it’s a system that can help them gain awareness of their emotional state and that allows us to measure the variations between the different groups.

Kat Agres –And one of the interesting directions we are exploring with the automatic music generation system is the trajectory of taking someone from a particular (current) emotional state to another, target emotional state. It will be interesting to compare whether the optimal trajectory through emotion space is similar for depressed patients and healthy adults.  

PLOS – Was there anything that particularly surprised you?

Stefan Ehrlich – A surprise for me was that without telling the listeners how to gain control over the feedback, when asked, all of them reported that they self-evoked emotions by thinking about happy/sad moments in their life. I want to emphasise that the system triggered people to engage with their memories and with their emotions in order to make the music feedback change. I was surprised that all of the subjects chose this strategy.

PLOS – What was the biggest challenge for you?

Stefan Ehrlich – The most difficult part was developing the music generation system and the mapping with continuous changes of brain activity. In the beginning we wanted to map brain activity features with musical features and the idea of focusing on emotions as the target only came during the development of the system. Constraining the system to emotional features and target variables helped to reduce the dimensionality and the complexity, while clarifying the main objective (emotion mediation) of the eventual system.

Kat Agres – Creating an automatic music generation system is not as easy as it might sound, especially when it has to be flexible to react to changes in brain state in real time. There’s a lot of structure and repetition in music. So when the participants try to push their emotion state up or down the music has to adapt in real time to their brain signals and sound continuous and musically cohesive.

Stefan Ehrlich – Yes, and there can’t be a big time-lag with the generated music, as this would compromise the sense of agency participants have over the system. If the system does not react or respond accordingly, people would lose faith that the system actually responds to their emotions.

PLOS – This work is very interdisciplinary with researchers from many different backgrounds. What are your thoughts on interdisciplinary research?

Stefan Ehrlich – I think it is more fun to work in an interdisciplinary setting. I’m really excited to hear and learn about the insight or the perspective of the other side on a topic or problem. It can be occasionally challenging. You have to establish a common ground, values and methodological approaches to a problem. You need to be able to communicate and exchange in an efficient way so that you can learn from each other. It’s important that all of the involved parties are willing to understand to a certain degree the mindset of the other side.

Kat Agres – I feel quite passionately about interdisciplinary research, especially as a cognitive scientist working at a conservatory of music. One of the obvious things that comes to mind when you’re working with people from different disciplines is how they use different terms, theoretical approaches, or methods. And yes, that can be a difficulty. But as long as everyone is clear on what the big challenges are, have the same high-level perspectives, values, and a shared sense of what the big goals are, it works well. In order to collaborate, you have to get on the same page about what you think is the most important issue, and then you can decide on the methods and how to get there.

PLOS – Considering your original research backgrounds, how did you end up doing such interdisciplinary research?

Stefan Ehrlich – I have a very non-interdisciplinary background in a way (electrical engineering and computer science). During my masters I attended a lecture called “Introduction to computational neuroscience” and it was really an eye opener for me. I realized that my background could contribute to research in neuroscience, engineering, and medicine. From then I started developing a strong interest in research at this intersection of topics.

Kat Agres – I specifically chose an undergrad institution that allowed me to pursue two majors within one degree programme: cognitive psychology and cello performance. I found it really difficult to choose one over the other and eventually I realised that I could study the cognitive science of music. And then I did a PhD in music, psychology, and cognitive science. I consider health to be yet another discipline that I’m interested in incorporating into a lot of my research. I am very grateful that recently I’ve been able to do more research at the intersection of music, technology, and health.

PLOS – In the field of affective computing and human-computer interactions, what do you think are the biggest challenges and opportunities?  

Stefan Ehrlich – I think one important aspect is the human in the loop. The human is at the centre of this technology, as important as the system itself. Often the transfer from the lab is very difficult to do due to the variables associated with humans. Ultimately, we want to see people using these technologies in the real world, and this is the main challenge. 

Kat Agres – I agree that human data can be messy. Physiological signals, like EEG, galvanic skin response, heart rate variability, etc., are all pretty noisy signals, and so it’s just difficult to work with the data in the first place. We see daily advancements in AI, medical technologies, and eHealth. I think the future is going to be about merging these computational and engineering technologies with the creative arts and music.

PLOS – Do you see Open Science practices, like code and data sharing, as important for these fields?

Stefan Ehrlich – Yes absolutely. When I started working in research there were not many data sets available that would have been useful for my work. I think researchers should upload everything – from data to code – to a public repository. I personally use GitHub, which currently has the limitation of not allowing very large files, e.g., EEG data. It’s not an ideal repository for this kind of data at the moment, but there are many other platforms being developed and will hopefully be adopted in the future.

Kat Agres – I wholeheartedly agree that Open Access is extremely important. I am glad that a discussion is happening around not all researchers having access to funds to make their work Open Access. I’m lucky that I’m attached to an academic institution where one can apply for funds for Open Access. My concerns is that policies requiring authors to pay might create elitism in publication. Academic partnerships with journals like PLOS ONE can help researchers publish Open Access.

PLOS – What would be your take home message for the general public?

Stefan Ehrlich & Kat Agres – I think that the public currently perceives music predominantly as a medium for entertainment, but music has a much bigger footprint in human history than this. Historically, music served many important roles in society, from social cohesion, to mother-infant bonding, to healing. In ancient Greece, Apollo was the god of Music and Medicine. He could heal people by playing his harp. They used to think that music had healing properties. The same is found in Eastern cultures, where for example the Chinese character for medicine is derived from the character for music. There is a very long-standing connection between these areas. In more recent years music has taken this more limited role in our society, but now more and more people are beginning to realise that music serves many functions in society, including for our health and wellbeing. We hope that music interventions and technologies such as our affective BCI system will contribute to this evolving landscape and provide a useful tool to help people improve their mental health and well-being.

References:

1. Ehrlich SK, Agres KR, Guan C, Cheng G (2019) A closed-loop, music-based brain-computer interface for emotion mediation. PLOS ONE 14(3): e0213516. https://doi.org/10.1371/journal.pone.0213516

Author Biographies


Stefan Ehrlich is a postdoctoral fellow in the Dystonia and Speech Motor Control Laboratory at Harvard Medical School and Massachusetts Eye and Ear Infirmary, Boston, USA. His current research is focused on brain-computer interfaces (BCIs) for the treatment of focal dystonia using non-invasive neurofeedback and real-time transcranial neuromodulation. Formerly, he was a postdoctoral researcher at the Chair for Cognitive Systems at the Technical University of Munich, where he also obtained his PhD in electrical engineering and computer science in 2020. His contributions comprise research works on passive brain-computer interfaces (BCI) for augmentation of human-robot interaction as well as contributions to the domain of easy-to-use wearable EEG-based neurotechnology and music-based closed-loop neurofeedback BCIs for affect regulation.

ORCID ID – 0000-0002-3634-6973.


Kat Agres is an Assistant Professor at the Yong Siew Toh Conservatory of Music (YSTCM) at the National University of Singapore (NUS), and has a joint appointment at Yale-NUS College. She was previously the Principal Investigator and founder of the Music Cognition group at the Institute of High Performance Computing, A*STAR. Kat received her PhD in Psychology (with a graduate minor in Cognitive Science) from Cornell University in 2013, and holds a bachelor’s degree in Cognitive Psychology and Cello Performance from Carnegie Mellon University. Her postdoctoral research was conducted at Queen Mary University of London, in the areas of Music Cognition and Computational Creativity. She has received numerous grants to support her research, including Fellowships from the National Institute of Health (NIH) and the National Institute of Mental Health (NIMH) in the US, postdoctoral funding from the European Commission’s Future and Emerging Technologies (FET) program, and grants from various funding agencies in Singapore. Kat’s research explores a wide range of topics, including music technology for healthcare and well-being, music perception and cognition, computational modelling of learning and memory, automatic music generation and computational creativity. She has presented her work in over fifteen countries across four continents, and remains an active cellist in Singapore.

ORCID ID – 0000-0001-7260-2447

Affective Computing and Human-Computer Interactions Collection
Submit now to PLOS ONE

The post Music based brain-computer interfaces – an interview with Stefan Ehrlich and Kat Agres appeared first on EveryONE.

Physical forces at the interface with biology and chemistry: a conversation with Kerstin Blank and Matthew Harrington

Posted on by

 

Cell behaviour, tissue formation/regulation, physiology and disease are all influenced by cellular mechanics and physical forces. The field of mechanobiology has for a long time striven to fully understand how these forces affect biological and cellular processes, as well as developing new analytical techniques. At the same time, the properties of advanced smart materials, such as self-healing, self-reporting and responsive polymers, have been determined by a complex interplay between the thermodynamics, kinetics and mechanics of dynamic bonding strategies. These are tightly connected to the field of mechanochemistry, which aims to elucidate and harness molecular level design principles and translate these to the bulk material level as emergent properties. At this interface between disciplines lies an emerging and exciting research area that has been strongly facilitated by the collaboration of physicists, chemists, engineers, materials scientists, and biologists.

We had the pleasure of speaking to Kerstin Blank and Matthew Harrington, who have been working on how mechanical forces influence biological systems, molecules and responsive biomaterials, about their views of the field and the recent ‘Multiscale Mechanochemistry and Mechanobiology’ conference of which PLOS ONE was one of the proud sponsors.

 

How did you first become interested in this topic?

 

Kerstin: When I started in this field in 2000, I was mostly impressed by the technical possibilities. I was working with Hermann Gaub, one of the leaders in single-molecule force spectroscopy. I found it fascinating that we could stretch a single biological molecule and observe its response. I did ask myself sometimes if this was just something that physicists like to play with or if one could solve biomedically relevant questions with this approach. Now, almost 20 years later, it has become very evident that a large number of biological systems are regulated by mechanical forces in many different ways.

 

Matt: My educational background was primarily in biology and biochemistry, but I became fascinated with the capacity of certain biological materials to exhibit self-healing responses in the absence of living cells. I reasoned that this must arise from specific chemical and physical design principles in the material building blocks themselves, and I became obsessed with figuring out how this works. This led me to the self-healing materials community, which was largely populated with chemists and materials engineers, but not so many biologists. When I began to see that many of the same principles at play in synthetic self-healing materials were present in nature, and that in some cases nature was going well beyond the state of the art in synthetic self-healing materials, I realized the enormous potential at the interface of mechanobiology and mechanochemistry. I haven’t looked back since.

 

Which areas are you most excited about?

 

Kerstin: I find it very intriguing how cells utilize mechanical information from their environment and then feed it into intracellular biochemical signalling cascades. Understanding these mechanosensing and mechanotransduction processes requires knowledge of the cellular players and their interactions. But to develop the complete picture, we also need to investigate how cells interact with their extracellular environment. This also involves understanding the microscopic and macroscopic mechanical properties of the extracellular environment. I am highly excited about the development of molecular force sensors that convert mechanical force into a fluorescent signal. This allows for the localized detection of cell traction forces and, in the future, will also enable us to visualize force propagation inside materials that mimic the natural extracellular matrix.

 

Matt: I am currently most excited about understanding how and why nature uses different transient interactions to control the fabrication and viscoelastic mechanical responses of biopolymeric materials and the potential this has for the development of sustainable advanced polymers of the future. Recent discoveries in the field clearly show that in contrast to traditional polymers, living organisms commonly use specific supramolecular interactions based on dynamic bonds (e.g. hydrogen bonding, metal coordination or pi-cation interactions) to guide the self-assembly and mechanical properties of protein-based materials. The thermodynamic and kinetic properties of these labile bonds enable a certain dynamicity and responsiveness in these building blocks that provides potential inspiration for environmentally friendly materials processing and active/tuneable material properties. These concepts are already being adapted in a number of exciting bio-inspired polymers.

 

What progress has the field made in the last years?

 

Kerstin: It is now well-established that cells are able to sense and respond to the elastic and viscoelastic properties of the material they grow in. We have also learned a lot about how the mechanical signal is converted into biochemical signalling on the intracellular side. This is a direct result of many new technological developments, including the molecular force sensors described above. It is further a result of the increasing development of extracellular matrix mimics with well-defined and tuneable mechanical properties and microstructures.

 

Matt: Due to recent technological advances it is becoming possible to link specific aspects of mechanical material responses directly to structural features at multiple length scales. The better we understand these structure-property relationships, the better we can optimize the material response. This provides an intimate feedback loop that has enabled major breakthroughs in the fields of active matter, including self-healing and self-reporting polymers.

 

What is the real-world impact?

 

Kerstin: It is widely accepted that mechanical information plays a key role in stem cell differentiation. It has further been shown that mutated cells, e.g. in cancer or cardiovascular diseases, have different mechanical properties and show alterations in processing mechanical information. Understanding the origin of these changes and being able to interfere with them will have direct impact in disease diagnostics and treatment. Engineering materials with molecularly controlled structures and mechanical properties will further enable the community to direct stem cell differentiation in a more defined manner for applications in tissue engineering and regenerative medicine.

 

Matt: Aside from biomedical impacts, the insights gained from understanding the structure-function relationships defining the mechanical response of molecules are also extremely relevant for the development and sustainable fabrication of next generation advanced polymers. Given the global threat of petroleum-based plastics processing and disposal, this is an extremely important aspect of the research in this field.

 

What are the challenges and future developments of the field?

 

Kerstin: At this moment, we usually try to relate the macroscopic material properties (measured in the lab) with the microscopic environment that cells sense. In my view, we are missing a key piece of information. We need to understand how the macroscopic properties of a material emerge from its molecular composition, topography and hierarchical structure. In combination, all these parameters determine the mechanical properties of a material and, more importantly, what the cells ‘see’. In fact, this is not only key for the development of new extracellular matrix mimics. The same questions need to be answered for understanding how nature assembles a wide range of structural and functional materials with outstanding properties, such as spider silk, cellulose composites and nacre. Here, I see a great potential for future collaboration between disciplines.

 

Matt: There are enormous challenges on the bio-inspiration side of the field involved with transferring design principles extracted from biological materials into synthetic systems. Biology is inherently complex, so there is a common tendency to distil the extracted concept to a single functional group or concept, while often there are collective effects that are lost by this more reductionist approach. On the biological side, a key challenge is ascertaining which are the relevant design principles. On the bio-inspired side, there are challenges in finding appropriate synthetic analogues to mimic the chemical and structural complexity of the natural system. Overcoming this barrier requires cross-disciplinary communication and feedback and is an extremely exciting and active area in our field.

 

Why and when did you decide to organize a conference on this topic?

 

Kerstin & Matt: While both working at the Max Planck Institute of Colloids and Interfaces, we quickly realized that the cell biophysics, biomaterials, mechanochemistry and soft matter communities are all interested in very similar questions while using similar methods and theoretical models; however, we had the impression that they hardly interact with each other. We thought of ways to change this and organizing a conference was clearly one way to do it. The first conference with the topic ‘Multiscale Mechanochemistry and Mechanobiology: from molecular mechanisms to smart materials’ took place in Berlin in 2017. When bringing this idea forward in our respective communities, we immediately realized that we hit a nerve. Now that the conference has taken place for the second time in Montreal in 2019, we really got the feeling that we are starting to create a community around this topic. There will be another follow up conference from August 23-25, 2021 in Berlin (@mcb2021Berlin).

 

What are the most interesting and representative papers published in PLOS ONE in this field?

 

Kerstin: The paper ‘Monodisperse measurement of the biotin-streptavidin interaction strength in a well-defined pulling geometry’, published by Sedlak et al., is a highly interesting contribution to the field of single-molecule force spectroscopy, which was also presented at the conference. This work highlights the methodological developments in single-molecule force spectroscopy since its very early days. The authors from the Gaub lab have re-measured the well-known streptavidin-biotin interaction, now with a very high level of control over the molecular setup. It clearly shows how far the field has come and also that protein engineering, bioconjugation chemistry, instrumentation development and data analysis all need to go hand in hand to obtain clear and unambiguous experimental results. Clearly, considering a defined molecular setup is not only crucial for this kind of measurement but also for the development of biomimetic materials with controlled mechanical properties.

 

Sedlak SM, Bauer MS, Kluger C, Schendel LC, Milles LF, Pippig DA, et al. (2017) Monodisperse measurement of the biotin-streptavidin interaction strength in a well-defined pulling geometry. PLoS ONE 12(12): e0188722, https://doi.org/10.1371/journal.pone.0188722 

 

Matt: Accurately detecting and measuring the mechanical forces at play inside living cells is one of the key challenges in the field of mechanobiology, given the small size and dynamic nature of the intracellular environment. However, this information is extremely important for understanding the role of mechanics in regulating cellular functions such as growth, differentiation and proliferation, as well as disease states. In the ‘Nuclei deformation reveals pressure distributions in 3D cell clusters’ paper from the Ehrlicher group, the authors address this challenge by using fluorescently labelled proteins in the cell nucleus coupled with confocal microscopy to measure compressive pressures within cells and cell clusters. Using this methodology, they explored the effect of cell number and shape of multicellular clusters on the internal compressive pressure within cells, providing potentially important insights for cellular signalling and function. These studies have potential applications in both in vitro and in vivo models, and provide a relatively simple methodology for acquiring intracellular mechanical data.

 

Khavari A, Ehrlicher AJ (2019) Nuclei deformation reveals pressure distributions in 3D cell clusters. PLoS ONE 14(9): e0221753, https://doi.org/10.1371/journal.pone.0221753

 

 Other PLOS ONE representative papers:

 

  • Huth S, Sindt S, Selhuber-Unkel C (2019) Automated analysis of soft hydrogel microindentation: Impact of various indentation parameters on the measurement of Young’s modulus. PLoS ONE 14(8): e0220281, https://doi.org/10.1371/journal.pone.0220281
  • Taufalele PV, VanderBurgh JA, Muñoz A, Zanotelli MR, Reinhart-King CA (2019) Fiber alignment drives changes in architectural and mechanical features in collagen matrices. PLoS ONE 14(5): e0216537. https://doi.org/10.1371/journal.pone.0216537
  • Wheelwright M, Win Z, Mikkila JL, Amen KY, Alford PW, Metzger JM (2018) Investigation of human iPSC-derived cardiac myocyte functional maturation by single cell traction force microscopy. PLoS ONE 13(4): e0194909. https://doi.org/10.1371/journal.pone.0194909
  • Opell BD, Clouse ME, Andrews SF (2018) Elastic modulus and toughness of orb spider glycoprotein glue. PLoS ONE 13(5): e0196972. https://doi.org/10.1371/journal.pone.0196972
  • Yalak G, Shiu J-Y, Schoen I, Mitsi M, Vogel V (2019) Phosphorylated fibronectin enhances cell attachment and upregulates mechanical cell functions. PLoS ONE 14(7): e0218893. https://doi.org/10.1371/journal.pone.0218893

 

Kerstin Blank studied Biotechnology at the University of Applied Sciences in Jena and obtained her PhD in Biophysics under the supervision of Prof Hermann Gaub at Ludwig-Maximilians Universität in Munich. After two postdocs at the Université de Strasbourg and the Katholieke Universiteit Leuven, she became an Assistant Professor at Radboud University in Nijmegen in 2009. In 2014, she moved to the Max Planck Institute of Colloids and Interfaces where she holds the position of a Max Planck Research Group Leader. Her research interests combine biochemistry and single molecule biophysics with the goal of developing molecular force sensors for biological and materials science applications.

 

Matthew J. Harrington is Canada Research Chair in Green Chemistry and assistant professor in Chemistry at McGill University since 2017. He received his PhD in the lab of J. Herbert Waite from the University of California, Santa Barbara. Afterwards, he was a Humboldt postdoctoral fellow and then research group leader at the Max Planck Institute of Colloids and Interfaces in the Department of Biomaterials. His research interests are focused on understanding biochemical structure-function relationships and fabrication processes of biopolymeric materials and translating extracted design principles for production of sustainable, advanced materials.

 

The post Physical forces at the interface with biology and chemistry: a conversation with Kerstin Blank and Matthew Harrington appeared first on EveryONE.

An Interview with Guest Editors for the Photovoltaic Materials Call for Papers

Posted on by

One of the most pressing challenges of the 21st century is meeting the ever-increasing demand for energy consumption whilst reducing the environmental impact of energy production and storage. Solar energy conversion devices have the potential

Open Biomaterials Research

Posted on by

  In this Guest Blog, Guest Editors from the Open Biomaterials Research Collection discuss the range of research topics featured in the collection, their contributions to open science and promotion of reproducibility via shared protocols in biomaterials research.