Subcortical Matters
Data Sonification for Augmented Reality Experience
2023
https://www.maximelecalve.com/on-going-fieldwork/2023/12/18/rubbing-as-fieldwork-workshop-on-site-writing-with-technologies-of-captures-with-melt-and-shauna-janssen-and-speclab-at-kunstgewerbemuseum
https://speculativerealitieslab.pubpub.org/pub/yp1wtprt/release/5
https://www.matters-of-activity.de/en/activities/12820/site-writing-with-technologies-of-capture
The subcortical matters projects aim to create an immersive experience based on new developments on neurological data obtained by fMRI. Through data processing techniques, it is possible to perform a tractography of neural paths within the white matter of the brain, thus identifying specific connection that relate to certain functions of the brain.
Within neuro-chirurgical practices, this is important in the removal of gliomas (tumors), so that such paths are not damaged in the process.
We are interested in exploring fMRI data to visualize subcortical tracks. Beyond the standard Diffusion tensor imaging (DTI) = One direction we aimed to include the Constrained spherical deconvolution (CSD) = multiple directions technique to acquire a more comprehensive visualization of said data. We take advantage of data processing of voxel (3D pixels) that signal not only the position of data but also its direction though spherical harmonics.
Sonically the project consists of three layers. First it should include a basic and pragmatic UX sound design that is coupled with the VR experience. This includes reinforcing certain movements and gestures that the user might encounter. Second it should include a semantically representative layer, where data configurations glioma location in relation to neve tracks. This includes the use of distortion to signify the sound of the glioma or certain modulations to signify its position. Finally, the overall configuration of the voxel data should create a soundscape that represents the configuration and nuances of different patients.
Brain in the Shell
for Brain Computer Interfaces, 4D sound System and 3 to 6 performers
Opening of the Syndicate of Space and Sound.
Budapest, Hungary
2023
Brain in the Shell is performance that explores the possibilities of consumer-level devices to create a musical experience. Beyond the sonification of brain frequency bands and the translation of data into “mental states,” the piece explores the performativity of wearing BCI devices as an act of intrusion and violence. The compatibility or incompatibility of the devices and the with the practical constrains of closed consumer-level made evident in the sound results and the unexpected glitches that it generates. The piece makes use of the raw data of the electroencephalograms to create dissonant soundscapes and eroding sonic decompositions in order to explore a more sensorial and physical translation of sound data. The conceptual boundaries between human and non-human agencies are challenged, as we embrace the possibilities of utopic places generated through the collaboration of humans and machines.
The first presentation of this performance took place on May 2023 at the Syndicate of Sound and Space (SSS) in Budapest using 4DSOUND technology. This is the development of a three-year long research on brainwave sonification, which has resulted in papers, conference talks and performances.
Multi_fractions
Sound installation for speaker cones, nylon wires, UV light, video and Plexiglass Structure
2023
A sound installation exploring the audiovisual perceptual properties of sonic particles through light, sound, and movement. Glissons, grainlets, trainlets, pulsars and wavelets disturbe the time-space architectural configurations generating a cybernetic sonic environment. By experiencing the different stages of sound as wave-particle entity through diffracting patters, ecological and phenomenological intra-actions emerge between material, bodies, space, and time.
It consists of a cybernetic environment of sonic particles that act as its constituent agents. Speaker cones are tied to tense nylon strings that vibrate with low frequencies of the different sonic particles which are visible through light diffractions of raster grid patterns.
The two main rooms in the gallery will be physically separated and will interact between each other via a main computer that controls the actions of the two different parts of the installation. One of the spaces consist of hanging grated panels, four speaker cones with a UV-painted nylon wire attached to the ceiling, and a UV light Rod in the center. Through the grates and the light configurations, the audience is able to perceive the particle-wave dualities of sound, which allows the viewer to peek inside the movement of the inner space.
On the second space speaker cones are tied to tense nylon strings that vibrate with low frequencies of the different sonic particles. In order to make these vibrations visible, raster patterns are projected on a wall behind them, so that the light diffractions reinforce the effect of the vibrating string. Through their translation to the kinetic and visual realm, the sonic particles are made perceptible.
The installation is inspired by quantum entanglement, a phenomenon in which the characteristics of atomic particles are interdependent regardless of the distance from one another, creating a “spooky action at a distance.” Karen Badar’s proposes a post-human account of quantum entanglement, in which phenomena and systems intra-act and resulting in emergent agency. Through immersive sonic experiences in parallel spaces, Multifractions explores entangled phenomena between human and non-human agents. The result is a sonic system where agents intra-act with each other to sonically enact themselves in an immersive environment.
Presentation and Program:
Location: Errant Sound
Rungerstr. 20
10179 Berlin
www.errantsound.net
Vernissage: Friday 10 of March 2023 at 6 pm
Opening days:
March 10 -12 and 16-18 form 18h to 22h
March 13-15 with appointment at nicodaleman@gmail.com
Diffractions
4D Sound Installation
As part of the SVS Collision Festival 2022 @ Muffathalle in Munich
2022
Inspired by
quantum particles, this set of pieces explores diverse interactions of sonic
particles in order to induce a series of disturbances in the virtual time-space
configuration of the sound system. The diffractions,
reflections, superpositions and collisions of sound material present a nuanced perspective
on the role of perception in speculative quantum physics. By considering the
different stages of sound as wave-particle entity through diffracting patters,
ecological and phenomenological relations emerge between material, bodies,
space and time
.
Hanging Wires of Babylon
Multichannel Sound Installation
As part of “Reflections” at the Schwartsche Villa in Berlin-Steglitz
2021
The Hanging Wires of Babylon is a multichannel sound installation that explores the possibilities of modular synthesizers, speakers and cables as sculptural material. Various Doepfer A-100 modules are single-positioned around the space, interconnected by audio and electric powers, generating a network of electromagnetic and sonic interaction. Through the use of different sensors, the installation reacts to the public and creates a generative composition that reflects on the perceptual capabilities of technology and their influence in our artistic experience. Speaker cones reproduce the processes and bring attention to the sounds that are always happening on top of our head. The audience is then confronted with a mediated perception of the space, that evoke the images of promised pasts and possible futures, generating a new virtual reality.
