Neural Harmony

Installation | Museum | Neural Science | 2024

This interactive installation introduces the concept of the neural syntax to the general public. Using language as a metaphor, it explains the hierarchical relationship between different brainwaves, referencing cross-frequency modulation and phase-locking.

Interaction Design and Unreal Engine Developer: Lois He

Product Design and Fabrication: Sofia Shen

Interaction and Visual Programming: Vio Zhu, Lois He

Visual of brainwaves: Vio Zhu

Typography Design: Zhang Xing

Collaboration with NYU Neuroscience Institute Buzsaki Lab. Meeting Documentation

The installation is collected by the New York Academy of Sciences.

Documentation Video

Brainwaves and Language: A Hierarchical Metaphor

Through the metaphor of language, the piece demonstrates how brain rhythms interact. Neural Syntax highlights the parallels between language and brainwave patterns. Just as letters form words and sentences, slower neuronal rhythms modulate faster ones, structuring how the brain processes information. This hierarchy organizes and structures neural activity, allowing for effective information processing.

Users can manipulate brainwave rhythms through a simple interface. By turning a knob, they control the speed and order of the rhythms. Slower waves align with sentences, middle waves with words, and the fastest with letters. This interaction allows users to see how these changes reorganize the text, revealing how brainwaves shape cognitive processes.

When the user turns the knob on the installation, letters rearrange within specific words. This mimics the faster neural oscillations nested within slower rhythms. Simultaneously, the according brainwaves adjust their frequency to synchronize with both the upper and lower layers.

Technical Information

TouchDesigner, p5.js and UE5 development for the inetractive brain wave and text animation content.
Connecting Phidget input to control p5.js and UE5 animated experience.
Projection Mapping and pepper ghost effect.