Skip to content

shawngraham/songsfromthegap

Repository files navigation

...couldn't get the bloody workflow to build properly (edit: damn package-lock.json), so built locally then threw it up on Netlify here:

https://songsfromthegap.netlify.app/

Songs from the Gap: A Semantic Topology Sonification

Songs from the Gap is a mathematical sonic journey that transforms the semantic distance between Wikipedia articles into a multi-voiced melodic landscape. By projecting knowledge structures into a 2D manifold, the application allows users to select two disparate points of information and listen to the "bridge" that connects them.

The Concept of the "Semantic Gap"

In traditional knowledge retrieval, we look at articles as isolated nodes. This application instead focuses on the latent space between them.

When you select an Origin and a Target, the app identifies the "Gap"—the mathematical and structural vacuum between these two concepts. It then synthesizes a musical composition where the melody is not derived from the articles themselves, but from the shared structural links that inhabit the space between them.

How it Works

1. Spatial Projection (The Manifold)

The application fetches articles within a specific radius of a geographic coordinate. It then performs a Deterministic Multidimensional Scaling (MDS) approach:

  • Structural Analysis: Every article's outgoing links are cross-referenced with every other article in the set.
  • Jaccard Similarity: We calculate a similarity matrix where $Sim(A, B) = \frac{|Links_A \cap Links_B|}{|Links_A \cup Links_B|}$.
  • Force-Directed Layout: Articles are projected into a 2D plane. Nodes that share a high degree of structural affinity "clump" together, while semantically distant concepts are pushed to the periphery.

2. Visual Symbology (The Web of Affinity)

When the articles load, you will notice a web of faint lines connecting various nodes. These are Structural Affinity Indicators:

  • Lines: A line indicates that two articles share at least one outgoing Wikipedia link.
  • Weight & Opacity: The visual strength of a line is mapped directly to the Jaccard Similarity Coefficient. Thick, bright lines represent strong conceptual bonds, while thin, ghostly lines represent tangential connections.
  • Topology: Clusters of densely connected lines reveal "Semantic Neighborhoods"—groups of articles that inhabit the same field of human knowledge.

3. The Orchestral Sonification Engine

The soundscape is composed of three distinct voices powered by high-quality acoustic samples (sourced from orchestral recording archives), with performance parameters driven by vector physics:

  • The Solo Cello (Bass): Rooted in the Origin node. Provides a rich tonal foundation. The pitch is transposed based on the $x$-coordinate of the gap's center.
  • The String Ensemble (Harmony): A lush pad whose pitch and pulsation rate (LFO-driven gain) are mapped to the Gap Center's $y$-coordinate. This creates a breathing atmospheric texture.
  • The Celesta (Melody): A bell-like pointillist voice that represents the "DNA" of the gap.
    • Note Selection: The sequence is generated by iterating through the characters of the Shared Links (the intersection set).
    • Rhythmic Elasticity (Jitter): The Gap Distance determines the rhythm's stability. A large vector distance (spanning dissimilar concepts) introduces "micro-timing jitter," making the rhythm feel unstable and syncopated.
    • Timbre (Brightness): The Semantic Affinity controls a low-pass filter. High-affinity gaps sound bright and sharp, whereas low-affinity gaps sound dark and diffused.

4. Saving Your Discoveries

Should you encounter a gap whose sonic signature resonates with you, use the Save to File button (the download icon). The application will use an OfflineAudioContext to high-fidelity render the 32-step sequence and export it as a 16-bit PCM WAV file directly to your device.

Technical Implementation

  • React & Tailwind: Responsive, glassmorphism interface optimized for high-density information display.
  • Web Audio API: Custom MultiVoicePlayer using recursive setTimeout for playback and OfflineAudioContext for rendering-to-file.
  • Wikipedia API: Real-time retrieval of geographic nodes and deep-link metadata for structural mapping.

Usage

  1. Anchor: Use your current location or enter specific coordinates (e.g., for a city or landmark) to generate a local semantic field.
  2. Select Origin: Tap any node in the structural map.
  3. Select Target: Tap a second node. The "Vector Bridge" will appear.
  4. Synthesize: Click "Synthesize Gap" to begin the sonification.
  5. Export: If you like the result, click the download icon to save the WAV file.

Created as a study in the intersection of knowledge topology and generative music.

Install locally:

Clone, and with node.js already installed:

npm install
npm run dev

About

topology sonification engine

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published