Sialk

Playing it

The audio

What Sialk hears and what it works out from it.

Sialk listens to a system input device and analyses it. That analysis reaches every layer through the audio contract, and from v1 it drives every audio modulator.

Getting sound to the input is its own page.

input mic · loopback analysis level · bassmid · high spectrum · onsets bpm · beatPhase window.sialk the published contract your sketch reads it, and moves compositor GPU, every layer the wall
Sialk analyses the input once and hands the result to every layer through the versioned contract. Your sketch reads it directly, there is no bridge to wire up.

What it measures

Value What it is
level Broadband loudness, smoothed. 0-1.
bass Roughly 20-250 Hz. 0-1.
mid Roughly 250-2000 Hz. 0-1.
high Roughly 2000-16000 Hz. 0-1.
spectrum 64 bins, log-spaced across 20 Hz - 16 kHz.
levelTrail The last 128 frames of level, newest first.
hits A counter that increases on every onset. Never resets mid-show.
onBeat 1 on the frame a beat lands, decaying towards 0 before the next.
beatPhase Ramps 0 → 1 between beats. 0 when there is no tempo.
bpm 0 when unknown.
bpmConfidence 0-1. 0 when unknown.
silent True when no signal has been present for one second.

Growing with the contract

The contract is versioned, and additions arrive as minor versions that break nothing, the published surface is on the contract page. Two have landed since the beta was cut, and both reach the app with v1:

  • percussive and harmonic (contract 1.1), how much of this moment is drums against notes. A share, not a loudness: multiply by level for how hard is the kick.
  • stems and stemCount (contract 1.2), real per-stem input, up to eight, each with its own level, bass, mid and high. They arrive from a DAW through Sialk: Connect or from a multichannel device, and read as zeros wherever no stem input is set up, which is a true statement, not an error.

Two things worth knowing before you use them

A band is energy, not an average. The three bands report how much is happening in that part of the spectrum, which is what you actually want when you are driving a visual with a kick drum.

bpm and bpmConfidence travel together. A tempo of 128 with a confidence of 0.2 is a guess. If you are switching a whole scene on the beat, read the confidence too, and remember that beatPhase is 0 when there is no tempo at all, which is a perfectly good signal to fall back to level.

Designing for a real room

The signal from a microphone in a loud room is not the signal from a loopback device, and the difference is mostly headroom. A visual that saturates at 0.9 will look identical at 0.9 and 1.0, which means a loud room and a very loud room look the same, and the last bit of the night has nowhere to go.

Leaving the top third of your range unused is not caution, it is what makes the peak mean something.

silent is the other half of this. A show that goes to black between tracks because nothing is driving it reads as broken. Use silent to hold a resting state rather than to stop.