npm · WebAssembly · noncommercial

life2film-engine

The analysis behind Life2Film, as a package. Finds the beat in music, the cuts in a video, and scores frames for picture quality — in a browser tab or in Node, with no server, no API key and nothing uploaded.

npm i life2film-engine

View on npm See it running →

What it is for

Cut to music

Get the tempo and every beat position, then place edits where the music turns instead of at fixed intervals.

Find the shots

Detect where a video changes shot, so clips begin at real boundaries rather than on a stopwatch.

Rank the footage

Score frames for sharpness, exposure, contrast and detail, and find what is worth keeping in a long recording.

Skill for coding agents

Paste this into Claude Code, Cursor or any agent that reads skills. It covers the API, how to get frames and PCM in a browser, the exports to editors — and the two mistakes that cost the most time.

Download

Read it here

Loading…

It also ships inside the package at node_modules/life2film-engine/SKILL.md, so an agent that installed it can find it without this page.

With Remotion

The natural pairing: this engine decides when things happen, Remotion decides what they look like. Analysis in Node writes a beat grid; a React composition reads it and renders a real MP4. Both halves run locally.

Analyse

import init, { detect_beats } from 'life2film-engine';
import { readFileSync, writeFileSync } from 'node:fs';
import { createRequire } from 'node:module';

const require = createRequire(import.meta.url);
await init({
  module_or_path: readFileSync(
    require.resolve('life2film-engine/va_wasm_bg.wasm'),
  ),
});

// mono f32 PCM at 22050 Hz
const { bpm, beats } = JSON.parse(detect_beats(pcm, 22050, null));
writeFileSync('beats.json', JSON.stringify({ bpm, beats, duration }));

Render

import { useCurrentFrame, useVideoConfig, interpolate } from 'remotion';

export const BeatFlash = ({ beats, bpm }) => {
  const { fps } = useVideoConfig();
  const seconds = useCurrentFrame() / fps;

  // Which beat are we on, and how long since it fired?
  const index = beats.findLastIndex((b) => b <= seconds);
  const since = index >= 0 ? seconds - beats[index] : 1;

  // Decay from each beat — a pulse, not a blink.
  const pulse = interpolate(since, [0, 0.18], [1, 0], {
    extrapolateRight: 'clamp',
  });

  return <Circle scale={1 + pulse * 0.35} glow={pulse} />;
};

Verified end to end, not sketched: a 128 BPM click track reads back as 128.4 BPM, and in the rendered MP4 the centre pixel measures 154 → 108 → 77 between beats, rising to 189 and 210 on the next two. The pulses land on the grid. npx remotion render produces the file; nothing is uploaded at any stage.

What it does not do

It does not decode or encode. Bring your own frames and PCM — in a browser, <video> plus a canvas and AudioContext.decodeAudioData are enough, and the tool pages show exactly that. For writing files, mediabunny is excellent and MIT licensed.

It reads no speech and tracks no faces. It looks at pictures and sound, which is what makes it useful for footage where nobody is talking.

Exports

`detect_beats`	Tempo and the position of every beat, from mono PCM.
`analyze_audio`	Silence, sections, spectral flatness, content type.
`beat_sync_timeline`	Place cuts on the beat.
`detect_scenes_slick`	Shot boundaries from centre pixels — 16 algorithms.
`detect_scenes_content`	Shot boundaries from HSV frame deltas.
`detect_scenes_features`	Shot boundaries from N-dimensional feature vectors.
`score_frame`	31 measurements per frame, as one score plus a garbage flag.
`score_segments`	Rank segments by the frames inside them.
`select_segments`	Choose which segments make the cut, under a duration budget.
`compose_montage`	Assemble a timeline from scored segments.
`build_otio`	Write OpenTimelineIO.
`parse_otio`	Read OpenTimelineIO.

Nineteen in total, typed in va_wasm.d.ts. Every one returns a JSON string.

Licence

PolyForm Noncommercial 1.0.0. Free for personal projects, study, research, charities, educational institutions and public bodies.

Anything commercial — including internal use inside a company — needs a separate licence. Write to [email protected]. This is a real offer rather than a formality: the licence is noncommercial so that the terms are a conversation instead of an assumption.

The package ships compiled WebAssembly. The Rust it is built from is not public.