Skip to content

[P1] Modulate sounds with imported audio or saved clips #20

Open

Description

@nosam1998

Parent plan: #1
Priority / gate: P1 / M2
Requirement: F18 in the PRD
Design: Technical design
Depends on: #19, #11, #12

Outcome

Use an uploaded or saved clip to shape another sound's amplitude, ring-modulated timbre, or pitch, then inspect, export, and save the result.

Scope

  • Add Carrier and Modulator selectors: carrier is a selected oscillator, current finite Timeline/mix, or frozen/imported clip; modulator is an F17 audio asset. Propose one active route/effect per project.
  • Provide volume follow (rectified attack/release envelope), bipolar ring modulation with dry/wet depth, and exponential pitch modulation in semitones.
  • Integrate effective oscillator frequency into continuous phase; use an integrated source readhead and measured antialias resampling for explicitly labeled clip varispeed.
  • Expose depth/bypass/A–B, numeric trim/start offset, independent modulator/carrier looping, attack/release, output duration, and source/result inspection while preserving oscillator colors and workspaces.
  • Use finite worker rendering with source-revision/asset/settings cache keys, progress/cancellation, measured oversampling/filtering, one global headroom stage, and one monitor multiplier.
  • Integrate complete function/data exports, asset-bearing project round trips, and soundboard Save/Open/Update through the foundation adapters; store both rendered output and a complete editable snapshot.

Acceptance criteria

  • Imported and saved-clip modulators work with each carrier target; only the selected route is affected, muted/excluded oscillator targets stay excluded, and the modulator is not mixed into output without intentional audition.
  • Volume follow implements c*((1-d)+de) with bounded attack/release envelope e; ring implements (1-d)c+dcm with signed m. A zero-depth render matches bypass from the same initial state.
  • A steady full-depth 440 Hz sine multiplied by a 100 Hz sine produces 340/540 Hz components of amplitude 0.5 before conditioning, with no original 440 Hz term; window/transport effects are excluded from the analytic fixture.
  • Pitch uses f_base2^(Dm/12), then integrates the effective supported frequency; constant +12 semitones doubles frequency when unclamped. Limits/clamping are reported and source settings remain intact.
  • Clip pitch uses q(t)=q0+integral(2^(D*m/12) dt); a +12-semitone plateau advances at twice the source rate. UI states that varispeed changes timing, and readhead/phase remains continuous across chunks and live bypass transitions.
  • Depth returns to zero with bounded ramps outside modulator availability; silence inside an active clip follows the selected effect. Mismatched durations, offsets, loops, source EOF, and carrier EOF follow the PRD without implicit time stretching or sudden phase reset.
  • A–B restarts from matching initial state. Stop, source changes, or newer render revisions defeat late completions; source audition obeys shared transport and never starts implicitly.
  • Near-Nyquist and maximum-depth fixtures pass the foundation's measured spectral contract. All-zero gain stays silent apart from declared boundary/filter behavior, intentional dynamics survive, and generated DC is reported without post-effect mean subtraction.
  • Saving/reloading/reopening a modulated sound preserves settings and immutable dependencies after source-pad update/removal; pad replay and WAV export apply neither the effect nor monitor gain twice.
  • Generated JS/Python with companion assets runs in a fresh environment and matches canonical numerical fixtures within 10^-6; rendered WAV follows the shared rate/conditioning/quantization contract.
  • Carrier/modulator/result and effective pitch/rate information are inspectable by labels/numeric text as well as color. Maximum four-voice/one-route rendering is measured against the 2-second render and 128 MiB app-owned buffer targets.

Validation

Independent ring-product spectrum, amplitude-follower attack/release, pitch phase and varispeed readhead references; silence/DC/depth/timing/loop fixtures; async Stop/cancel races; fresh-environment function/project round trips; matched-settings preview/export/soundboard checks; and browser/performance evidence.

Boundaries

Recursive/live feedback graphs, multiple simultaneous modulation routes, time-preserving pitch shifting, vocoding, convolution, and continuous live microphone processing are deferred. Repeated stages may be explicitly rendered to saved clips.

Implementation evidence

Volume-follow, ring and pitch/varispeed routes, immutable clip sources, bound oscillator targets, timing/loop/bypass/audition, exports and saved snapshots are implemented. Numerical and fresh-storage tests pass. Maximum-workload performance/memory qualification remains in #16.

Merged work: #23, #27, #28, #29, #30.

Current CI · Measured validation report.

Keep this issue open for the remaining performance/memory acceptance described in #16; implementation availability is not being confused with a fully qualified release.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions