Code for Encode Once, Decode Never: Reusing Audio LM Internals for Efficient Temporal Localization (COLM 2026)

Setup

Environment Setup

1. First, modify the prefix argument in environment.yaml to point to your desired directory (or current directory):

   prefix: /path/to/your/desired/environment/location

2. Create the environment from the YAML file:

   conda env create -f environment.yaml

3. Activate the environment:

   conda activate ./env # or whatever you set prefix to

Download NLTK Data

Run the following Python commands to download required NLTK data:

import nltk
nltk.download('stopwords')

Datasets

The datasets can be found on HuggingFace

1. LibriSpeech
   https://huggingface.co/datasets/gilkeyio/librispeech-alignments

2. LibriCount
   https://huggingface.co/datasets/enyoukai/libricount-timings

3. AudioSet
   https://huggingface.co/datasets/enyoukai/audioset-humans-reprocessed