Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
431 lines (294 loc) · 12 KB

CONTRIBUTING.md

File metadata and controls

431 lines (294 loc) · 12 KB
Raw

Contributing to VIDEOContext

Thank you for your interest in contributing to VIDEOContext.

VIDEOContext is an open-source project focused on making video easier for AI systems to understand and use. The project transforms video into structured, searchable, temporal, and task-optimized context.

Contributions of all kinds are welcome, including bug fixes, documentation improvements, tests, new integrations, processing improvements, semantic profiles, and new ideas.

Before You Start

Before making significant changes, please take a moment to understand the existing architecture and codebase.

If you are planning a large feature, architectural change, or major refactor, it is recommended to first open an issue describing the proposed change. This helps avoid duplicate work and allows discussion before significant implementation effort is invested.

For smaller fixes and improvements, feel free to open a pull request directly.

Ways to Contribute

You can contribute to VIDEOContext in several ways.

Report Bugs

If you find a bug, please open an issue and include as much useful information as possible.

Helpful information includes:

  • A clear description of the problem
  • Steps to reproduce the issue
  • Expected behavior
  • Actual behavior
  • Relevant error messages or logs
  • Python version
  • Operating system
  • VIDEOContext version or commit

Before opening a new issue, please check whether the problem has already been reported.

Suggest Features

Feature ideas are welcome.

VIDEOContext is evolving around the idea of converting video into useful representations for AI, including:

  • Searchable video context
  • Timestamped evidence
  • OCR and transcript extraction
  • Temporal understanding
  • Scene and event understanding
  • Semantic profiles
  • UI and design analysis
  • Application interaction understanding
  • Context compression
  • Task-aware context selection
  • AI agent and MCP integrations

If you have an idea that could improve the project, open an issue describing:

  • The problem you are trying to solve
  • Your proposed solution
  • Possible alternatives, if relevant
  • How the feature could fit into VIDEOContext

A detailed implementation plan is not required.

Improve Documentation

Documentation contributions are extremely valuable.

You can help improve:

  • Installation instructions
  • Usage examples
  • API documentation
  • SDK documentation
  • MCP documentation
  • CLI documentation
  • Architecture explanations
  • Tutorials
  • Examples
  • Typographical errors and clarity

If something in the project is difficult to understand, that is often a good sign that the documentation can be improved.

Improve Video Processing

Contributions related to video processing are welcome.

Possible areas include:

  • Frame sampling
  • Scene detection
  • OCR
  • Speech extraction
  • Transcription
  • Object detection
  • Event detection
  • Visual understanding
  • Temporal segmentation
  • Semantic deduplication
  • Processing performance

The goal should generally be to improve useful information extraction while avoiding unnecessary or repetitive processing.

Build Semantic Profiles

VIDEOContext is moving toward supporting multiple semantic representations of video.

Examples of potential profiles include:

  • UI and website design analysis
  • Application interaction analysis
  • Product demonstrations
  • Educational videos
  • Tutorials
  • Meetings
  • Presentations
  • Gameplay
  • General video understanding

A semantic profile should ideally transform lower-level video evidence into a reusable structured representation.

Profiles should remain traceable to their source evidence where possible.

Improve Context Routing

One of the long-term goals of VIDEOContext is to determine what type of video context an AI actually needs for a particular task.

For example:

Question:
"What was the revenue?"

Context:
Relevant transcript or OCR evidence

Task:
"Recreate this website design."

Context:
Design profile
+
Representative visual evidence
+
Layout patterns
+
Interaction patterns

Potential contributions in this area include:

  • Task classification
  • Context selection
  • Context budgeting
  • Token estimation
  • Context compression
  • Evidence ranking
  • Representative frame selection

Improve Integrations

VIDEOContext can benefit from additional integrations.

Potential areas include:

  • AI models
  • Vision providers
  • Speech providers
  • Vector databases
  • Agent frameworks
  • MCP clients
  • Developer tools
  • Web frameworks

Please try to keep integrations modular and avoid unnecessarily coupling the core architecture to a single provider.

Add a Source Adapter

New video origins (object storage, platforms, catalogs) plug in behind the source capability without touching the processing engine:

  1. Implement the SourceAdapter protocol (can_handle / inspect / materialize / cleanup) in src/videocontent/sources/ — pure acquisition, never extraction.
  2. Register it (@register_source("my-provider")) with lazy imports so optional dependencies stay optional.
  3. Resolve through videocontent.sources.resolve so the SSRF/redirect/limits boundary applies; never fetch URLs with a separate code path.
  4. Add tests: resolution, canonical identity, inspection, and adversarial cases (private IP, redirect-to-private, oversized, wrong content-type).
  5. Document the source as supported only when it actually works end to end.

Add a Temporal View or Entity Detector

Derived intelligence (chapters, changes, states, entities) lives in src/videocontent/temporal.py / src/videocontent/entities.py as pure functions over the document:

  1. Copy timestamps from facts — never generate them. Instant spans use end == start.
  2. Cite evidence: every object carries reference IDs to the facts behind it.
  3. Flag inference: derived titles, heuristic links, and low-confidence calls must say so (inferred, ambiguous, confidence) rather than read as ground truth.
  4. Keep it cheap: no models, no network, no new required dependencies.
  5. Add unit tests on hand-built documents, including empty and missing-modality cases.

Add a Graph Relation, Intent, or MCP Tool

  • Graph relations go in src/videocontent/graph.py with exactly one documented construction rule per edge, provenance IDs, and a rule string a human can audit. Temporal adjacency is TEMPORAL_SEQUENCE, never causation.
  • Query intents go in src/videocontent/queryplan.py with keyword rules, a retrieval strategy, graph operations, and a fallback. Every new temporal phrasing needs a parser test and an executor test.
  • MCP tools must be read-only, validate arguments, cap output (_bounded), preserve video IDs and timestamps, and never trigger processing. Mirror every tool with a CLI command or SDK method that the tool delegates to — no logic in the transport layer.

Development Setup

Start by cloning your fork of the repository.

git clone https://github.com/AAGAM17/VIDEOContext.git
cd VIDEOContext

Create a new branch for your work.

git checkout -b feature/my-feature

Install the project dependencies according to the setup instructions in the repository README.

Before making changes, verify that the existing project runs correctly in your environment.

Making Changes

When contributing code:

  • Keep changes focused
  • Avoid unrelated refactoring
  • Follow existing project structure where practical
  • Prefer clear and maintainable code
  • Add tests when appropriate
  • Update documentation when behavior changes
  • Preserve backwards compatibility where possible

For larger changes, consider breaking the work into smaller pull requests.

Preserve Existing Functionality

VIDEOContext already contains multiple components and interfaces.

When adding new features, avoid breaking existing functionality unless the change is explicitly intended and documented.

Where applicable, preserve existing:

  • Processing pipelines
  • Search behavior
  • APIs
  • SDK interfaces
  • CLI behavior
  • MCP functionality
  • VCTX compatibility

New functionality should generally extend the project rather than unnecessarily replacing working functionality.

Code Quality

Please keep contributions readable and maintainable.

Prefer:

  • Clear naming
  • Small focused functions
  • Explicit behavior
  • Reusable components
  • Meaningful error messages
  • Minimal duplication

Avoid adding complexity when a simpler solution is sufficient.

Testing

If your contribution changes behavior, please add or update tests when practical.

Examples of areas that may benefit from testing include:

  • Video processing
  • Search and retrieval
  • Context routing
  • Task classification
  • Semantic profile generation
  • Context budgeting
  • API behavior
  • CLI behavior
  • MCP tools
  • VCTX serialization and compatibility

Before opening a pull request, run the relevant tests available in the repository.

Pull Requests

When opening a pull request, please provide:

  • A clear title
  • A concise description of the change
  • The problem being solved
  • Relevant testing information
  • Any important limitations or follow-up work

Keep pull requests focused whenever possible.

Large pull requests that combine unrelated changes are more difficult to review.

Pull Request Checklist

Before submitting a pull request, check the following:

  • My changes are focused on a specific improvement
  • I have tested the relevant functionality
  • I have updated documentation where necessary
  • I have not intentionally broken existing functionality
  • I have added or updated tests where practical
  • My code follows the existing project structure where possible

Architecture Principles

When contributing to VIDEOContext, keep the following principles in mind.

Evidence First

High-level understanding should remain traceable to video evidence whenever possible.

A semantic conclusion should ideally be connected to:

  • A timestamp
  • A video segment
  • A frame
  • A transcript span
  • OCR output
  • Other source evidence

Avoid Redundant Context

Video contains large amounts of repeated information.

A useful representation should avoid repeatedly describing information that has not changed.

Prefer:

Persistent state
+
Meaningful changes

over:

Repeated descriptions of nearly identical frames

Task-Aware Context

Not every AI task needs the same representation of a video.

A factual question may need only a small evidence span.

A design analysis task may require global visual understanding and representative frames.

A contribution that improves the system's ability to select the appropriate context for a task is aligned with the long-term direction of VIDEOContext.

Provider Independence

Where practical, avoid tightly coupling the core system to a single external AI, vision, speech, or infrastructure provider.

Integrations should be modular where possible.

Backwards Compatibility

Existing VCTX files and existing interfaces should not be broken unnecessarily.

If a breaking change is required, document it clearly.

Commit Messages

Clear commit messages are appreciated.

Examples:

Add UI design semantic profile

Improve temporal scene deduplication

Fix OCR timestamp alignment

Avoid vague commit messages when possible.

Questions and Discussions

If you are unsure about an implementation approach, opening an issue before starting a large change is encouraged.

Questions, ideas, experiments, and discussions are welcome.

The project is still evolving, and constructive discussion can help shape its direction.

Code of Conduct

Please be respectful and constructive when interacting with other contributors.

Harassment, discrimination, personal attacks, and disruptive behavior are not welcome.

The goal is to build a collaborative environment where people can learn, experiment, and contribute.

License

By contributing to VIDEOContext, you agree that your contributions will be distributed under the same license as the project.

Thank you for contributing to VIDEOContext.

Every contribution, whether it is a bug report, documentation improvement, test, feature, integration, or discussion, helps make video more useful for AI systems.