Added everything

.github/workflows/pre-merge-checks.yml

@@ -0,0 +1,46 @@
+name: Pre-merge Checks
+
+on:
+  pull_request:
+    branches:
+      - main
+  workflow_dispatch:
+
+jobs:
+  checks:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.9'
+
+      - name: Install Poetry 2.1.2
+        run: |
+          curl -sSL https://install.python-poetry.org | python3 - --version 2.1.2
+          poetry config virtualenvs.create true
+          poetry config virtualenvs.in-project true
+
+      - name: Load cached venv
+        id: cached-poetry-dependencies
+        uses: actions/cache@v3
+        with:
+          path: .venv
+          key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
+
+      - name: Install dependencies
+        if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
+        run: poetry install --no-interaction --no-root --all-groups
+
+      - name: Install pre-commit
+        run: |
+          pip install pre-commit
+          pre-commit install
+
+      - name: Run pre-commit hooks
+        run: pre-commit run --all-files --show-diff-on-failure
+
+      - name: Run tests
+        run: poetry run pytest tests/ -vvv

.gitignore

@@ -0,0 +1,49 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+.ruff_cache/
+.mypy_cache/
+
+# Virtual Environment
+.env
+.venv
+env/
+venv/
+ENV/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+.DS_Store
+
+# Testing
+.coverage
+htmlcov/
+.pytest_cache/
+.tox/
+coverage.xml
+*.cover
+
+# Project specific
+*.log
+.cache/

.pre-commit-config.yaml

@@ -0,0 +1,42 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-added-large-files
+      - id: check-ast
+      - id: check-json
+      - id: check-merge-conflict
+      - id: detect-private-key
+
+  - repo: local
+    hooks:
+      - id: ruff
+        name: ruff
+        entry: poetry run ruff check . --fix
+        language: system
+        types: [python]
+        pass_filenames: false
+
+      - id: ruff-format
+        name: ruff-format
+        entry: poetry run ruff format .
+        language: system
+        types: [python]
+        pass_filenames: false
+
+      - id: mypy
+        name: mypy
+        entry: poetry run mypy .
+        language: system
+        types: [python]
+        pass_filenames: false
+
+      - id: poetry-lock
+        name: poetry lock check
+        entry: poetry check --lock
+        language: system
+        types: [toml]
+        pass_filenames: false

CONTRIBUTING.md

@@ -0,0 +1,112 @@
+# Contributing to AI Data Stream
+
+Thank you for your interest in contributing to AI Data Stream! This project implements the Vercel AI Data Stream protocol, which enables streaming AI chat responses to clients. There are two main types of contributions you can make:
+
+## Framework Requirements
+
+Regardless of whether you're implementing an agent framework or an API framework, you must:
+
+1. Add a dependency group in `pyproject.toml` for your framework:
+   ```toml
+   [tool.poetry.group.your-framework]
+   optional = true
+
+   [tool.poetry.group.your-framework.dependencies]
+   your-framework = ">=1.0.0,<2.0.0"
+   ```
+
+2. Create a README.md file in your framework's directory (e.g., `ai_datastream/agent/your-framework/README.md` or `ai_datastream/api/your-framework/README.md`) that includes:
+   - Installation instructions
+   - Basic usage examples
+   - Any framework-specific configuration or requirements
+   - Links to relevant documentation
+
+## 1. Agent Framework Contributions
+
+Agent frameworks are responsible for handling the AI agent's logic and streaming its responses. Currently, we support LangGraph as an agent framework.
+
+### Implementing a New Agent Framework
+
+To implement a new agent framework, you'll need to:
+
+1. Create a new directory under `ai_datastream/agent/` for your framework
+2. Implement at least one of the following interfaces (implementing both is highly preferred):
+   ```python
+   class Streamer(abc.ABC):
+       @abc.abstractmethod
+       def stream(self, prompt: str, messages: Sequence[ChatMessage]) -> Generator[DataStreamPart, None, None]:
+           pass
+
+   class AsyncStreamer(abc.ABC):
+       @abc.abstractmethod
+       def async_stream(self, prompt: str, messages: Sequence[ChatMessage]) -> AsyncGenerator[DataStreamPart, None]:
+           pass
+   ```
+
+Your implementation should handle:
+- Message parsing and conversion
+- Tool calls and their results
+- Streaming text responses
+- Proper start and finish steps for each message
+
+Note: While the LangGraph implementation uses additional classes like message parsers and stream converters, these are implementation details and not mandatory for new agent frameworks. You can structure your implementation in whatever way best suits your framework's needs, as long as you implement at least one of the above interfaces.
+
+## 2. API Framework Contributions
+
+API frameworks provide the web interface for the streaming functionality. Currently, we support FastAPI as an API framework.
+
+### Implementing a New API Framework
+
+To implement a new API framework, you'll need to implement all the necessary code to handle requests in the following format:
+
+```
+# Request
+POST <URL>
+Content-Type: application/json
+{
+    "messages": [{"role": "user", "content": "Hello, how are you?"}]
+}
+
+# Response
+Content-Type: text/event-stream
+Transfer-Encoding: chunked
+x-ai-data-stream: 1
+{type}:{data}
+{type}:{data}
+...
+```
+
+To do so, you'll need to:
+1. Offer a request type that converts from the framework's request format to our internal `ChatMessage` format
+2. Offer a way to stream the response of the `Streamer` or `AsyncStreamer` implementation (preferably both)
+3. Set the headers to the correct values
+
+## Development Setup
+
+1. Install dependencies:
+   ```bash
+   poetry install
+   ```
+
+2. Install development dependencies:
+   ```bash
+   poetry install --with dev
+   ```
+
+3. Run tests and linting:
+   ```bash
+   poetry run ruff check .
+   ```
+
+## Pull Request Process
+
+1. Fork the repository
+2. Create a new branch for your feature
+3. Make your changes
+4. Add tests for your changes
+5. Ensure all tests pass and linting is clean
+6. Submit a pull request with a clear description of your changes
+
+## License
+
+By contributing to this project, you agree that your contributions will be licensed under the project's Apache 2.0 license.