Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
b6daa0b
feat(psl)!: remove `directUrl` and `shadowDatabaseUrl` from schema (#…
aqrln Oct 27, 2025
44e0881
Merge branch 'main' into next
aqrln Oct 27, 2025
a06eb11
chore: fix building and testing with `next` branch of prisma (#5667)
aqrln Oct 28, 2025
5d7e37b
Merge branch 'main' into next
aqrln Oct 28, 2025
5c75710
feat!: remove deprecated `metrics` preview feature (#5665)
aqrln Oct 28, 2025
3ab778d
chore: update binaryen to v124 (#5666)
aqrln Oct 28, 2025
94d4927
feat: stop supporting Postgres 9.6, 10, 11
jkomyno Oct 29, 2025
ae8f137
feat: stop supporting MySQL 5.6
jkomyno Oct 29, 2025
f46797a
feat: support more recent CockroachDB versions
jkomyno Oct 30, 2025
6e7f7fe
chore: fix leftover from PostgreSQL 9, 10, 11
jkomyno Oct 30, 2025
bc2d133
chore: fix cockroachdb port for quaint tests
jkomyno Oct 30, 2025
b29052e
chore: fix cockroachdb user for quaint tests
jkomyno Oct 30, 2025
e8ca086
fix(schema-engine): postgres 16 snapshots
jkomyno Oct 30, 2025
0afa923
fix: attempted fix of cockroachdb URLs
jkomyno Oct 30, 2025
20b7cf9
fix: attempted fix of cockroachdb URLs
jkomyno Oct 30, 2025
2a401b4
[integration]
jkomyno Oct 30, 2025
8f44482
Merge branch 'main' into next
aqrln Oct 30, 2025
ac95b58
chore(deps): bump indoc from 2.0.6 to 2.0.7 (#5676)
dependabot[bot] Oct 30, 2025
020b0fb
fix(schema-engine): cockroachdb
jkomyno Oct 30, 2025
2f59862
Merge branch 'next' into feat/bump-database-versions-2
jkomyno Oct 30, 2025
bc04346
fix(schema-engine): fix some cockroachdb errors
jkomyno Oct 30, 2025
f00d01c
Merge branch 'feat/bump-database-versions-2' of github.com:prisma/pri…
jkomyno Oct 30, 2025
3a8386f
fix(schema-engine): fix some cockroachdb errors
jkomyno Oct 30, 2025
3a81b42
fix(schema-engine): cockroachdb migration tests
jkomyno Oct 31, 2025
1f9a8c0
fix(schema-engine): fixes?
jkomyno Oct 31, 2025
2c6d587
fix(schema-engine): cockroachdb migration tests
jkomyno Oct 31, 2025
807cf5d
fix(schema-engine): cockroachdb migration tests
jkomyno Oct 31, 2025
74e6e42
fix(schema-engine): cockroachdb 25 fixes
jkomyno Nov 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-engines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ on:
- '*.*.x'
- 'integration/*'
paths-ignore:
- '!.github/workflows/build-engines*'
- '.github/**'
- '!.github/workflows/build-engines*'
- '*.md'
- 'LICENSE'
- 'CODEOWNERS'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-prisma-schema-wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ on:
- main
pull_request:
paths-ignore:
- '.github/**'
- '!.github/workflows/build-prisma-schema-wasm.yml'
- '!.github/workflows/include/rust-wasm-setup/action.yml'
- '.github/**'
- '*.md'
- 'LICENSE'
- 'CODEOWNERS'
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/include/rust-wasm-setup/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ runs:
uses: jaxxstorm/action-install-gh-release@v1.14.0
with:
repo: WebAssembly/binaryen
tag: version_122
binaries-location: binaryen-version_122/bin
tag: version_124
binaries-location: binaryen-version_124/bin
cache: true

- name: Install bc
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-compilation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ name: 'All crates'
on:
pull_request:
paths-ignore:
- '.github/**'
- '!.github/workflows/test-compilation.yml'
- '!.github/workflows/test-compilation-template.yml'
- '.github/**'
- '*.md'
- 'LICENSE'
- 'CODEOWNERS'
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test-quaint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
pull_request:
paths:
- 'quaint/**'
- '!.github/workflows/test-quaint.yml'
- '.github/workflows/test-quaint.yml'

jobs:
tests:
Expand All @@ -23,7 +23,7 @@ jobs:
TEST_MYSQL_MARIADB: 'mysql://root:prisma@localhost:3308/prisma'
TEST_PSQL: 'postgres://postgres:prisma@localhost:5432/postgres'
TEST_MSSQL: 'jdbc:sqlserver://localhost:1433;database=master;user=SA;password=<YourStrong@Passw0rd>;trustServerCertificate=true;isolationLevel=READ UNCOMMITTED'
TEST_CRDB: 'postgresql://prisma@127.0.0.1:26259/postgres'
TEST_CRDB: 'postgresql://root@127.0.0.1:26259/postgres'

steps:
- uses: actions/checkout@v4
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-query-compiler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:
- '.github/**'
- '!.github/workflows/test-query-compiler.yml'
- '!.github/workflows/test-query-compiler-template.yml'
- '!.github/workflows/include/rust-wasm-setup/action.yml'
- '*.md'
- 'LICENSE'
- 'CODEOWNERS'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-query-engine-black-box.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ on:
- main
pull_request:
paths-ignore:
- '!.github/workflows/test-query-engine-black-box.yml'
- '.github/**'
- '!.github/workflows/test-query-engine-black-box.yml'
- '*.md'
- 'LICENSE'
- 'CODEOWNERS'
Expand Down
15 changes: 6 additions & 9 deletions .github/workflows/test-query-engine.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,6 @@ jobs:
- name: 'mysql_5_7'
version: '5.7'
relation_load_strategy: '["query"]'
- name: 'mysql_5_6'
version: '5.6'
relation_load_strategy: '["query"]'
- name: 'mysql_mariadb'
version: 'mariadb'
relation_load_strategy: '["query"]'
Expand All @@ -109,12 +106,12 @@ jobs:
fail-fast: false
matrix:
database:
- name: 'cockroach_23_1'
version: '23.1'
- name: 'cockroach_22_2'
version: '22.2'
- name: 'cockroach_22_1_0'
version: '22.1'
- name: 'cockroach_24_3'
version: '24.3'
- name: 'cockroach_25_1'
version: '25.1'
- name: 'cockroach_25_2'
version: '25.2'
name: 'CockroachDB v${{ matrix.database.version}}'
uses: ./.github/workflows/test-query-engine-template.yml
with:
Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/test-schema-engine.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,6 @@ jobs:
database:
- name: mssql_2019
url: 'sqlserver://localhost:1433;database=master;user=SA;password=<YourStrong@Passw0rd>;trustServerCertificate=true;socket_timeout=60;isolationLevel=READ UNCOMMITTED'
- name: mysql_5_6
url: 'mysql://root:prisma@localhost:3309'
- name: mysql_5_7
url: 'mysql://root:prisma@localhost:3306'
- name: mysql_mariadb
Expand All @@ -108,12 +106,14 @@ jobs:
url: 'postgresql://postgres:prisma@localhost:5437'
- name: postgres15
url: 'postgresql://postgres:prisma@localhost:5438'
- name: cockroach_23_1
url: 'postgresql://prisma@localhost:26260'
- name: cockroach_22_2
url: 'postgresql://prisma@localhost:26259'
- name: cockroach_22_1_0
url: 'postgresql://prisma@localhost:26257'
- name: postgres16
url: 'postgresql://postgres:prisma@localhost:5439'
- name: cockroach_24_3
url: 'postgresql://root@localhost:26260'
- name: cockroach_25_1
url: 'postgresql://root@localhost:26261'
- name: cockroach_25_2
url: 'postgresql://root@localhost:26262'
- name: vitess_8_0
url: 'mysql://root:prisma@localhost:33807/test'
shadow_database_url: 'mysql://root:prisma@localhost:33808/shadow'
Expand Down
2 changes: 0 additions & 2 deletions .test_database_urls/cockroachdb_22_1

This file was deleted.

2 changes: 0 additions & 2 deletions .test_database_urls/cockroachdb_22_2

This file was deleted.

2 changes: 0 additions & 2 deletions .test_database_urls/cockroachdb_23_1

This file was deleted.

2 changes: 2 additions & 0 deletions .test_database_urls/cockroachdb_24_3
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export TEST_DATABASE_URL="postgresql://root@localhost:26260"
unset TEST_SHADOW_DATABASE_URL
2 changes: 2 additions & 0 deletions .test_database_urls/cockroachdb_25_1
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export TEST_DATABASE_URL="postgresql://root@localhost:26261"
unset TEST_SHADOW_DATABASE_URL
2 changes: 2 additions & 0 deletions .test_database_urls/cockroachdb_25_2
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export TEST_DATABASE_URL="postgresql://root@localhost:26262"
unset TEST_SHADOW_DATABASE_URL
2 changes: 0 additions & 2 deletions .test_database_urls/mysql_5_6

This file was deleted.

185 changes: 185 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# Agent Playbook — Prisma Engines

## 1. Big Picture
- This repo hosts the **Prisma Engines**: PSL (schema parser/validator), schema-engine (migrate, introspect), query components (legacy query engine, new query compiler), driver adapters, and utilities shared with Prisma Client.
- Prisma 7 roadmap status:
- `directUrl` and `shadowDatabaseUrl` are **invalid** in PSL.
- `url` remains temporarily (legacy query engine still consumes it); removal is a follow-up.
- CLI/tests override connection info via schema-engine CLI (`--datasource`) or shared `TestApi::new_engine_with_connection_strings`.
- Reference commit: `34b5a692b7bd79939a9a2c3ef97d816e749cda2f` (driver adapter override plumbing).
- Prisma is deprecating the **native Rust query engine** in favor of the **Query Compiler (QC)** architecture:
- Query planning happens in Rust (`query-compiler` crate). Output: an expression tree (“query plan”).
- Query interpretation/execution runs in Prisma Client TypeScript using driver adapters. The interpreter has no knowledge of connection strings or even whether it talks to a real DB.
- A compatibility harness (`qc-test-runner.ts` in the main repo) emulates legacy query engine behavior for the test suite until QE removal is complete.
- MongoDB support is not yet implemented for QC; Prisma 7 will ship without MongoDB, to be added later.

---

## 2. Repository Orientation
Key directories:
- `psl/` – Prisma Schema Language parser, validator, config tooling.
- `schema-engine/` – Migration/introspection engine plus test suites.
- `prisma-fmt/` – Language server & formatter entry point (tests rely on `expect!` snapshots).
- `schema-engine/sql-migration-tests` / `sql-introspection-tests` – Heavy integration suites (require DBs).
- `query-engine/` – Legacy query execution stack (Rust).
- `query-compiler/` – New query planner + associated WASM + playground.
- `libs/` – Shared libraries (value types, driver adapters, test setup).
- `driver-adapters/` – Rust-side adapter utilities for the new query interpreter.

Supporting infra:
- Tests use Rust `cargo test`. Some suites expect database URLs in env (see §5).
- `test-setup` crate provisions databases when env vars are defined (Docker-based in CI).
- `UPDATE_EXPECT=1 cargo test …` regenerates `expect!` snapshots (common when diagnostics shift).

---

## 3. Current Domain Knowledge
### Datasource URLs
- PSL rejects `directUrl`/`shadowDatabaseUrl` with targeted diagnostics (`DatamodelError::new_datasource_*_removed_error`).
- Parser still records `url` (and uses span for override fallbacks).
- `Datasource::override_urls()` now fakes spans because overrides bypass PSL parsing.
- Schema-engine tests must supply overrides via `TestApi::new_engine_with_connection_strings(connection_string, Some(shadow_connection))`. The wrapper returns an `EngineTestApi`.
- Old fixtures relying on `directUrl` inside PSL must be rewritten or deleted.
- Query compiler already assumes datasource URLs are supplied externally (from Prisma Client).

### Text Completions
- `prisma-fmt` completions now only offer `url` (no more direct/shadow suggestions).
- Completion scenarios removed for the deprecated properties. Expect JSON fixtures to change if docs/completions change again.

### Diagnostics / Tests
- Many tests assert on colored output via `expect!`. Always regenerate expectations when diagnostics wording changes.
- Integration tests around multi-schema migrations still need real DB URLs; without them they skip/fail early.
- Query compiler tests use insta snapshots (`query-compiler/tests`). Regenerate with `UPDATE_EXPECT=1 cargo test -p query-compiler`.
- Query engine connector tests rely on `cargo insta` snapshots too (see `connector-test-kit-rs` README).

---

## 4. Typical Workflows
### Linting / Formatting
- Rustfmt + cargo fmt (standard). JSON fixtures kept raw (no formatter).
- Full lint pass (formatting + clippy warnings as errors):
```bash
make pedantic
```
This runs `cargo fmt -- --check` and `cargo clippy --all-features --all-targets -Dwarnings`. Fix the compiler/clippy diagnostics first, then formatting.

### Running Tests
1. **Fast PSL/LSP suites**
```bash
cargo test -p prisma-fmt -F psl/all
```
Use `UPDATE_EXPECT=1` to refresh snapshots.

2. **Unit tests in PSL**
```bash
cargo test -p psl -F all
```

3. **Unit tests for the whole workspace**
```bash
make test-unit
```
Use this one if you can't figure out the correct cargo features for a specific crate.
Some library crates may be tricky to compile in isolation without feature unification.
Unit tests are very fast so there's no problem running them for the whole workspace.
Note that `cargo test` for the whole workspace won't work because of the Node-API
symbol dependencies in the `query-engine-node-api` crate, use the makefile target.

3. **Schema engine SQL tests**
Require DB env vars (see `.test_database_urls/` in repo root). Example:
```bash
source .test_database_urls/postgres
cargo test -p sql-migration-tests migration_with_shadow_database -- --nocapture
```

4. **Schema engine integration**
Similar pattern; rely on generated DB URLs. Without env vars tests will refuse to run (by design).

5. **Query compiler snapshots**
```bash
UPDATE_EXPECT=1 cargo test -p query-compiler
```
Graphviz (`dot`) optional; set `RENDER_DOT_TO_PNG` for visuals (requires Graphviz installed).

6. **Query engine connector tests**
```bash
make dev-postgres15 # or appropriate make target to spin up DB & config
cargo test -p query-engine-tests -- --nocapture
```
Requires `.test_config` or env vars; see `query-engine/connector-test-kit-rs/README.md`.

7. **Query engine Node API / C-ABI builds**
- Node addon: `cargo build -p query-engine-node-api`.
- C-ABI (used by React Native): `cargo build -p query-engine-c-abi`.

### Updating expect! snapshots
```bash
UPDATE_EXPECT=1 cargo test -p prisma-fmt [optional::test::path]
```
Ensure diffs make sense and rerun without `UPDATE_EXPECT` to confirm.

---

## 5. Environment Essentials
- **Databases**: env vars follow `TEST_DATABASE_URL`, `TEST_SHADOW_DATABASE_URL`, etc. Use the `.test_database_urls/` helper scripts or docker-compose setup from team docs.
- **Linear tickets**: two key Prisma 7 projects – *Breaking Changes* and *New Features*. Search via Linear MCP server if context needed.
- **Feature flags**: driver adapters live behind configuration (`prisma.config.ts` with `engine: 'classic' | 'js'`). Schema engine CLI accepts `--datasource` JSON payload – reuse the structure from commit `34b5a69…`.
- **Graphviz (`dot`)**: optional but useful for rendering query graphs (required if `RENDER_DOT_TO_PNG` set in QC tests/playground).
- **Node.js**: required when working with query-engine Node bindings or QC interpreter harness.
- **Docker**: used for local DBs via `docker-compose.yml`; make targets (`make dev-postgres15`, `make start-mongo6`, etc.) orchestrate containers + config files.

---

## 6. Common Gotchas
- Running prisma-fmt tests after updating diagnostics **without** refreshing expect files will cause failures. Always run with `UPDATE_EXPECT=1`.
- Some fixtures expect **CRLF** endings (`create_missing_block_composite_type_crlf`). Avoid rewriting line endings when not necessary. If Git warns, restore file from `HEAD`.
- Integration tests bail with “Missing TEST_DATABASE_URL”. Set env vars or skip running them locally.
- `TestApi` inside `sql-migration-tests` exposes `new_engine_with_connection_strings`; use it to pass overrides.
- When touching overrides, update both PSL and schema-engine sides; they share assumptions about spans and optional URLs.
- Query compiler shares substantial code with query engine (e.g., `query_core`, `query_structure`). Changes in shared crates affect both paths—be mindful of feature flags.
- Many query engine tests still assume native QE; the `qc-test-runner` harness (in main repo) ensures QC behaves like QE for now. Expect follow-up cleanup once QE removal completes.
- MongoDB currently runs only on legacy QE; QC MongoDB support is pending. Avoid regressing existing QE tests until QC parity is achieved.

---

## 7. Useful Commands & Snippets
- Show diff for specific file:
`git diff path/to/file.rs`
- Re-run single Rust test:
`cargo test -p prisma-fmt validate::tests::validate_direct_url_direct_empty -- --nocapture`
- Search for legacy attributes:
`rg "directUrl"`, `rg "shadowDatabaseUrl"`
- Build schema-engine CLI:
`cargo build -p schema-engine-cli`
- Build query compiler WASM:
`make build-qc-wasm`
- Build legacy query engine binary:
`cargo build -p query-engine`
- Query compiler playground (generate plan + graph):
`cargo run -p query-compiler-playground`

Prefer using Makefile targets that take care of setting up the environment correctly or running prerequisite commands.

---

## 8. Open Themes / Future Tasks
- Removing `url` from PSL will be a follow-up; expect similar pattern (PSL error + override path).
- Query engine removal: remaining QE dependencies/tests need to migrate to QC or be deleted once QE is gone.
- Additional schema-engine tests may need migration to the new override helper.
- Documentation updates (internal + public) should mirror code changes; check when editing diagnostics to keep docs consistent.
- Query compiler MongoDB support: implement translation path + driver adapters, update tests once ready.
- Post-QE cleanup: strip QE-specific branches in shared crates (`query_core`, `query_structure`), simplify driver adapter plumbing.

---

## 9. External References
- Prisma Config (`prisma.config.ts`) implementation lives in the main Prisma repo (`@prisma/config` package).
- Linear roadmap items for Prisma 7 (Breaking Changes, New Features) hold context.
- Commit `34b5a69…` – canonical example for datasource override wiring.
- Query engine connector test guide: `query-engine/connector-test-kit-rs/README.md`.
- QC playground usage: `query-compiler/query-compiler-playground/`.
- QC harness in Prisma repo: `packages/cli/src/__tests__/queryCompiler/qc-test-runner.ts` (mirrors QE behavior).

---

**When modifying anything involving diagnostics or fixtures:** run relevant tests, refresh expectations, and ensure Git diffs are readable (no accidental CRLF/encoding swaps). Keep this file updated whenever we discover new traps.***
Loading
Loading