Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
815e844
docs: add PostgreSQL support design spec
johnford2002 Apr 12, 2026
01b7261
docs: add PostgreSQL support implementation plan
johnford2002 Apr 12, 2026
2494232
feat(db): add DATABASE_DIALECT and PostgreSQL connection config
johnford2002 Apr 12, 2026
9717fb3
refactor(db): move SQLite migrations to migrations/sqlite/
johnford2002 Apr 12, 2026
cc8917d
refactor(db): extract relations into schema.relations.ts
johnford2002 Apr 12, 2026
4dd700a
refactor(db): rename schema to schema.sqlite.ts with entry point
johnford2002 Apr 12, 2026
d83c42d
feat(db): add PostgreSQL schema (schema.pg.ts)
johnford2002 Apr 12, 2026
c371c12
refactor(db): abstract error handling with isUniqueConstraintError
johnford2002 Apr 12, 2026
5bff0fb
feat(db): rewrite drizzle.ts as dialect factory with PostgreSQL support
johnford2002 Apr 12, 2026
7635c81
refactor(db): rename instrumentDatabase to instrumentSqliteDatabase
johnford2002 Apr 12, 2026
034fb9f
feat(db): add per-dialect Drizzle Kit configs and updated scripts
johnford2002 Apr 12, 2026
2a8595d
feat(db): add PostgreSQL baseline migration
johnford2002 Apr 12, 2026
008eb8f
feat(db): add domainFromUrl SQL helper for cross-dialect URL parsing
johnford2002 Apr 12, 2026
8c644ac
docs: add PostgreSQL configuration environment variables
johnford2002 Apr 12, 2026
cad6b69
docs: add PostgreSQL setup guide
johnford2002 Apr 12, 2026
972ff82
feat(db): add SQLite-to-PostgreSQL data migration script
johnford2002 Apr 12, 2026
bcb16fa
fix(db): resolve PostgreSQL runtime issues found during integration t…
johnford2002 Apr 12, 2026
6c2e4d4
fix(docker): update Dockerfile to copy migrations from new directory
johnford2002 Apr 12, 2026
b428bd4
fix(db): replace createRequire with dynamic import for bundler compat…
johnford2002 Apr 12, 2026
fe3cfa1
chore: remove planning docs and simplify error detection
johnford2002 Apr 12, 2026
5483ae3
fix(db): alias .changes on PG results and extract connection string h…
johnford2002 Apr 12, 2026
881cdf7
fix(db): correct listCollaborators timestamp column name in migration…
johnford2002 Apr 12, 2026
30fb179
fix(db): URL-encode credentials in PostgreSQL connection string builder
johnford2002 Apr 12, 2026
182a53b
fix(db): stabilize postgres.js integration and add graceful shutdown
johnford2002 Apr 12, 2026
cd64c52
fix(db): replace process.exit(0) with graceful close() in migrate.ts
johnford2002 Apr 12, 2026
da8ab58
fix(docker): add workers and prep to PostgreSQL compose overlay
johnford2002 Apr 12, 2026
2bf2bbc
fix: add trailing newlines to .env.sample files
johnford2002 Apr 12, 2026
64b5a89
fix(db): use DML statement for .changes verification guard
johnford2002 Apr 12, 2026
ece15e0
fix(db): add missing GROUP BY columns for PostgreSQL strict mode
johnford2002 Apr 12, 2026
71377e5
test(db): add schema sync validation between SQLite and PostgreSQL
johnford2002 Apr 12, 2026
ec547d5
fix(db): add compile-time type safety check for PostgreSQL DB cast
johnford2002 Apr 12, 2026
acc743d
Add superpowers
johnford2002 May 3, 2026
01330e3
Merge pull request #4 from johnford2002/feat/postgresql-support
johnford2002 May 18, 2026
287b3f1
chore(github): rebrand fork's workflows and templates for johnford2002
johnford2002 May 18, 2026
1c6d417
Merge pull request #6 from johnford2002/chore/fork-rebrand-github-config
johnford2002 May 19, 2026
6a37fba
Merge upstream karakeep-app/main into postgresql-supporting fork
johnford2002 May 19, 2026
03b782b
feat(db): add PostgreSQL migration for upstream schema sync
johnford2002 May 19, 2026
ffdfeed
fix(docker): pin postgres overlay to v18 and fix volume mount
johnford2002 May 19, 2026
f63457a
ci(docker): strip leading v from release image tags
johnford2002 May 19, 2026
ae2e01c
chore(workflows): namespace fork workflows and shelve upstream ones
johnford2002 May 19, 2026
6ec4953
Merge pull request #8 from johnford2002/chore/workflows-reorg
johnford2002 May 19, 2026
0c68d6a
chore: sync upstream/main #9
johnford2002 May 20, 2026
7292e0c
Merge upstream/main into fork
johnford2002 May 20, 2026
3c9ec6c
Merge pull request #11 from johnford2002/chore/sync-upstream-main
johnford2002 May 21, 2026
2e579ab
Merge pull request #12 from karakeep-app/main
johnford2002 May 21, 2026
fb1e4d6
Merge upstream/main into chore/sync-upstream-2026-06-13
johnford2002 Jun 13, 2026
a429906
Merge pull request #14 from johnford2002/chore/sync-upstream-2026-06-13
johnford2002 Jun 13, 2026
75b44d9
feat(db): add optional SQL query logging + connection lifecycle logs
johnford2002 Jun 14, 2026
098688a
Merge pull request #15 from johnford2002/feat/db-query-logging
johnford2002 Jun 14, 2026
f377696
feat(db): eagerly pre-warm the Postgres connection pool at startup
johnford2002 Jun 14, 2026
aa46f20
Merge pull request #16 from johnford2002/worktree-db-pool-warmup
johnford2002 Jun 14, 2026
3d3c4da
docs(video): spec for first-class video downloading
johnford2002 Jun 14, 2026
2fa9bb1
feat(video): backend for first-class video downloading
johnford2002 Jun 14, 2026
53e201b
feat(video): UI for first-class video downloading
johnford2002 Jun 14, 2026
be856e2
Merge pull request #17 from johnford2002/feat/video-first-class
johnford2002 Jun 14, 2026
bd74cf8
build(shared): add @anthropic-ai/sdk dependency
johnford2002 Jun 15, 2026
d3e69a1
feat(inference): add AnthropicInferenceClient text inference
johnford2002 Jun 15, 2026
23daa4d
test(inference): cover Anthropic image content block
johnford2002 Jun 15, 2026
e562e0e
test(inference): assert Anthropic embeddings are unsupported
johnford2002 Jun 15, 2026
e2e2d84
feat(inference): select Anthropic provider via ANTHROPIC_API_KEY
johnford2002 Jun 15, 2026
ad44ae8
docs: document the native Anthropic inference provider
johnford2002 Jun 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
440 changes: 440 additions & 0 deletions .claude/superpowers/plans/2026-04-12-postgresql-review-fixes.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# PostgreSQL PR Review Fixes -- Design Spec

**Date:** 2026-04-12
**PR:** #4 (feat: add PostgreSQL as configurable database backend)
**Scope:** One targeted fix from code review -- DB type safety. Plus documentation of a reviewed-and-validated design decision.

## Issue 1 (resolved -- no change needed): `schema.relations.ts` imports from `./schema.sqlite`

### Original concern

`schema.relations.ts` imports table objects from `./schema.sqlite`, meaning relations are always defined against SQLite table objects, even when PostgreSQL is active.

### Why this is correct

Drizzle resolves relations to tables by **SQL table name** (via `getTableUniqueName` which returns `"schema.tableName"`, e.g. `"public.user"`), **not** by JavaScript object identity. When `drizzle()` receives `{ ...pgSchema, ...relations }`, it matches `userRelations` (defined with SQLite's `users` table) to PG's `users` table because both share the SQL name `"public.user"`.

Changing the import to `./schema` was attempted but creates a **circular dependency**: `schema.relations.ts` -> `./schema` -> `export * from "./schema.relations"`. Node.js returns partially-initialized exports, causing `undefined` table objects at runtime.

### No change needed

The existing import from `./schema.sqlite` is the correct design.

## Issue 2: Type safety of the `BetterSQLite3Database` cast

### Problem

In `drizzle.ts:133-136`, the PostgresJsDatabase instance is cast to `BetterSQLite3Database<FullSchema>` via `as unknown as`:

```ts
export const db: BetterSQLite3Database<FullSchema> =
dialect === "postgresql"
? ((await createPostgresDB()) as unknown as BetterSQLite3Database<FullSchema>)
: ((await createSqliteDB()) as unknown as BetterSQLite3Database<FullSchema>);
```

Drizzle has no shared base type between SQLite and PG. The cast works at runtime because the query builder APIs are structurally compatible. However, the `as unknown as` erases all type checking -- a future Drizzle upgrade could break the PG instance's compatibility with the `BetterSQLite3Database` interface and TypeScript wouldn't catch it.

### Approach: keep the cast, add compile-time compatibility check

A wrapper class or union type was considered but rejected:
- **Union type** would require narrowing at all ~250 db method call sites across 47 files
- **Wrapper class** can't properly type the return values of `.select()`, `.insert()`, etc. because they return dialect-specific builder types, and the `.query` relational API is especially hard to wrap
- **Runtime cost** of either alternative is zero -- this is purely a compile-time concern

Instead, add a type-level assertion in `drizzle.ts` that verifies the PostgresJsDatabase type satisfies the key methods the codebase uses. This catches Drizzle upgrades that break structural compatibility at compile time.

### Implementation

Add a compile-time compatibility check after the type definitions in `drizzle.ts`:

```ts
import type { PostgresJsDatabase } from "drizzle-orm/postgres-js";

// Compile-time check: verify that PostgresJsDatabase exposes the same
// core methods as BetterSQLite3Database. If a Drizzle upgrade breaks
// structural compatibility, this block will produce a type error.
type _PgDB = PostgresJsDatabase<FullSchema>;
type _AssertHas<T, K extends keyof T> = K;
type _PgCheck =
| _AssertHas<_PgDB, "select">
| _AssertHas<_PgDB, "selectDistinct">
| _AssertHas<_PgDB, "insert">
| _AssertHas<_PgDB, "update">
| _AssertHas<_PgDB, "delete">
| _AssertHas<_PgDB, "query">
| _AssertHas<_PgDB, "transaction">
| _AssertHas<_PgDB, "$count">;
// Suppress unused-type warning
type _Unused = _PgCheck;
```

This is zero-cost at runtime (types are erased) and will fail `pnpm typecheck` if any of these methods are removed or renamed in a future Drizzle version.

Also improve the existing TSDoc comment on the `db` export to explain the cast rationale and point to the compatibility check.

### Files changed

- `packages/db/drizzle.ts` -- add `PostgresJsDatabase` type import, add compile-time assertion block, improve TSDoc on `db` export

## Verification

- `pnpm typecheck` -- ensure no type errors from new assertions
- `pnpm test` -- ensure existing tests pass (in-memory SQLite path unchanged)
6 changes: 5 additions & 1 deletion .env.sample
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# See https://docs.karakeep.app/configuration for more information
DATA_DIR=<path>
NEXTAUTH_SECRET=<secret>
NEXTAUTH_SECRET=<secret>

# Database (optional, defaults to SQLite)
# DATABASE_DIALECT=postgresql
# DATABASE_URL=postgresql://user:password@host:5432/karakeep
3 changes: 1 addition & 2 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# These are supported funding model platforms

buy_me_a_coffee: mbassem
github: MohamedBassem
github: johnford2002
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ body:
attributes:
label: Have you searched for an existing open/closed issue?
description: |
To help us keep these issues under control, please ensure you have first [searched our issue list](https://github.com/karakeep-app/karakeep/issues?q=is%3Aissue) for any existing issues that cover the fundamental benefit/goal of your request.
To help us keep these issues under control, please ensure you have first [searched our issue list](https://github.com/johnford2002/karakeep/issues?q=is%3Aissue) for any existing issues that cover the fundamental benefit/goal of your request.
options:
- label: I have searched for existing issues and none cover my fundamental request
required: true
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/question.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ body:
- type: markdown
attributes:
value: |
We use Github discussions for anything that's not a bug report or a feature request. Please ask your question in the [Q&A section](https://github.com/karakeep-app/karakeep/discussions/categories/q-a) and someone will answer it soon!
We use Github discussions for anything that's not a bug report or a feature request. Please ask your question in the [Q&A section](https://github.com/johnford2002/karakeep/discussions/categories/q-a) and someone will answer it soon!
File renamed without changes.
61 changes: 61 additions & 0 deletions .github/workflows/fork-claude.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Claude Code

on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]

jobs:
claude:
if: |
github.actor == 'johnford2002' && (
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
)
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1

- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}

# Optional: Specify model (defaults to Claude Sonnet 4, uncomment for Claude Opus 4)
# model: "claude-opus-4-20250514"

# Optional: Customize the trigger phrase (default: @claude)
# trigger_phrase: "/claude"

# Optional: Trigger when specific user is assigned to an issue
# assignee_trigger: "claude-bot"

# Optional: Allow Claude to run specific commands
allowed_tools: "Bash(pnpm install),Bash(pnpm typecheck),Bash(pnpm test),Bash(pnpm lint:fix)"

# Optional: Add custom instructions for Claude to customize its behavior for your project
# custom_instructions: |
# Follow our coding standards
# Ensure all new code has tests
# Use TypeScript for new files

# Optional: Custom environment variables for Claude
# claude_env: |
# NODE_ENV: test
158 changes: 158 additions & 0 deletions .github/workflows/fork-docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
name: Build and Push Docker
on:
release:
types:
- created
push:
branches:
- main

jobs:
build:
strategy:
fail-fast: false
matrix:
platform: [linux/amd64, linux/arm64]
image: [web, workers, cli, mcp, aio]
include:
- platform: linux/amd64
suffix: amd64
os: ubuntu-latest
- platform: linux/arm64
suffix: arm64
os: ubuntu-24.04-arm
- image: web
name: karakeep-web
target: web
image_name: ghcr.io/johnford2002/karakeep-web
- image: workers
name: karakeep-workers
target: workers
image_name: ghcr.io/johnford2002/karakeep-workers
- image: cli
name: karakeep-cli
target: cli
image_name: ghcr.io/johnford2002/karakeep-cli
- image: mcp
name: karakeep-mcp
target: mcp
image_name: ghcr.io/johnford2002/karakeep-mcp
- image: aio
name: karakeep-aio
target: aio
image_name: ghcr.io/johnford2002/karakeep
runs-on: ${{ matrix.os }}
permissions:
packages: write
contents: read

steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Github Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Prepare tags
id: tags
env:
IMAGE: ${{ matrix.image_name }}
SUFFIX: ${{ matrix.suffix }}
EVENT_NAME: ${{ github.event_name }}
RAW_VERSION: ${{ github.event.release.name }}
run: |
set -euo pipefail

# Strip leading 'v' so image tags follow docker convention (1.0.0, not v1.0.0)
VERSION="${RAW_VERSION#v}"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"

all_tags=""

if [[ "${EVENT_NAME}" == "push" ]]; then
all_tags="${IMAGE}:latest-${SUFFIX}"
fi

if [[ "${EVENT_NAME}" == "release" ]]; then
all_tags="${IMAGE}:${VERSION}-${SUFFIX},${IMAGE}:release-${SUFFIX}"
fi

echo "all=${all_tags}" >> "$GITHUB_OUTPUT"

- name: Build ${{ matrix.name }}
uses: docker/build-push-action@v5
with:
context: .
build-args: SERVER_VERSION=${{ github.event_name == 'release' && steps.tags.outputs.version || 'nightly' }}
file: docker/Dockerfile
target: ${{ matrix.target }}
platforms: ${{ matrix.platform }}
push: true
tags: ${{ steps.tags.outputs.all }}
cache-from: type=registry,ref=ghcr.io/johnford2002/karakeep-build-cache:${{ matrix.target }}-${{ matrix.suffix }}
cache-to: type=registry,mode=max,ref=ghcr.io/johnford2002/karakeep-build-cache:${{ matrix.target }}-${{ matrix.suffix }}

manifest:
needs: build
runs-on: ubuntu-latest
permissions:
packages: write
contents: read
strategy:
fail-fast: false
matrix:
include:
- name: karakeep-web
image_name: ghcr.io/johnford2002/karakeep-web
- name: karakeep-workers
image_name: ghcr.io/johnford2002/karakeep-workers
- name: karakeep-cli
image_name: ghcr.io/johnford2002/karakeep-cli
- name: karakeep-mcp
image_name: ghcr.io/johnford2002/karakeep-mcp
- name: karakeep-aio
image_name: ghcr.io/johnford2002/karakeep
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Github Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Create manifests for ${{ matrix.name }}
env:
IMAGE: ${{ matrix.image_name }}
IS_RELEASE: ${{ github.event_name == 'release' }}
IS_PUSH: ${{ github.event_name == 'push' }}
RAW_VERSION: ${{ github.event.release.name }}
run: |
set -euo pipefail

# Strip leading 'v' so image tags follow docker convention (1.0.0, not v1.0.0)
VERSION="${RAW_VERSION#v}"

create_manifest() {
local tag="$1"
docker buildx imagetools create \
-t "${tag}" \
"${tag}-amd64" \
"${tag}-arm64"
}

if [[ "${IS_PUSH}" == "true" ]]; then
create_manifest "${IMAGE}:latest"
fi

if [[ "${IS_RELEASE}" == "true" ]]; then
create_manifest "${IMAGE}:${VERSION}"
create_manifest "${IMAGE}:release"
fi
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
name: Android App Release Build

# Disabled in this fork. The upstream workflow publishes a karakeep-branded build.
# Trigger manually via workflow_dispatch if you intend to produce your own build.
on:
push:
tags:
- 'android/v[0-9]+.[0-9]+.[0-9]+-[0-9]+'
workflow_dispatch:

jobs:
build:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
name: Publish CLI Package to npm
# Disabled in this fork to avoid conflicting with upstream @karakeep/cli on npm.
# Trigger manually via workflow_dispatch if you publish your own package.
on:
push:
tags:
- "cli/v[0-9]+.[0-9]+.[0-9]+"
workflow_dispatch:

permissions:
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
name: Extension Release Build

# Disabled in this fork. The upstream workflow publishes a karakeep-branded build.
# Trigger manually via workflow_dispatch if you intend to produce your own build.
on:
push:
tags:
- 'extension/v[0-9]+.[0-9]+.[0-9]+'
workflow_dispatch:

jobs:
build:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
name: iOS App Release Build

# Disabled in this fork. The upstream workflow publishes a karakeep-branded build.
# Trigger manually via workflow_dispatch if you intend to produce your own build.
on:
push:
tags:
- 'ios/v[0-9]+.[0-9]+.[0-9]+-[0-9]+'
workflow_dispatch:

jobs:
build:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
name: Publish MCP Package to npm
# Disabled in this fork to avoid conflicting with the upstream MCP package on npm.
# Trigger manually via workflow_dispatch if you publish your own package.
on:
push:
tags:
# This is a glob pattern not a regex
- "mcp/v[0-9]+.[0-9]+.[0-9]+"
workflow_dispatch:

permissions:
id-token: write # Required for OIDC
Expand Down
Loading