Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/thick-months-wave.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@workflow/next": patch
---

Allow specifying workflow directories
39 changes: 39 additions & 0 deletions docs/content/docs/api-reference/workflow-next/with-workflow.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,42 @@ export default async function config(
return nextConfig;
}
```

## Configuration

The second argument to `withWorkflow` accepts the following options:

### `workflows.dirs`

Directories containing files with `use workflow` directives.

By default, `withWorkflow` uses Next.js entrypoint directories (`pages`, `app`, `src/pages`, `src/app`) as starting points and traces their imports to discover workflow files. If your workflows live in dedicated directories, use this option to point directly at them — avoiding the need to crawl the entire app's import tree.

- **Type:** `string[]`

```typescript title="next.config.ts" lineNumbers
import { withWorkflow } from "workflow/next";
import type { NextConfig } from "next";

const nextConfig: NextConfig = {};

export default withWorkflow(nextConfig, {
workflows: {
dirs: ['workflows', 'src/workflows'], // [!code highlight]
},
});
Comment on lines +76 to +80
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should do dirs (for other reasons - like @ijjk mentioned - the lazy approach is the correct answer here).

but if we were to do dirs or add new configs, we shouldn't be putting it in a enw config options at the end of withWorkflow I think. instead the signature of withWorkflow should become withWorkflow(NextConfig & { workflow: WorkflowConfig }) imo (including the local port/datadir options and future config options)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the lazy approach works, then sure, I'll use it. Just began testing it again.

But what would the call to withWorkflow look like then? Keep in mind that the config might be a function, so we can't easily pass {...nextConfig, workflow: {...}}. Putting the option in the "raw" object literal would require typing it as NextConfigOrFn & WorkflowConfig, which again wouldn't work for functions and would be an extra step. I don't see what's wrong with the current signature, withSentryConfig does it the same way. Maybe Next.js could give an easier way for multiple modules to add config, but then you'd also need to care about namespaces, ...

```

### `workflows.local.port`

Port for the local workflow server during development.

- **Type:** `number`
- **Default:** Uses the `PORT` environment variable

### `workflows.local.dataDir`

Directory for storing local workflow data during development.

- **Type:** `string`
- **Default:** `'.next/workflow-data'`
39 changes: 39 additions & 0 deletions docs/content/docs/getting-started/next.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,45 @@ Check the [Deploying](/docs/deploying) section to learn how your workflows can b

## Troubleshooting

### Out of Memory (OOM) during build

When using `withWorkflow()` in a large Next.js application, the "Discovering workflow directives" phase may consume excessive memory, causing builds to fail with OOM errors on standard build machines (e.g., 16 GB RAM).

**Solution:** Use the `workflows.dirs` option to point directly at the directories containing your workflow files. By default, `withWorkflow` uses entrypoint directories (`pages`, `app`, `src/pages`, `src/app`) as starting points and traces their imports to discover workflows, which in large applications can involve crawling thousands of files.

If your workflows live in a dedicated directory, configure `dirs` to only scan that directory:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the other problem with "dirs" is that it causes split brain I think:

the client mode/webpack transformation still has to happen in the actual code (api routes/server actions) that trigger workflows. we can't skip that - and that code won't live in the workflows directory

dirs feels like it misleads you into believing you've limited the search space to specific directory when you're only limiting the search space of actual workflows/steps imo


```typescript title="next.config.ts" lineNumbers
import { withWorkflow } from "workflow/next";
import type { NextConfig } from "next";

const nextConfig: NextConfig = {};

export default withWorkflow(nextConfig, {
workflows: {
// Only scan the workflows directory instead of the entire app
dirs: ['workflows'], // [!code highlight]
},
});
```

If your workflows are spread across multiple directories:

```typescript title="next.config.ts" lineNumbers
import { withWorkflow } from "workflow/next";
import type { NextConfig } from "next";

const nextConfig: NextConfig = {};

export default withWorkflow(nextConfig, {
workflows: {
dirs: ['workflows', 'src/background-jobs'], // [!code highlight]
},
});
```

This significantly reduces memory usage and build times by avoiding scanning unrelated application code.

### Next.js 16.1+ compatibility

If you see this error when upgrading to Next.js 16.1 or later:
Expand Down
1 change: 1 addition & 0 deletions packages/builders/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ export interface VercelBuildOutputConfig extends BaseWorkflowConfig {
*/
export interface NextConfig extends BaseWorkflowConfig {
buildTarget: 'next';
dirsAreEntrypoints: boolean;
// Next.js builder computes paths dynamically, so these are not used
stepsBundlePath: string;
workflowsBundlePath: string;
Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/lib/config/workflow-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export const getWorkflowConfig = (
) => {
const config: WorkflowConfig = {
dirs: ['./workflows'],
dirsAreEntrypoints: false,
workingDir: resolveObservabilityCwd(),
buildTarget: buildTarget as BuildTarget,
stepsBundlePath: './.well-known/workflow/v1/step.js',
Expand Down
6 changes: 6 additions & 0 deletions packages/next/src/builder-deferred.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import type { NextConfig } from '@workflow/builders';
import { createHash } from 'node:crypto';
import { constants, existsSync, realpathSync } from 'node:fs';
import {
Expand Down Expand Up @@ -65,6 +66,8 @@ export async function getNextBuilderDeferred() {
)) as typeof import('@workflow/builders');

class NextDeferredBuilder extends BaseBuilderClass {
protected declare config: NextConfig;

private socketIO?: SocketIO;
private readonly discoveredWorkflowFiles = new Set<string>();
private readonly discoveredStepFiles = new Set<string>();
Expand Down Expand Up @@ -953,6 +956,9 @@ export async function getNextBuilderDeferred() {

protected async getInputFiles(): Promise<string[]> {
const inputFiles = await super.getInputFiles();
if (!this.config.dirsAreEntrypoints) {
return inputFiles;
}
return inputFiles.filter((item) => {
// Match App Router entrypoints: route.ts, page.ts, layout.ts in app/ or src/app/ directories
// Matches: /app/page.ts, /app/dashboard/page.ts, /src/app/route.ts, etc.
Expand Down
6 changes: 6 additions & 0 deletions packages/next/src/builder-eager.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import type { NextConfig } from '@workflow/builders';
import { constants } from 'node:fs';
import { access, copyFile, mkdir, stat, writeFile } from 'node:fs/promises';
import { extname, join, resolve } from 'node:path';
Expand All @@ -23,6 +24,8 @@ export async function getNextBuilderEager() {
)) as typeof import('@workflow/builders');

class NextBuilder extends BaseBuilderClass {
protected declare config: NextConfig;

async build() {
const outputDir = await this.findAppDirectory();
const workflowGeneratedDir = join(outputDir, '.well-known/workflow/v1');
Expand Down Expand Up @@ -390,6 +393,9 @@ export async function getNextBuilderEager() {

protected async getInputFiles(): Promise<string[]> {
const inputFiles = await super.getInputFiles();
if (!this.config.dirsAreEntrypoints) {
return inputFiles;
}
return inputFiles.filter((item) => {
// Match App Router entrypoints: route.ts, page.ts, layout.ts in app/ or src/app/ directories
// Matches: /app/page.ts, /app/dashboard/page.ts, /src/app/route.ts, etc.
Expand Down
4 changes: 3 additions & 1 deletion packages/next/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export function withWorkflow(
}: {
workflows?: {
lazyDiscovery?: boolean;
dirs?: string[];
local?: {
port?: number;
dataDir?: string;
Expand Down Expand Up @@ -87,7 +88,8 @@ export function withWorkflow(
return new NextBuilder({
watch: shouldWatch,
// discover workflows from pages/app entries
dirs: ['pages', 'app', 'src/pages', 'src/app'],
dirs: workflows?.dirs || ['pages', 'app', 'src/pages', 'src/app'],
dirsAreEntrypoints: !workflows?.dirs,
workingDir: process.cwd(),
distDir: nextConfig.distDir || '.next',
buildTarget: 'next',
Expand Down
2 changes: 1 addition & 1 deletion packages/nitro/src/builders.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export class LocalBuilder extends BaseBuilder {
watch: nitro.options.dev,
dirs: ['.'], // Different apps that use nitro have different directories
}),
buildTarget: 'next', // Placeholder, not actually used
buildTarget: 'standalone', // Placeholder, not actually used
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'standalone; means something else. this could be nitro but honestly better to leave it out of this PR since it's not used anyway

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless this was needed for some reason?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It complained about the missing dirsAreEntrypoints, but yeah, could just pass that.

});
this.#outDir = outDir;
}
Expand Down