Merge pull request #67 from pavelhov/codex/xai-x-search-0.1.15

feat: add Grok 4.7 X search through xAI

Author: pavelhov

docs-site/src/content/docs/guides/sidecars.md

@@ -67,6 +67,40 @@ request. The effective bridge watchdog is
 stall is not a total generation timeout. Failures before SSE starts return non-2xx JSON; generation
 failures after response headers have started are delivered as `response.failed` SSE.
 
+### X search for Grok 4.7
+
+When a turn routes to `xai/grok-4.7` on the xAI provider, CodexCommander also gives the model a
+synthetic `x_search` function tool. The model calls it only when a task needs live X (Twitter)
+research, such as what people are posting about a topic or what a specific account said. Other
+providers and other Grok models are unchanged, and the turn itself keeps using the existing xAI chat
+route, so Codex tools such as `apply_patch` and shell keep working.
+
+Each `x_search` call runs one request to xAI `/v1/responses` with xAI's server-side
+[`x_search` tool](https://docs.x.ai/developers/tools/x-search). It uses the same xAI credential and
+transport as the routed turn, either your Grok login or an xAI API key. The model can narrow a
+search with `allowed_x_handles` or `excluded_x_handles` (up to 20 handles, not both) and
+`from_date` / `to_date` (`YYYY-MM-DD`).
+
+The answer returns to the model as an untrusted tool result with direct `x.com` post links. Codex
+shows the call as a web-search cell whose sources are those post links. X search counts against the
+same `maxSearchesPerTurn` budget and forced-answer rule as web search. Its per-search deadline is
+`max(webSearchSidecar.timeoutMs, 90000)` milliseconds.
+
+xAI bills X search on top of tokens: $5 per 1,000 posts and $10 per 1,000 profiles fetched (xAI
+pricing as of 2026-09-21; check the xAI page above for current rates). The search query text is sent
+to xAI. To turn X search off, set:
+
+```json
+{
+  "webSearchSidecar": {
+    "xSearch": false
+  }
+}
+```
+
+If your client already defines its own `x_search` function, CodexCommander leaves it alone and does
+not add the synthetic tool.
+
 ## Vision sidecar
 
 When the routed model is listed in its provider's `noVisionModels` and a request carries an image,

docs-site/src/content/docs/reference/configuration/server.md

@@ -170,6 +170,7 @@ provider-selection behavior.
 | `maxSearchesPerTurn?` | `number` | `3` | Real searches allowed per main-model turn. |
 | `routedModelStallTimeoutMs?` | `number` | `200000` | Config-file-only routed-model raw-body inactivity deadline. Integer 1–2147483647; every non-empty chunk resets it. |
 | `timeoutMs?` | `number` | `60000` | Deadline for one hosted search. |
+| `xSearch?` | `boolean` | `true` | Adds the xAI `x_search` tool for Grok 4.7 turns on the xAI provider. `false` turns it off. See [X search for Grok 4.7](/guides/sidecars/#x-search-for-grok-47). |
 
 The OpenAI backend requires a ChatGPT login and enabled ChatGPT `forward` provider. Claude-inbound
 routed replays inject main ChatGPT auth into the internal request. The Anthropic backend uses the

docs/plans/2026-09-25-xai-x-search-release.md

@@ -0,0 +1,38 @@
+---
+artifact_contract: ce-unified-plan/v1
+artifact_readiness: implementation-ready
+execution: code
+created: 2026-09-25
+---
+# Grok X Search and macOS update
+
+## Goal Capsule
+Enable a Grok 4.7 Codex task to search live X posts through xAI's Responses API, retain direct source links, and ship the change through development, main, and the next authenticated macOS update.
+
+## Requirements
+- R1: Requests routed to xAI Grok 4.7 can opt into server-side `x_search` when the task asks for X research.
+- R2: Normal function tools, streaming, cancellation, errors, and citations remain usable.
+- R3: General web search for other providers remains unchanged.
+- R4: User-facing documentation explains when X search is available and its xAI billing.
+- R5: Release assets use the existing signed appcast, an incremented immutable build, and verified prior inventory.
+
+## Key Technical Decisions
+- KTD-1: Use xAI's `/v1/responses` endpoint for X research. Its documented `x_search` tool is not available on `/chat/completions`; avoid implying model selection alone enables it. Governs R1-R2.
+- KTD-2: Preserve Codex's client tool calls and existing provider routes while adding the hosted xAI tool only for eligible xAI requests. Governs R1-R3.
+- KTD-3: Complete feature PR to `development`, then promote via PR to `main` before the release. Governs R5.
+
+## Implementation Units
+### U1: xAI hosted search path
+Inspect `src/providers/registry.ts`, `src/adapters/openai-chat.ts`, `src/adapters/openai-responses.ts`, `src/responses/parser.ts`, `src/bridge.ts`, and `src/web-search/`. Add the smallest sound xAI Responses routing and `x_search` handling. Add focused regression tests in `tests/` for tool injection, X source links, non-X routes, errors, and streaming/cancellation. Depends on no other unit. Covers R1-R3.
+
+### U2: User documentation
+Update the relevant `docs-site/` provider/search documentation and translations so they accurately describe availability, cost, and setup. Depends on U1. Covers R4.
+
+### U3: Integration and update
+Run `bun run typecheck`, focused tests, `bun run test:parallel`, `bun run privacy:scan`, docs build, and applicable app checks. Review security implications of sending task content to xAI and preserving citations. Merge the feature to `development`, promote to `main` after CI, package the signed macOS update, generate and verify its appcast, then publish and verify release assets. Depends on U1-U2. Covers R5.
+
+## Verification Contract
+The existing non-xAI adapter behavior is preserved. Focused tests prove that a Grok X research request carries `x_search` to xAI Responses and returns verifiable post URLs. Full repository checks and PR CI pass. The release asset is signed, feed verification passes, and the published fixed appcast URL serves the new version.
+
+## Definition of Done
+The feature is on both `development` and `main`; a new macOS release is published with an authenticated update; the user can update and test X search from the app.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "codexcommander",
-  "version": "0.1.14",
+  "version": "0.1.15",
   "private": true,
   "description": "CodexCommander — universal provider proxy for OpenAI Codex & Claude Code — use any LLM with Codex CLI/App/SDK and Claude Code",
   "type": "module",

src/config.ts

@@ -997,6 +997,7 @@ const webSearchSidecarSchema = z.object({
   reasoning: z.string().optional(),
   maxSearchesPerTurn: z.number().int().nonnegative().optional(),
   timeoutMs: z.number().int().positive().optional(),
+  xSearch: z.boolean().optional(),
   routedModelStallTimeoutMs: z.number().int().min(1).max(2_147_483_647).optional(),
 }).strict();
 

src/providers/registry.ts

@@ -214,6 +214,12 @@ export interface ProviderRegistryEntry {
   modelReasoningEffortMap?: Record<string, Record<string, string>>;
   reasoningWireFormat?: CodexCommanderProviderConfig["reasoningWireFormat"];
   noVisionModels?: string[];
+  /**
+   * Models whose xAI Responses endpoint accepts the server-side `x_search` hosted tool. Routed chat
+   * turns on these models get a synthetic `x_search` function tool that the proxy executes against
+   * `/v1/responses` (see src/web-search/xai-x-search.ts). Registry-only: never seeded into config.
+   */
+  xSearchModels?: string[];
   noReasoningModels?: string[];
   noTemperatureModels?: string[];
   noTopPModels?: string[];
@@ -941,7 +947,7 @@ export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [
     // grok-composer-2.5-fast is retained as account-verified despite being absent from public docs.
     // grok-4.20-multi-agent-0309 is intentionally absent: the OAuth chat-completions
     // transport returns 400 ("Multi Agent requests are not allowed on chat completions").
-    models: ["grok-4.6", "grok-4.5", "grok-4.3", "grok-4.20-0309-reasoning", "grok-4.20-0309-non-reasoning", "grok-build-0.1", "grok-composer-2.5-fast"],
+    models: ["grok-4.7", "grok-4.6", "grok-4.5", "grok-4.3", "grok-4.20-0309-reasoning", "grok-4.20-0309-non-reasoning", "grok-build-0.1", "grok-composer-2.5-fast"],
     defaultModel: "grok-4.5",
     // Vision lineup per docs.x.ai model-capabilities/images/understanding: the grok-4.x chat
     // models accept image input (JPEG/PNG, URL or base64). Without this the catalog leaves
@@ -950,6 +956,7 @@ export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [
     // the app blocks attachments client-side. grok-build-0.1 / grok-composer-2.5-fast stay out
     // (they are already listed in noVisionModels below).
     modelInputModalities: {
+      "grok-4.7": ["text", "image"],
       "grok-4.6": ["text", "image"],
       "grok-4.5": ["text", "image"],
       "grok-4.3": ["text", "image"],
@@ -961,7 +968,7 @@ export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [
     // reasoning_content as the top cause of prompt-cache misses on multi-turn conversations
     // (docs.x.ai prompt-caching/multi-turn, verified 2026-07-13).
     // Models that never emit reasoning simply have no thinking parts to replay (no-op).
-    preserveReasoningContentModels: ["grok-4.6", "grok-4.5", "grok-4.3", "grok-4.20-0309-reasoning"],
+    preserveReasoningContentModels: ["grok-4.7", "grok-4.6", "grok-4.5", "grok-4.3", "grok-4.20-0309-reasoning"],
     // grok reasoning is always-on with low/medium/high control (no off tier upstream);
     // grok-4.6+ extends the ladder with xhigh per docs.x.ai. xAI rejects max/ultra outright
     // (400 "Invalid reasoning effort"), so every grok reasoning model clamps ultra/max down
@@ -970,11 +977,13 @@ export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [
     modelReasoningEfforts: {
       "grok-4.5": ["low", "medium", "high"],
       "grok-4.6": ["low", "medium", "high", "xhigh"],
+      "grok-4.7": ["low", "medium", "high", "xhigh"],
     },
     // Provider default for live-discovered reasoning models: clamp to the verified xAI
     // ladder unless a per-model entry raises it (noReasoningModels stay effort-free).
     reasoningEfforts: ["low", "medium", "high"],
     modelContextWindows: {
+      "grok-4.7": 500_000,
       "grok-4.6": 500_000,
       "grok-4.5": 500_000,
       "grok-4.3": 1_000_000,
@@ -983,6 +992,9 @@ export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [
       "grok-build-0.1": 256_000,
     },
     noVisionModels: ["grok-build-0.1", "grok-composer-2.5-fast"],
+    // Server-side X search (docs.x.ai/developers/tools/x-search) exists only on /v1/responses.
+    // Live-verified 2026-09-25 for grok-4.7 on both the Grok-login CLI proxy and api.x.ai.
+    xSearchModels: ["grok-4.7"],
   },
   {
     id: "command-code",

src/server/responses/core.ts

@@ -65,7 +65,7 @@ import {
   resolveAnthropicAccountForSession,
   rotateAnthropicAccountOn429,
 } from "../../oauth/anthropic-routing";
-import { buildWebSearchTool, planWebSearch, runWithWebSearch, shouldResolveOpenAiWebSearchSidecar } from "../../web-search";
+import { buildWebSearchTool, buildXSearchTool, planWebSearch, planXaiXSearch, planXSearchOnlyLoop, runWithWebSearch, shouldResolveOpenAiWebSearchSidecar, X_SEARCH_TOOL_NAME } from "../../web-search";
 import { describeImagesInPlace, planVisionSidecar, resolveOpenAiVisionModel, shouldResolveOpenAiVisionSidecar, stripImagesInPlace } from "../../vision";
 import { createAdapterEventQueue, preflightAdapterEvents } from "../../adapters/run-turn-queue";
 import {
@@ -2539,27 +2539,41 @@ async function handleResponsesInner(
   const wsPlan = !routedCompaction
     ? planWebSearch(config, parsed, false, route.provider, route.modelId, openAiSidecar)
     : undefined;
+  // xAI hosted X search: Grok models listed in the registry `xSearchModels` get a synthetic x_search
+  // function the loop runs on xAI /v1/responses (same transport + credential as this routed turn).
+  const xPlan = !routedCompaction && !adapter.runTurn
+    ? planXaiXSearch(config, parsed, route.providerName, route.provider, route.modelId)
+    : undefined;
+  const xOnlyLoopPlan = xPlan ? planXSearchOnlyLoop(config, xPlan.timeoutMs) : undefined;
+  const loopPlan = wsPlan ?? xOnlyLoopPlan;
   // Web-search sidecar: Codex enabled web_search but this is a routed (non-OpenAI) model that can't
   // run it server-side. Expose web_search as a function tool and run searches via the gpt-mini sidecar
   // through the ChatGPT passthrough, looping until the model answers. Otherwise take the normal path.
   // Placed BEFORE the runTurn early-return for non-runTurn adapters so dual-tool turns dispatch
   // through web-search instead of being swallowed. runTurn adapters never enter this branch.
-  if (wsPlan && !adapter.runTurn) {
-    const tools = parsed.context.tools ?? [];
-    if (!tools.some(tool => !tool.namespace && tool.name === "web_search")) {
-      parsed.context.tools = [...tools, buildWebSearchTool()];
+  if (loopPlan && !adapter.runTurn) {
+    if (xPlan) parsed.options.parallelToolCalls = false;
+    let tools = parsed.context.tools ?? [];
+    if (wsPlan && !tools.some(tool => !tool.namespace && tool.name === "web_search")) {
+      tools = [...tools, buildWebSearchTool()];
+    }
+    if (xPlan && !tools.some(tool => !tool.namespace && tool.name === X_SEARCH_TOOL_NAME)) {
+      tools = [...tools, buildXSearchTool()];
     }
+    parsed.context.tools = tools;
     const wsResponse = await runWithWebSearch({
       parsed, adapter,
       incomingMeta: { headers: selectedForwardHeaders, abortSignal: options.abortSignal, translatorBudget, dispatch: requestDispatchContext(logCtx, options.abortSignal ?? req.signal) },
       toolParameterSchemas: toolBridgeMaps.toolParameterSchemas,
-      backend: wsPlan.backend,
-      forwardProvider: wsPlan.forwardSidecar?.provider,
-      anthropicSidecar: wsPlan.anthropicSidecar,
-      hostedTool: wsPlan.hostedTool,
-      selectedForwardHeaders: wsPlan.forwardSidecar?.headers ?? selectedForwardHeaders,
-      settings: wsPlan.settings,
-      maxSearches: wsPlan.maxSearches,
+      backend: loopPlan.backend,
+      forwardProvider: loopPlan.forwardSidecar?.provider,
+      anthropicSidecar: loopPlan.anthropicSidecar,
+      hostedTool: loopPlan.hostedTool,
+      selectedForwardHeaders: loopPlan.forwardSidecar?.headers ?? selectedForwardHeaders,
+      settings: loopPlan.settings,
+      maxSearches: loopPlan.maxSearches,
+      ...(xPlan ? { xSearch: xPlan } : {}),
+      webSearchEnabled: !!wsPlan,
       forceEmptyResponseId: true,
       abortSignal: options.abortSignal,
       ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}),
@@ -2573,10 +2587,10 @@ async function handleResponsesInner(
           if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage;
         }
       },
-      recordSidecarOutcome: wsPlan.forwardSidecar?.recordOutcome,
+      recordSidecarOutcome: loopPlan.forwardSidecar?.recordOutcome,
       connectTimeoutMs: config.connectTimeoutMs ?? 200_000,
-      routedModelStallTimeoutMs: wsPlan.routedModelStallTimeoutMs,
-      stallTimeoutSec: wsPlan.stallTimeoutSec,
+      routedModelStallTimeoutMs: loopPlan.routedModelStallTimeoutMs,
+      stallTimeoutSec: Math.max(loopPlan.stallTimeoutSec, xOnlyLoopPlan?.stallTimeoutSec ?? 0),
       on429: retryAfter => {
         const rotated = rotateProviderTransportOn429(config, route.providerName, route.provider, {
           retryAfter,
@@ -2586,6 +2600,8 @@ async function handleResponsesInner(
         });
         if (!rotated) return null;
         route.provider = rotated;
+        // Keep the X-search sidecar on the rotated credential too.
+        if (xPlan) xPlan.provider = rotated;
         return resolveAdapter(
           resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire),
           config.cacheRetention,

src/types.ts

@@ -166,6 +166,8 @@ export interface CodexCommanderTool {
   loadedFromToolSearch?: boolean;
   /** Synthetic web_search tool: the model's call is executed by the gpt-5.4-mini sidecar, not relayed to Codex. */
   webSearch?: boolean;
+  /** Synthetic xAI x_search tool: executed by the proxy against xAI /v1/responses, not relayed to Codex. */
+  xSearch?: boolean;
 }
 
 /**
@@ -953,6 +955,11 @@ export interface CodexCommanderWebSearchSidecarConfig {
   maxSearchesPerTurn?: number;
   /** Sidecar fetch timeout (ms). */
   timeoutMs?: number;
+  /**
+   * xAI X search for Grok models listed in the registry `xSearchModels`. Default on: the routed
+   * Grok model gets an `x_search` tool it calls only for X research. Set false to hide it.
+   */
+  xSearch?: boolean;
   /**
    * Config-file-only deadline (ms) for continuous routed-model response-body raw-byte inactivity
    * during a web-search turn. Default 200000. Must be an integer from 1 through 2147483647.

src/web-search/index.ts

@@ -6,6 +6,16 @@ import { getAccountSet } from "../oauth/store";
 import { DEFAULT_STALL_TIMEOUT_SEC } from "../stall-timeout";
 
 export { runWithWebSearch } from "./loop";
+export {
+  buildXSearchTool,
+  isXSearchModel,
+  parseXSearchArgs,
+  parseXSearchResponse,
+  planXaiXSearch,
+  runXaiXSearch,
+  X_SEARCH_TOOL_NAME,
+  type XaiXSearchSidecar,
+} from "./xai-x-search";
 export { buildWebSearchTool, extractHostedWebSearch, WEB_SEARCH_TOOL_NAME } from "./synthetic-tool";
 export { runAnthropicWebSearch, parseAnthropicSidecarSSE } from "./anthropic-executor";
 
@@ -194,3 +204,22 @@ export function planWebSearch(
     stallTimeoutSec,
   };
 }
+
+/**
+ * Loop settings for a turn that exposes ONLY the xAI x_search tool (Codex did not enable web_search,
+ * or no web-search backend is usable). The web backend never runs because no web_search tool is
+ * exposed; these values only bound the loop and its stall watchdog.
+ */
+export function planXSearchOnlyLoop(config: CodexCommanderConfig, xSearchTimeoutMs: number): SidecarPlan {
+  const cfg = config.webSearchSidecar ?? {};
+  const routedModelStallTimeoutMs = resolveRoutedModelStallTimeoutMs(cfg.routedModelStallTimeoutMs);
+  const connectTimeoutMs = config.connectTimeoutMs ?? 200_000;
+  return {
+    backend: "openai",
+    hostedTool: {},
+    settings: { model: cfg.model ?? DEFAULT_SIDECAR_MODEL, reasoning: cfg.reasoning ?? DEFAULT_SIDECAR_REASONING, timeoutMs: xSearchTimeoutMs, describeImages: false },
+    maxSearches: cfg.maxSearchesPerTurn ?? DEFAULT_MAX_SEARCHES,
+    routedModelStallTimeoutMs,
+    stallTimeoutSec: webSearchStallTimeoutSec(config.stallTimeoutSec, connectTimeoutMs, routedModelStallTimeoutMs, xSearchTimeoutMs),
+  };
+}