Skip to content

TAVERN-COMPATIBILITY.md

Latest commit

 

History

History
287 lines (232 loc) · 14.6 KB

TAVERN-COMPATIBILITY.md

File metadata and controls

287 lines (232 loc) · 14.6 KB
Raw

Tavern Compatibility and Imports

1. Enduring vision, principles, and rationale

WorldSharp should support Tavern-style content as a first-class compatibility format without making Tavern Cards the canonical domain model. Tavern content may combine one or more Characters, scenario, author instructions, example dialogue, alternate greetings, and lorebooks. A durable WorldSharp Universe has separate Characters, Instances, graph entities, Interfaces, messages, summaries, chronology, and history.

The compatibility model therefore distinguishes:

  1. Tavern Card — source content used according to Tavern semantics without requiring immediate decomposition into native entities.
  2. Ephemeral Chat Workspace — a non-Universe chat context for direct card use. Persistent chat data is allowed, but it has no canonical world graph or history.
  3. Content Pack — a deterministic native WorldSharp recipe containing deliberately structured reusable material.
  4. Universe — an instantiated mutable world whose runtime state never leaks back into source material.

"Content Pack" is the preferred native term. "World Seed" may describe a pack primarily intended to create a Universe, but not every pack has that destination. "Ephemeral Chat Workspace" is preferable to a fake global Universe because reuse of chat UI must not accidentally grant temporary content durable world semantics.

Source preservation

Compatibility import should preserve the original source payload, format, and provenance before producing a potentially lossy CharacterCard projection. Unknown or ambiguous data should remain inspectable rather than disappear. Re-import needs source identity/version semantics and must never silently overwrite mutable native state.

Prompt semantics are not world semantics. Author notes, system prompts, post-history instructions, examples, and lorebook activation rules influence context construction; they are not automatically canonical facts.

Embedded lorebooks should first remain source knowledge with keys, triggers, content, and metadata. Conversion into world nodes or relationships is a reviewable proposal. A flat lore import must not require speculative hierarchy.

2. Proposed and future architecture, including alternatives

Direct Tavern use

An Ephemeral Chat Workspace should load a card without an AI import call, support character-only, scenario-only, and multi-character material, retain source instructions/lore, and remain usable even when clean native mapping is impossible. Merely opening such a chat must not create World Nodes, Character Instances, relationships, or history events.

This is intentionally an alternative to forcing every Tavern session into a Universe. Scenario-only sources do not satisfy the current Character Card name invariant, and multi-character sources need explicit participant mapping rather than flattening.

Native Content Packs

A Content Pack is declarative, deterministic, and composable. It may contain:

  • metadata and source provenance;
  • reusable Character Cards;
  • scenario or setting text;
  • world-node and containment templates;
  • typed relationship templates;
  • lorebook entries and activation metadata;
  • suggested Interfaces and initial configuration;
  • optional seed messages/opening Scene;
  • compatibility notes and unmapped fields.

Entries use stable local IDs. Import resolves them into new durable identities; it never reuses mutable IDs from another Universe. Existing-Universe import must be explicit and report collisions/ambiguous references rather than overwrite. Content Packs are not transcripts, live shared Universes, or AI guesses.

Promotion from ephemeral chat

"Promote to Universe" is an assisted reviewed migration, not Content Pack import. It should preserve the original card/chat, ask an extraction Driver for proposed Characters, structured fields, setting/lore, nodes, and relationships, present accepted/rejected/uncertain/source-only sections, and create native material only after approval.

The proposed operation is:

  1. capture Tavern Card and chat provenance;
  2. ask a configured extraction Driver for structured Characters, attributes, setting/scenario material, lore, and possible graph entities;
  3. present an editable conversion proposal;
  4. create or update reusable Character Cards only after approval;
  5. create an approved scenario/template or Content Pack;
  6. create a new Universe or apply the approved pack to an explicitly selected existing Universe;
  7. preserve the original card and chat unchanged.

High-confidence information should enter focused fields; ambiguity should remain source text or an editable proposal. A Smart Connection may be stronger than the normal chat Provider, but changing Provider never changes the review-first rule. No AI result directly mutates canonical state.

Scenario versus world seed

"Scenario card" remains useful compatibility language, but it is not sufficient as the native type. The same scenario may be used directly in ephemeral chat, applied to an existing Universe, used to create a new Universe, or combined with Characters and lore from other packs. Native terminology should describe composition and destination explicitly; a scenario-heavy Content Pack may be presented as a Universe starter without making every scenario a seed.

Import Wizard and reconstruction

The Import Wizard is broader than one-card promotion. Source adapters should cover Tavern/SillyTavern, Marinara, native Content Packs, and folders combining cards with chat history. Adapters should produce common staged records rather than separate domain models.

Two explicit workflows are useful:

  • Collection to Universe: import reusable cards, lorebooks, multiple Interfaces, summaries, feed history, and source metadata.
  • Cards and one roleplay to Universe: import attached cards and one long transcript, preserve setup/scenario, propose day ranges, and offer reviewed world reconstruction.

Ingestion and reconstruction remain separate:

  1. ingest sources without canonical world mutation;
  2. normalize cards, messages, summaries, lore, attachments, and provenance;
  3. import or detect candidate time ranges;
  4. confirm summary/boundary semantics;
  5. run bounded extraction per exact range;
  6. review and commit proposed entities, relationships, facts, states, and memories chronologically;
  7. reconcile imported lore against the accepted message-backed records;
  8. review proposed duplicate merges, complementary context, and contradictions;
  9. retain every source record, range, evidence link, and execution provenance.

SillyTavern markers such as "Day 1" are candidates, not authoritative boundaries. When a source has both summary and exact messages, retain both: the summary is efficient context and the transcript is evidence. Pause for a decision between ranges, but allow the UI to prepare the next range automatically after that decision. Failure in one range must not erase earlier accepted work or turn a large import into one unreviewable operation. An AI reconciliation pass may propose how lore and extracted records fit together, but it must not rewrite or discard either source silently.

The Smart Connection should work incrementally with modest local models by using bounded schemas/ranges and already accepted state rather than an entire transcript. Deterministic import must remain useful without it.

Compatibility roadmap

  1. Deepen versioned source boundaries and unknown-field preservation.
  2. Add a general import staging/review area.
  3. Add the Ephemeral Chat Workspace without Universe coupling.
  4. define and validate a versioned Content Pack schema.
  5. support explicit pack import into new or existing Universes.
  6. expand collection/single-roleplay adapters and reviewed reconstruction.
  7. support feed/swipe and richer source artifacts as evidence.
  8. export approved material to Tavern-compatible forms only where mapping is honest and sufficiently lossless.

Risks to resolve

  1. Lossy import: reducing a payload immediately to CharacterCard can lose lorebooks, author notes, greetings, and creator/source metadata needed later.
  2. Scenario-only and multi-character sources: the current Card invariant requires a name, while Tavern source semantics may not identify one Character.
  3. Prompt versus world semantics: instructions and activation behavior must not become history merely because they are present in source text.
  4. Re-import and identity: repeated Chub/source import needs explicit revision/update behavior instead of accidental overwrite or unexplained duplication.
  5. Review granularity: conversion needs independently accepted, rejected, uncertain, and source-only fields; one all-or-nothing model response is not a usable correction workflow.

3. Current implementation, status, and known gaps

Character Card compatibility

The Library imports Tavern Character Card V1/V2 JSON and PNG into CharacterCard. PNG artwork is copied into managed host storage; valid remote HTTP(S) artwork may remain a URL. Parsed native fields and source format/payload are retained when supplied by the importer. This is Character Card compatibility, not general Tavern Card or Content Pack support.

Character prompt fields resolve the identity macros {{char}}/{{charName}} and {{user}}/{{userName}} immediately before provider generation. The character macros use the current Character Instance name, while the user macros use the selected Persona name or User when no Persona is active. Roleplay rosters resolve each Character Card against that Character independently, so the same card template works correctly in group scenes.

Folder discovery, upload, and Chub import are Library workflows. Their current depth, count, and request limits are documented in Architecture.

Source imports

SourceImportService supports saved locations for:

  • SillyTavern-style JSONL roleplay plus selected Character Cards;
  • Marinara Engine table discovery, one-roleplay import, and collection import.

Marinara collection import can create a Universe, reusable Cards and Instances, Convo/Roleplay Interfaces, timestamped source messages and attachments, optional source Personas, staged lorebook knowledge, and exact historical day replay ranges. It can also project Marinara Noodle Feed accounts, posts, replies, quotes, likes, reposts, follows, and timestamps into the destination Social Feed. Imported messages and feed records retain source identity and are read-only. Their raw source payload also preserves Marinara display/generation metadata, reactions, translations, and linked swipe alternatives even where WorldSharp has no native editing surface for those fields yet.

Noodle Feed media is projected into the native Social Feed post-media relation when it has a safe HTTP(S) URL, while the complete record remains in the durable social_import_records provenance table as raw source payload. Canonical feed rows are projected when their referenced account and post relationships can be resolved; malformed or cyclic reply-parent references are broken deterministically by keeping the reply content as a top-level post rather than aborting the collection. Ambient accounts are created for source accounts that are not part of the selected Character/Persona set. Re-running the same collection import is idempotent by (Universe, source kind, entity type, source ID). Imported media is currently read-only.

Marinara metadata.summaryEntries are imported with their exact source message IDs, title, and summary content. These are arbitrary bounded source summaries, not assumed to be calendar days; they are injected as sealed historical-range evidence when their exact replay range is reconstructed. If a roleplay has no native summary entries, explicit transcript separators such as [Day 2] are staged as candidate ranges and the summary is generated from that bounded transcript during replay. If no separator is available, the whole transcript is staged as an unsegmented generated-summary range rather than being silently omitted.

Marinara branch chats (metadata.branchName) are included in collection imports and discovery so alternate history is not silently lost. The source format does not consistently provide a parent chat or fork message, so those records are imported as explicitly labelled alternate Interfaces until that linkage can be recovered. Replay is chronological, review-first, and stops at each proposed tracker run. The reconstruction UI can automatically prepare the next range after the current proposal is accepted or discarded, and lets the user choose the saved OpenAI-compatible connection used for new runs.

Accepted lorebook knowledge currently projects its whole text into canonical node content and also remains Tracker context. Character knowledge is bound to the imported Instance and its linked World Node; Universe knowledge is global. This is intentionally documented as a transitional limitation: similar entries can duplicate prose, and the current model has no atomic fact/state/memory, temporal applicability, or message-backed source navigation. The intended replacement ingests lore as immutable context, reconstructs chat history first, then performs a reviewed provenance-preserving reconciliation into atomic knowledge records.

Marinara Conversation weekSummaries are preserved over their exact Monday-to-Sunday message interval, including consolidated key details. They are evidence-only ranges and are not queued for a second Tracker replay because the constituent Story Days already own historical reconstruction.

Failure behavior

Import into a new Universe compensates body failure by deleting that Universe and dependent database rows transactionally. This is not a distributed transaction. Reusable/global records and managed artwork have separate lifetimes, so failure does not guarantee that no global artifact remains. If cleanup fails, both import and cleanup failures are reported.

Current gaps

There is no Ephemeral Chat Workspace, Tavern scenario/multi-character runtime, native Content Pack schema/import/export, Tavern export, generic lorebook trigger injection, rich manual range-splitting editor, or AI promotion from arbitrary chat. Atomic provenance-aware knowledge reconciliation, temporal states, Character memories, and source-message navigation are also not yet implemented. Unknown Tavern fields and all creator metadata are not guaranteed to survive every importer.

Marinara support is limited to consumed table layouts and fields. Swipes, reactions, and Noodle Feed media payloads are provenance-preserved, while swipes and imported media do not have native editing surfaces. Day and week summaries require matching timestamped messages to produce exact evidence ranges.