Skip to content

UN-3332 [FIX] Use default profile for export validation when single-pass is enabled #1991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pk-zipstack wants to merge 4 commits into
base: main
Choose a base branch
from
Open

UN-3332 [FIX] Use default profile for export validation when single-pass is enabled #1991

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
92d9bfe
UN-3332 [FIX] Validate single-pass exports against the default profile
pk-zipstack May 19, 2026
c7affd4
UN-3332 [TEST] Pin export validation profile lookup for single-pass mode
pk-zipstack May 25, 2026
4ff2b3c
UN-3332 [FIX] Use default profile for per-prompt export JSON in singl…
pk-zipstack May 25, 2026
356f795
UN-3332 [MISC] Tighten output_profile inline comment
pk-zipstack May 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 26 additions & 16 deletions backend/prompt_studio/prompt_studio_registry_v2/prompt_studio_registry_helper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,45 +318,55 @@ def frame_export_json(
if not prompt.profile_manager:
prompt.profile_manager = default_llm_profile

# Single-pass execution always runs prompts via the tool's

@chandrasekharan-zipstack chandrasekharan-zipstack May 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pk-zipstack make the comment more concise and generic

@pk-zipstack pk-zipstack May 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 356f795 — collapsed the 8-line comment down to two lines stating just the rule:

# Single-pass runs every prompt via the tool's default profile;
# otherwise each prompt uses its own profile_manager.

@pk-zipstack pk-zipstack May 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 356f795 — collapsed the 8-line comment down to two lines stating just the rule:

# Single-pass runs every prompt via the tool's default profile;
# otherwise each prompt uses its own profile_manager.

# default profile (see
# OutputManagerHelper.handle_prompt_output_update). Both
# validation lookup AND the per-prompt export payload must
# therefore source their settings from the default profile in
# single-pass mode — otherwise the exported tool would embed
# the stale prompt-level profile and the validation lookup
# would miss the rows actually produced by the run.
if tool.single_pass_extraction_mode:
output_profile = default_llm_profile
else:
output_profile = prompt.profile_manager

if not force_export:
prompt_output = PromptStudioOutputManager.objects.filter(
tool_id=tool.tool_id,
prompt_id=prompt.prompt_id,
profile_manager=prompt.profile_manager,
profile_manager=output_profile,
is_single_pass_extract=tool.single_pass_extraction_mode,
).all()
if not prompt_output:
invalidated_outputs.append(prompt.prompt_key)
continue

vector_db = str(prompt.profile_manager.vector_store.id)
embedding_model = str(prompt.profile_manager.embedding_model.id)
llm = str(prompt.profile_manager.llm.id)
x2text = str(prompt.profile_manager.x2text.id)
adapter_id = str(prompt.profile_manager.embedding_model.adapter_id)
vector_db = str(output_profile.vector_store.id)
embedding_model = str(output_profile.embedding_model.id)
llm = str(output_profile.llm.id)
x2text = str(output_profile.x2text.id)
adapter_id = str(output_profile.embedding_model.adapter_id)
embedding_suffix = adapter_id.split("|")[0]

output[JsonSchemaKey.PROMPT] = prompt.prompt
output[JsonSchemaKey.ACTIVE] = prompt.active
output[JsonSchemaKey.REQUIRED] = prompt.required
output[JsonSchemaKey.CHUNK_SIZE] = prompt.profile_manager.chunk_size
output[JsonSchemaKey.CHUNK_SIZE] = output_profile.chunk_size
output[JsonSchemaKey.VECTOR_DB] = vector_db
output[JsonSchemaKey.EMBEDDING] = embedding_model
output[JsonSchemaKey.X2TEXT_ADAPTER] = x2text
output[JsonSchemaKey.CHUNK_OVERLAP] = prompt.profile_manager.chunk_overlap
output[JsonSchemaKey.CHUNK_OVERLAP] = output_profile.chunk_overlap
output[JsonSchemaKey.LLM] = llm
output[JsonSchemaKey.PREAMBLE] = tool.preamble
output[JsonSchemaKey.POSTAMBLE] = tool.postamble
output[JsonSchemaKey.GRAMMAR] = grammar_list
output[JsonSchemaKey.TYPE] = prompt.enforce_type
output[JsonSchemaKey.NAME] = prompt.prompt_key
output[JsonSchemaKey.RETRIEVAL_STRATEGY] = (
prompt.profile_manager.retrieval_strategy
)
output[JsonSchemaKey.SIMILARITY_TOP_K] = (
prompt.profile_manager.similarity_top_k
)
output[JsonSchemaKey.SECTION] = prompt.profile_manager.section
output[JsonSchemaKey.REINDEX] = prompt.profile_manager.reindex
output[JsonSchemaKey.RETRIEVAL_STRATEGY] = output_profile.retrieval_strategy
output[JsonSchemaKey.SIMILARITY_TOP_K] = output_profile.similarity_top_k
output[JsonSchemaKey.SECTION] = output_profile.section
output[JsonSchemaKey.REINDEX] = output_profile.reindex
output[JsonSchemaKey.EMBEDDING_SUFFIX] = embedding_suffix
# Webhook postprocessing settings
output[JsonSchemaKey.ENABLE_POSTPROCESSING_WEBHOOK] = (
Expand Down
Empty file added backend/prompt_studio/prompt_studio_registry_v2/tests/__init__.py
View file
Open in desktop
Empty file.
Loading