Skip to content

Support fire-and-forget async statement submission without blocking pool slots #197

Closed

Description

@jeremyowensboggs

Problem

We're using Snowflex for direct Snowflake invocations in our scenario-planning
system. However, Snowflex's async execution (via Snowflex.query_async/2) holds
a connection pool slot for the full query duration, blocking other requests from
acquiring that slot while waiting for results.

For fire-and-forget operations where we only need the statement handle and don't
need to block on execution, this is wasteful — we submit work to Snowflake but
must hold the pool slot until the query completes.

Solution We Built

We created a small in-house module (AsyncStatement) that:

  1. Submits a Snowflake statement directly via Snowflake's HTTP SQL REST API
  2. Returns the statement handle immediately without acquiring a pool slot
  3. Allows callers to check status or fetch results later via the handle

This unblocks the pool slot right after submission, letting other queries proceed
in parallel.

Trade-off

Our implementation currently borrows Snowflex's internal transport-option builder
(Snowflex.HttpClient.transport_options/1) to construct HTTP requests. This
creates a maintenance burden: we must re-verify compatibility on every Snowflex
upgrade.

Proposal

We'd like to discuss whether fire-and-forget async submission (statement handle
without blocking) should be a feature in Snowflex itself, rather than in every
consumer's codebase. If so:

  • Snowflex could expose a public API (e.g., Snowflex.submit_async/2) that
    submits via the HTTP SQL API and returns a handle
  • Consumers could use that instead of reimplementing
  • Maintenance would centralize in Snowflex

Questions

  1. Is this a feature direction you'd consider for Snowflex?
  2. If yes, is there existing work or an API design we should align with?
  3. If not, any guidance on best practices for shipping our in-house solution
    without tight coupling to Snowflex internals?

We're currently in production with the in-house module, so this is a longer-term
improvement opportunity rather than a blocker.

Activity

  1. Ch4s3 commented on Oct 8, 2026

    @Ch4s3
    Contributor

    done in #198

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions