Skip to content

reorg: remove golang depends#1339

Open
MaojiaSheng wants to merge 7 commits intovolcengine:mainfrom
MaojiaSheng:main
Open

reorg: remove golang depends#1339
MaojiaSheng wants to merge 7 commits intovolcengine:mainfrom
MaojiaSheng:main

Conversation

@MaojiaSheng
Copy link
Copy Markdown
Collaborator

remove golang depends
and rewrite #1333

openviking and others added 5 commits April 9, 2026 16:16
- Keep deletion of third_party/agfs (deleted by us)
- Merge other upstream changes
Port of PR volcengine#1333 from Go version to Rust:

- Add disable_batch_delete config option to S3Client
- When enabled, use sequential single-object deletes instead of DeleteObjects
- This is for S3-compatible services like Alibaba Cloud OSS that require
  Content-MD5 for DeleteObjects but AWS SDK v2 does not send it by default
- Add documentation and config example for OSS
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ MaojiaSheng
❌ openviking


openviking seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2026

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🏅 Score: 78
🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections
🔀 Multiple PR themes

Sub-PR theme: Add disable_batch_delete option for S3-compatible services

Relevant files:

  • crates/ragfs/src/plugins/s3fs/client.rs
  • crates/ragfs/src/plugins/s3fs/mod.rs

Sub-PR theme: Remove Go dependencies and AGFSManager

Relevant files:

  • openviking/agfs_manager.py
  • openviking/pyagfs/init.py
  • openviking/pyagfs/binding_client.py
  • openviking/service/core.py
  • openviking/storage/viking_fs.py
  • openviking/utils/agfs_utils.py
  • openviking_cli/utils/config/agfs_config.py
  • setup.py
  • Dockerfile
  • Makefile
  • MANIFEST.in
  • pyproject.toml
  • .github/workflows/*.yml

⚡ Recommended focus areas for review

Config Compatibility Break

Removed config fields (mode, port, url, impl, etc.) with model_config = {"extra": "forbid"} will cause existing config files with those fields to fail validation, breaking existing deployments.

class AGFSConfig(BaseModel):
    """Configuration for RAGFS (Rust-based AGFS)."""

    path: Optional[str] = Field(
        default=None,
        description="[Deprecated in favor of `storage.workspace`] RAGFS data storage path. This will be ignored if `storage.workspace` is set.",
    )

    backend: str = Field(
        default="local", description="RAGFS storage backend: 'local' | 's3' | 'memory'"
    )

    timeout: int = Field(default=10, description="RAGFS request timeout (seconds)")

    # S3 backend configuration
    # These settings are used when backend is set to 's3'.
    # RAGFS will act as a gateway to the specified S3 bucket.
    s3: S3Config = Field(default_factory=lambda: S3Config(), description="S3 backend configuration")

    model_config = {"extra": "forbid"}
Public API Break

Renamed AGFSBindingClient to RAGFSBindingClient and removed the old name without an alias, breaking existing code that imports AGFSBindingClient. Also removed config_impl parameter from get_binding_client().

def get_binding_client():
    """Get the RAGFS binding client class.

    Returns:
        ``(RAGFSBindingClient_class, BindingFileHandle_class)``
    """
    try:
        client, fh = _load_rust_binding()
        _logger.info("Loaded RAGFS Rust binding")
        return client, fh
    except ImportError as exc:
        raise ImportError("ragfs_python native library is not available: " + str(exc)) from exc


# Module-level defaults
# Ensure module import never fails, even if bindings are unavailable
try:
    RAGFSBindingClient, BindingFileHandle = get_binding_client()
except Exception:
    _logger.warning(
        "Failed to initialize RAGFSBindingClient during module import; "
        "RAGFSBindingClient will be None. Use get_binding_client() for explicit handling."
    )
    RAGFSBindingClient = None
    BindingFileHandle = None

__all__ = [
    "AGFSClient",
    "RAGFSBindingClient",
    "FileHandle",
    "BindingFileHandle",
    "get_binding_client",
    "AGFSClientError",
    "AGFSConnectionError",
    "AGFSTimeoutError",

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2026

PR Code Suggestions ✨

No code suggestions found for the PR.

openviking added 2 commits April 9, 2026 20:55
Add disable_batch_delete to the s3_plugin_config dict in _generate_plugin_config
so that the Python config can properly control the Rust S3FS plugin's behavior.
Copy link
Copy Markdown
Collaborator

@qin-ctx qin-ctx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Porting #1333's disable_batch_delete behavior into the Rust S3 path looks directionally correct, but this PR also removes the Go/AGFS compatibility layer. I found a couple of blocking gaps in that migration: schema field removals currently break checked-in configs/tests/examples, and cross-config validation still expects storage.agfs.url even though this PR removes that field. I also left one non-blocking suggestion about adding a regression test for the new sequential-delete path.

description="[Deprecated in favor of `storage.workspace`] RAGFS data storage path. This will be ignored if `storage.workspace` is set.",
)

backend: str = Field(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Bug] (blocking) This schema drop removes mode, port, log_level, retry_times, etc. while the repository still contains configs/tests that rely on them (for example tests/server/test_api_sessions.py, tests/session/test_session_context.py, and tests/test_prompt_manager.py). On this branch those suites now fail during config loading with Unknown config field 'storage.agfs.mode'. If the migration to binding-only RAGFS is intentional, the callers/docs/tests need to be migrated in the same PR; otherwise AGFSConfig still needs a compatibility layer.

description="Path to AGFS binding shared library. If set, use python binding instead of HTTP client. "
"Default: third_party/agfs/bin/libagfsbinding.{so,dylib}",
)
timeout: int = Field(default=10, description="RAGFS request timeout (seconds)")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Bug] (blocking) Removing url here is incomplete because openviking_cli.utils.config.open_viking_config.is_valid_openviking_config() still reads config.storage.agfs.url for the vectordb.backend == "http" and agfs.backend == "local" case. A minimal config in that mode now raises AttributeError during validation instead of a user-facing config error. This PR needs to update that cross-config validation at the same time as the schema change.

"prefix": "",
"use_ssl": true
"use_ssl": true,
"disable_batch_delete": false
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Bug] (blocking) This example now advertises disable_batch_delete, but the surrounding agfs block still contains port, log_level, and retry_times above. After the AGFSConfig schema change in this PR those fields are invalid, so copying this example into ov.conf will fail before the new S3 option can even be used.

.send()
.await
.map_err(|e| Error::internal(format!("S3 DeleteObjects error: {}", e)))?;
if self.disable_batch_delete {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] (non-blocking) Please add a regression test that exercises this branch end-to-end. #1333 existed because the old stack silently missed forwarding this flag once already, and this PR rewrites the whole Go path away. Without a test that proves disable_batch_delete=true reaches the sequential-delete path, this is easy to regress again in a future refactor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

3 participants