[Data] Fix SSRF vulnerability in DatabricksUCDatasource external URL …

[hoamgh]

Fix SSRF vulnerability in DatabricksUCDatasource (fixes #65669).

databricks_uc_datasource.py unconditionally calls requests.get(external_url)
with a URL from the Databricks API response JSON, without any SSRF validation.
A compromised or MITM'd server could redirect cluster nodes to internal services
(e.g., cloud metadata at 169.254.169.254) for data exfiltration.

Changes:

• Add _validate_external_url() that enforces HTTPS-only and blocks
  private/loopback/link-local/reserved IPs via DNS resolution
• Add 9 unit tests covering scheme validation, IP blocking, DNS failure,
  and end-to-end SSRF prevention

Related issue number

Fixes #65669

Checks

• I've signed all my commits
• I've run scripts/format.sh to lint the changes in this PR
• I've included any doc changes needed for https://docs.ray.io/en/master/
• I've added any new APIs to the API Reference
• I've made sure the tests are passing

[gemini-code-assist]

Code Review

This pull request introduces SSRF validation for external URLs in the Databricks UC datasource by verifying that URLs use HTTPS and do not resolve to private, loopback, link-local, or reserved IP addresses. Comprehensive unit tests are also added to cover these validation scenarios. The review feedback highlights a critical security vulnerability where the validation can be bypassed via HTTP redirects, as requests.get follows redirects by default. It is recommended to disable automatic redirects by setting allow_redirects=False and explicitly raising an error for 3xx status codes.

[gemini-code-assist]

By default, requests.get automatically follows HTTP redirects (3xx status codes). If a compromised or malicious Databricks server returns an external URL that redirects to an internal/private IP address (e.g., http://169.254.169.254), the initial _validate_external_url check will be bypassed because it only validates the initial URL. This leads to a redirect-based SSRF vulnerability.

To prevent this, we should disable automatic redirects by setting allow_redirects=False and explicitly raise an error if a 3xx redirect status code is returned.

Additionally, note that there is a potential Time-of-Check to Time-of-Use (TOCTOU) DNS Rebinding vulnerability because the DNS resolution in _validate_external_url is separate from the DNS resolution performed by requests.get. A malicious DNS server could return a public IP during validation and a private IP during the actual request. While resolving DNS rebinding completely in Python requests requires a custom transport adapter or DNS resolver, disabling redirects is a critical first step that mitigates the most common redirect-based SSRF vectors.

Suggested change

	raw_response = requests.get(
	external_url, auth=None, headers=None
	)
	raw_response.raise_for_status()
	raw_response = requests.get(
	external_url, auth=None, headers=None, allow_redirects=False
	)
	if 300 <= raw_response.status_code < 400:
	raise ValueError(
	f"Redirection is not allowed for Databricks external URLs "
	f"to prevent SSRF attacks, got status: {raw_response.status_code}"
	)
	raw_response.raise_for_status()

[cursor]

DNS rebinding bypasses URL validation

Medium Severity

Hostname resolution in _validate_external_url is not bound to the later requests.get. A compromised Databricks response can return an attacker-controlled host that answers with a public IP during validation and a private or link-local IP when the fetch runs, bypassing the SSRF checks.

 

^{Reviewed by Cursor Bugbot for commit d99c0f0. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{Reviewed by Cursor Bugbot for commit 9115c1a. Configure here.}

[cursor]

SSRF filter misses CGNAT address range

Medium Severity

_validate_external_url treats an address as safe unless it is loopback, link-local, reserved, multicast, unspecified, or is_private. In Python, 100.64.0.0/10 is neither is_private nor is_global, so CGNAT and similar internal addresses pass, including well-known cloud metadata such as 100.100.100.200. A compromised Databricks response can still send workers at those hosts.

 

^{Reviewed by Cursor Bugbot for commit 9115c1a. Configure here.}