Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions attestation-service/docs/grpc-as.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,36 @@ PCCS are usually supported by cloud providers, you can find the steps to configu
- IBM Cloud: [Attestation with Intel SGX and Data Center Attestation Primitives (DCAP) for Virtual Servers for VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-about-attestation-sgx-dcap-vpc)
Or you can [set-up a PCCS yourself](https://download.01.org/intel-sgx/sgx-dcap/1.9/windows/docs/Intel_SGX_DCAP_Windows_SW_Installation_Guide.pdf).

When the TDX verifier is built with the pure-Rust `tdx-dcap-rust` backend,
verification collateral is cached in the AS process. The following environment
variables configure PCCS resilience:

See [PCCS Configuration and TDX Collateral Resilience](./pccs-resilience.md)
for the full configuration precedence, cache state machine, and alerting guide.

| Variable | Default | Description |
|----------|---------|-------------|
| `PCCS_URLS` | unset | Ordered, comma-separated PCCS base URLs. The verifier falls back to the next URL when an endpoint fails. |
| `PCCS_URL` | QCNL/default | Backward-compatible single PCCS URL used when `PCCS_URLS` is unset. |
| `PCCS_HTTP_TIMEOUT_SECS` | `60` | Per-request timeout before the next PCCS is tried. |
| `VERIFY_COLLATERAL_CACHE_REFRESH_HOURS` | `72` | Proactively refresh cached collateral after this age. Must be shorter than cache expiry. |
| `VERIFY_COLLATERAL_CACHE_EXPIRE_HOURS` | `168` | Age at which cached collateral becomes stale. `0` disables the cache. |
| `VERIFY_COLLATERAL_CACHE_REFRESH_RETRY_SECS` | `3600` | Backoff after a failed refresh. |

Proactive refresh is asynchronous on native AS builds. If every PCCS endpoint
is unavailable after the cache expiry, AS continues using the cached collateral
until its signed TCB/QE `nextUpdate` is reached. Refresh failures are logged as
`tdx_collateral_refresh_failed` and exposed through the generic status RPC:

```shell
grpcurl \
-plaintext \
-import-path protos \
-proto attestation.proto \
-d '{}' 127.0.0.1:50004 \
attestation.AttestationService/GetAttestationServiceStatus
```

Then an attestation request can be used to request the server. We provide an [example request of validating a SGX quote](../tests/coco-as/request.json).

You can use the [tool](https://github.com/confidential-containers/guest-components/tree/main/attestation-agent/attester#evidence-getter-tool) to generate a report on
Expand Down
117 changes: 117 additions & 0 deletions attestation-service/docs/pccs-resilience.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# PCCS Configuration and TDX Collateral Resilience

This document describes PCCS selection, failover, and collateral caching for
the pure-Rust TDX verifier selected by the `tdx-dcap-rust` or
`all-verifier-rust` feature. The Intel DCAP FFI verifier continues to use the
quote-provider/QCNL implementation and is not affected by these settings.

## PCCS endpoints

Configure an ordered fallback list with `PCCS_URLS`:

```shell
export PCCS_URLS="http://sgx-dcap-server-vpc.${region_id}.aliyuncs.com,https://sgx-dcap-server.cn-hangzhou.aliyuncs.com"
export PCCS_HTTP_TIMEOUT_SECS=5
```

Each value may be either a host base URL or a URL ending in
`/sgx/certification/v4` or `/tdx/certification/v4`. AS normalizes the suffix,
removes duplicates, and tries the endpoints in the configured order. A failure
while fetching any item in a collateral set causes AS to retry the complete set
against the next PCCS. `PCCS_HTTP_TIMEOUT_SECS` is the timeout for each PCCS
HTTP request.

PCCS configuration is resolved in this order:

1. `verifier::tdx::set_pccs_urls` when the verifier is embedded as a library.
2. The comma-separated `PCCS_URLS` environment variable.
3. The backward-compatible single `PCCS_URL` environment variable.
4. `pccs_url` or `PCCS_URL` in `/etc/sgx_default_qcnl.conf`.
5. The built-in default PCCS.

Environment variables therefore override the QCNL file. Restart AS after
changing its environment or QCNL configuration.

## Collateral cache policy

The cache is local to each AS process and is keyed by the ordered PCCS list,
FMSPC, and CA type. The PCK certificate chain embedded in a quote is not shared
through the cache.

| Variable | Default | Meaning |
|----------|---------|---------|
| `VERIFY_COLLATERAL_CACHE_REFRESH_HOURS` | `72` | Schedule an asynchronous refresh after a successful cache fill. |
| `VERIFY_COLLATERAL_CACHE_EXPIRE_HOURS` | `168` | Mark the local cache entry stale after this age. `0` disables caching. |
| `VERIFY_COLLATERAL_CACHE_REFRESH_RETRY_SECS` | `3600` | Minimum delay before another refresh after PCCS failure. |

These numeric settings are read from the environment first, then from
`/etc/sgx_default_qcnl.conf`. If the refresh age is not shorter than the cache
expiry, AS clamps it to half of the expiry and emits a warning.

The three time boundaries have different purposes:

1. **Refresh age**: a native AS schedules a background timer as soon as it
stores collateral. When the timer fires, verification continues with the
current cache while one refresh runs for that cache key. A request that
notices an overdue refresh also triggers it as a safety net.
2. **Cache expiry**: after this local TTL, the next request attempts a blocking
refresh. If every PCCS is unavailable, AS falls back to the cached value.
3. **Collateral validity**: stale fallback is allowed only while the earliest
signed TCB Info or QE Identity `nextUpdate` is still in the future. Intel
collateral is commonly issued with a validity window of about 30 days, but
AS uses the actual signed dates instead of assuming a fixed 30-day value.

Refreshes for the same key are coalesced. A slow PCCS for one FMSPC does not
block refreshes for other cache keys. Stable log event names are available for
alerting:

- `tdx_pccs_endpoint_failed`
- `tdx_pccs_failover_succeeded`
- `tdx_collateral_refresh_succeeded`
- `tdx_collateral_refresh_failed`
- `tdx_collateral_stale_fallback`

## Status API

AS exposes a generic dependency status model so other verifier dependencies can
reuse it in the future. Reading status never contacts PCCS.

For REST AS:

```shell
curl http://127.0.0.1:8080/status
```

For gRPC AS:

```shell
grpcurl \
-plaintext \
-import-path protos \
-proto attestation.proto \
-d '{}' 127.0.0.1:50004 \
attestation.AttestationService/GetAttestationServiceStatus
```

Each TDX cache entry reports a stable `kind` of `tdx-collateral-cache` and one
of these states:

| Dependency state | Meaning |
|------------------|---------|
| `not_initialized` | This AS process has not verified TDX evidence yet. |
| `ready` | Cache age is below the proactive refresh threshold. |
| `refresh_due` | Collateral is usable and refresh is due or running. |
| `degraded` | Refresh failed or local cache TTL elapsed, but signed collateral remains valid. |
| `unhealthy` | Signed collateral has passed `nextUpdate`; PCCS recovery is required. |

Useful detail fields include `pccs_urls`, `last_successful_pccs`,
`last_refresh_attempt_at`, `last_refresh_error`, `refresh_in_progress`,
`cache_age_seconds`, `collateral_expires_at`, and
`collateral_valid_for_seconds`. Timestamp fields are Unix seconds.

For alerting, trigger an early operational warning when any dependency becomes
`degraded` or `last_refresh_error` appears. Trigger a page when it becomes
`unhealthy` or when `collateral_valid_for_seconds` approaches the team's repair
SLO. With the defaults, a failed refresh is visible around day 3, the local
cache TTL is day 7, and stale fallback remains bounded by the collateral's
signed validity.
30 changes: 26 additions & 4 deletions attestation-service/docs/restful-as.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,32 @@ PCCS are usually supported by cloud providers, you can find the steps to configu
Or you can [set-up a PCCS yourself](https://download.01.org/intel-sgx/sgx-dcap/1.9/windows/docs/Intel_SGX_DCAP_Windows_SW_Installation_Guide.pdf).

When the TDX verifier is built with the pure-Rust `tdx-dcap-rust` backend,
verification collateral is cached in the AS process. The cache lifetime uses
`VERIFY_COLLATERAL_CACHE_EXPIRE_HOURS` from the environment or QCNL config and
defaults to 168 hours. Set the value to `0` to disable the cache. Concurrent
cache misses are coalesced to avoid overloading PCCS during startup.
verification collateral is cached in the AS process. The following environment
variables configure PCCS resilience:

See [PCCS Configuration and TDX Collateral Resilience](./pccs-resilience.md)
for the full configuration precedence, cache state machine, and alerting guide.

| Variable | Default | Description |
|----------|---------|-------------|
| `PCCS_URLS` | unset | Ordered, comma-separated PCCS base URLs. The verifier falls back to the next URL when an endpoint fails. |
| `PCCS_URL` | QCNL/default | Backward-compatible single PCCS URL used when `PCCS_URLS` is unset. |
| `PCCS_HTTP_TIMEOUT_SECS` | `60` | Per-request timeout before the next PCCS is tried. |
| `VERIFY_COLLATERAL_CACHE_REFRESH_HOURS` | `72` | Proactively refresh cached collateral after this age. Must be shorter than cache expiry. |
| `VERIFY_COLLATERAL_CACHE_EXPIRE_HOURS` | `168` | Age at which cached collateral becomes stale. `0` disables the cache. |
| `VERIFY_COLLATERAL_CACHE_REFRESH_RETRY_SECS` | `3600` | Backoff after a failed refresh. |

Proactive refresh is asynchronous on native AS builds, so verification requests
continue using the last successful collateral. If all PCCS endpoints are down
after the cache expiry, the stale cache remains usable until the signed TCB/QE
`nextUpdate` is reached. Refresh failures are logged with the stable event
`tdx_collateral_refresh_failed` and exposed by the status API.

`GET /status` returns AS and verifier dependency state without contacting PCCS:

```shell
curl http://127.0.0.1:8080/status
```

Then an attestation request can be used to request the server. We provide an [example request of validating a SGX quote](../tests/coco-as/request.json).

Expand Down
41 changes: 38 additions & 3 deletions attestation-service/src/bin/grpc/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ use tonic::{Request, Response, Status};
use crate::as_api::attestation_service_server::{AttestationService, AttestationServiceServer};
use crate::as_api::{
list_policies_response::PolicyInfo, AttestationRequest, AttestationResponse, ChallengeRequest,
ChallengeResponse, DeletePolicyRequest, DeletePolicyResponse, GetPolicyRequest,
GetPolicyResponse, ListPoliciesRequest, ListPoliciesResponse, SetPolicyRequest,
SetPolicyResponse,
ChallengeResponse, DeletePolicyRequest, DeletePolicyResponse,
DependencyStatus as GrpcDependencyStatus, GetAttestationServiceStatusRequest,
GetAttestationServiceStatusResponse, GetPolicyRequest, GetPolicyResponse, ListPoliciesRequest,
ListPoliciesResponse, SetPolicyRequest, SetPolicyResponse,
};

use crate::rvps_api::reference_value_provider_service_server::{
Expand Down Expand Up @@ -87,6 +88,40 @@ impl AttestationServer {

#[tonic::async_trait]
impl AttestationService for Arc<RwLock<AttestationServer>> {
async fn get_attestation_service_status(
&self,
_request: Request<GetAttestationServiceStatusRequest>,
) -> Result<Response<GetAttestationServiceStatusResponse>, Status> {
let status = self.read().await.attestation_service.status().await;
let dependencies = status
.dependencies
.into_iter()
.map(|dependency| GrpcDependencyStatus {
kind: dependency.kind,
name: dependency.name,
status: dependency.status,
message: dependency.message.unwrap_or_default(),
details: dependency
.details
.into_iter()
.map(|(key, value)| {
let value = match value {
serde_json::Value::String(value) => value,
value => value.to_string(),
};
(key, value)
})
.collect(),
})
.collect();

Ok(Response::new(GetAttestationServiceStatusResponse {
service: status.service,
status: status.status,
dependencies,
}))
}

async fn set_attestation_policy(
&self,
request: Request<SetPolicyRequest>,
Expand Down
6 changes: 5 additions & 1 deletion attestation-service/src/bin/restful-as.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ use tokio::sync::RwLock;

use crate::restful::{
attestation, delete_policy, get_certificate, get_challenge, get_jwks, get_openid_configuration,
get_policies, set_policy,
get_policies, get_status, set_policy,
};

mod restful;
Expand Down Expand Up @@ -76,6 +76,9 @@ enum WebApi {

#[strum(serialize = "/.well-known/openid-configuration")]
OpenIdConfiguration,

#[strum(serialize = "/status")]
Status,
}

#[derive(Error, Debug)]
Expand Down Expand Up @@ -178,6 +181,7 @@ async fn main() -> Result<(), RestfulError> {
web::resource(WebApi::Certificate.as_ref()).route(web::get().to(get_certificate)),
)
.service(web::resource(WebApi::Jwks.as_ref()).route(web::get().to(get_jwks)))
.service(web::resource(WebApi::Status.as_ref()).route(web::get().to(get_status)))
.service(
web::resource(WebApi::OpenIdConfiguration.as_ref())
.route(web::get().to(get_openid_configuration)),
Expand Down
7 changes: 7 additions & 0 deletions attestation-service/src/bin/restful/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,13 @@ fn parse_init_data(data: InitDataInput) -> Result<InnerInitDataInput> {
Ok(res)
}

/// Return transport-neutral AS and verifier dependency status. This endpoint
/// does not contact PCCS; it only snapshots the in-process state.
pub async fn get_status(cocoas: web::Data<Arc<RwLock<AttestationService>>>) -> impl Responder {
let status = cocoas.read().await.status().await;
web::Json(status)
}

/// This handler uses json extractor
pub async fn attestation(
request: web::Json<AttestationRequest>,
Expand Down
29 changes: 29 additions & 0 deletions attestation-service/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,17 @@ pub struct AttestationService {
token_broker: Box<dyn AttestationTokenBroker + Send + Sync>,
}

/// Transport-neutral runtime status exposed by REST and gRPC AS binaries.
/// Verifier dependencies use a generic list so future KDS, RIM, registrar, or
/// other cache-backed integrations can report state without changing the
/// top-level API shape.
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct ServiceStatus {
pub service: String,
pub status: String,
pub dependencies: Vec<verifier::DependencyStatus>,
}

#[derive(serde::Serialize, Debug, Clone)]
struct Jwk {
kty: String,
Expand Down Expand Up @@ -256,6 +267,24 @@ impl AttestationService {
})
}

/// Return AS and verifier dependency status without performing network I/O.
pub async fn status(&self) -> ServiceStatus {
let dependencies = verifier::dependency_statuses().await;
let status = if dependencies.iter().any(|status| status.is_unhealthy()) {
"unhealthy"
} else if dependencies.iter().any(|status| status.is_degraded()) {
"degraded"
} else {
"ok"
};

ServiceStatus {
service: "attestation-service".into(),
status: status.into(),
dependencies,
}
}

/// Set Attestation Verification Policy.
pub async fn set_policy(&mut self, policy_id: String, policy: String) -> Result<()> {
self.token_broker.set_policy(policy_id, policy).await?;
Expand Down
2 changes: 2 additions & 0 deletions deps/verifier/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ tdx-dcap-rust = [
"asn1-rs",
"urlencoding",
"chrono",
"tokio/rt",
"tokio/time",
]
sgx-verifier = ["scroll", "intel-tee-quote-verification-rs"]
az-snp-vtpm-verifier = ["az-snp-vtpm", "sev", "snp-verifier"]
Expand Down
19 changes: 19 additions & 0 deletions deps/verifier/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ use log::debug;
mod error;
pub use error::Error as VerifierError;

pub mod status;
pub use status::DependencyStatus;

pub mod sample;
pub mod sample_device;

Expand Down Expand Up @@ -247,6 +250,22 @@ pub trait Verifier {
}
}

/// Return runtime status for verifier dependencies that support introspection.
///
/// An empty list means no enabled verifier currently exposes dependency state;
/// it does not mean the verifier service is unhealthy.
pub async fn dependency_statuses() -> Vec<DependencyStatus> {
#[cfg(feature = "tdx-dcap-rust")]
{
tdx::dependency_statuses().await
}

#[cfg(not(feature = "tdx-dcap-rust"))]
{
Vec::new()
}
}

#[allow(dead_code)]
/// Padding or truncate the given data slice to the given `len` bytes.
fn regularize_data(data: &[u8], len: usize, data_name: &str, arch: &str) -> Vec<u8> {
Expand Down
Loading
Loading