Skip to content

ADR devices export api

Jump to bottom
ShradhaGupta31 edited this page Aug 10, 2026 · 11 revisions

Design: /api/v1/devices/export — Device Data Export API

GitHub Issue: device-management-toolkit/console#1076

Parent: Console Discovery Dashboard & Device Data Export #1093

Status: Proposed

Overview

Introduce a new Console API endpoint that exports all device data (including deviceInfo) in JSON format. This enables fleet operators to extract a complete snapshot of managed and discovered devices for reporting, auditing, and tooling integration.

Endpoint: GET /api/v1/devices/export

Proposed JSON Schema

Three options were evaluated. Option C is the chosen approach.

Option A — Legacy Flat Model (Current DB Shape)

{
  "guid": "143e4567-e89b-12d3-a456-426614174000",
  "hostname": "lab-pc-01",
  "friendlyName": "Lab PC Alpha",
  "tags": ["campus-lab", "shared-device"],
  "tenantId": "",
  "dnsSuffix": "corp.example.com",
  "connectionStatus": true,
  "lastConnected": "2026-07-07T05:30:00Z",
  "lastSeen": "2026-07-07T06:00:00Z",
  "lastDisconnected": null,
  "deviceInfo": {
    "amtEnabledInBIOS": true,
    "currentMode": "Admin",
    "discovered": true,
    "fwVersion": "16.1.32",
    "fwBuild": "3400",
    "fwSku": "16392",
    "features": "AMT Pro Corporate",
    "lmsInstalled": true,
    "lmsVersion": "2410.5.0.0",
    "meInterfaceVersion": "16.1.25.2124",
    "tlsMode": "TLS 1.2",
    "ipAddress": "10.0.0.12",
    "osName": "linux",
    "osVersion": "6.8.0-51-generic",
    "osDistro": "Ubuntu 24.04 LTS",
    "osIpAddress": "10.49.76.163",
    "cpuModel": "Intel(R) Core(TM) Ultra 7 165H",
    "dhcpEnabled": true,
    "ethernetAdapterCount": 2,
    "monitorConnected": true,
    "ieee8021xEnabled": false,
    "certHashes": ["a1b2c3", "d4e5f6"],
    "upid": {
      "csmeId": "4A45A39C5ED9462082510000",
      "oemId": "",
      "oemPlatformIdType": "Not Set (0)"
    },
    "lastUpdated": "2026-05-21T00:00:00Z"
  }
}

Decision for Option A: All AMT, OS, and peripheral fields are mixed into a single flat blob. As more peripherals are added (e.g. BMC, future hardware interfaces), the flat list grows unmanageably. Consumers of the export have no easy way to understand which fields belong to which subsystem without reading documentation.

Option B — Nested Subsystem Model (Initial Draft)

{
  "guid": "143e4567-e89b-12d3-a456-426614174000",
  "hostname": "lab-pc-01",
  "friendlyName": "Lab PC Alpha",
  "tags": ["campus-lab", "shared-device"],
  "dnsSuffix": "corp.example.com",
  "deviceInfo": {
    "currentMode": "Admin",
    "amt": {
      "amtEnabledInBIOS": true,
      "fwVersion": "16.1.32",
      "fwBuild": "3400",
      "fwSku": "16392",
      "features": "AMT Pro Corporate",
      "tlsMode": "TLS 1.2",
      "lmsInstalled": true,
      "lmsVersion": "2410.5.0.0",
      "meInterfaceVersion": "16.1.25.2124",
      "dhcpEnabled": true,
      "certHashes": ["a1b2c3", "d4e5f6"],
      "upid": {
        "csmeId": "4A45A39C5ED9462082510000",
        "oemId": "",
        "oemPlatformIdType": "Not Set (0)"
      },
      "network": {
        "wired": {
          "ipAddress": "10.0.0.12",
          "dhcp": true,
          "static": false
        },
        "wireless": {
          "ipAddress": "192.168.1.20"
        }
      }
    },
    "os": {
      "name": "linux",
      "version": "6.8.0-51-generic",
      "distro": "Ubuntu 24.04 LTS",
      "cpu": "Intel(R) Core(TM) Ultra 7 165H",
      "monitorConnected": true,
      "ethernetAdapterCount": 2,
      "ieee8021xEnabled": false,
      "network": {
        "wired": [
          {
            "name": "eth0",
            "ipAddress": "10.49.76.163",
            "macAddress": "AA:BB:CC:DD:EE:FF"
          },
          {
            "name": "eth1",
            "ipAddress": "10.49.76.164",
            "macAddress": "AA:BB:CC:DD:EE:00"
          }
        ],
        "wireless": {
          "ipAddress": "10.0.2.25"
        }
      }
    },
    "bmc": {
      "vendor": "string",
      "model": "string",
      "firmwareVersion": "string"
    }
  }
}

Fields are grouped by subsystem (amt, os, bmc). Adding new peripheral data tomorrow (e.g. BMC firmware version, future HW interfaces) means adding a new top-level section under deviceInfo — no consumer needs to guess which fields belong where. Network details are scoped to the subsystem that reported them (amt.network vs os.network).

Decision for Option B: Better than a flat blob, but this needs slight modifications. Things like CPU model and physical network adapters don't really belong under os, they're platform info. Also, calling the section amt is misleading since it also covers ISM devices. We took this structure and cleaned it up in Option C.

Option C — Refined Nested Model with Platform Split (Chosen)

Decision: Option C is chosen. This is the cleaned-up version of Option B. amt is renamed to me so it works for both AMT and ISM devices. Detailed interface/IP data now lives under os.network, while platform.adapters provides a adapter-name summary (wired and wireless). Each section has a clear purpose, and adding support for new hardware down the line just means adding a new key — nothing else needs to change.

Data Grouping Overview

The export organizes device information into logical subsystems for clarity and extensibility:

graph TB
    Export["Device Export Response"]
    Export --> Metadata["Metadata<br/>exportedAt, filtersApplied<br/>swVersion"]
    Export --> Summary["Summary Statistics<br/>totalCount"]
    Export --> Data["Data Array<br/>Individual Device Records"]

    Data --> Device["Device Record<br/>guid, hostname, friendlyName<br/>tags"]
    Device --> DeviceInfo["deviceInfo Object"]

    DeviceInfo --> ME["Management Engine - me<br/>AMT/ISM firmware, TLS config<br/>network settings, certificates, UPID"]
    DeviceInfo --> OS["Operating System - os<br/>OS name, version, distro<br/>LMS Info, monitors, 802.1X<br/>network interfaces and IPs"]
    DeviceInfo --> Platform["Platform Details - platform<br/>CPU model, adapter count<br/>adapter-name summary"]
    DeviceInfo --> BMC["Baseboard Mgmt - bmc<br/>High-level summary"]

    ME --> MENetwork["ME Network<br/>wired IP, DHCP, wireless IP"]
    OS --> OSNetwork["OS Network<br/>wired interfaces with IPs and MACs<br/>wireless IP"]
    Platform --> PlatformAdapters["Platform Adapter Summary<br/>wired adapter name<br/>wireless adapter name"]
Loading

Overall Export Response (All Groups)

{
  "metadata": {
    "exportedAt": "2026-07-22T16:45:00Z",
    "filtersApplied": {
      "status": "all",
      "tags": ["vPro", "lab"]
    },
    "swVersion": "console v1.38.1"
  },
  "summary": {
    "totalCount": 2
  },
  "data": [
    {
      "guid": "143e4567-e89b-12d3-a456-426614174000",
      "...": "..."
      "deviceInfo": {
        "...": "..."
      }
    },
    {
      "guid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
       "...": "..."
      "deviceInfo": {
        "...": "..."
      }
    }
  ]
}

Example Device Record - ME Device

{
  "guid": "143e4567-e89b-12d3-a456-426614174000",
  "hostname": "lab-pc-01",
  "friendlyName": "Lab PC Alpha",
  "tags": ["campus-lab", "shared-device"],
  "tenantId": "tenant1",
  "firstDiscovered": "2025-06-15T10:30:00Z",
  "lastSynced": "2026-07-20T14:22:15Z",
  "lastUpdated": "2026-07-20T14:22:15Z",
  "deviceInfo": {
    "me": {
      "dnsSuffix": "corp.example.com",
      "currentMode": "Admin",
      "mebxEnabledInBIOS": true,
      "fwVersion": "16.1.32",
      "fwBuild": "3400",
      "fwSku": "16392",
      "features": "AMT Pro Corporate",
      "tlsMode": "TLS 1.2",
      "dhcpEnabled": true,
      "certHashes": ["a1b2c3xxx", "d4e5f6xxx"],
      "upid": {
        "csmeId": "4A45A39C5ED9462082510000",
        "oemId": "",
        "oemPlatformIdType": "Not Set (0)"
      },
      "network": {
        "wired": {
          "ipAddress": "10.0.0.12",
          "dhcpEnabled": true,
          "dhcpMode": "active",
          "linkStatus": "up",
          "macAddress": "AA:BB:CC:DD:EE:10"
        },
        "wireless": {
          "ipAddress": "192.168.1.20",
          "dhcpEnabled": true,
          "dhcpMode": "passive",
          "linkStatus": "up",
          "macAddress": "AA:BB:CC:DD:EE:20"
        }
      }
    },
    "os": {
      "dnsSuffix": "vpro.demo.com",
      "name": "linux",
      "version": "6.8.0-51-generic",
      "distro": "Ubuntu 24.04 LTS",
      "lmsInstalled": true,
      "lmsVersion": "2410.5.0.0",
      "meInterfaceVersion": "16.1.25.2124",
      "monitorConnected": true,
      "ieee8021xEnabled": false,
      "network": {
        "wired": [
          {
            "name": "eth0",
            "ipAddress": "10.49.76.163",
            "dhcpEnabled": true,
            "linkStatus": "up",
            "macAddress": "AA:BB:CC:DD:EE:FF"
          },
          {
            "name": "eth1",
            "ipAddress": "10.49.76.164",
            "dhcpEnabled": false,
            "linkStatus": "up",
            "macAddress": "AA:BB:CC:DD:EE:00"
          }
        ],
        "wireless": {
          "name": "wlan0",
          "ipAddress": "10.0.2.25",
          "dhcpEnabled": true,
          "linkStatus": "up",
          "macAddress": "AA:BB:CC:DD:EE:30"
        }
      }
    },
    "platform": {
      "cpu": "Intel(R) Core(TM) Ultra 7 165H",
      "ethernetAdapterCount": 2,
      "adapters": {
        "wired": "Intel Corporation Ethernet Controller I226-LM (rev 04)",
        "wireless": "Intel Corporation Meteor Lake PCH CNVi WiFi (rev 20)"
      }
    },
    "bmc": null
  }
}

Note on BMC fields: Currently restricted to high-level summary (vendor, model, firmware version). Future versions will expand to include sensors, event logs, firmware upgrade capabilities, and detailed health status.

Example Device Record — Non-ME Device

For devices without Management Engine capabilities (e.g., standard Intel systems, non-vPro workstations, or non-Intel platforms), the me field is null:

{
  "guid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "hostname": "standard-pc-05",
  "friendlyName": "Standard Workstation",
  "tags": ["standard-fleet"],
  "tenantId": "tenant1",
  "firstDiscovered": "2025-08-10T09:15:00Z",
  "lastSynced": "2026-07-15T11:45:30Z",
  "lastUpdated": "2026-07-15T11:45:30Z",
  "deviceInfo": {
    "me": null,
    "os": {
      "dnsSuffix": "corp.example.com",
      "name": "linux",
      "version": "6.8.0-51-generic",
      "distro": "Ubuntu 24.04 LTS",
      "lmsInstalled": true,
      "lmsVersion": "2410.5.0.0",
      "meInterfaceVersion": "16.1.25.2124",
      "monitorConnected": true,
      "ieee8021xEnabled": false,
      "network": {
        "wired": [
          {
            "name": "eth0",
            "ipAddress": "10.49.76.163",
            "dhcpEnabled": true,
            "linkStatus": "up",
            "macAddress": "AA:BB:CC:DD:EE:FF"
          },
          {
            "name": "eth1",
            "ipAddress": "10.49.76.164",
            "dhcpEnabled": false,
            "linkStatus": "up",
            "macAddress": "AA:BB:CC:DD:EE:00"
          }
        ],
        "wireless": {
          "name": "wlan0",
          "ipAddress": "0.0.0.0",
          "dhcpEnabled": true,
          "linkStatus": "down",
          "macAddress": "00:00:00:00:00:00"
        }
      }
    },
    "platform": {
      "cpu": "Intel(R) Core(TM) Ultra 7 165H",
      "ethernetAdapterCount": 2,
      "adapters": {
        "wired": "Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)",
        "wireless": "Realtek RTL8821CE 802.11ac PCIe Wireless Network Adapter"
      }
    },
    "bmc": null
  }
}

Field Gap Analysis — Entities Not Currently Sent by rpc-go

The Option C schema is the target shape. However, several fields it defines are not currently collected/populated by rpc-go (rpc amtinfo --sync) and therefore do not exist in the stored deviceInfo today. The list below was derived by comparing the Option C schema against the live GET /api/v1/devices payload from the database.

Fields NOT sent by rpc-go (gaps to close)

Group Schema field Status Notes
record lastUpdated Missing Only firstDiscovered and lastSynced exist today. Needs a writer to stamp last-update time.
me tlsMode Missing Not collected. DB only exposes top-level useTLS / allowSelfSigned, not a resolved TLS mode string.
me network.wired (ipAddress, dhcpEnabled, dhcpMode, linkStatus, macAddress) Missing No ME wired network object today; only a single flat ipAddress. Requires extended ME interface collection.
me network.wireless (ipAddress, dhcpEnabled, dhcpMode, linkStatus, macAddress) Missing No ME wireless network data collected at all.
os dnsSuffix Missing Only a single top-level dnsSuffix exists; a distinct OS-reported DNS suffix is not collected.
os ieee8021xEnabled Missing Not collected by rpc-go.
os network.wired[] (name, per-interface ipAddress, dhcpEnabled, linkStatus, macAddress) Missing DB stores only a single osIpAddress string — no per-adapter list, names, MACs, or link status. Requires extended OS telemetry.
os network.wireless (name, ipAddress, dhcpEnabled, linkStatus, macAddress) Missing No OS wireless interface data collected.
platform adapters.wired (adapter name) Missing Adapter model/name string not collected; only ethernetAdapterCount exists.
platform adapters.wireless (adapter name) Missing Wireless adapter model/name string not collected.

API Proposal

Request

GET /api/v1/devices/export
Authorization: Bearer <token>
Accept: application/json

Filtering

Note: Filtering is not implemented in the initial release but is planned for a future iteration.

Parameter Type Default Description
tags string (none) Comma-separated list; returns devices matching ANY tag
status string all Values: connected, disconnected, all
discovered boolean N/A If true, returns only discovered devices

HTTP Endpoint Specification

Endpoint: GET /api/v1/devices/export

Authentication: Bearer JWT token from POST /api/v1/authorize

Headers:

Header Value Required Notes
Authorization Bearer <jwt_token> Yes Signed JWT; expiration enforced per auth.jwtExpiration

Query Example:

GET /api/v1/devices/export?tags=vPro,production&status=connected

Response (Status 200 - JSON):

Conforms to the JSON Schema defined in Option C. Returns an object with metadata, summary, and data array.

Error Responses:

  • 401 Unauthorized — Missing or invalid JWT token
  • 400 Bad Request — Invalid query parameter (e.g., malformed tag list)
  • 403 Forbidden — Authenticated user lacks export permissions
  • 500 Internal Server Error — Database or export generation failure

Open Questions

1. Where will wired/wireless network fields come from?

The proposed JSON example includes me.network.wired, me.network.wireless, platform.adapters.wired[], and platform.adapters.wireless. These fields do not currently exist in DeviceInfo.

  • Does this require extending rpc amtinfo --sync to collect network adapter details?
  • Does this require a DB schema update (new fields in the deviceInfo JSON blob)?

Sudhir:

  • For Management Engine (me) network: wired/wireless IP addresses come from existing AMT fields already collected by rpc amtinfo --sync. These are part of the standard ME interface query.
  • For Operating System (os) network: detailed adapter and IP information requires extended OS telemetry collection. This may require:
    • Enhancement to rpc amtinfo --sync to collect detailed OS network adapter list
    • DB schema update to store the expanded OS network details in deviceInfo

2. DB schema alignment

The proposed export JSON uses a nested structure (me.{}, os.{}, platform.{}, bmc.{}), but the DB today stores deviceInfo as a flat JSON blob.

  • Do we need to update the DB schema to match the nested structure, or do we keep the flat blob and transform it at the API layer on export?
  • If we keep the flat DB schema and only reshape on export: any future change to the DB field names or structure will silently break the export API. Who is responsible for keeping the mapping in sync?
  • If we update the DB to store the nested structure: this is a breaking migration — all existing deviceInfo rows need to be migrated, and every writer (rpc amtinfo --sync, Console API) needs to be updated.

Sudhir:

  • Strategy: Keep the database schema flat (minimal breaking changes), and perform the nested transformation at the API layer (in the export endpoint).
  • Responsibility: The export API handler is responsible for mapping flat DB fields → nested JSON response. A mapping helper will document the flat→nested transformation.
  • Future Migration Path: If Console adopts nested deviceInfo at the DB level, the export API will need to evolve accordingly. This will be driven by a separate ADR when the time comes.

3. Do we need export metadata?

Should the response include metadata fields such as:

  • exportedAt — timestamp when export was generated
  • exportedBy — authenticated user ID
  • exportFormat — schema version for versioning
  • filterApplied — which filters were active for this export

Sudhir:

  • Yes, these metadata fields are included in the response schema.
  • exportedAt and swVersion are required for auditing and version traceability.
  • filtersApplied is optional to support scenarios where the export runs without filter context.
  • This enables downstream tooling to understand when the export occurred, which software version produced it, and what filters were applied.

4. Do we need a summary section?

Should the export include aggregated counts?

  • Total devices
  • activatedCount vs. not activated count
  • discoveredCount
  • vProCount vs. ISMCount

Sudhir:

  • Current: Include only totalCount for simplicity. This is the most critical metric and unblocks the feature.
  • Future enhancements (Phase 2+): Add activatedCount, discoveredCount, and distribution breakdowns (byOperatingSystem, byTlsMode) once operational dashboards require them.

5. Export DTO — Do not reuse the existing Device DTO?

Do not reuse the existing Device DTO. It contains sensitive fields such as mpsPassword and mebxPassword. Please use a dedicated export DTO with an explicit allowlist to ensure sensitive data is never exported.

Shradha:

Agreed, the existing Device entity will not be reused here. The problem is it carries mpsPassword, mebxPassword, and other credential fields that have no business appearing in an export file. A dedicated DeviceExportDto with an explicit allowlist is the right approach.

The allowlist covers exactly what Option C defines: guid, hostname, friendlyName, tags, tenantId, firstDiscovered, lastSynced, lastUpdated, and the nested deviceInfo sub-objects. Nothing else gets through. The standing rule: any new field added to the device model has to be explicitly opted into DeviceExportDto. It is never included by default.

6. Why was tenant ID removed? It was included in Option A but is missing in Option C. Please clarify whether the export is tenant-scoped and how tenant isolation will be enforced.

Shradha:

tenantId was accidentally dropped during the Option B → Option C cleanup. It has been added back to the schema.

On scoping: the export is tenant-scoped. Each call returns only the devices that belong to the caller's tenant, resolved from the tenantId claim in the JWT. Tenant isolation is enforced server-side in the repository query (a WHERE tenant_id = ? predicate for SQL, and the equivalent filter for MongoDB).

7. Keep the export output consistent across all databases.

SQL and MongoDB should return the same fields and follow the same export schema so consumers get a consistent experience regardless of the underlying datastore.

Shradha:

Schema shaping should happen in exactly one place: the export handler. Both the SQL and MongoDB repositories just return the raw stored record, they don't shape anything. A flatToNested mapping function inside the export handler is responsible for converting that raw data into the final DeviceExportDto. This way, consumers always get the same JSON structure regardless of which database is running underneath.

8. Why was pagination removed?

Existing device APIs support pagination, but the export endpoint does not. If the intent is to export the full fleet, the ADR should explicitly justify this decision and define limits, large-export handling, and failure behavior.

Shradha:

Pagination was intentionally left out because this endpoint serves a different purpose than the regular device list API. When an operator hits /export, they want a complete snapshot of their tenant's fleet. Pagination would mean making multiple calls, stitching pages back together, and hoping nothing changed between requests which might add complexity.

That said, returning an unbounded dataset is not safe either, so we will put guardrails in place: a hard cap on the number of records returned, a request timeout, and a response header that tells the client how many records are coming. If the query times out or the database throws an error mid-way, the entire response is discarded and a 503 is returned so that we never send a partial export file.

9. Have you considered server-side audit logging for exports (who exported, source IP/server, device count, timestamp, etc.)?

I know metadata is planned in the JSON output, but server-side logs provide a more persistent audit trail. This may be worth discussing during the demo.

Shradha:

Yes, this is worth adding. The plan is to log each export call to the same structured logger Console already uses. Here is what gets captured per call:

Field Source
timestamp Server clock (UTC)
userId sub claim from the JWT
tenantId tenantId claim from the JWT
sourceIp X-Forwarded-For / RemoteAddr (proxy-aware)
serverHostname os.hostname() or pod name in Kubernetes
filtersApplied Parsed query parameters
deviceCount Number of records included in the export
durationMs Time taken to generate the export

Failed attempts like auth failures, over-limit rejections, timeouts etc. are also logged with an outcome field so they are easy to distinguish. None of this ends up in the HTTP response; it stays server-side only.

Console Wiki

ADRs

Clone this wiki locally