# node_exporter pcidevice collector: NUMA, SR-IOV and PCIe power state metrics

> PCIe link width, NUMA node, SR-IOV virtual functions, power state and vendor names as Prometheus metrics: what each one reports and how to turn them on.

- URL: https://blog.asama.ai/blog/pci-device-metrics-in-prometheus-node-exporter/
- Author: Jain Johny
- Published: 2025-09-29
- Tags: Field notes

node_exporter's new pcidevice collector turns the PCIe state in sysfs into Prometheus metrics. It reports link width and speed, and our PR #3425 adds the NUMA node a device is attached to, SR-IOV virtual function counts, power state and, optionally, vendor and device names. Those are the signals behind the PCIe problems dashboards miss, such as [a link that trained at x8 instead of x16](https://blog.asama.ai/blog/article-02-asama-where-the-loop-always-starts).

## Where the data comes from: procfs PR #748

node_exporter reads sysfs through [prometheus/procfs](https://github.com/prometheus/procfs), and we contribute our fixes upstream rather than carry forks. Our [PR #748](https://github.com/prometheus/procfs/pull/748) extends the library's `PciDevice` struct with fields read from `/sys/bus/pci/devices/<address>/`:

- `numa_node`: the NUMA node the device is attached to, or -1 when the firmware does not say
- the SR-IOV attributes: `sriov_numvfs`, `sriov_totalvfs`, `sriov_drivers_autoprobe`, `sriov_offset`, `sriov_stride`, `sriov_vf_device` and `sriov_vf_total_msix`
- `power_state`: the device's current PCI power state, from D0 to D3cold
- `d3cold_allowed`: whether the device may be put into D3cold

## What PR #3425 adds to the pcidevice collector

[PR #3425](https://github.com/prometheus/node_exporter/pull/3425) builds on those fields in three ways.

### 1. Extended PCI Metrics
The pcidevice collector is now enriched with several new fields:

- NUMA node: which NUMA node the device is attached to.
- SR-IOV details: the virtual functions enabled, the most the device supports, and the MSI-X vectors available to them.
- SR-IOV drivers autoprobe: whether newly enabled virtual functions are bound to a driver automatically.
- Power state and D3cold: the device's current power state, and whether it may be put into D3cold.

Previously, collecting this data required a custom textfile collector. With this change, these attributes become first-class metrics in node_exporter.

### 2. ID → Name Conversion
A highly ergonomic addition: numeric PCI IDs (vendor, device, class) can now be optionally mapped to human-readable names.

Example:

Before → `{vendor_id="0x8086"}`
After → `{vendor_id="0x8086", vendor_name="Intel Corporation"}`

This mapping relies on the system's `pci.ids` file (or a user-specified alternative). It's disabled by default to minimise overhead, but for dashboards, debug logs, and alerts, the readability boost is huge.

### 3. Nil-Pointer Checks
Alongside the new features, PR #3425 adds nil-pointer checks across optional fields in the `sysfs.PciDevice` struct.

Why this is important:

- Not all sysfs entries exist consistently across kernels, drivers, or hardware types.
- Without these checks, collector could panic when trying to read missing fields.
- With them, the collector gracefully skips unavailable data, keeping metrics flowing reliably.

## Why This Matters (Especially for Infra Teams)

### Better Observability & Context
These enhancements unlock deeper insights into machine hardware, particularly in **high-performance, virtualised, or containerised environments**:

- Detect and diagnose **NUMA locality issues** that impact performance.
- Monitor the **power state** of every PCI device, and whether it may enter D3cold.
- Validate and observe **SR-IOV setups**, critical for NICs and accelerators.
- Build **cleaner dashboards** with vendor and device names instead of cryptic hex codes.
- Rely on **robust collectors** that won't break due to missing sysfs entries.

### Opt-In by Design
The optional nature of ID-to-name conversion is deliberate: users can enable richer context where needed, without forcing additional dependencies on minimal setups.

## Conclusion
At Asama.ai, we believe in strengthening the open-source ecosystem we rely on. Contributing improvements like these ensures broader community benefit and reduces the need to carry private forks.

And yes, we're always looking for developers who care about **infrastructure, observability, and open source.**

## Update, October 2026: shipped in node_exporter 1.10.0

Both changes shipped: the procfs fields in [procfs v0.18.0](https://github.com/prometheus/procfs/releases/tag/v0.18.0), and PR #3425 in [node_exporter 1.10.0](https://github.com/prometheus/node_exporter/releases/tag/v1.10.0) on 24 October 2025. The collector itself, with the link width and speed metrics, came in the same release from [PR #3339](https://github.com/prometheus/node_exporter/pull/3339). It is off by default:

```bash
node_exporter --collector.pcidevice

# with vendor, device and class names
node_exporter --collector.pcidevice --collector.pcidevice.names \
  --collector.pcidevice.idsfile=/usr/share/hwdata/pci.ids
```

Without `--collector.pcidevice.idsfile`, the collector looks for `/usr/share/misc/pci.ids`, then `/usr/share/hwdata/pci.ids`. Every metric carries the device's address as `segment`, `bus`, `device` and `function` labels.

| Metric | What it reports |
| --- | --- |
| `node_pcidevice_info` | Always 1. Labels carry the parent bridge's address, the class, vendor, device and subsystem IDs and the revision, plus names when `--collector.pcidevice.names` is set |
| `node_pcidevice_current_link_width`, `node_pcidevice_max_link_width` | Link width in lanes, now and at most; -1 when the kernel gives no value, and an empty slot's port typically reports 0 |
| `node_pcidevice_current_link_transfers_per_second`, `node_pcidevice_max_link_transfers_per_second` | Link speed in transfers per second, now and at most; -1 when the kernel gives no value or reports Unknown, as an empty slot's port does |
| `node_pcidevice_numa_node` | The device's NUMA node; not emitted when the kernel reports -1 |
| `node_pcidevice_power_state` | 1 for the current state in the `state` label (D0, D1, D2, D3hot, D3cold, unknown or error), 0 for the others; emitted only when the device exposes power_state |
| `node_pcidevice_d3cold_allowed` | 1 if the device may be put into D3cold; 0 otherwise, including when the device has no such file |
| `node_pcidevice_sriov_numvfs`, `node_pcidevice_sriov_totalvfs` | VFs enabled, and the most the device supports; 0 for devices without SR-IOV |
| `node_pcidevice_sriov_drivers_autoprobe` | 1 if newly enabled VFs bind to a driver automatically; 0 for devices without SR-IOV |
| `node_pcidevice_sriov_vf_total_msix` | MSI-X vectors available to all the device's VFs; 0 for devices without SR-IOV |

Queries worth alerting on:

```promql
# endpoints whose link trained narrower than they support: an x16 card at x8
# (expected if the slot itself is x8); bridges are left out, since a root port
# reports the slot's width, and 0 when the slot is empty
node_pcidevice_current_link_width < node_pcidevice_max_link_width
  and node_pcidevice_current_link_width > 0
  and on (instance, segment, bus, device, function) node_pcidevice_info{class_id!~"0x0604.."}

# endpoints below their top link speed; many GPUs downshift when idle, so check under load
node_pcidevice_current_link_transfers_per_second < node_pcidevice_max_link_transfers_per_second
  and node_pcidevice_current_link_transfers_per_second > 0
  and on (instance, segment, bus, device, function) node_pcidevice_info{class_id!~"0x0604.."}

# devices out of D0; normal for idle devices under runtime power management
node_pcidevice_power_state{state="D0"} == 0

# SR-IOV capable devices with no VFs enabled; alert only on hosts that should have VFs
node_pcidevice_sriov_totalvfs > 0 and node_pcidevice_sriov_numvfs == 0
```

Asama's fleet visibility is generally available, and detection built on it has been validated; investigation and remediation are expanding. To see these signals read across a whole fleet, book a live comparison against your stack at [asama.ai](https://www.asama.ai), or write to contactus@asama.ai.