# The hardware intelligence gap: BMC, SMART and PCIe data that nothing joins (Solution Landscape 4/4) > BMCs, SMART and PCIe counters already report on every server. Why the readings stay in separate tools and vendor dialects, and what joining them takes. - URL: https://blog.asama.ai/blog/issue-14-asama-the-hardware-intelligence-gap/ - Author: Jain Johny - Published: 2026-04-17 - Tags: Frameworks The hardware intelligence gap is not missing instrumentation. BMCs report sensors and event logs over IPMI and Redfish, drives report SMART attributes, PCIe errors are counted through AER, and GPUs report through DCGM. The gap is that each reading stays in its own tool and its own vendor's dialect, and each vendor's console sees only its own servers. Nothing joins the readings across vendors and layers, ties them to the [infrastructure state](https://blog.asama.ai/blog/issue-10-asama-the-firmware-layer-nobody-owns) that explains them (what each part is, what it should be and what last changed it), or reasons over them the way a senior engineer does. Cloud infrastructure evolved sophisticated monitoring, tracing, and alerting capabilities over two decades. The reason was not sophisticated foresight. It was that the cloud was built on the assumption that things would break, frequently, unpredictably and at scale, and the tooling had to catch failures and recover automatically. Physical infrastructure was built on a different assumption: hardware is reliable. Cloud-era tools also grew up on rented machines, where the provider owns the BIOS, the BMC and the firmware, so they never had to read that layer. If you run your own servers, you inherited tools built for a world where that layer belonged to someone else. The monitoring that exists for physical infrastructure reflects the assumption that hardware is reliable: it watches for hard failures, threshold breaches and unambiguous faults. It was not designed to catch the [gradual, below-threshold degradation](https://blog.asama.ai/blog/issue-01-asama-uptime-is-not-reliability) that drives much of the operational overhead of running a physical fleet. > "The gap is not instrumentation. The counters exist on every server. The gap is that nothing joins them into one picture of the fleet." --- ## The instruments already exist | Signal | Where it comes from | Where it often ends up | | :--- | :--- | :--- | | **BMC sensors and event log** | IPMI and Redfish | Each vendor's console, which sees only that vendor's servers | | **Drive health** | [SMART attributes](https://blog.asama.ai/blog/issue-06-asama-what-smart-data-isnt-telling-you) and the NVMe health log | smartd alerts on vendor thresholds, or a smartctl run during an incident | | **PCIe link errors** | [AER counters in sysfs](https://blog.asama.ai/blog/issue-07-asama-the-correctable-error-trap) | The kernel log, or read by hand if anyone looks | | **GPU health** | [DCGM health watches](https://blog.asama.ai/blog/gpu-health--diagnostics----why-they-matter) and XID events | A separate GPU dashboard | | **Firmware versions and BIOS settings** | Redfish, vendor tools and dmidecode | A spreadsheet, if anywhere | Each source is useful on its own. They name the same part differently, refresh on different clocks and live in different tools, and [each vendor speaks its own dialect](https://blog.asama.ai/blog/issue-09-asama-the-heterogenous-problem-nobody-admits). None of them knows what the server is supposed to be. Joining them is work an engineer does by hand, during an incident. --- ## What hardware intelligence actually means Hardware intelligence is not better dashboards. It is not more metrics pushed to a Grafana instance. It is the capacity to reason about physical infrastructure state the way a senior engineer with ten years of operational experience would: understanding which signals matter in which context, what combinations of signals indicate which failure modes, and what actions are appropriate given the specific hardware model, firmware version, and workload profile of affected nodes. --- > ### Hardware-layer ingestion > > **Operational capability** > BMC telemetry, PCIe error counters, SMART attributes, CPU topology, and kernel module states collected directly at the hardware layer, with the BMC also read out-of-band, so a server can still be checked when its OS can't answer. > > **Why it matters** > The signals that explain infrastructure degradation rarely originate at the application layer. --- > ### Cross-signal correlation > > **Operational capability** > Signals spanning firmware state, component health, configuration drift, and workload behaviour, normalised so one physical part is one record whichever vendor built it, and connected into a topology graph. > > **Why it matters** > Cause and effect become traceable across the full hardware and software stack. --- > ### Detection to remediation > > **Operational capability** > Intelligence that does not stop at alerting. > > **Why it matters** > Root cause identified, remediation generated, execution tracked with engineers in the loop for approval. --- ## The investment case The investment case for hardware intelligence is not primarily about preventing dramatic failures. It is about the steady, unglamorous cost of operating infrastructure that is partially visible and manually managed: - 40 to 60% of senior engineering time spent on debugging and firefighting, on the fleets Asama has measured - 7 to 15% of the fleet [sitting in maintenance queues](https://blog.asama.ai/blog/issue-05-asama-the-maintenance-queue-is-a-capex-problem) at any given point, in Asama's fleet observations - Over-provisioning accumulated as insurance against unreliability that is never fully resolved The cost is not one catastrophic incident. The cost is the operational drag that compounds every week. --- ## From Asama > ### Built from the BMC up > > Asama is designed to close this gap: > > - Hardware and OS layer collection, in-band and out-of-band from the BMC > - One normalised record per physical part, whichever vendor built it > - Cross-signal reasoning across firmware, configuration, and workload state > - Topology-aware root cause analysis > - Detection connected directly to remediation workflows, with an engineer approving each change > > Detection is not the endpoint. > > Operational resolution is. --- ## Food for thought > What percentage of the signals your physical infrastructure generates are actually being used to make operational decisions, and what is the cost of the ones being ignored? **Update, October 2026.** Asama's fleet visibility is generally available. A host agent reads the kernel, eBPF probes, /proc, /sys and the system logs; each BMC is read out-of-band over Redfish and IPMI for the SEL, sensors, firmware versions, BIOS attributes and audit log; GPUs report through DCGM and XID events, and drives through SMART and the NVMe health log. Raw SEL, XID, AER and SMART codes are decoded into a part, a fault and a severity, and one model covers Dell, HPE, Lenovo, Supermicro and NVIDIA hardware, with each physical part resolved to one object. Detection built on it has been validated on Dell, HPE and Lenovo hardware. About 70% of root-cause work is deterministic correlation and statistical inference, with agentic reasoning writing the causal chain from those results; investigation and remediation are expanding. To see what your hardware is already telling you, book a live comparison against your stack at [asama.ai](https://www.asama.ai), or write to contactus@asama.ai.