NICs with optical ports: an overview
Đội ngũ của chúng tôi đang thực hiện bản dịch. Bài viết này tạm thời chưa có bản tiếng Việt và đang được hiển thị bằng tiếng Anh.
The other end of every server link is a network interface card (NIC), and it plays the same game as the switch — cages, SerDes lanes, a management bus to the module, a policy about which modules it accepts — with two differences: the policy lives in a driver, not a network OS, and the bottleneck is often the PCIe bus, not the port. This section covers NICs with SFP/QSFP-class cages: the families and vendors, how ports relate to PCIe, transceiver compatibility and its driver-level overrides, the tools that read a module from the server side, and the programmable adapters (SmartNICs, DPUs) that blur the line between card and switch.
What a NIC is, in optics terms
| Element | On a switch | On a NIC |
|---|---|---|
| Cages | 24–64 per box | 1–4 per card (1–2 typical) |
| SerDes | forwarding ASIC | NIC controller ASIC (Intel E810, Broadcom BCM57508, Nvidia ConnectX-7…) |
| Module management | NOS over I²C mux and CPLD | driver/firmware over the controller's I²C |
| Policy | NOS whitelist (How each NOS validates a module) | driver/firmware whitelist (Optics compatibility on NICs) |
| Speed/FEC config | CLI | ethtool, vendor tools, BIOS/UEFI, OS device manager (Tools) |
| Backhaul | fabric ASIC | PCIe lanes to the CPU (Ports & PCIe) |
| Power | 3.3 V per cage from the chassis | PCIe slot (25–75 W) plus auxiliary connector on high-end cards |
Form factors of the card
| Card type | Where | Ports |
|---|---|---|
| PCIe add-in card (half-height half-length, full-height) | any server slot | 1–2 × SFP28/QSFP28, 1–2 × QSFP56/QSFP112/QSFP-DD/OSFP |
| OCP NIC 3.0 (SFF / LFF mezzanine) | OCP slot at the rear of the server | same cages, front-accessible, hot-swap on some chassis |
| LOM / embedded | on the motherboard | usually RJ45; SFP+ on some |
| Mezzanine / blade | blade servers | connects to blade chassis switch, no cages |
| SmartNIC / DPU | PCIe or OCP | 2 × 100/200/400G, own Arm cores (SmartNICs & DPUs) |
Families and vendors
| Vendor | Families (optical-port models) | Speeds | Cages | Driver (Linux) |
|---|---|---|---|---|
| Intel | X520 / X710 / XXV710 / E810 / E830 | 10G / 10G / 25G / 25–100G / 200G | SFP+, SFP28, QSFP28, QSFP56 | ixgbe, i40e, ice, idpf |
| Nvidia (Mellanox) | ConnectX-4 Lx / -5 / -6 Dx / -6 Lx / -7 / -8 SuperNIC | 25G / 100G / 100–200G / 25–50G / 200–400G / 800G | SFP28, QSFP28, QSFP56, QSFP112, OSFP | mlx5_core |
| Broadcom | NetXtreme-E BCM5741x / 5750x (P225, P425, P2100, P1400GD) | 25G / 100–200G / 400G | SFP28, QSFP56, QSFP-DD | bnxt_en |
| Marvell (QLogic) | FastLinQ 41000 / 45000 | 10–25G / 100G | SFP28, QSFP28 | qede |
| AMD (Solarflare, Pensando) | X2522 / X3522; Pensando DSC | 10–25G; 100–200G | SFP28; QSFP56 | sfc; ionic |
| Chelsio | T6 | 10–100G | SFP28, QSFP28 | cxgb4 |
| Others | Silicom, Napatech (FPGA), Xilinx/AMD Alveo, Netronome | 10–100G | mixed | — |
Speeds per card model are fixed by controller and cages; "25G capable" means 10G and 25G, rarely 1G (Ports & PCIe).
What the NIC needs from a module
- Compliance codes for a speed the controller supports at that cage.
- Identity the driver accepts — Intel families are the strict ones (Optics compatibility on NICs).
- FEC/AN behaviour the driver can match: DAC at 25G+ requires Clause 73 AN on both ends; optics need forced speed and matching FEC (FEC & AN).
- Power class within the card's cage budget — most NICs allow QSFP28 class 4 (3.5 W) and QSFP-DD up to 12–15 W; check for ZR/tunable modules (Power & consumption).
- Cooling: server airflow is front-to-back through the card; modules on rear-facing cards sit in exhaust air (Temperature grades).
Typical roles and their optics
| Role | NIC | Link |
|---|---|---|
| General virtualisation host | 2 × 25G SFP28 | DAC to ToR (Roles & topologies) |
| Storage / NVMe-oF | 2 × 100G QSFP28 | DAC/AOC, RoCE with PFC (Lossless Ethernet) |
| GPU server | 4–8 × 400G OSFP/QSFP112 (one per GPU) | AOC/DR4 to rail leaves (Cabling an AI cluster) |
| HPC | InfiniBand HCA 1–2 × NDR/XDR | DAC/AOC (IB cables) |
| Edge / branch | 2 × 10G SFP+ | LR to a campus core |
| Management / OOB | 1G RJ45 or SFP | — |
Further reading
- NIC ports, speeds and PCIe bandwidth — cages on NICs, dual rate, breakout cables, PCIe generation × lanes vs port totals, OCP 3.0, power and cooling.
- Transceiver compatibility on NICs — driver whitelists by vendor, Intel
allow_unsupported_sfp, Windows/ESXi behaviour, FEC/AN with DAC and optics, what to code. - NIC tools and diagnostics —
ethtool -m, mlxlink, bnxtnvm, Intel tools, Windows and ESXi commands, reading DDM and FEC counters from the server. - SmartNICs and DPUs — BlueField, Pensando, IPU: ports and optics on programmable adapters.
In CodingBox
A module that a NIC rejects is diagnosed the same way as one a switch rejects: read the identity and compliance codes on the bench, compare with the driver's list, code if the policy demands it (Check transceiver, Vendor lock).