SAN fabrics & distance

How a SAN is laid out decides which optics it needs — from short SW links inside a row to LW/ELW inter-switch links between buildings and DWDM for replication between sites.

Fabric basics

Dual fabrics. Production SANs run two completely independent fabrics (A and B);

every host and array connects to both. A single failed optic never takes storage offline — but it does halve the path, so it must be found.

Zoning controls which initiators see which targets; it does not affect optics

but does mean a "missing LUN" complaint can hide a physical-layer problem on one path.

ISLs (inter-switch links) carry traffic between switches; several are usually

grouped into a trunk. ISLs are the links most worth monitoring with DDM.

Buffer credits and distance

FC is lossless through buffer-to-buffer credits: a port may only send when the far end has advertised a free buffer. Over distance the round trip eats credits — at 16GFC a frame occupies roughly 1 km of fibre — so a long ISL needs many more credits than a short one, or throughput collapses even though the link is "up". Long-distance ports on FC switches let you allocate extra credits per link; the optics still have to close the loss budget.

Choosing optics by distance

Span	Optics	Notes
Within a row / room	SW on OM4	100–190 m depending on generation
Between rooms / buildings	LW on single-mode	10 km
Campus / metro	ELW on single-mode	25–40 km, credits and budget matter
Metro replication	FC over DWDM	channels on a DWDM system; latency limits synchronous replication to roughly 100 km

See Fibre Channel transceivers & optics for the SW/LW/ELW reach figures.

Common physical-layer faults in a SAN

Slow-drain / credit stalls traced to one marginal link with rising CRC — check

levels and clean connectors first (link flapping).

Mod_Inv on a replacement optic — the switch did not validate the module's

identity; see Vendor lock.

One fabric path degraded, the other fine — the classic case of a dying laser on

one side; compare DDM across the A and B paths.

In CodingBox

Before an optic goes into a fabric, read it on the bench: confirm the FC rate codes, the wave type and the identity the switch expects, and log its DDM as the baseline you will compare against later. For replacements on strict platforms, code the module to the qualified identity in the EEPROM editor.