Fibre Channel Essentials

Author: Raymond B.

 Abstract

Fibre Channel is an ANSI standard-based protocol for the movement of data between servers and storage devices. FC is defined by port addresses and names and each device port connected to a switch port creating a FC Fabric. FC represents the first three layers of the OSI reference model and has a defined frame format and interface with upper layer protocols and drivers. The most common payload for FC frame is SCSI.

Fibre Channel defines a strict flow control process using buffer credits. VSANs, Zones, and Sub-zones are also defined to create data architecture and secure environment. The entire process is controlled by sophisticated fabric switches from Cisco, Brocade, McData and others. These services include domain services, name services, management services, and routing. This white paper will teach you the basics of Fibre Channel.

Introduction

As a major component of any major enterprise network, the corporate data center has evolved into an integral and highly complex environment. The data center is now important enough to be a separate strategic focus for Cisco and many other networking companies. The demand is high for skilled personnel in this area.

But the objective of this white paper is not to describe the design or operations of the data center. To establish a preamble to our objective, consider all the options that fill the role of a server in the data center:

• Dell
• IBM
• HP
• Sun
• Linux
• Red Hat
• IBM Mainframe
• Blade Servers
• NIC Teams with VMware

This is not an exhaustive list. All of the servers listed have one thing in common: the need to retrieve data from storage media such as disks or optic memory arrays. The protocol utilized to transport data between servers and storage for High Performance Computing (HPC) and to define a Storage Area Network (SAN) is Fibre Channel (FC). Fibre Channel is an "invisible" protocol, because it is only known to the technicians that implement it and keep it working. Just as a PC user is unaware of the underlying technology with which the PC retrieves data from the hard drive, the data center client is likewise in the dark.

To provide an explanation of the essentials of this technology, an overview of the following will be provided:

• Fibre Channel/SAN topology
• Flow control
• FC ports
• Error control
• FC vs. OSI
• FC fabric components
• FC port addressing and naming
• VSANs, zones, services
• Classes of service

Fibre Channel Essentials

In the graphic above, the server could be any of the ones listed. The Fibre Channel fabric is composed of Fibre Channel switches such as the Cisco MDS 9000 series, the Nexus 5000 series, as well as switches from Brocade, McData and others. Modern topologies include:

• Point-to-point
• Switched fabric

Point-to-point is any two FC ports connected together. When a switch is inserted, multiple point-to-point connections are made to the switch, creating a switched fabric. Switched fabric is the most common technology in SANs and allows thousands of devices and up to 239 meshed switches. The devices are:

• Server (host)
• Tape storage
• Storage array or just a bunch of disks (JBOD)
• Fibre switch

Each device manufacturer provides Fibre Channel port technology for the device. An FC port is not just a physical interface; it is an intelligent component of the fabric. The servers use a Host Bus Adapter (HBA). The disk, tape storage devices, and the switch have a port/s as well.

FC is documented in American National Standards Institute (ANSI) specifications. In older implementations an arbitrated loop topology along with an appropriate hub device was supported. That technology is not included here.

• A node port (N Port) is a port on a node that connects to a fabric.
• A fabric port (F port) is a port on a switch that connects to an N port.
• An extension port (E port) is a port on a switch that connects to an E Port on another switch. Another name of this connection is an Inter-Switch Link (ISL).

There are other ports for connectivity between the switch and the older arbitrated loop hubs not defined here.

World Wide Names (WWNs) are unique identifiers that are assigned to manufacturers by the International Electrical and Electronic Engineers (IEEE) and hard-coded into FC devices. The names are usually 128 bits long (it could be 64 if older). Since one server may have dual ports, the WWN has two formats:

• nWWN (node WWN)
• pWWN (port WWN)

Each port has a Fibre Channel identifier (FCID), which is 24 bits in length and consists of three parts (domain, area, and port) so that many devices can be identified within a domain. FCIDs typically are assigned dynamically by the switch during start-up.

Obviously, the port is much more than a physical interface. With all the parts named and addressed, a name service is necessary (provided by switch) to resolve WWNs (names) to FCIDs (port addresses). This is somewhat analogous to IP and Domain Name Service in a TCP/IP network.

Fibre Channel defines the lower three layers of the OSI Reference Model: physical; data link; and network.

Related Courses

DCNID - Data Center Network Infrastructure Design v2.0
DCSNS - Designing Cisco Storage Networking Solutions 3.2