Development of the FDDI Physical Layer

second. The physical layer The engineering development of FDDI-the topic of this of the FDDI physical layer paper connects many resulted in the delivery of stations, each of which components, specifications, may transmit information and protocols. The to any other station in the development presented new network. As in other LANs, design problems related to packets of user data are the technology and to the encoded according to the operation of token rings. physical layer protocol and The choice of the most are transmitted as a serial appropriate technologies data stream over a physical for the chip set was based media to other stations on technology issues, risk of the LAN. FDDI, however, control, and costs. The is unique in its use of chip set that emerged hundreds of individual, after the physical layer point-to-point, fiberfunctions were partitioned optic connections that uses both ECL and CMOS form a ring network in technology. Further, the physical layer. The three design problems of resulting LAN offers both a general interest arose high data rate and a total during development: the physical extent of up to elasticity buffer and 100 kilometers (km). circuitry related to the The development of physical distributed clocks in an layer hardware, used in all FDDI LAN, the multimode FDDI products, included fiber-optic link using the physical protocol light emitting diodes, and (encoding/decoding) the media error processes device, a receive clock as related to correctness recovery device, a local and fault isolation. clock generator, and Introduction optical transmitters and receivers. This paper The fiber distributed focuses on development of data interface (FDDI) is the physical layer hardware a multiaccess, packetand describes some aspects switching local area of the design in detail. We network (LAN) that operates first review the operation at 100 megabits (Mb) per of the physical layer and Digital Technical Journal Vol. 3 No. 2 Spring 1991 1 Development of the FDDI Physical Layer the functional partitioning topologies for FDDI. Each of the implementation. bit of information received We then present detailed from one physical link is discussions of the transmitted onto another distributed clock scheme, physical link until the the design of an optical information travels around link, and the methods the loop and returns to to control the effects where it started. The FDDI of bit errors in the protocols provide for a physical layer. Some single originator of data of the results of the packets; other stations development to improve the repeat the data so that performance, correctness, each station on the ring and reliability of FDDI receives the packet of described here have information. The collection been incorporated in the of many point-to-point American National Standards links forms the ring, which Institute (ANSI) FDDI is viewed as a multiaccess standards. medium by the users. The basic element in the Operation of the Physical topology of an FDDI LAN is Layer the physical connection. The FDDI physical layer A physical connection is a collection of pointcontains two physical to-point links forming a links, also shown in "ring." The operation of Figure 1. Within the the layer is described in station, the circuitry terms of physical links, that implements physical physical connections, and layer functionality needed the functions of individual for one physical connection stations. The many station is called the PHY port. types allowed by the The physical connection is ANSI FDDI standards are a full duplex connection constructed with a simple between exactly two PHY physical layer functional ports. Neighbors in the block called the PHY port. ring directly exchange the control information for A physical link contains each connection, allowing a transmitter, a receiver, control protocols in FDDI and a segment of physical station management (SMT) medium which conducts the to establish the shared bits of a packet from states for a connection: one station to a second in-use, starting, and station. The topology of disconnected. The status FDDI is arranged so that "in-use" indicates that a the collection of physical connection is part of the links forms a closed ring; other states indicate path, or ring, as shown it is not. The control in Figure 1. This simple information exchanged over topology illustrates the the physical connection is basic concepts common to used to autoinitialize even the most complicated and autoconfigure the 2 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer connections in the LAN, from the coded signal) and a method of operation bounds the low-frequency currently unique to FDDI components of the signal rings. spectrum. The code bits are There are several types then converted to a serial of FDDI stations, and stream and transmitted different types can support as optical pulses on the different numbers of fiber-optic media. physical connections.[1] The station is coupled A single attachment station to the media with (SAS) (as seen in Figure a media interface 1) can establish one connector (MIC). The MIC physical connection with provides the concrete a single neighbor. The interface necessary for dual attachment station interoperability between (not shown) has two PHY equipment from multiple ports and may establish vendors. The FDDI Physical physical connections Layer Media Dependent with two neighbors. A (PMD) standard specifies concentrator (CON) is a mechanical and optical type of station that can properties of the MIC.[3] establish connections with The MIC includes both a many neighbors, thereby transmit and a receive providing attachment points interface. for other stations. The Signals received from a CON shown in Figure 1 connection are decoded by interconnects its PHY ports the PHY port for processing internally to configure a by the station. The optical single ring. input signal is translated Figure 2 shows the to an electrical signal. functions of and flow of The remote bit clock is data through a PHY port extracted from the signal which implements the FDDI and used to recover logic physical layer protocol levels corresponding to (PHY) standard.[2] Data the individual bits. A packets to be transmitted framer then establishes over the LAN are passed as the original code group a stream of bytes from the boundaries and converts data link to the physical the serial code bit stream layer. Each byte contains into parallel form. Also, two PHY symbols, and each the elasticity buffer symbol represents 4 bits synchronizes the received of user data. The FDDI data to the local clock coding scheme, called 4B reference and accounts for /5B encoding, translates the frequency difference each symbol into a code between the local and group containing 5 code remote clock references. bits. This encoding Finally, code groups of limits the maximum time 5 code bits are decoded between transitions on into symbols, and symbols the media (allowing clock are correctly paired to information to be derived form the data bytes which Digital Technical Journal Vol. 3 No. 2 Spring 1991 3 Development of the FDDI Physical Layer represent the received o FOT (fiber-optic data. These data bytes transmitter) are passed either to o FOR (fiber-optic another PHY port or to the receiver) data link layer. A later section, Operation of the Our choices for the Distributed Clock Scheme, appropriate partitioning expands on the elasticity and technology were buffer design. founded on our decision to We have so far described develop a highly integrated the FDDI physical layer and low-cost chip set. in terms of PHY ports and After examining several the physical connections alternatives, we chose a between them. These basic partitioning that enabled elements form the physical us to use mostly CMOS layer for all types of technology (complementary FDDI stations. Different metal oxide semiconductor), types of FDDI stations a minimal amount of custom have one or many PHY ports, ECL (emitter coupled but the operation of an logic), and no ECL gate individual PHY port and array technology. Although physical connection is a 125-megahertz (MHz) independent of station serial channel requires ECL type and topology. In circuitry in the system, the next section, we we wanted to minimize discuss the functional the amount of custom ECL partitioning of the PHY technology. ECL consumes port and the reasons a substantial amount of behind the partitioning current and is relatively chosen. Subsequent sections expensive as compared describe the distributed to CMOS technology. clock scheme, the design We also considered ECL of the physical link, and gate array technology, the impact of physical link but decided against it errors on the LAN. because it was not a mature technology, lacked requisite analog functions Functional Partitioning for clock extraction, and In this section, we was available from only a describe the partitioning few vendors. of the functions of the We determined that the PHY port into the following CDCR and CDCT were the components: only functions that had o PHY (physical protocol to be implemented in chip, also referred to ECL technology. This as the ELM chip) determination was based on the need for the high o CDCT (clock and data transmission rates and for conversion transmitter) quick conversion to and o CDCR (clock and data from serial and parallel conversion receiver) data streams. The CDCR receives a 125-megabaud 4 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer ECL serial data stream clock, and convert the 5from the FOR. Using a phase bit parallel bus to the lock loop, CDCR extracts serial stream. With this a receive clock to recover method, the highest clock the data bits and then rate distributed on our converts the serial data boards was a 25-MHz clock. to a 5-bit parallel bus. As a consequence of The CDCT receives a 25selecting the transmit MHz, 5-bit-wide parallel phase lock loop, we chose bus; then, by using a phase to specify and build lock loop, it generates an separate transmit (CDCT) internal 125-MHz transmit and receive (CDCR) devices clock in phase with the in custom ECL technology. local 25-MHz clock. CDCT We were very concerned then converts the 5-bitthat the combination of wide parallel bus to a two asynchronous phase 125-megabaud ECL serial bit lock loops on a single stream that is transmitted chip would induce cross by the FOT. talk. Cross talk could We selected the 5-bit cause false locking of width for the parallel the phase lock loops to bus to obtain a 25-MHz one another, resulting bus rate. This rate is a in lost data. Therefore convenient divisor of the making a single chip was 125-MHz serial rate and considered too risky for is within the operating the initial implementation. range of the CMOS gate Our solution was to specify array technology used in the transmit and receive the connected chips. The devices, thus eliminating 5-bit bus also offered the the possibility of cross advantage of enabling us talk. to maintain a low pin count The balance of the logic on the devices to which for the physical layer the bus is interfaced, thus protocols could now be further containing costs. designed in CMOS gate Another complication array technology. The relative to the clock use of CMOS gate arrays component was how to was important in meeting distribute a 125-MHz clock schedule since it allowed signal. As noted earlier, us to quickly implement some FDDI products have changes. Changes were many PHY ports, and those inevitable and therefore PHY ports must have a had to be accommodated common clock line for the because the ANSI standard transmission of the serial was not finished and stable data stream. We decided to during our design cycle. add another phase lock loop All of the physical in the transmit component layer functions such as that would lock onto the encoder, decoder, the 25-MHz local clock, elasticity buffer, generate the 125-MHz serial framer, and smoother Digital Technical Journal Vol. 3 No. 2 Spring 1991 5 Development of the FDDI Physical Layer were implemented in Operation of the Distributed CMOS gate arrays. We had Clock Scheme simulated these functions In the FDDI distributed in software; however, we clocking scheme, were now able to build each station uses an them using CMOS gate arrays independent, local and actually analyze their clock reference when behavior in real networks. transmitting or repeating With the hardware, we data packets. The station quickly verified the must synchronize the protocols defined in the receive data with its FDDI standards. Proper own reference clock prior PHY operation is best to further processing. confirmed by testing actual Although this distributed implementations. clock reference scheme The fiber-optic transmitter simplifies many problems, (FOT) converts a 125it also can give rise to megabaud electrical signal data integrity problems to light pulses to be and packet loss rate issues transmitted to a receiving that must be solved for the station. The fiber-optic scheme to work effectively. receiver (FOR) receives the Data must be synchronized pulses from a transmitting to the local clock station and converts them reference in a way that to an electrical data prevents detected and stream. We decided not undetected errors caused to develop the FOT and the by metastability problems. FOR components ourselves. Further, interpacket Instead we chose to gap shrinkage that can influence the specification result in an unacceptable of the system's functional packet loss rate must be requirements in the ANSI controlled. In the sections FDDI Committee and then Elasticity Buffer and depend on external vendors Smoother below, we describe to develop the components. how these problems are It was important to addressed in the physical encourage the optical layer protocol. vendors to standardize their components so costs would decrease, and so that more than one source Elasticity Buffer of optical components would be available to Each PHY port of a station us. Accordingly we did must accept data packets not combine the optical from another station with transmitter and receiver a slightly different with any other physical clock frequency and layer functions. The bit transmission rate. optical link design is the It is the function of subject of a later section the elasticity buffer in this paper. within the PHY port to synchronize the incoming 6 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer data to the local clock reference. The buffer is also designed to control during the idle time synchronizer metastability, between data packets, known a source of undetected data corruption. As a as the interpacket gap. result of the elasticity When a minimum interpacket buffer operation, the size gap time is detected by of the gap between two the input state machine, data packets varies as the a reset control signal packets are repeated around is sent to the output the ring. state machine. The reset The elasticity buffer is signal is synchronized by a collection of storage the output state machine registers that are written to avoid metastability. to and read from at When an input signal to different rates. (See a register is changing at Figure 3.) The buffer forms the time the register is a circular queue due to the clocked, its output may movement of two independent become indeterminant and pointers: the input pointer assume multiple values over selects the register to be a time called a period of written to and moves at the metastability.[4] The reset recovered clock rate; the signal could be changing output pointer selects the when it is sampled by the register to read from and output state machine, so moves at the local clock this signal is synchronized rate. The location of the by waiting an interval pointers is based on the after each sample for the gray code counters. The sample value to settle input pointer is controlled before the sampled value by the input state machine is utilized, or considered and the output pointer by valid. The reset signal is the output state machine. delayed by this process. These state machines The circuitry guarantees position the pointers at that the present address of a controlled distance from the input pointer has been one another. Therefore in the holding register pointer control prevents on the reset condition data from being written for a sufficient duration while it is being read from and thus its stability is the same register, even ensured. The output state though the pointers are machine then loads the moving at different rates. address that the input state machine stored in During normal operation, the holding register. the input and output The output pointer moves pointers approach to that location plus an each other and must be offset in order to keep a periodically repositioned. minimum distance from the The repositioning occurs input pointer. Digital Technical Journal Vol. 3 No. 2 Spring 1991 7 Development of the FDDI Physical Layer This approach to repositioning pointers ensures that all data buffer has overflowed or or signals are stable in underrun. their respective registers The input and output before being sampled by pointer counters are the local clock. As a compared using the local consequence, we were able clock; that is, the input to specify in the design pointer counter bits can that only one control line change with respect to be synchronized in the the local clock, as shown elasticity buffer. The in Figure 4. Gray code signal that crosses the counters limit to one clock boundaries is the the number of bits that reset signal, which is can change in the pointer generated by the remote counters at any sample clock and sampled by the interval. The comparator local clock. The reset circuit is sampled twice signal triggers all events using the local clock and required for the elasticity the local clock shifted by buffer to correctly receive 90 degrees. If one sample data. Since there is only of the D flipflop notes a one control line within change, the changing bit the elasticity buffer that settles down before the needs synchronization, other sample happens; thus the implementation is very metastability problems robust. are controlled. A logical We also had to anticipate AND of the sampled outputs occurrences outside normal signifies that the two operation. Therefore we pointers have had the designed circuitry that same address for at least detects when the pointers one quarter of the local point to the same register clock interval. When the for more than a minimum addresses are the same for amount of time.[5] This at least one quarter of circuitry prevents the the local clock time, the buffer from reading the error flag is raised. When register while its contents the error flag is raised, are changing; if the buffer an overflow or underrun were read, data corruption is imminent. The cause is and consequently undetected somewhere in the network. errors could result during For instance, the clocks abnormal operation. The are out of specification, input and output pointers or a downstream station are 3-bit, asynchronous is sending larger data gray code counters. The packets than are permitted. pointers are compared to In any case, the condition one another to determine is detected and prevented whether they coincide, from increasing the risk of indicating that data in the undetected data corruption. 8 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer Smoother o One hundred one elasticity buffers Another function in the o Maximum size data packet physical layer, called the length smoother, prevents the loss o Pseudorandom clock of data packets that can distribution result from shrinkage of the interpacket gap. Data The solution is to monitor packets can be lost, or the interpacket gap.[6] If discarded, at two places. it falls below the 7-byte The decision to discard minimum size, the smoother can be made at the physical adds to the interpacket layer as the result of an gap. The addition of elasticity buffer overflow interpacket gap to the or underflow. Also, packets output stream causes the can be discarded at the elasticity buffer to use media access control (MAC) a buffer to delay the layer. The MAC layer is not output data and then send required to copy a packet an interpacket gap byte. that has less than 6 idle The amount of buffering bytes of interpacket gap within the elasticity preceding the packet. buffer is finite so that The decision to implement the delay within a station the smoother came after is not long. The smoother simulations of the also reclaims storage elasticity buffer revealed elements by deleting bytes the then current draft of interpacket gap one ANSI PHY protocol would byte at a time from long result in an unacceptable preambles. This reclamation packet loss rate. In a occurs any time 8 bytes series of nodes with a or more of interpacket gap random distribution of appear at a station. clocks, some stations add The smoother is a to and others delete from distributed algorithm the interpacket gap. If that cannot be adequately nodes add or delete without proven in simulation. We regard to the size of the built a gate array that interpacket gap, we found contained the elasticity that the interpacket gap buffer with the added could be deleted entirely smoother function to prove or reduced to a minimum in hardware that our new size. At this size, the algorithm would operate MAC is not required to copy properly. We built a 200a frame, and data packets node ring of elasticity are lost. Our simulation buffers that could handle showed that an unacceptable at least the test case 10 percent packet loss (101 elasticity buffers) would occur due to 6 or used in the simulation. less bytes of interpacket Each elasticity buffer gap under the following had a variable oscillator conditions: to allow control of the distribution of clock Digital Technical Journal Vol. 3 No. 2 Spring 1991 9 Development of the FDDI Physical Layer frequencies that we felt the PHY protocol, the FDDI would induce interpacket distributed clocking scheme gap shrinkage. We also does not significantly included special test contribute to packet loss. features in the chip to monitor the interpacket FDDI Optical Link Design gap at every node in the tester. As noted in the earlier During four weeks of ring section, Operation of operation, we observed the Physical Layer, the the interpacket gap of physical connection is 6.72 billion maximum size the basic element in packets (4500 bytes). The the FDDI LAN topology. minimum interpacket gap The physical connections observed was 7 bytes, which between FDDI stations are resulted in no packet loss. full duplex, fiber-optic The experiment indicated links that deliver a serial a packet loss rate of less code bit stream from one than 2E-10. Since no packet station to another with loss occurred, the actual a bit error rate (BER) loss rate is unknown; less than 2.5E-10. Each but this result gives us station has a separate confidence that the loss transmit and receive rate predictions made by link, and both links are analysis are correct. cabled together to the same destination. The The 200-node hardware optical link requirements test bed demonstrated are defined and measured that our algorithm worked at the MIC. Any set of effectively. The smoother conforming FDDI stations protocol was adopted as a connected together with mandatory part of the final a compliant cable plant ANSI FDDI PHY standard.[2] in a legal topology are The standard allows a guaranteed to provide variety of different the required transmission designs, some having as service. Conformance to the little as one byte of optical requirements can smoothing, depending on be measured independently the number of preamble of both the interconnecting bytes required by the media and the attached MAC implementation in station. Measurements can the same station. All be taken from either end of allowed designs have worsta physical connection. case loss rates below The technology choices we 1E-14 (by analysis) in confronted and the design a homogeneous ring. The methods we used in the worst-case packet loss rate development of the optical in a heterogeneous ring, link are summarized in one with multiple types the following sections. of smoother designs, has a These methods can be packet loss rate below 1Eapplied to any transmission 10 (by analysis). Given the system design problem with addition of the smoother to 10 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer similar requirements. The multimode fiber prevailed Physical Layer Medium over lasers operating with Dependent (PMD) Working single-mode fiber because Group of the FDDI committee at the design time the adopted these methods, and former was more reliable Digital played a leading and had a better chance of role in the design of achieving the cost goals the PMD Standard. The required by short-distance Working Group developed office interconnection the design in a manner spans. The selection of that combined theoretical appropriate technology analysis with empirical was especially difficult modifications in an because the technology iterative process to arrive was rapidly evolving. at the specifications for The Working Group made the system. The full detail the basic technology of the models has been choices in the 1984-1986 documented previously in time frame; the chosen the literature.[7,8] technology represented the Technology Choices best compromise between available technology and The FDDI distance and bit reasonable anticipated rate requirements clearly improvements. The FDDI mandate the use of a fibercommittee later addressed optic transmission system. the long-distance However, the choices requirements (greater than are not equally obvious 2 km) of a campus LAN with between laseror LEDa single-mode fiber and based transmitters, between laser transmitter PMD (SMF850-nanometer (nm) and PMD). That development 1300-nm operation, and effort is not addressed in between single-mode and this paper. multimode fiber operation. Optical Link Overview FDDI development initially focused on the transmission The optical link is distance requirements of composed of three basic LANs which serve as local elements: a transmitter, office networks as well a cable plant, and a as network backbones. receiver. The transmitter Accordingly the technology is provided with a serial chosen for the optical 125-megabaud code bit link should be coststream and creates an effective and capable of amplitude modulated 1300spanning approximately nm optical version of the 2-km distances. For bit stream. The code bit these applications, the stream has previously been superior bandwidth and loss encoded with a 4-bit into characteristics of 1300-nm 5-bit (4B/5B) non return to LED systems prevail over zero invert (NRZI) coding 850-nm LED technology; scheme that ensures that 125-megabaud transmission the serial sequence has over 2 km is not possible sufficient transitions with 850-nm LEDs. LEDs and to allow recovery of the Digital Technical Journal Vol. 3 No. 2 Spring 1991 11 Development of the FDDI Physical Layer transmit station's timing Bandwidth Allocation clock at the distal end and Models Nyquist of the link. The cable communications theory plant uses a glass, gradedrequires a system bandwidth index, multimode optical of at least one half the wave guide to ferry the baud rate to prevent error signal to the receiver; rate degradation due to the cable has an arbitrary intersymbol interference. number of junctions (e.g., Practical systems require a connectors). The cable somewhat greater bandwidth. plant is described by its We determined the LEDoptical loss and bandwidth. fiber bandwidth for FDDI by The receiver in turn measuring the sensitivity converts the optical signal of commercial 125-megabaud back into a logic-level optical receivers as a code bit stream. When a function of increasing station is not sending input rise time (decreasing data, the transmitter bandwidth) and by observing is provided with special when the channel bandwidth code bit sequences which started to cause a penalty ensure that there is in the measured receiver always an optical signal BER performance. The on the medium between 0.5 decibel (dB) optical packet transmissions. power penalty point was Thus, whenever the link found at 95 MHz. That is a part of a ring, the point is the bandwidth optical system stays in requirement for the LED its equilibrium operating and fiber combination in a conditions, and the clock worst-case maximum length recovery circuit is always link; lower bandwidth synchronized with the causes increasingly incoming stream. higher penalties in BER Design Methods performance and must be prevented. The design of any digital The bandwidth of an LED transmission system must and multimode fiber optical provide sufficient end-tosystem is modeled with end bandwidth and signal three components which power. Further, the design add in a root mean square must demonstrate bounded (RMS) fashion as shown jitter characteristics in equation 1 in Figure in order to provide the 5. The design problem required data transfer confronted is as follows: at the desired BER. The how are the three different bandwidth allocation, bandwidth components jitter budget, and loss rationally allocated to budget for the FDDI optical meet the 95-MHz LED-fiber system are described next. requirement, and what is the maximum distance that can be achieved and still meet this requirement? Although the electrical and 12 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer modal bandwidth limitations bandwidth; the equation for are well known (equations the model is 4 in Figure 5. 2 and 3 in Figure 5), Equations 1 through 4 the chromatic bandwidth are the complete model limitation caused by the for the bandwidth of the LED and fiber combination FDDI optical system. The was not well understood. inputs to the model are Chromatic bandwidth the transmitter spectral limitation is caused by center wavelength, spectral the interaction of the LED width, transmitter rise spectral width with the and fall times, the fiber wavelength dispersion of length, the fiber modal the glass fiber. Thirteen bandwidth, the fiber's hundred-nm LEDs are not zero dispersion wavelength monochromatic; their ((0)), and the zero emission spectrum is dispersion slope (S(0)). typically 170-nm wide at These parameters completely the half optical power define the constituents point. The propagation of the bandwidth of a velocity of light in multimode fiber-optic glass is a function of transmission system. In the wavelength of the an iterative sequence light; light of different of calculations with the wavelengths experiences model, we evaluated a differential delay or trade-off of fiber length, dispersion. Accordingly fiber modal bandwidth, a signal of appreciable LED chromatic attributes, optical spectral width and LED rise and fall experiences dispersion that times to arrive at a 2-km causes an increase in the maximum fiber length with signal transition times transmitter chromatic and and limits the bandwidth. temporal requirements that The amount of dispersion could reasonably be met by experienced by a pulse is a vendors. The transmitter function of the length of requirements were described the fiber, of the optical by a series of curves spectral width, and of the that balanced transmitter separation of the pulse rise and fall times and central wavelength from the chromatic attributes. These zero dispersion wavelength requirements guarantee of the fiber. The wide the 95-MHz LED-fiber spectral width of 1300-nm bandwidth requirement LEDs is sufficient to cause for a 2-km fiber. Thus systems based on their transmitters are allowed use to be distance limited slow rise times if they by chromatic dispersion, have narrow spectral widths even though the system or central wavelengths is operating at 1300 nm, that match the minimum which is the nominal zero dispersion wavelength of dispersion window of fiber. the fiber. Transmitters A model was developed and with wider spectral widths verified for the chromatic and central wavelengths Digital Technical Journal Vol. 3 No. 2 Spring 1991 13 Development of the FDDI Physical Layer displaced from the zero a probability equal to the dispersion wavelength BER requirement. have fast rise time A jitter budget tracks requirements. The curves the accumulation of jitter in ANSI FDDI PMD Figure in the bit stream edge 9 show the final allowed position and allocates it transmitter spectral and to different components. temporal trade-offs.[3] The budget ensures there is They were generated with a jitter-free opening, or a slight modification to window, for the placement the basic model described of the sampling clock. above that used explicit Jitter consists of three fast Fourier transform basic types: (FFT) descriptions of the LED electrical bandwidth o Duty cycle distortioncomponent. The transmitter DCD requirements depend on o Data dependent jitterthe fiber meeting modal DDJ bandwidth and chromatic o Random jitter-RJ dispersion specifications. We empirically established DCD is static and is the 500 MHz·km minimum caused by switching modal bandwidth distance threshold variation product requirement and the and mismatched rise and allowed range of dispersion fall times in driver parameters shown in the circuits. DDJ is caused ANSI PMD Figure 14.[3] by bandwidth limitations Jitter Budget In most in transmission components high-speed serial digital and is also a function communications systems, of the transmitted code the clock used to recover bit stream. We developed the received data must be a worst-case test pattern extracted from the bit that evinces high-frequency stream. The recovered DDJ components caused by clock is used to sample local run length variations the data, and the sampling in the transmitted bit transition is nominally stream and low-frequency in the middle of the bit DDJ components caused by interval. If the sampling variations in the average clock location overlaps power of the unbalanced with the signal transition 4B/5B code bit stream. between bits, errors occur. RJ is caused primarily by Jitter is time dither thermal noise corrupting of the bit stream signal the signal in receivers and transitions; the measured is apparent at low optical value is a function of powers. RJ adds in a rootthe probability of its mean-square fashion with occurrence. Because jitter other RJ components; DDJ is the predominant source and DCD add linearly to RJ. of communications system error, it is measured at 14 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer The FDDI jitter budget jitter-free window is the tracks these three measured receiver component components of jitter requirement. through the optical link. Table 1 FDDI Jitter Budget The budget ensures a Example (ns Peak to Peak) sufficient allocation for the clock recovery Measurement DCD DDJ RJ implementation to place Point the clock correctly in the jitter-free window PHY out 0.4 0.0 0.32 to retime the data. The specific values of jitter PMD out 1.0 0.6 0.76 allotted to each link element were determined PMD in 1.0 1.2 0.76 largely by empirical methods. The sum of all PHY in 1.4 2.2 2.27 jitter allocations must not Optical Loss Budget The exceed the code bit width optical loss budget for (8 ns). Table 1 summarizes FDDI is the difference the jitter budget, showing between the minimum optical the totals for each jitter power launched into the component as it adds fiber and the minimum through the link. optical power required Only the jitter components at the receiver. Decreased visible at the PMD MIC optical power in a receiver (PMD out and PMD in) are causes a reduction in the enforceable parts of the signal-to-noise ratio and standard. Note the sum is evinced on the serial of the jitter components data stream as an increase at PHY in (the exit of in its RJ. The optical the receiver function) is power requirements are 5.87 ns, leaving a 2.13defined in terms of the ns jitter-free window performance measured with remaining in the 8-ns 62.5-micron-core multimode bit cell. This window is fiber. The fiber core allocated to the static size is specified because alignment error and RJ the launch power of a of the clock recovery particular transmitter is a implementation. Digital function of the fiber type developed specialized used. This fiber type is test equipment to generate the prevalent standard for and receive the DDJ test fiber-optic LANs and with pattern and to signal it an 11-dB loss budget is received bit errors; the provided for the optical error rate at the worstlink. case optical conditions (minimum power, maximum jitter) was measured as a function of clock sampling position to measure the jitter-free window at the receiver exit. The 2.13-ns Digital Technical Journal Vol. 3 No. 2 Spring 1991 15 Development of the FDDI Physical Layer The 11-dB loss budget Physical Link Error Process is apportioned by a user An important part of the between bulk fiber losses physical layer development (1.5 dB/km typical, 2.5 dB was the analysis of the /km worst case), connector media bit error processes. losses (0.6 dB typical, 1.0 In the previous section, dB worst case) and splice we presented the design losses (0.2 dB typical, of the optical link to 0.5 dB worst case). With control the bit error rate. this loss budget, users can This section considers construct cable plants of the effect of the error up to 2 km in length with process and the isolation any number of connectors of certain types of faults and splices, provided the that cause errors. total loss is less than 11 dB. There is no minimum To evaluate the error loss required because process, we had to know the maximum launch power the source of the errors is equal to the maximum and study their effect on input power; stations may the protocols for the FDDI. be operated back to back The error process must be without saturating the considered in light of two receiver function. metrics: In summary, the design o Correctness of the methods we used guaranteed protocol. Error events the bit error rate of may lead to undetected the serial data stream corruption of user data. transmission between Detected errors reduce stations. The optical performance, but an bandwidth was allocated undetected error may and guaranteed by design to have nearly unbounded prevent BER degradation bad effects for the due to intersymbol user. Therefore the interference; the jitter undetected error rate accumulation from different must be very low. link elements was budgeted o Isolation of error to prevent BER degradation source to a component due to received data of the network when sampling errors, and the error rate is too an optical power budget high. Error rates may was defined to control exceed acceptable levels BER degradation due to as the result of a inadequate receiver signalmisconfigured network to-noise ratio. or a fault. Isolation of the problem is the first step in a repair process. 16 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer A discussion of protocol As discussed earlier for correctness and fault the design of an optical isolation must consider link, bit errors are more sources of errors caused by transition jitter than the normal bit error resulting from bandwidth process discussed in and power budget limits. the previous section. The important faults and To provide correctness misconfigurations reduce and fault isolation, the channel bandwidth the design must account or increase the optical for misconfigured links loss beyond the design and common faults. A limits. The error rate may misconfigured LAN may exceed the design limit but provide poor performance, the physics of the error but it is always process remains the same. unacceptable for a data We analyzed the impact of packet to be delivered with this error process given undetected errors. the FDDI encoding/decoding Good examples of and error-detecting misconfigured links include protocols. An example of those with cables that an error event is shown are too long and that use in Figure 6 to illustrate too many connectors in the the effect of media noise cable plant. Common faults on the FDDI encoding in the system include schemes. The code bits transmitters that are too on the media are encoded dim, dirty or partially as NRZI, where a signal plugged connectors, and transition represents cables that are kinked (for a code bit 1 and a lack example, by a misplaced of transition (for a bit chair leg). One can write time) represents a code bit an endless list of possible 0. With this encoding, a faults and can posit a single noise event results fault with an arbitrarily in two code bit errors complex symptom. The faults where the resulting pair of listed above are important bits are the complement because they are likely of the original bits. to occur during normal use In Figure 6, the pair of of the components. Many of code bits 0,1 are changed these faults can be traced to 1,0. There are four to a careless or uninformed possible pairs of code user. A design must ensure bits-00, 10, 01, 11 that that these external causes change to 11, 01, 10, 00, of abnormal error rate do respectively, by an error no lasting damage and that event. The FDDI PHY uses a they can be detected and block code in which 5 code isolated. bits represent a symbol, The error process resulting and a symbol contains 4 from important faults data bits. In the example, is similar to the error the single error event process of a correctly changes 2 code bits, which operating optical link. results in a decoded symbol Digital Technical Journal Vol. 3 No. 2 Spring 1991 17 Development of the FDDI Physical Layer with 4 incorrect data This enhancement results bits. The number of data in an undetected error bits affected by an error rate of 5E-24 for the event is multiplied by the protocols, allowing decoding process. significant margin for Error detection is provided actual implementations.[10] by redundancy in the data To isolate a faulty packet. Errors are detected physical link, we need by the MAC protocol based to know which of many on a frame check sequence links exceeds the design(FCS). The probability specified error rate. of an undetected error Each error event must be is related to the number detected and counted at one of error events in the point in the topology. packet and to the specific Using a traditional symbols created. Our method, we would isolate analysis, based on a draft faults based on the of the FDDI MAC protocol, information provided by indicated that undetected the MAC FCS error counters. data corruption could occur Although this method works with high probability.[9] reasonably well for a In the important case, bus topology, it is more a new frame was created difficult to use with FDDI when a noise event changed topologies. The quantity of a data symbol into an physical links may greatly ending delimiter and outnumber the MACs in the created a smaller frame. topology. The errors from This truncation process more than a single physical resulted in an undetected link may be counted by packet error rate of 3Eone MAC, thus masking 14 for large rings (500 which links exceed the stations).[10] Our design error rate. For example, requirements include in a wrapped dual ring of the much more strict single-MAC, dual attachment limit of 1E-21 on this stations, data errors rate. For this reason, an occurring in only half enhancement to strengthen the physical links in the the ending delimiter was network would be counted proposed and accepted by by a single event counter. ANSI X3T9.5 for the MAC A similar situation occurs protocol.[11] In accord in an FDDI tree topology. with this enhancement, a The MAC error counters frame is valid only if are not associated with a its ending delimiter is particular physical link. followed by a symbol that Fault isolation must be cannot be created by the based on facilities present noise event that created for each physical link. For the ending delimiter. this purpose, we developed Thereby, undetected a protocol called link corruption was greatly error monitor (LEM). LEM reduced in the final, takes advantage of the standard MAC protocol.[12] requirement in the PHY 18 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI Physical Layer standard that a set of code MAC FCS error counters as bit groups representing well. For instance, the violation symbols and FCS-based estimate of BER certain sequences of also depends on packet control symbols not be length and additionally transmitted (repeated) on ring utilization. The onto a physical link. Our FCS error counters count study of the error process errors in valid packets indicated that roughly 30 only, so estimates of error percent of the error events rate are strongly affected could be detected by the by ring utilization. The physical layer decoder.[10] LEM estimate includes This accuracy is acceptable error events in tokens, as BER variations of many stripped frames, and those orders of magnitude are that occur during the idle often the most important. period between packets. LEM counts the decode The LEM protocol counts violations that are errors and provides a BER received only at one point estimate for each link in the LAN immediately in the FDDI LAN. Network after the error event management applications occurs. Errors not counted may collect this data by LEM are those in which and identify marginal the created symbol may be links within the LAN. In repeated by a PHY port, addition, LEM provides a such as when a data symbol mechanism to automatically is changed to another data eliminate faults from symbol. An instance of the network. This faultLEM protocol may observe recovery procedure each PHY port and detect preserves the integrity events associated with a of the ring when physical particular physical link. links would otherwise The accuracy of a LEM BER prevent ring operation. estimate is comparable The LEM protocol was to other methods and proposed and included in has the advantage of the draft FDDI Station providing better fault Management (SMT) proposed isolation. The accuracy standard.[13] of a LEM estimate is The analysis of the affected by the statistics physical layer error given above for the error process resulted in two process and the length important changes that of packet transmitted on reduce the impact of the ring. Generally we errors. A change proposed only assign significance and adopted in the FDDI to the order of magnitude MAC protocol greatly of the estimate, i.e., reduced the rate of the exponent of the BER undetected corruption. written in scientific The isolation of components notation. This type of contributing to a high accuracy problem is shared error rate is facilitated by BER estimates based on by LEM, now a part of Digital Technical Journal Vol. 3 No. 2 Spring 1991 19 Development of the FDDI Physical Layer the draft FDDI StationFinally, the analysisManagement standard.of the physical linkThese developments haveerror process resultedimproved the correctnessin increased correctnessand maintainability of thethrough a reduction of theFDDI LAN.undetected error rate andenhanced fault isolationSummaryprovided by the link errormonitor, LEM.Our development work onthe FDDI physical layerAcknowledgmentsprovided physical layercomponents, specificationsThe authors have reportedand new protocols. Thison the results and work ofpaper has described themany individuals. Specialoperation of the FDDIacknowledgment is extendedphysical layer and theto Raj Jain, Don Knudson,functional partitioningCharlie Kaufman, and Hermanof the chip set. TheLevenson for their originalfunctional partitioningcontributions to theresulted in greaterdevelopment and for theirintegration and lowerhelp in writing this paper.cost for the chip set.Much of the work on theReferencesphysical layer centeredon the need to control1. P. Hayden, R. Graham,the error characteristicsand W. Hawe, "Fiberof both the constituentDistributed Datalinks and interplayInterface Overview,"of many asynchronousDigital Technicallinks as a system. ThreeJournal, vol. 3, no.important design problems2 (Spring 1991, thiswere solved during theissue): 10-18.development effort. First,the elasticity buffer2. FDDI PHY, ANSI X3.148-and smoother protocols,1988, Token Ringwhich were developed forPhysical Layer Protocolthe distributed clocking(New York: Americanscheme, resolve dataNational Standardsintegrity and data lossInstitute, 1988).problems. Second, the3. FDDI PMD, ANSI X3.166-design of the fiber-1990, Physical Layeroptic link for FDDIMedium Dependent (Newrequired methods toYork: American Nationalallocate system bandwidthStandards Institute,and power margins. The1990). bandwidth, jitter, andloss budgets provided ameans to allocate channelmargin between individualcomponents and can beapplied to the design ofmany transmission systems. 20 Digital Technical Journal Vol. 3 No. 2 Spring 1991 Development of the FDDI PhysicalLayer 4. T. Chaney and C. Molnar,9. Draft of proposed"Anomalous Behaviourstandard for FDDI MAC,of Synchronizer andAccredited StandardsArbiter Circuits,"Committee (ASC) X3T9.5IEEE Transactions on/83-16, Token Ring MediaComputers (April 1973):Access Control, Rev. 10421-422.(1986). 5. B. Thompson and J.10.R. Jain, "ErrorIannarone, Method andCharacteristics ofApparatus for DetectingFiber DistributedImpending OverflowData Interface,"and/or Underrun ofIEEE Transactions onElasticity Buffer, U.S.Communications, vol. 38,patent no. 4,945,548.no. 8 (1990): 1244-1252. 6. C. Kaufman, M. Kempf,11.R. Jain, "FDDI Errorand J. Hutchison,Analysis," FDDI X3T9.5Method and ApparatusWorking Group on SMT,for Nodes in NetworksCommittee Document SMT-to Avoid Shrinkage of80 (1987). an Interframe Gap, U.S.12.FDDI MAC, ANSI X3.136-patent no. 4,878,219.1987, Token Ring Media7. J. Hutchison and D.Access Control (NewKnudson, "DevelopingYork: American NationalStandards for a FiberStandards Institute,Optic LAN FDDI," SPIE1987). Proceedings, vol. 71513.H. Levenson, "Link Error(1986).Monitor (LEM)," FDDI8. D. Hanson and J.X3T9.5 Working Group onHutchison, "LED SourceSMT, Committee Documentand Fiber SpecificationX3T9.5/88-251 (OctoberIssues for the FDDI1989). See also "LEMNetwork," IEEE ComputerError Monitor," DocumentSociety ProceedingsX3T9.5/88-198 (AugustCOMPCON (1987).1989). Digital Technical Journal Vol. 3 No. 2 Spring 199121=============================================================================Copyright 1991 Digital Equipment Corporation. Forwarding and copying of thisarticle is permitted for personal and educational purposes without feeprovided that Digital Equipment Corporation's copyright is retained with thearticle and that the content is not modified. This article is not to bedistributed for commercial advantage. Abstracting with credit of DigitalEquipment Corporation's authorship is permitted. All rights reserved.=============================================================================