Instruction/ maintenance manual of the product 9077 IBM
Go to page of 342
IBM 9 077 SP Switc h Router: Get Conne cte d to the SP Swit ch S G24 -51 57 -00 Inter nat ional T echnical Suppor t Organizati on http:/ /www.re dbooks .
.
IBM 9077 S P Swit ch Ro uter : Ge t Con ne cte d to th e SP Sw itc h Nove mber 19 98 SG24-5 157-00 Int ernatio na l T echnica l Sup po rt O rganiz atio n.
© Copyright Internatio nal Business Ma chines Co rporation 1998 . All rights rese rved Note to U.S G overnm ent User s – Doc umen tation r elated to restri cted rights – Use, duplic ation or dis clos ure is subject to r estriction s set for th in G SA ADP S che dule Contr act with IB M Cor p.
© C opyri gh t I BM C or p. 19 98 iii Contents Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix Ta b l e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iv IBM 9077 SP Switch R outer: G et Connec ted to the SP Switch 2.4.1 S DR Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.4 . 2 N e w Co m m a nd s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
v 3.14.2 Check Media Card S tatus Using grcard . . . . . . . . . . . . . . . . . . 9 8 3.1 4 . 3 R e s e t Me d ia C a rd U sin g g rre s e t . . . . . . . . . . . . . . . . . . . . . . . . 99 3.1 4 . 4 U s i n g g r s ta t to D i sp la y G R F S ta t ist ic s .
vi IBM 9077 SP Switch R outer: G et Connec ted to the S P Switch 4.5.3 P hysical and Logic al Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 139 4.5.4 Conf iguration Files and Profiles . . . . . . . . . . . . . . . . . . . . . . . . 140 4.
vii Appen dix A. Labo ra tory H ardwa re a nd So ftware Conf ig uration . . . . 233 A.1 Node and Cont rol Workstation Configuration . . . . . . . . . . . . . . . . . . . . . 233 A.1 .1 Hard Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . .
viii IBM 907 7 SP Switch R ou ter: Get C onnec ted to the SP Switch Appendix D. Special Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Appen dix E. R el a ted P ubl icat ion s . . . . . . . . . . . . . . . . . . . . . .
© C opyri gh t I BM C or p. 19 98 ix Figures 1. SP Swi tc h Ro uter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2. Fu nctio nal Compari son . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
x IBM 9077 SP Switch R oute r: Get Connec ted to the SP Switch 41. Master/Slave Connecto rs for SAS Interfaces . . . . . . . . . . . . . . . . . . . . . 1 22 42. A/B Connectors for DAS Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . .
© C opyri gh t I BM C or p. 19 98 xi Ta b l e s 1. Memo ry Con fig urati on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2. Depend entNode Att ributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xii IBM 907 7 SP Switch Rou ter: Get Connec ted to the S P Sw itch 41. Software Levels on CWS and All Nodes Part 10 of14 . . . . . . . . . . . . . . . 248 42. Software Levels on CWS and All Nodes Part 11 of 14 . . . . . . . . . . . . . . 2 49 43. Software Levels on CWS and All Nodes Part 12 of 14 .
© C opyri gh t I BM C or p. 19 98 xiii Prefac e The GRF is a high-performance switched IP Router which provides high-speed data communication links betw een IBM RS/6000 SP and external network s or hosts. It acts as a s pecial-purpos e SP node that routes IP traf fic between SP nodes on t he SP S witch and the out side world.
xiv IBM 9077 SP Sw itch Router: G et Connected to the SP Switch for eight years. Hi s areas of ex perti se in clude RS/6000 S P , SMP , and Benchmarks. He now specializes in SP S ystem Management, SP Perf ormance T uni ng and SP hardware. Dr Steffen Eisenblätter is an AIX Software Specialis t in the RS/6000 SP Software S upport Cent er , Germany .
xv Comme nts W elc ome Y our comments are imp ortant to us! We want our redbooks to be as helpful as possible. Please send us your comments about thi s or other red books i n one of t he foll owing wa ys: • Fax the ev aluation form found in “ ITSO Redbook Evaluation” on page 323 to the fax number shown on the form.
xvi IBM 9077 SP Sw itch Router: G et Connected to the SP Switch.
© C opyri gh t I BM C or p. 19 98 1 Part 1. Introducing and I nstalling the G RF.
2 IBM 9077 SP Switch R oute r: Get Connec ted to the SP Switch.
© C opyri gh t I BM C or p. 19 98 3 Chapter 1 . Dependent Node This chapter provides an ov erview of a dependent node in RS/6000 SP. We start by defining the dependent node and the rationale behind its design.
4 IBM 9077 SP Switch R oute r: Get Connec ted to the SP Switch • The fault serv ice daemon runs on all sw itch nodes in t he RS/6000 SP, but not on the d ependent node. T herefor e, the dependent node does not have the f ull funct ionality of a normal R S/6000 SP Switch no de.
© C opyri gh t I BM C or p. 19 98 5 Chapter 2 . Router Node The first dependent node is ac tually a new SP S witch Router Adapter in a router . This chapter offer s m ore details about the implementation. Section 2.1, “Overview” on page 5 gives you an ov erview of SP S witch Router .
6 IBM 9077 SP Switch R oute r: Get Connec ted to the SP Switch Figure 1. SP S witch Router The RS/6000 SP software treats this adapter as an extension node.
Ro ut er N od e 7 packet size. This would only enable a wide node t o handle appr oximately 7.5 MB /s o f I P tr a ffic . Since Ascend’ s business depends on k eeping pace with networking technology , they already support the major interfaces today .
8 IBM 9077 SP Switch R oute r: Get Connec ted to the SP Switch 2.1 .2 D esign Objecti v es Because the dependent node is part of the RS /6000 SP, it had to be packaged and assigned some roles consistent with other RS/6000 SP nodes. Changes wer e made t o the RS/6000 SP t o incorporat e management requirements for the dependent node.
Ro ut er N od e 9 Figure 3. T ypical Router Configur ation Routers help to reduc e the amount of pr ocessing r equired on local systems, sin ce they perform the computation of routes to remote systems. For example, a system can communicate with a remot e system by pas sing the message (or packets) to the router .
10 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch • Mini mal r out ing A net work c ompletely is olated from all ot her TCP/ IP networks requires only m inimal r outing. A m inimal routing t able is usually bui lt by ifconfig when the net work int erfaces ar e conf i gured.
Ro ut er Nod e 11 situat ions more quickly and accurately than a system administrator can do. Routing protocols are designed not only to switch to a bac kup route when t he primary route becomes inoper able; they are al so des igned to decide which is the "best" route to a destination.
12 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch The second case has proven to be very expensive as well. The RS/6000 SP node was not des igned for rout ing. It i s not a cost- ef fec tiv e way to route traf f i c for the following reas ons: • It takes m any CPU cycles to process routing.
Router Node 13 The GRF uses a cr osspoint switch (see Figure 7) instead of an I/O bus to interconnec t its adapters . This s witch is capabl e of 4 or 16 Gbit/s ( model dependent) and gives better perf ormance than the M CA bus. Figure 7. GRF 4 00 In conventional routers, each packet is processed at each gat eway (also called hop ) along a path.
14 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Figure 8. Conv entional Rou ters The SP Sw itch Router provides near wire- speed packet forwardi ng while usi ng standard routing pr otocol s. This ensures in teroper ability with other network technologies and does not require a spec ific networ k architecture, such as A TM.
Router Node 15 Figure 9. Sw itched Router s Other advantage s of using G RF are as follows: • Av ailability of a redundant power supply • Av ailability of a redundant fan • Availabi lity of a ho.
16 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Multicast IP M ulticast and OSPF Multicast EG P Exterio r Gateway Pr otocol BGP B order Gateway Protocol V ers ion 3 or 4 (BGP 3 or 4) More details about the various prot ocols are in Section 2.
Router Node 17 adapter . As a resul t, ot her network adapt ers ar e br ought down as well. Bringin g down the rout er will imp act all the networ ks in the location. Each RS/6000 SP is allowed t o connect to mul tiple SP Switc h Router Adapters, and it does not m atter if these adapters are on differ ent GRFs.
18 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Figure 10 . Price Compar ison These pric e comparisons are based on US prices as of March 1998.
Router Node 19 Manager updates the system routing t ables and performs ot her administrative functi ons, the int elli gent proc essors on eac h media c ard perfor m all routing functi ons. This design supports effi cient distribut ed processing of router operations.
20 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Subnet Masking/Supernetti ng V ariable length subnet masking is a classless addressing scheme for interdom ain IP pack et routing. It is a way t o more effic iently manage the current 32-bit IP addressing method.
Router Node 21 inform ation for the autonomous system. Here is the list of interior protoco ls support ed by t he GRF: • R IP The Routing Information Pr otocol (RIP), as delivered wit h most UNIX systems, is run by the routing deamon routed . Dur ing the s tartup of routed a r equest f or routing updat es i s i ssued.
22 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Every area m ust connect to the backbone, because the back bone is responsible for the distribution of routi ng information between areas. The backbone itself has all the properties of an area.
Router Node 23 Host extensions f or IP multicasting as described in RFC 1 1 12 are also prov ided. The Router Manager ac ts as a host and uses the Inter net Group Managem ent Protocol (IGMP), version 2, t o add and delete its membership in multicast groups.
24 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch • Source address • Des tination addr ess • Protocol port number (single number , or range, or ranges) for TCP and UDP • E stablished TCP connections 2.
Router Node 25 Figure 1 1. GRF Models The SP Sw itch Router model 9077 04S (GRF 400) can ac commodate up to four media adapters. The SP Sw itch Router model 9077 16S (GRF 1600) can ac commodate up to 16 media adapters. Each adapter enables the GRF to connect to one or more networks.
26 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Power S upp ly The left side of the chassis is reserved for the two power suppl ies that are requi red f or redundancy . The failed power supply can be hot-swapped out of the GRF chassis.
Router Node 27 If the temper ature exceeds 57.5 ° C (137 ° F), the GRF does an automatic system shutdown. • Hot -swappabl e fan For the GRF 16S model, t he cooling fan can be replaced whi le the GRF i s in operati on. • Hot -swappabl e adapters There are two types of adapters on the GRF : the media adapt ers and the IP Switch Control Board.
28 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 1. Normally , al l media cards have a 4 MB send buf fer and a 4 MB r eceive buf fer , except the SP S witch Adapter card, which has a 16 MB buf fer size for each buff er . See also S ection 2.
Router Node 29 Figure 13 . Data Packet T ransfer The routing can be divided into the following steps: 1. A data packet is received by the m edia c ard. 2. The packet is trans ferred t o the receive buf fer by t he DMA engine. 3. The CPU examines the header and gives the destination address to the route lookup hardware.
30 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 10.The header is examined by the CP U, whic h uses the infor mati on to buil d a new header that will deliv er the data across the m edia interface. 1 1. T he DMA engine transfers the packet to the media interface.
Router Node 31 6. The packet is then transferred to the Combus interface by the DMA engine. 7. The packet is sent to the IP S witch Control Board’ s Router Manager across the Combus. 8. The Route Manager receives the packet and passes i t to the dy namic routing sof tware.
32 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Figure 15 . Side View of GRF 400 Cha ssis with Slo ts Number ed The GRF 1600 has 16 media slot s.
Router Node 33 2.3.3.1 Route Man ag er As already mentioned, the router management takes place on t he IP Switch Control Board. Specific functions of t he Route M anager are: • I t proc esses all dynamic r outing pack ets. • It synchroniz es the route tables on the media cards.
34 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch ItemDescription Memory T he IP Sw itch Control Board comes standard with 128 M B of memory ( the f our shaded bloc ks of 32 MB of memory in the upper left corner ). The memory can be upgraded to 256 M B, in increment s of 64 MB (t he f our white bl ocks of m emory).
Router Node 35 Additional ly , the RS232 port (which is not shown in the figure) allows y ou to connect t he VT100 cons ole by using an RS232 nul l modem cable. The console and cable must be supplied by the user . 2.3.4 Mem ory Gu idelines for t he IP Switc h Contro l Board As alread y mentioned, t he GRF base s ystem comes wit h 128 MB of memory .
36 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Figure 18 . System R AM 2.3.5 Ch aracterist ics of GRF Media Cards All GRF media cards (media adapt ers) are self -contained and independent of other media adapters. Each media card has an onboard processor that is responsible for IP forwar ding on the m edia adapter .
Router Node 37 Figure 19 . SP Switch R outer A dapte r The SP Switch Router Adapter i s made up of two parts: t he media board and a serial daughter card. The serial daughter card is an interface for the media board into the crosspoi nt switch. This sw itch is the m edium by which the GRF (m edia) adapters talk to eac h other .
38 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Receive Controller and Pr ocessor This component recognizes the SP Switch segments and assembles them into IP packets in the 16 MB buffer . Up to 256 IP datagrams can be handled simultane ously .
Router Node 39 • It is able to transfer up to 30,000 packets per second. At 20,000 packets per second, each packet needs t o be at 5 KB in order to achieve the 100 MB per second transfer rate mentioned. • As previously mentioned, each adapter stores its own route tables in memory .
40 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch IP/S O NE T The IP /SONET OC-3c is a single-por ted card that allows the user to c onnect t o a di gital network using a transmis sion form at known as Synchronous Optical Network prot ocol (SONET).
Router Node 41 The attribut es of t he DependentNode class are described in detail as follows: Attr ibuteDescripti on node _numbe r This user-suppl ied node num ber represents the node position of an unus ed S P S witch por t used for the SP Switch router adapter .
42 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 2.4.1.2 Depen dentAd apter Attribut es The attribut es of t he DependentAdapter class are described in T abl e 3: T able 3.
Router Node 43 Attr ibuteDescripti on node_type This attribute is set to dependent for GRF and to standard for a ll other RS /6000 SP nodes . s witch_max_ltu This specifies the maximum packet length of data on the SP Switch; the def ault is 1024. Do not change this value for any reason .
44 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch The enadmin command is used t o change t he administrat ive state of a dependent node in the GRF ; it has the following charact eristics: • It is part of the ssp.spmgr fileset. • It must only be executed on the Control Wor kstation.
Router Node 45 T able 7. en defnod e Co mma nd Op tions This command adds at tribute information for the extension node. The endefadap ter command adds IP information, such as the IP address and netmask for the ex tension node. T oget her , t hese two commands define t he extension node.
46 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 2.4.2.2 The en rmn ode Comm an d The enrmnod e c ommand is used t o remove an extension node from the SDR DependentNode class and can also be e xecuted us ing smit . The f ast path for smit is dele te_extnode .
Router Node 47 2.4.2.3 The en defadap ter Com mand The endefad apter command is us ed to add or change the extension node adapter IP inform ation in the SDR DependentAdapter object, and can be executed using s mit . T he fas t path f or smit is ente r_exta dapter .
48 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 2.4.2.5 The en adm in C omma nd The enadmin command is used t o change t he status of t he SP Switch route r adapter in the GRF and can also be ex ecuted using sm it. The fast path for smit is mana ge_extnode .
Router Node 49 2.4.2.6 The spls tnodes Co mmand The sp lstnodes command is used to list the node attribut es of all node s in the SDR, and can also be ex ecuted using smit. The fast path for smit is list_extn ode . See all command opt ions in T abl e 1 1.
50 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 2.4.2.7 The sp lstadapter s Comma nd The splstadapers command is us ed to list the adapter attributes of all nodes in the S DR, and can al so be ex ecuted us ing smit . The fast path for smit is list_exta dapter .
Router Node 51 2.4 .3 En hanced Commands The following commands (see T abl e 13) have been m odified due t o the introducti on of the dependent node: T able 13 .
52 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 2. 4.4 Ha rd ware Pe rsp ect ive s In P erspectiv es IP Nod e is used as a convenient and short descriptive term easily displ ayed in the GUI. It conv eys the role and functions of the dependent node.
Router Node 53 3. Nodes pane (Fram e or I con V iew) 4. Informati on area The most obvious change is t he addition of the IP Node icon as seen in the Nodes pane. (The figure above shows the Frame View .) The default label for this icon is IP Node <node number>.
54 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Figure 21 . Action Men u •V i e w This will bring up t he IP Node’s hardware notebook, shown in the next figure. • F ence/Unfence... This will bring up another window to allow us to either fence or unfence an IP Node.
Router Node 55 This will bring up a w indow to show the three-digit disp lay of all R S/6000 SP standard nodes in t he current partit ion. T his action does not apply t o the I P Node, ev en though it is sel ectable. • Open Administrative Session...
56 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch These are the attributes listed in the Configurat ion tab: • Node number • Hos tname • M anagement agent hostname • SNM P comm.
Router Node 57 Figure 23 . System P artition Aid Perspectiv es The IP Nodes can only be ass igned to a partiti on here. This i s done either by using t he Assig n icon in the toolbar (2), or by selecting Action ->Nodes->Assign Nodes to S ystem Partiti on on the menu bar (1).
58 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Figure 24 . System P artition Aid Notebook These attributes are list ed in t he Node Information tab: • Node number • S witch port number • Assigned to system partition 2.4.5 SP Extens ion Node SNMP M a n ager The SP E xtens ion Node S NMP manager is contained in the ssp.
Router Node 59 •stopsrc •lssrc •traceson •tracesoff 2.4.6 Depe nden t Node MIB IBM has defined a dependent node SNMP Management Informati on Base (MIB) called ibmSPDepNode. This MIB contains definitions of objects represent ing configuration attribut es of each dependent node and its state.
60 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch ibmSPDepSwT oken A combination of sw itch_number , switch_chip and switch_chip_port attribut es from the DependentNode c lass. ib mSPD epS wAr p The arp_enabled attribute in the S witch_partit ion class.
Router Node 61 PSSP 2.4 to represent coexistence. Also, note that Node 16 is empty , because the SP Switc h port for this node is us ed by the SP Sw itch router adapter in t he GRF . Figure 25 . Coexistenc e The dependent node is only supported in PSSP 2.
62 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch • All RS/6000 SP nodes with a v ersion less than PSSP 2.3 in the partition need to maintain the right level of fixes (PTF s) in order for coexistence with PSSP 2.4 to take place. • The ssp.
Router Node 63 Figure 26 . Partitioni ng Normally , RS/6000 SP nodes in different partitions cannot communicat e with each ot her thr ough the S P Switch.
64 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch Next, ensure that the followi ng parameters are defined: ParametersDescriptions GRF IP add ress T he IP address for the GRF adm inistr ative Ethernet. GRF netmask The Netmask for the GRF administrat ive Ethernet.
Router Node 65 2.6 Plannin g for the De pendent Node Next, f or each dependent node on the RS/6000 SP, define t he following: ParametersDescriptions Node # A user-supplied dependent node number representing the node position of an unused SP S witch port to be used by t he SP Switch Router Adapter .
66 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch 2.7 Conc lusion The SP S witch Router 9077-04S has an aggregate bandwidth of 800 MB/s.
© C opyri gh t I BM C or p. 19 98 67 Pa r t 2. Sc enarios This part presents some sample configurations of an RS/6000 SP s ystem with an SP Switch Router . It is beyond the scope of this book to re present all possible appl ications of an SP S witch Router .
68 IBM 907 7 SP Switch Rou ter: Get C onnec ted to the SP S witch.
© C opyri gh t I BM C or p. 19 98 69 Chapter 3 . I nstall a tion and Configuration The SP S witch Router functions as an IP router to provide high-speed data communication li nks between SP processor nodes and ex ternal networks or hosts.
Inst al l atio n an d Co nf igur a tio n 70 The intent of this chapter is to provide, or refer you to, the necessary inform ation to enable you to attach an SP S witch Router to an IBM SP system.
Inst al l atio n an d Co nf igur a tio n 71 UNIX systems. On most of the UNIX systems you are working on the shell lay er aft er you logged ont o the s ystem. Many system managem ent and configuration commands are now availabl e. Enter a question mark (?) to retriev e a list of CLI commands.
Inst al l atio n an d Co nf igur a tio n 72 system and enter these basic configuration parameters. Procedures for starti ng and s etting up the SP Switch R outer are f ound in GRF 400/1600 Getting Start ed V .
Inst al l atio n an d Co nf igur a tio n 73 Figure 29 . Conn ecting th e GRF to the Fram e • Y ou are ready t o configur e media c ards. Proce dures to configure media cards are in this redbook; complete information is in the GRF Configurat ion Guide 1.
Inst al l atio n an d Co nf igur a tio n 74 Figure 30 . Conn ecting th e GRF Conso le • The IBM SP system is up and operating. • The SP system adm inistrator has given you one of these pieces of i.
Inst al l atio n an d Co nf igur a tio n 75 4. Methods t o determi ne node num ber and SP S witch por t for an SP S witch Router Adapter card 5. A step-by-st ep configuration of an SP Switch Router Adapter card 6. A list of ways to verify that the SP Switch Router Adapter card is correc tly install ed in the SP Sw itch Router 7.
Inst al l atio n an d Co nf igur a tio n 76 4. Route the Ethernet twisted-pair cable betw een the SP Switch Router unit and the Ethernet hub, then connect the c able to the SP Sw itch Router control board and the Ethernet hub. 5. V erify that the SP CWS has a connect ion to this same Ethernet hub.
Inst al l atio n an d Co nf igur a tio n 77 3. Edit the file /etc/rc .boot and see if the line mount /dev/ wd3a /var/log is present ; if not , add this line at the end of the file.
Inst al l atio n an d Co nf igur a tio n 78 5. Edit the file /etc/syslog. conf to specify the location where the logs will be kept. Uncomment the local log conf iguration li nes in the “ Log messages to Dis k” section by removing #disk# from each line, and specify /var/log as the dir ectory for eac h log.
Inst al l atio n an d Co nf igur a tio n 79 7. Save all changes and reboot: # grwrite -v # reboot 8. After the SP Switch Router is up and running again, use csconfig -a to verify that the PCMCIA interface i s available and t he PCMCIA disk are up. For a quick test, run grconslo g -vf.
Inst al l atio n an d Co nf igur a tio n 80 Figure 31 . SP Sys tem Admi nistrative Ethe rnet Connectio ns 3.4 .2 SP Sw itch Cabl e The SP S witch Router Adapter card provi des one full-duplex attachment and requires a s pecific c able with 50-pin c onnector ends, obt ainable from IBM.
Inst al l atio n an d Co nf igur a tio n 81 2. Using appr opriat e frame entr y and exit hol es f or c able management, route the SP Switch cable between the SP S witch Router unit and the S P Swi tch .
Inst al l atio n an d Co nf igur a tio n 82 3.5.1 D eter mining the Switch Conne ction fo r a Depe ndent Nod e The SP S witch Router Adapter connection replaces an SP node connection to the SP Sw itch.
Inst al l atio n an d Co nf igur a tio n 83 Figure 32. Sw itch Port Assi gnmen ts in S upp orted Frame C onfigu rations Swit c h Frame n Swit c h Frame n No Sw i tch Frame n+1 Swit c h Frame n No Sw i.
Inst al l atio n an d Co nf igur a tio n 84 Figure 33 . Node Numbering for an SP Syst em 3.5 .2 Pr ocedu re to Get th e Jack N umber Followi ng are t he steps r equired to get the jac k number: 1. From the SP control wor kstation, dete rmine the dependent node’ s number by ent ering SDRGetObjects DependentNo de node_numbe r.
Inst al l atio n an d Co nf igur a tio n 85 In this case, the primary node host name is s p21n01. 4. Log into the primary node by entering telnet no de_hostname, where node_hostname is the hos t name of t he primary node. 5. From the primary node, enter pg /var/a dm/SPlogs/css /out.
Inst al l atio n an d Co nf igur a tio n 86 • The SP Switch Rout er card c onnect ed to port J31 of SP Switch A2 is node number 25. • The SP Switch Rout er card c onnect ed to port J31 of SP Switch A3 is node number 41. • The SP Switch Rout er card c onnect ed to port J15 of SP Switch A1 is node number 16.
Inst al l atio n an d Co nf igur a tio n 87 3.7.1.1 Overv i ew of the St eps to Conf igure a Me dia Card A det ailed dis cussion of t hese st eps follows thi s overvi ew . 1. Edit the SNMP c onfigurati on file and s tart the SNM P daemon on the SP Switc h Router .
Inst al l atio n an d Co nf igur a tio n 88 • As an alternative, you can log on to the SP S witch Router and use a UNIX edit or to enter the parameters in the /etc/grdev 1.conf file. 5. Reboot the SP Switch Router unit so that the altered configur ation files are install ed and used.
Inst al l atio n an d Co nf igur a tio n 89 3.9 Put SN MP Chang es into Effe ct T o have changes to /etc/snmpd.conf take ef f ect, k ill snmpd. It will be automatically res tarted. Log in as r oot, find the snmpd P ID (process I D), and th en kill the SNMP daemon, as follows: 3.
Inst al l atio n an d Co nf igur a tio n 90 Note: The E xtension Node Identifier (t he number of the slot in the G RF , the SP Switch Router A dapter c ard is seat ed) mus t be given as a two-digit nu.
Inst al l atio n an d Co nf igur a tio n 91 • Command: smitty annotator /etc/SP/ expected.top.annotated now has a new line: COM MAND S TATUS Comm and: O K s tdout: yes std err: no Befo re com mand c ompletio n, add itiona l inst ructio ns may a ppear below.
Inst al l atio n an d Co nf igur a tio n 92 • Command: smitty manage_extnod e Use SDRGetOb jects switch_ responds to see the actual status of t he S P Swi tch : Before the extended node can be successfully Eunfen ced, more work has to be done. Ext ension Node Manage ment Type or se lect v alues in entry field s.
Inst al l atio n an d Co nf igur a tio n 93 3.10.2 Me thod 2: Ed it /etc/grifcon fig.conf - O ptiona l Edit the /etc/grif config.conf file t o assign an IP address to each logical SP Switch Router inter face. Y ou can also provide other information about the logical I P network to which that interf ace is physically attached.
Inst al l atio n an d Co nf igur a tio n 94 Inter net address The Internet address is the 32-bit IP address for the specified logical interf ace. The address is i n stand ard dott ed-decimal ( octet) notation: xxx.
Inst al l atio n an d Co nf igur a tio n 95 Default MTUs for fr aming protocols are: • Frame Relay: 4352 bytes • HDLC: 4352 bytes • Point-to-Poi nt Protocol: 1496 bytes MT U dis covery facility MTU sizes are generally selected at the host end of the route.
Inst al l atio n an d Co nf igur a tio n 96 Nevert heless, c heck the f ile / etc/gr dev1.conf. It must cont ain an entry for t he slo t in whi ch t he media c ard is install ed. A s we ha ve our card in slot 3, the entry looks as follows: These entries show up after the SP Switch Router Adapter card gets configured on the SP Switch.
Inst al l atio n an d Co nf igur a tio n 97 T o save an alternat e configurat ion on the int ernal flash bas ed upon the current ly running configuration on the internal flash device, use grsnapshot -si -di=revis ion,ver sion. For more i nf orm ation ab out these c ommands, see GRF Referenc e Guide 1.
Inst al l atio n an d Co nf igur a tio n 98 2. Enter a ping command. Specify the appropriate media card by its chassis slot number; for example, to act on the SP S witch Router Adapter media card in slot 3, enter pin g -c4 -P grid 3. This is what you see when the m edia card responds: T o act on the GRF cont rol bo ard, ent er ping -P grid 66.
Inst al l atio n an d Co nf igur a tio n 99 3.14.3 Re set Media Card Using g rreset Use t he grreset command to reset a media card from the UNIX prompt: 1. Log in as root on the SP S witch Router . 2. Enter t he gr reset command. Specify t he appropri at e m edi a card by it s chassis slot number .
I nsta lla tion an d Con figur ati on 100 Below i s an actual example: # grstat -w70 all gt030 gt030 ipstat co unt descripti on 11095 886 total pac kets receive d 51 packets d ropped 3678 330 packets .
I nsta lla tion an d Con figur ati on 101 one of the f ollowing ac tions t o br ing the S P Switch Router A dapter card online: • A s wit ch initializat ion • A n unfencing sequence • A nother S witch management s equence The appropriat e action depends on what state the S P system is in with respect to the dependent node.
I nsta lla tion an d Con figur ati on 102 Each SP Switch Router Adapter media card is considered a dependent node for the SP System. Eac h dependent node has a node_number and ot her configurat ion and status information that is unique to that dependent node.
I nsta lla tion an d Con figur ati on 103 Maintenance I nformation M anual Dependent Node MAP , r eturn t o that procedure. For more inform ation about configuration as related to the SP , see RS/6000 SP: Administrat ion Guid e V ersion 2 Release 4, GC23-3897 and RS/600 0 SP: Command and T echnical Reference Vers ion 2 Release 4, GC23-3900.
I nsta lla tion an d Con figur ati on 104.
© C opyri gh t I BM C or p. 19 98 105 Chapter 4 . Configuration of IP-Forw ardi ng Media Cards This chapter covers the installation and configur ation of selected IP Forwar ding media c ards in an SP Switc h Router . F or detail ed inf ormation ref e r t o GRF Co nfiguration Gui de 1.
106 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Figure 36 . Componen ts of the Ether net Interfac e N ame The interface name is used in the /etc/grif config.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 107 • C hange run-time binaries • Change dump v ariables 3. Load profil e Global executable binari es are set in t he Load profile in the hw-t able field. These only change when you want to execut e new run-time code in ev ery Ethernet card.
108 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 4.1 .5 Sp ecif y Ethe rne t Card Paramet ers As already mentioned, modifying the following profil es is optional.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 109 if-config : Ethernet interf ace configura tion. Enumerate d field, valu es: autonegot iate: autoneg otiate 10-half: 10 BaseT Half Duplex 10-.
11 0 IBM 9077 SP Switch R oute r: Get Connec ted to the S P Swi tch you s ee the out put f or al l eight. The input port side is report ed on firs t. maint 5 -to retur n GRF swi tch stat ist i cs.
Con fig ur ation o f IP -Fo rw ard in g Me di a Card s 111 See Fi gure 38 f or the nami ng conventions of an A TM int erface. Figure 38 . Com ponen ts in the A TM OC-3c In terfa ce Name V irtu al Circu its A virtual circ uit (VC) exists between two A TM devices.
11 2 IBM 9077 SP Switch R oute r: Get Connec ted to the S P Swi tch VPIs 0 through 15 are available for configuration use. VPIs are assigned in the /etc/ gratm.conf fil e with regard to their V CI. VCI s VCI s name VCs. VCIs are also ass igned in the /etc/gratm.
Co nf igur atio n of IP-F orw ar ding M edia Ca rds 11 3 4.2.2 Inst alling Co nfiguratio ns or Change s In the command line interface (CLI), use set and write commands t o install configurat ion parameters . T o save the /etc configurat ion direc tory , us e grwrite - v .
11 4 IBM 9077 SP Switch R oute r: Get Connec ted to the S P Swi tch 4. Load profil e (optional). Global executable binari es are set in t he Load profile in the hw-t able field. These only change when you want to execut e new run-time code in ev ery AT M c a r d .
Co nf igur atio n of IP-F orw ar ding M edia Ca rds 11 5 interf ace basis. This field is also used to specify ISO when an ISO address is being added to an i nterf ace’s I P address. Specify the MTU v alue as mtu xxxx . Leave the arguments field blank if you are not us ing it.
11 6 IBM 9077 SP Switch R oute r: Get Connec ted to the S P Swi tch • I nterface s ection Define t he traffic shapi ng profile for the logic al int erface t o whi ch the media c ard’ s PVCs ar e assigned.
Co nf igur atio n of IP-F orw ar ding M edia Ca rds 11 7 The following message is retur ned along with the chan ged prompt : Current p ort card is 3 GR 3> . • T o leave any main t prompt and return to the shell, enter quit . Foll owing are just a few maint commands we hav e found useful; of course, your ex periences m ay vary .
11 8 IBM 9077 SP Switch R oute r: Get Connec ted to the S P Swi tch 4.2.8 Using grrt to Dis play the Ro ute T able Use t he grrt - S -p<slot > command to di splay t he c urrent content s of t he A TM OC-3c c ard’ s rout e tabl e. The f ollowi ng is an act ual screen shot: # grrt -S -p1 default 1 192.
Co nf igur atio n of IP-F orw ar ding M edia Ca rds 11 9 4.2 .9 U sing gr stat t o Disp lay GRF St atis tics Use t he gr stat -w70 a ll <interface > command to di s play t he curr ent st atist ics of the A TM OC-3c card’s IP stack.
120 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Figure 40 . A TM OC-12c P hys ical and Logic al Interf aces Phys ical Interfaces The A TM OC-12c media card s upports one phy sical interf ace. It suppor ts the assignment of 220 logical interf aces out of a range of 256.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 121 1. Identify each logical i nterface. Edit /etc/gri fconfi g.conf, t o identi fy ea ch logic al int erface by assigning: • A n IP address • T he GRF i nterface nam e • A netmask, as requir ed • A destinat ion or broadcast addr ess, as required • An MTU, if needed 2.
122 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch It might be useful to give a short overview of possible FDDI connecti on options, namely SA S, DAS, optical bypass and dual homing, and show a picture explaini ng these scenarios. Sing le Atta ch (S AS) Single attac h FDDI interfaces can be either master (M) ports or slave (S) ports.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 123 Figure 42 . A/B Connec tor s for DAS Interfac es Co nfi g uri ng SA S v ers u s D AS Only the top or bot tom pair of FDDI interfaces c an be set t o dual attach. Interfac es 1 and 2, for example, m ust not be paired.
124 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch super> li st ports 0 port_num = 0 cisco-hdl c = { off on 10 3 } fddi = { single off } sonet = { "" "" 1 sone t .
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 125 super> wr ite CARD/0 wr itten super> Opt ical b ypa ss Optical bypass c apability has to be provided externally . The F DDI f ace plate has a six-pin DIN connector to direc tly attach a single bypass switch.
126 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Configure the FDDI m edia card f or dual at tach, but use t wo single att ach (SAS) cables to connect to two M ports. As shown in Figure 45, the M ports can be on either one or two F DDI conc entrators on the ring.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 127 • By a logical inter face number assigned after the SAS/DAS settings are numbered (used in the / etc/ grif config.
128 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Figure 47 . Physica l Interf ace Numb er ing on the FDDI Med ia C ard The diagram shows physical inter face numbering to be 0-based (0–3), whereas SNMP numbering is 1-based (1–4). 4.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 129 Edit / e tc /gr ifc o n fi g . c onf to identify each logical interface by assigning: • A n IP address • T he GRF i nterface nam e • A netmask, as requir ed • A destinat ion or broadcast addr ess, as required • An MTU, if needed 2.
130 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch The arguments field is also used to s pecify ISO when an ISO address is being added to an i nterf ace’s I P address. Specify the MTU v alue as mtu xxxx . Leave the arguments field blank if you are not us ing it.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 131 This completes the procedure to configure FDDI cards, and as with the A T M card, we would li ke to int roduce some of the ma int c ommands we found to be useful.
132 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 4.4.10 Us ing grrt to D isplay the Route T able Use t he grrt -S -p<slot> com mand to display the current contents of the FDDI card’ s route table, as f ollows : # grrt -S -p0 default 1 192.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 133 4.4 .1 1 Usi ng gr stat t o Disp lay G RF St atist ics Use t he gr stat -w70 a ll <interface > command to di s play t he curr ent st atist ics of the FDDI card’s IP stack.
134 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch these modes. Hosts must pass on the appropriate informati on for t he GRF media c ards and ot her HIPPI devices t o operate in t he desired way . HIPPI off ers many configuration options .
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 135 header . The media card reads the header only if told to do s o by information in the HI PPI I -f i eld. I f the I -field tells t he c ard to read t he IP header , then an I P connection is established.
136 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 01 Lo gic al A ddre ss When PS is set to 01, that is, l ogi cal addres s, the host does not know or want to speci fy the act ual phys ical rout e t o the tar get endpoint. The host supplies a logical addres s for the endpoint host.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 137 4.5.1.6 Direc tion Bi t HIPPI hosts set the di rection bit (D) . This bit determines how a s witch or router reads the 24- bits of des tinati on addr ess infor mati on.
138 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch of t his addres s to slot 64 in the f ile /etc/grl amap.conf indic ates the I P connection to the receiving media card and causes i t to read the IP header . The logical address 0xfc0 is pr eset as a default in the logi cal addres s table.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 139 Section 7.3, “HIPPI Backbone Connection” on page 227, des cribes the steps to configur e the GRF ’s HI PPI media card to do IP forwarding, so you have t o refer to t he Ascend documentation if you need t o set up a di f ferent configurat ion.
140 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 4.5.4 Co nfiguratio n Files an d Profile s The following are the steps to configure HIP PI cards. For detailed inform ation, s ee GRF Co nfiguration Gui de 1.4, GA22-7366. 1. Identify each logical i nterface.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 141 4.5.5 Inst alling Co nfiguratio ns or Change s In t he command line interf ace (CLI), w hich is the wor king environment on t he GR F w i th t h e super> pr ompt, use the set and write commands to install configurat ion parameters onto the media card.
142 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 4.6 Conf iguring Brid ging This Chapter describes the GRF bridging implementation and provides configurat ion information. 4.6 .1 GR F Brid ging I mplemen ta tion The GRF im plements IEEE 802.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 143 4.6.2 Sim ultaneo us Rou ting and Brid ging Ascend’ s transparent bridging does not preclude t he use of IP packet r outing on the same physical int erface. Bridgi ng as well as IP version 4 (IPv4) routi ng can both be enabled on the same physical interf ace.
144 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 4.6.4 Inte roperability The following table giv es an over view of the GRF s interoperability features. FDDI Fram e forwarding is compatible with any station sending and receiving FDDI LLC frames.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 145 A frame may be too large for the maximum transmissi on unit (M TU) of the sending GRF interf ace. One example is when for warding a 4500-byte frame from FDDI to an Ethernet interface wit h an M TU of 1500 byt es.
146 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 4.6.9.3 Editing U tility – Bredit The bredi t utility is used t o ac cess and edit the / etc/bridged. conf configurat ion file. At t his point , bredit runs a script in which you are asked if you want to make the changes permanent.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 147 In terf ace Port ID Con Stat e C ost Co st Brid ge Port ------ --- ------- --- ---- ---- -- ----- ----- ---- ---- ---- -------- --- ---- ---.
148 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Flag s: 0 xb04 3 u p br oadc ast runn ing lin k0 l ink1 mul tica st Brid ging med ia: fddi bpd u MAC addr ess: 0:c 0:80 :89: 2d:f 5 Br.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 149 If you are going to configure an enc apsulated bridge on an A TM circuit, edit the /etc/ gratm.conf fil e to create a PVC on the A TM OC-3c logical interf ace. 5. Specify ARP ser vice in t he /et c/grarp.
150 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 4. 6.1 1 . 2 C rea ti ng B ri dge Gr ou ps i n bri dge d. co nf The only required parameter is the list of F DDI, A TM or Ethernet interfac es you are assi gning to t he group.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 151 • V C-Based Mul tiplex ing (R FC 1483, se ction 5) When LLC Encapsulation is used, a single PVC is configured to carry all bridged traf fic. The same PVC can also c arry nonbr idged t raff ic suc h as routed IP datagram s.
152 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch A bridging PVC is assigned a protocol value. This value must be consist ent with the bridging method defined for the logical interf ace. Bridgi ng PVCs ar e assi gned either of the f ollowing pr otocol values: 1.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 153 Followi ng are s ome PV C configurat ion examples: LLC Encapsulated: # T raf fic shape T raf fic_Shape name=high_speed_high_qual ity peak=15.
154 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch PVC ga030 0/32 proto=vcmux_bridge,ether PVC ga030 0/33 proto=vcmux_bridge,ether_fc s PVC ga030 0/34 proto=vcmux_bridge,bpdu # PV C for IP ( RFC 1577) PVC ga030 0/35 proto=llc Note: IP routed traf fic is transmi tted on its own P VC.
Con fig ur ati on of IP -F orw ard in g Me di a Car d s 155 4.6.13 Brid ging FDDI T ransparen t bri dging is especial ly useful f or day-t o-day c ustomer envi ronments, where s everal FDDI back bones meet i n t he c omputing center and must be connected to an SP .
156 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch.
© C opyri gh t I BM C or p. 19 98 157 Chapter 5 . Si ngle R S/6 000 SP and Single SP Sw itch R outer This section provides several sample configurati ons that are possible with a single RS/6000 SP and a single SP Sw itch Router .
158 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Configu ration assu mptions: • The SP S witch Router Ethernet media card has been installed according to Section 4.1, “Ethernet 10/100Base- T Configuration” on page 105 and works proper ly .
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 159 T able 14 . Con fi guration of SP Sw itch - Eth ernet Connec tion T o successfully run this configuration, no routes need to be set on the S P Switch Rout er . The F50 and the proc essor nodes i n SP 21 require a ttention, though.
160 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch -mtu 1500 10.20.30.1 2. Check for correct routing entry: 3. On the nodes in SP21 t hat are supposed to comm unicate with the F50, add the following route: route add -net 10.20.3 0.50 -netmask 255.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 161 On the chosen nodes in SP 21, ping the A TM interface of the F50: If these ping commands fai l, check r outing settings again. If everythi ng is as it shoul d be, t ry to p ing the GRF Ethernet media card or the GRF S P Switc h media card to find out which part is faili ng: ping 192.
162 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch As you see in the screen shot, the F50’s 9.2 MB/s comes v ery close to the theoret ical max im um throughput of about 10- 1 1 MB/s.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 163 • If ARP is disabled on the S P Switch net work , the IP addres ses ass i gned t o the nodes must be determined by the Switc h node num bers. Refer to PSSP Planning, Volum e 2, Control Work station and Software E nvironment for details .
164 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch though. T o add the needed routing infor mation and c heck for proper work, foll ow these steps: 1. On node 10 in SP21, add the following r oute to the swi tch network of SP2: route add -net 192.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 165 6. On the CWS of S P2 check if the SP S witch Router Adapter c ard is configured. S ee if the SP Switch Router Adapter card shows up green in perspecti ves or enter SDRGetO bjects switch_respo nds .
166 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch BRL, and ported to A IX by Prof. Peter Haas, University of Stuttgart ). tsock is a program that uses the Ultr aNet soc ket emulation l.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 167 5.1 .3 SP Sw itch - A TM Connect ion This scenario might be used quite often to attach a single computer with an A TM interface to the SP Switch of an SP system.
168 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Figure 55 . SP Switch - A TM Con nectio n T able 16 shows the I P addres ses used in our configura tion. T able 16 . Con fi guration of SP Sw itch - A T M Connection T o successfully run this configuration, no routes need to be set on the S P Switch Rout er .
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 169 of t he l ack of an ARP server; signal ing protoc ols should onl y matt er wi th SVCs and not with PVCs. 3. On the F50, use smitty chg_a tm , select the A TM device and change the fiel d SVC UNI Version fr om au to_detect to uni3.
170 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 5. Have sm itty carry out its work, exit smitty and then run smit ty mkatmpvc : In t he optional PVC De scription field, do not use blanks or underscores. In the VPI:VCI field, use the numbers you get from t he PVC statement in the /etc/gr atm.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 171 T o add the needed routin g info rmation, f ollow t hese st eps: 1. On the F50, add the following route to the nodes i n SP21: route add -net 192.168 .14 -netmask 255.255.255. 0 -mtu 9180 1 0.
172 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch The - mtu parameter is opti onal but should be set to ensure optimal pac ket size on this route.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 173 If these ping commands fai l, check r outing settings again. If everythi ng is as it should be, t ry to ping the S P S witc h Rout er A TM media card or the SP Switch R outer Adapter car d t o w hich part is failing: ping 192.
174 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch An A TM adapt er establishes a duplex connection t o it s part ner , so the 16. 5 MB/s wri te throughput should be accompanied by another 16.5 MB/s read throughput . T o prove t his, we st art ed tw o ft p put commands fr om the same SP21 node.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 175 page 175 and T able 17 on page 175. The netmask for all interfaces is 255.255.255 .0. Figure 56 . SP Switch - FDDI C onne ction T able 17 shows the I P addres ses used in our configura tion.
176 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T o successfully run this configuration, all four ports of the SP Switch Router FDDI media card have t o be assigned IP addresses in dif ferent subnets. Otherwis e, only port A0 will be activated.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 177 3. On the nodes in SP21 that are s upposed to comm unic ate with t he diff erent FDDI bac kbones, add the nec essary routes: route add -net 10.2.1 -netmask 255. 255.255.0 -m tu 4352 192.1 68.
178 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 7. On the CWS of SP21, check i f t he SP Switch Router Adapter card is configured. T o perf orm this check, look if the SP Switch Router Adapter card shows up green in perspectives or enter SDRGetObj ects switch_re sponds .
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 179 If these ping commands fail, check routing settings and IP address assignment again. If everything i s as it should be, try to ping the GRF FDDI media c ard port s or the GRF SP Switc h media c ard to find t he fail ing part: ping 10.
180 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Configu ration assu mptions: • A n SP Switch Router FDDI media c ard has be en instal led accordi ng t o Section 4.4, “FDDI Configurati on” on page 121 and works properly . • The S P S witch Router Adapter card has been installed accordi ng to Section 3.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 181 T able 18 shows the I P addres ses used in our configura tion. T able 18 . Con fi guration of SP Sw itch - FDDI C onnec tion (B ridging) T o successfully run this configuration, make s ure that all four dif ferent FDDI backbones are logically locat ed in one subnet.
182 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch bridge_gr oup bg0 { port gf00 0 gf001 gf002 gf003; }; This is the simplest br idge definiti on that wor ked in our scenario. M any additi onal parameters can be set. See Section 4.6.9.3, “E di ting Ut ility – Bredit” on page 146 for further details.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 183 3. Add the route to the Switch net work of S P21 on all four nodes of SP2 wi th an FDDI inter face. On node 9- 12 in SP2, add the f ollowing route: route add -net 192.168. 14 -netmas k 255.
184 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 7. On the CWS of SP21, check i f t he SP Switch Router Adapter card is configured. T o perf orm this check, look if the SP Switch Router Adapter card shows up green in perspectives or enter SDRGetObj ects switch_re sponds .
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 185 vice versa. We sent these files to /dev/null to eli minate any hard disk infl uence on t he receiver side. The hardware requis ites for this test are the same as described in Section 5.1.4.
186 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Configu ration assu mptions: • The S P S witch Router FDDI media card has been installed accordi ng to Section 4.4, “FDDI Conf igurat ion” on page 121, and works pr operly . • The S P S witch Router Adapter card has been installed accordi ng to Section 3.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 187 5.2 Sin gle SP Par ti tio n and Mu ltip le SP Sw itch Rout er Adapt er Ca rds It is frequently necessary to maintain the data t ransfer even when an S P Switch Rout er Adapte r fails.
188 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 1. Each SP Switch Router Adapter card inter face has to be in a different subnet. 2. Netmasks have to be us ed to c reate diff erent subnets on the router side. 3. Logical subnett ing is only required on t he router .
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 189 If the SP Switch Rout er Adapter car ds work, the can both be us ed to route I P traff ic from the SP Switch. Add the following routes to the SP nodes of S P21: route add -net xxx.xxx .xxx.
190 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 5.2.2 Co mplex C onfigurat ion In this section, we des cribe the setup of a c omplete scenario wi th a dual SP Switch Router connection, and ex plain why node 8 in SP21 in this setup is able to route IP t raf f ic via IP address 192.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 191 T able 20 . Con fi guration of a D ual SP Switch R outer - SP Switch Con nectio n Co nfi g ura ti on: For this scenario we chose the following route settings on our three test nodes in SP21 and on node 1 in SP2: 1.
192 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 3. On node 6 in S P21, add the f ollowing r oute to the S witch net work of S P2: route add -net 192.168 .13 -netmask 255.255.255. 0 -mtu 65280 192.168.1 4.4 The -m tu parameter is again optional but should be s et to ensure optimal packet size on this route.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 193 8. Check for correct routing entry: 9. On GRF 1600, check /etc/grif config.conf for the following entries (or sim ilar): gt030 1 92.168.14.4 255.255.255 .128 - mtu 65520 gt050 1 92.168.14.
194 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch On node 1 in SP2, ping the SP S witch interfaces of the chosen nodes in SP21, for example: If these ping commands fai l, check r outing settings again.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 195 Figure 61 . IP T r affic Flow When Is suing p ing 192 .168.13.1 o n No de 6 Figure 63 on page 196 shows the I P traf fic flow when issuing the ping 192.168.1 3.1 on node 10 in SP21. All packets are first f orwarded to the SP Switc h Adapter card wit h IP address 192.
196 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Figure 63 shows the I P traf fic flow when iss uing ping 192.168 .13.1 on node 8 in SP21. All packets are first for warded to the SP Switch Adapter card with IP address 192.168.14.129 corres ponding to the routing settings.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 197 T o rec over from a failing SP Switch Adapter card, you have to define an alia s IP address for the surviving car d. Follow these steps when gt030 fails : 1. Login as root to the SP S witch Router .
198 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Configu ration assu mptions: • T he RS/ 6000 SP i s partit ioned in t o two or more parti tions (t wo part itions for thi s scenario). Note: Be c areful when choosing the par tition l ayout.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 199 T able 21 shows the I P addres ses used in our configura tion. T able 21 . Con fi guration of a Par tition - Partition C onn ection Co nfi g ura ti on: For this scenario, the followi ng routes have to be set: 1.
200 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch The -m tu parameter is again optional but should be s et to ensure optimal packet size on this route. 4. Check for correct routing entry: 5. On GRF 400, check /etc/grif config.conf for the following entry (or one similar ): gt020 19 2.
Si ng le RS / 60 00 SP and S in gl e SP Sw i t ch Ro ut er 201 If these ping commands fai l, check r outing settings again. If everythi ng is as it shoul d be tr y to ping the SP Switch Router Adapter c ards to find the fail ing part: ping 192. 168.13.
202 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch.
© C opyri gh t I BM C or p. 19 98 203 Cha pter 6. Mult iple R S/6000 S Ps an d On e SP Switch R outer In t his conf iguration, two RS /6000 SP s ystems are con nected to a single SP Switch Rout er . This enables bot h SPs to c ommunicate deploying the S P Switc h data transfer rat e and/or to share other network resources .
204 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch /etc/s nmpd.conf on the GRF 1600. T o check, open this file and look for a stanz a similar to the following : MANAGER 192.16 8.4.137 SEND A LL TRAPS TO POR T 162 WITH C OMMUNITY spe nmgmt This stanza is needed twi ce , one for e ach C WS.
Mult ip le R S/6 000 SP s and One SP Swi t ch Ro ut er 205 route add -net 192.168 .13 -netmask 255.255.255. 0 -mtu 65280 192.168.1 4.4 2. Check for correct routing entry: 3. On nodes in S P2, add t he following route to the switch network of S P21: route add -net 192.
206 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch cards shows up green in perspectives or enter SDRGetObjects switch_re sponds . Use Eunfence if needed.
Mult ip le R S/6 000 SP s and One SP Swi t ch Ro ut er 207 With this scenari o and without further tuning (ref er to A ppendix A, “Laborat ory Hardware and Software Conf iguration” on page 233 for actual no parameters) , we measured a stable cumulativ e transfer r ate of up to 83 MB/s (observ ed with the freeware tool monitor ).
208 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch interf erence between data transfer s from each S P t o different SP Sw itch Router Adapter media cards except t he o ne caused by l imited bandwidth of the dif ferent network ty pes (FDDI, A TM).
© C opyri gh t I BM C or p. 19 98 209 Cha pter 7. Mult iple R S/6000 S Ps an d Mu ltiple GRFs In t his section, s ample configurations with two SP systems c onnected wit h two SP Switch Routers are presented. The routers, in turn, are connected by any kind of high speed net work supported by the GRF .
210 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch fast network . (The spare second port might be in use for example for a connection to an A T M at tached device that needs a fast path to the SP Switch network ). T o make us e of bot h ports, four possible solutions come to mind: 1.
Mult i pl e RS /6 00 0 S P s an d Mu lt i pl e GRF s 21 1 • If ARP is disabled on the S P Switch net work , the IP addres ses ass i gned t o the nodes must be determined by the s witch node num bers. Note: The SP Switch Router Adapter c ard will not properly fo rward IP data to nodes as signed with an IP address t hat is in another s ubnet.
212 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 23 . Con fi guration of SP Sw itch - A T M - SP Switch T o successfully run this configuration, a route to the distant SP S witch network has to be set on every SP S witch Router .
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 213 1. The fi le /etc/grat m.conf needs the configurat ion statements f or the port used: Traffic_S hape name=hig h_speed_high_ quality p eak=155000 su s.
214 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 6. Check for correct routing entri es on all nodes in SP2: 7. I ssu e som e ping commands to check t he connection: On the SP21 nodes, ping the SP S witch interface of nodes in SP; on nodes in SP2, ping the SP Switch interface of nodes in SP21.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 215 Pe rf orm anc e : T o get a rough overview of the data transfer rat es that c an be achieved in this scenario, the following test was perform ed: 1.
216 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 24 shows the I P addres ses used in our configura tion. T able 24 . Con fi guration of SP Sw itch - A T M Bridged - SP Switch B.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 217 Router , so ther e was no techni cal reas on for i t . The :wq ! in t he sc reen shot is there t o remind you how to end editing of the file, and we give an n (no) as answer to the last question, as the A TM ports are already in use and theref ore cannot be m odified.
218 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Now it is time to look at the GRF 400: 1. The following screen shot gives you the minimum required data to be put into / etc/bri dged.conf: Remarks : On the GRF 400, bridging was never defined before, so we have t o creat e /etc/br idged.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 219 This two lines were changed from t he basic configuration! PVC ga020 0/132 proto= llc,bridging PVC ga028 0 0/134 proto =llc,bridging Because the GRF supports InA TMARP , there is no need t o have any entries in / e tc /g r a r p.
220 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Issue some ping commands to check the c onnection: On the SP21 nodes, ping the SP S witch interface of nodes in SP2, on nodes in SP2 pin g t he SP Swit ch inter face of nodes in SP21. If these ping commands fai l, check r outing settings again.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 221 So what happened t o the expected doubling of the aggr egate t hroughput? As it tur ns ou t, even wi th br idging ac tivated, only one A TM port is allowed to s end and rec eive data.
222 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch When port 0 is pl ugged in again, it res umes and takes ov er por t 1, whi ch in turn falls bac k to the blocked s tate, as the follow.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 223 Configu ration assu mptions: • The S P S witch Router A TM m edia card has been installed according to Section 4.3, “A TM OC-12c Configuration” on page 1 19 on both GRF routers and works properly .
224 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 25 shows the I P addres ses used in our configura tion. T able 25 . Con fi guration of SP Sw itch - A T M OC-12c - SP Switch T o successfully r un th is conf iguration, on ev ery SP S witch Rout er a r oute to the dist ant SP Switch network has to be set.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 225 1. The fi le /etc/grat m.conf needs the configurat ion statements f or the port used: Traffic_S hape name=big g_speed_high_ quality p eak=622000 su s.
226 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 5. Check for correct routing entri es on all nodes in SP2: 6. I ssu e som e ping commands to check t he connection: On the SP21 nodes, ping the SP S witch interface of nodes in SP2; on nodes in SP2 pin g t he SP Swit ch inter face of nodes in SP21.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 227 Pe rf orm anc e : T o get a rough overview of the data transfer rat es that c an be achieved in this scenario, the following test was perform ed: 1.
228 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch Co nfi g ura ti on: In this scenario, we have the SP Sw itch of SP21 connected to an GRF 1600. The GRF 1600 has its HIPPI media card’ s ports cross-c onnected to the GRF 400 HIPPI media card’ s ports.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 229 The media card adapters on the GR F routers should already be up and running (according to Section 3.7, “Step-by-St ep Media Card Configuration” on page 86 and S ec tion 4.5, “HIPPI Configurati on” on page 133).
230 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 5. The SP Sw itch Router Adapt er card is connected to the SP Switch and configured. Check with SDRGetObj ects switch_ responds on the CWS and use Eun fence if needed. The following tas ks are performed on the respective SP nodes: 1.
Mult ipl e RS/6 00 0 SPs and M ult iple GR Fs 231 5. I ssu e som e ping commands to check t he connection: On the SP21 nodes, ping the SP S witch interface of nodes in SP2; on nodes in SP2 pin g t he SP Swit ch inter face of nodes in SP21. If these ping commands fai l, check r outing settings again.
232 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch We saw up to about 48 MB/s with just one s ide sending data. With all nodes s ending and receivi ng, we achieved a duplex throughput of about 54 MB/s on the HIPP I port .
© C opyri gh t I BM C or p. 19 98 233 Appendix A. Laborat ory Hardware a nd Softw are Con figuration This appendix contains a detailed description of the hardware and software configurat ion used to test scenarios described in the s econd part of this redbook.
234 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 27 . Con fi guration of SP 21 Node No de T ype Host na me Adap ter I P addr es s Node 0 ( CWS) RS /600 0 5 70 s p21en 0 ent 0 tok0 19 2 .16 8. 4.1 37 9. 12. 1. 137 Node 1 hi gh n ode 1 12 M hz, 6 p ro c.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 235 T able 28 . Con fi guration of SP 2 A.1.1 Hard Di sks All nodes and CWSs include one or more internal SCSI disks. Some nodes are addi tionally equi pped with S SA disks ( T able 29 on page 2 36).
236 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch selected nodes. Refer to t he scenario des criptions in second par t of this redbook to see which disks were used. T able 29 . Har d Disk E quipm ent of SP 21 Nod e Disk s D es cr ipt ion Node (C WS) hd is k0 hdis k1 hdis k2 hdis k3 1.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 237 T able 30 . Hard D isk Equipm ent of SP 2 Part 1 of 2 Nod e Disk s D es cr ipt ion Node0 ( CWS) hdis k0 hdis k1 hdis k2 hdis k3 hdis k4 hdis k5 2.0 GB SCSI Disk Driv e 2.0 GB SCSI Disk Driv e 2.
238 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 31 . Hard D isk Equipm ent of SP 2 Part 2 of 2 Nod e Disk s D es cr ipt ion Node13 hd is k0 hdis k1 hdis k2 hdis k3 hdis k4 hdis k5 1.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 239 A.1.2 Software Co nfigurat ion Both CWSs and every SP node are install ed wi th AIX 4.3.1, inc luding all f ixes available on May 20th, 1998, and P SSP 2. 4 PTF Set 1 (See T able 32). T able 32 .
240 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 33 . So ftware Le vels o n CW S and All N odes Par t 2 of 14 File set Leve l D escriptio n X1 1. compat.l ib.X1 1R3 4.3.0.0 AIXwind ows X1 1R3 Librar ies Com patibility X1 1. compat.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 241 T able 34 . So ftware Le vels o n CW S and All N odes Par t 3 of 14 File set Leve l D escriptio n bos.adt.i nclude 4.3.1 .1 Base A pplic ation Dev elopment I nclude F iles bos.adt.l ib 4.
242 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 35 . So ftware Le vels o n CW S and All N odes Par t 4 of 14 File set Leve l D escriptio n bos.loc.i so.en_US 4.3.1.0 Base Sys tem Loca le ISO C ode Set - U .S. Englis h bos.mh 4.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 243 T able 36 . So ftware Le vels o n CW S and All N odes Par t 5 of 14 File set Leve l D escriptio n bos.r te.bind_cmds 4.3.1 .1 Binder and Loader Co mman ds bos.rte.boot 4.3.1.0 Boo t Comm ands bos.
244 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 37 . So ftware Le vels o n CW S and All N odes Par t 6 of 14 File set Leve l D escriptio n bos.rte.meth ods 4.3.1. 0 Device C onfig Me thods bos.r te.misc_cm ds 4.3.1. 0 Miscell aneous Co mmand s bos.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 245 T able 38 . So ftware Le vels o n CW S and All N odes Par t 7 of 14 File set Leve l D escriptio n bos.txt .spell 4.3.1.0 Writer ’ s T ools Commands bos.txt .spell.data 4.3.0.0 Writer ’s T ools Data bos.
246 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 39 . So ftware Le vels o n CW S and All N odes Par t 8 of 14 File set Leve l D escriptio n devices. common.r spcbase. rte 4.3.1.0 RISC PC C ommon B ase Syste m Devic e Softwar e devices.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 247 T able 40 . So ftware Le vels o n CW S and All N odes Par t 9 of 14 File set Leve l D escriptio n devices. mca.8f67.c om 4.3.1 .0 Comm on Turboways A T M Sof tware devices. mca.8f67.d iag 4.
248 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 41 . So ftware Le vels o n CW S and All N odes Par t 10 o f14 File set Leve l D escriptio n devices. msg.en_US .base.c om 4.3.0 .0 Base S ystem De vice Softwa re Msg s - U.S. Eng lish devices.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 249 T able 42 . So ftware Le vels o n CW S and All N odes Par t 1 1 of 14 File set Leve l D escriptio n devices. ssa.netwo rk_agent.r te 4.3.1 .0 SSA Netw ork Ag ent Suppor t devices. ssa.tm.
250 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 43 . So ftware Le vels o n CW S and All N odes Par t 12 o f 14 File set Leve l D escriptio n printers.hpl j-c.rte 4.3.0.0 Hewl ett-Packard L aserJet Color printers.ib m2380 .rt e 4.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 251 T able 44 . So ftware Le vels o n CW S and All N odes Par t 13 o f 14 File set Leve l D escriptio n printers.le xOptraE. rte 4.3.0.0 Lexmark Optra E Las er Prin ter printers.le xOptraEp .
252 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 45 . So ftware Le vels o n CW S and All N odes Par t 14 o f 14 File set Leve l D escriptio n sysmgt.s guide.rt e 4.3.1.0 T askGu ide Runtim e Environ ment sysmgt.web sm.a pps 4.3.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 253 A.1 .3 N e twor k O pti ons an d T u ni ng T able 46 shows t he network opti ons on t he CWS and all part icipating SP nodes. Options can be changed with t he /etc/no command. T able 46 .
254 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 47 . Netw ork O ptions of CW S and All No des Par t 2 of 3 Parameters V alue medt hresh 95 psecache 1 subnets arelocal 1 maxttl.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 255 T able 48 . Netw ork O ptions of CW S and All No des Par t 3 of 3 A.2 SP Switch Pool Size Settings T w o important switch parameters were changed from their default values: rpoolsi ze and s poolsize.
256 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch A.3 7025-F50 Conf iguration A 7025-F50 was used f or some A TM and Ethernet network tests. This machine is equipped with two166 MH z-604e processors, three 4500 MB 16-bit SCSI Disk Driv es, an IBM 155 M bps A TM PCI A dapter and an IBM 100/10 Mbps Ethernet PCI A dapter .
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 257 T able 50 . Netw ork O ptions of 7025-F50 Part 2 o f 3 Param eter V alue psebufc alls 20 strturnc nt 15 pseintrs tack 1228 8 lowt h res.
258 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch T able 51 . Netw ork O ptions of 7025-F50 Part 3 o f 3 A.4 SP IP S wit ch Router Configuration A GRF 400 and/or a GRF 1600 SP Switch Router were used for all the tests. Ascend Embedded/OS 1.
Labo r ato ry Har dwa re an d S oft war e Co nf igu ra tio n 259 The applied IP addresses vary with the scenari o and are specified in the corres ponding chapter . For specific SP Switch Router sample configurat ion fil es, refer to Appendix B, “GRF Configuration Files ” on page 261.
260 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch.
© C opyri gh t I BM C or p. 19 98 261 Appendix B. GRF Configuration Fi les This appendix contains relevant SP Switch Router configuration files . Some of them are here just for inform ation, some of them were worked out manually duri ng setup of hardware or software and some of them were created using Ascend-supplied tools.
262 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch # that we may cons ider enh anci ng it i n fu ture sof twar e # rele ases to supp ort your nee ds.
GRF C on f ig ur at i on File s 26 3 # if [ -t 0 ] then # # Ask for a t ermi nal type ; th e d efau lt i s th e ca noni cal "vt1 00". # i f [ X${T ERM} = X ] t hen T ERM= vt10 0 f i e val ‘tse t -s -m ?$TE RM‘ e xpor t TE RM # # It’ s in tera ctiv e, s o ex ec t he new CLI shel l fo r th e GR F.
264 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch B.3 /etc/bridg ed.conf This file holds the configurati on data for transparent bridging. It is created using the utilit y br edit . # # N etSt ar $ Id$ # # Configu rati on file for Brid ge D aemon (brid ged) .
GRF C on f ig ur at i on File s 26 5 # t o be a l ead or at l east low in the span ning tre e. # T he c lose r a LAN is to b eing a l eaf in t he t ree, the # l ess thro ugh tra ffic it will be aske d to car ry.
266 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch # t urne d on . # # S etti ng t he f orwa rd d ela y to o sm all woul d re sult in temp orar y # l oops as the span nign tr ee a lgor ithm con verg es. Sett ing this # v alue too lar ge r esul ts in l onge r pa rtit ions aft er t he # s pann ing tree rec onfi gur es.
GRF C on f ig ur at i on File s 26 7 # For ATM - A TM t wo p orts Tes t Ch ap.7 .1. 2 bridg e_gr oup bg1 { p ort ga01 0 ga 0180 ; # s pann ing_ tree dis able d; }; B.4 /e tc/ fs tab This file holds the filesystem mount information. This file was changed when adding a PCM CIA hard dis k in Section 3.
268 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch # f ollo wing for mats : # # 4 8-bi t MA C ad dres s fo r # E ther net or F DDI: xx :xx :xx: xx:x x:xx # wh ere ’xx ’ ar e he xade cima l di gits .
GRF C on f ig ur at i on File s 26 9 # The PVC sec tion is wher e Pe rman ent Vir tual Cir cuit s ar e de fine d, # using traffic shapes defi ned in the Traf fic Shaping sect ion, # a long wit h ot her para mete rs s pec ific to PVC conf igur atio n.
270 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch #Serv ice name =bc0 typ e=bc ast addr =198 .17 4.20 .1 a ddr= 198. 174. 22.1 # a ddr= 198.
GRF C on f ig ur at i on File s 27 1 # [ mode =SDH |SON ET] [clo ck=E xt|I nt] # # The ’card’ and ’con nect or’ spec ificati on are manda tory. # # The car d sh ould be iden tifi ed b y a hex idec imal dig it r epre sent ing # the slot number of the card in the GigaRou ter chas sis.
272 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch # the pr otoc ol # # v c_mu ltip lexe d - Use a sepa rate PVC for eac h pr otoc ol # # LLC enc apsu late d br idgi ng a llow s an y L AN f rame typ e to be tran smit ted, and # als o al lows IP data gram s to be sent di rect ly o n th e VC .
GRF C on f ig ur at i on File s 27 3 # (exc ept for RFC 148 3 br idgi ng) # ’ llc, brid ging ’ a ny L LC-e ncap sul ated pro toco l, i nclu ding RFC 148 3 # brid ging [Th is is t he P VC t ype for an i nter face usi ng # brid ge_m etho d=l lc_e ncap sula ted.
274 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch p eak= 6220 00 s usta in=6 2200 0 bu rst =204 8 qo s=hi gh Signa llin g ca rd=1 con nect or=t op p roto col =NON E Signa llin g ca rd=.
GRF C on f ig ur at i on File s 27 5 # DEF AULT S s ets all of t he a bove ke ywor ds t o th e ab ove defa ults . ##### #### #### #### ######## #### #### ####### #### #### #### ######## #### #### ######## ### # ##### #### #### #### ######## #### #### ####### #### #### #### ######## #### #### ######## ### # # log fil es.
276 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch # Log files that used to be archi ved by the /etc /{da ily|week ly|m onthly} # scripts .
GRF C on f ig ur at i on File s 27 7 DEFAU LTS hold= 2 local =y size= 1000 0 logfi le=/ var/ log/ grclean. log B.9 /etc/grde v1.conf This file normall y gets updated automatically by the SNM P daemon running on the CWS.
278 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 2.21. 2.1. 1.12 2 # No de S tate 2.21. 2.1. 1.13 0 # Sw itch Chi p Li nk 2.21. 2.1. 1.14 6 4 # No de D elay (cy cles ) 2.21. 2.1. 1.15 1 # Ad min Stat us # # CA RD 2 I nter face 0 # 2.21.
GRF C on f ig ur at i on File s 27 9 2.21. 6.1. 1.3 " 00:0 0:00 :01: 00: 00:0 0:07 :00: 03" # Sw itch Tok en 2.21. 6.1. 1.4 2 # Sw itch ARP 2.21. 6.1. 1.5 1 5 # Sw itch Nod e Nu mber 2.21. 6.1. 1.6 x 192. 168. 14.1 29 # IP Add ress 2.21. 6.1.
280 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 2.21. 9.1. 1.12 2 # No de S tate 2.21. 9.1. 1.13 0 # Sw itch Chi p Li nk 2.21. 9.1. 1.14 6 4 # No de D elay (cy cles ) 2.21. 9.1. 1.15 1 # Ad min Stat us # # CA RD 9 I nter face 0 # 2.21.
GRF C on f ig ur at i on File s 28 1 2.21. 13.1 .1.3 " 00:0 0:00 :00: 00: 00:0 0:00 :00: 00" # Sw itch Tok en 2.21. 13.1 .1.4 2 # Sw itch ARP 2.21. 13.1 .1.5 0 # Sw itch Nod e Nu mber 2.21. 13.1 .1.6 x 0.0. 0.0 # IP Add ress 2.21. 13.1 .1.7 x 0.
282 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch 2.21. 16.1 .1.1 2 2 # No de S tate 2.21. 16.1 .1.1 3 0 # Sw itch Chi p Li nk 2.21. 16.1 .1.1 4 6 4 # No de D elay (cy cles ) 2.21. 16.1 .1.1 5 1 # Ad min Stat us B.10 /e tc/g rifconfig.
GRF C on f ig ur at i on File s 28 3 # F DDI card . N OTE: Th e l ogic al i nter face num ber # m ay b e di ffer ent from th e ph ysic al i nter face on # t he c ard, dep endi ng o n t he s ingl e- o r du al- # a ttac hedn ess of t he v ari ous inte rfac es.
284 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch #gt02 0 0 .0.0 .0 2 55.2 55.2 55.0 - mtu 6552 0 gt030 1 92.1 68.1 4.4 2 55.2 55.2 55.1 28 - mtu 6552 0 #gt04 0 1 92.1 68.1 4.12 9 2 55.2 55.2 55.1 28 - mtu 6552 0 gt050 1 92.1 68.1 4.12 9 2 55.
GRF C on f ig ur at i on File s 28 5 # exit. # If only 3 v alue s ar e de sign ated , th e f irst val ue w ill be r epea ted as t he # fourth valu e. # # No whit espa ce i s al lowe d in any fie ld.
286 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch # A n etma sk i s re quir ed f or a ll e ntri es in t his conf igur atio n fi le. # # The net mask is norm ally the mas k of th e re mote net work : # # 1 92.0 .2.0 2 55.2 55.2 55.0 123.
GRF C on f ig ur at i on File s 28 7 #uucp d s trea m t cp n owai t r oot /usr /lib exec /tcp d uucp d #fing er s trea m t cp n owai t n obod y /usr /lib exec /tcp d fing erd #tftp d gram u dp w ait n.
288 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch #c a nd i s pr otec ted by U .S. and oth er c opyr ight law s an d th e #c l aws prot ecti ng t rade sec ret and con fide ntia l in form atio n.
GRF C on f ig ur at i on File s 28 9 # [ SNMP | S MUX] # [ OVER [UN IX | UD P | TCP] [SO CKET | T LI]] # [ AT < addr >] # # M ANAG ER < addr > [O N TR ANS PORT <na me>] # [ SEND [AL .
290 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch # UN SPEC IFIE D S UBAG ENTS # # < host Spec > :: = HO ST < host id> | # UN SPEC IFIE D HO ST[ S] # # < pass word Spec .
GRF C on f ig ur at i on File s 29 1 B.18 /e tc/sy slog.conf Use this file to specify the type of logging in the system. If no additional hard disk is installed, logging can be directed to another network attached system; otherwis e local f iles ar e used.
292 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch #net# cron .inf o @ser ver. doma in.c om #net# loca l0.i nfo @ser ver. doma in.c om #net# loca l1.i nfo @ser ver. doma in.c om #net# loca l2.* @ser ver. doma in.c om #net# loca l3.* @ser ver.
GRF C on f ig ur at i on File s 29 3 # Changes to the order of entries or number of ttys shou ld o nly be made in # single- user mode. # # nam e g etty type stat us comm ents # conso le " /usr /l.
294 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch ttyr9 n one netw ork ttyra n one netw ork ttyrb n one netw ork ttyrc n one netw ork ttyrd n one netw ork ttyre n one netw ork ttyrf n .
© C opyri gh t I BM C or p. 19 98 295 Appendix C. Hardware an d Softwa re Information Appendix C gives an ov erview of the LEDs on the front panel of the SP Switc h Adapter card and shows tables with the meaning of the LEDs’ bli nking patterns.
296 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch C.2 SP Switch Route r Adapter Media Card LE Ds LED acti vities during operati ons are listed in T able 52 on page 296 and T able 53 on page 296. LED activities during bootup are described in T able 54 on page 297.
Hardware and S oftware In form ation 297 C.3 SP Switch Route r Adapter Media Card - Bootup T able 54 shows the settings for the Switch Router Adapter Media Card LEDs during bootup.
298 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch C.4 Connectors and Re ceptacles for Differe nt Medi a T able 55 gives you a comprehensive overv i ew of al l the support ed cables and connect ors for the various media cards. T able 55 .
Hardware and S oftware In form ation 299 Figure 73. The SP Switch Boa rd C.6 Updating Route r Software This part pr ovides general information about obt aining and installing new operating sof tware (hereaft er refe rred to as machine code) fo r the SP Switch Router .
300 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch SP Switch Routers are delivered wi th the c urrent level of machine code already installed. Customers who wish to upgrade to new releases of the machine code should contact their IBM representat ive.
Hardware and S oftware In form ation 301 ftp> cd r eleases ftp> cd A 1_4_6 ftp> cd p atches ftp> cd A 1_4_6_4 4. Set the file format and download the fil es: ftp> bin ftp> get grf_update ftp> get RN1_4_6_4.pdf ftp> get RN1_4_6_4.
302 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch - Ftp acc ess to servic e2.boulder.i bm.com from t estbox.site.c om. - Approxi mately 30MB o f disk space in /flash. Please be aware: - testbox.s ite.com will be REBOO TED if any sy stem software and/ or patch file is i nstalled.
Hardware and S oftware In form ation 303 To tempor arily disable the directe d bcast setti ng later on, use: # sysctl -w net.inet.ip.fwdi rbcast =0 To verify that the bca st setting i s one of the sysctl execut ables, use: # sysctl net.inet.ip.f wdirbc ast IBM GRF u pgrade: testb ox.
304 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch.
© C opyri gh t I BM C or p. 19 98 305 Appen dix D . Speci al Notices This publicat ion is intended to help IBM c ustomers, Business Partners , IBM System Engineers and other RS/6000 SP specialist who.
306 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch inform ation or the implementation of any of these techniques is a custom er responsibility and depends on the customer’ s ability to evaluat e and integr ate them into the customer’ s operat ional environment.
Sp ec i al N ot ic es 307 Pentium, MMX, ProShare, LANDesk, and ActionMedia ar e trademarks or register ed trademarks of Intel Corporat ion in the U.S. and other countries . UNIX is a registered trademark in the United States and other countries licensed exclusively through X/Open Company Limited.
308 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch.
© C opyri gh t I BM C or p. 19 98 309 Appendix E. Relate d Publicat ions The publications lis ted in this section are consider ed particularly suitable for a more detailed discussion of the topics covered in this redbook.
310 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch • RS/6000 SP: Diagnosis and M essages Guide Vers ion 2 Release 4 , GC23-3899 • RS/6000 SP: Command and T echnical Ref erence V ers.
© Copyrigh t IBM Corp. 19 98 31 1 How to G et ITSO R edbooks Thi s s ect i on exp l ains ho w both cu stom er s an d IBM em ploy ee s can fin d ou t abo ut I TSO r ed book s, CD -ROM s, wo rk shop s, an d res ide nc ies. A f orm f or ord er ing boo ks an d CD- ROMs is a l so pr ovi ded .
312 IBM 9 077 SP Switch R outer: Get Conn ected to th e SP Sw itch How C ust ome rs Can G et ITS O Redbo oks Cus tom e rs m ay r e qu es t IT S O de l ive ra ble s ( re db oo ks , B o ok M an ag er BO.
313 IBM Re dbook Order Form Please se nd me the following: We accept American E xpress, D iners, Euroca rd, Master Card, and Visa . Payment by cr edit card not available in all countries.
314 IBM 9 077 SP Switch R outer: Get Conn ected to th e SP Sw itch.
315 List of Abbreviations ACL Acce s s C on tro l L i st AIX Adva nc ed Int er act ive Exec ut iv e AM G Adap ter Memb ershi p Gro up ANS Abs tra ct N o tat ion Synt a x AP A Al l P oi n t s Ad dr es .
316 IBM 90 77 SP S witch Ro uter: Get Conne cted to the SP Switch LRU Last R ec en tl y Us ed LSC Link Switch Chip LV M Logi ca l V ol u m e Ma nager MB Me gaby te s MIB Ma nagem ent Inform ation Ba s.
© C opyri gh t I BM C or p. 19 98 317 Index Symb ols /et c/bri dged.co nf 144 , 145 , 14 6 , 148 , 149 , 181 , 216 , 218 , 264 /etc / fsta b 77 , 26 7 /etc / gra rp.c on f 11 2 , 115 , 138 , 149 , 154 , 217 , 219 , 229 , 267 /etc / gra tm. con f 11 1 , 112 , 113 , 115 , 121 , 151 , 212 , 217 , 218 , 224 , 225 , 268 /et c/grcl ean.
318 IBM 9077 SP Sw itch R outer: Ge t Connected to th e SP Switch confi g_n etstat 70 csconf i g 79 dev1con f ig 87 , 95 Eannot ato r 82 Efence 51 , 99 Efence - autoj oin 99 Epri mary 51 Estart 51 , 9.
319 F faul t servi ce dae mon 4 FDDI 6 , 16 , 18 , 39 , 66 , 69 , 121 , 122 , 12 5 , 129 , 142 , 157 , 163 , 174 , 209 backbon e 126 , 15 5 , 174 , 177 , 179 , 181 concent r ator 126 dual at tached st.
320 IBM 9077 SP Sw itch R outer: Ge t Connected to th e SP Switch 33 , 34 , 36 slot 66 31 IP tra f fic 7 , 8 , 11 , 12 , 15 , 194 IPv 4 142 , 143 IS- IS 15 , 23 J jac k 82 Jack N um ber 84 K keep- count 107 L Layer 2 14 Layer 3 14 , 19 LLC 14 2 , 144 , 150 , 151 load pr ofil e 107 , 108 log file s gr.
321 disk 75 , 76 , 79 inte rfa c e 79 mode m card 34 sl ot 34 perf orman ce li m it at i on 7 Permanent vi rt ual cir cui ts 112 persp ect ives 52 , 56 physi cal int erfac e 112 , 127 physi cal int er.
322 IBM 9077 SP Sw itch R outer: Ge t Connected to th e SP Switch 16-por t 3 8-por t 3 adapt er card 28 cabl e 81 networ k 12 port 41 , 63 , 75 , 81 rout er 5 , 25 , 66 rout er adapter 5 , 12 , 36 , 3.
© C opyri gh t I BM C or p. 19 98 323 IT SO Redb oo k E valua t ion IBM 9077 S P S witch R outer : G et C on necte d to the S P Switc h SG2 4 -51 57 -0 0 Y ou r feed bac k is ve ry impo rta nt to he l p us m ain ta in t he qua l it y of I TSO re dbook s.
Prin ted in the U .S.A . SG2 4-5 15 7-00 IBM 9077 SP Swi tch Router: Get Connected to the SP Switch SG24-5157 -00.
An important point after buying a device IBM 9077 (or even before the purchase) is to read its user manual. We should do this for several simple reasons:
If you have not bought IBM 9077 yet, this is a good time to familiarize yourself with the basic data on the product. First of all view first pages of the manual, you can find above. You should find there the most important technical data IBM 9077 - thus you can check whether the hardware meets your expectations. When delving into next pages of the user manual, IBM 9077 you will learn all the available features of the product, as well as information on its operation. The information that you get IBM 9077 will certainly help you make a decision on the purchase.
If you already are a holder of IBM 9077, but have not read the manual yet, you should do it for the reasons described above. You will learn then if you properly used the available features, and whether you have not made any mistakes, which can shorten the lifetime IBM 9077.
However, one of the most important roles played by the user manual is to help in solving problems with IBM 9077. Almost always you will find there Troubleshooting, which are the most frequently occurring failures and malfunctions of the device IBM 9077 along with tips on how to solve them. Even if you fail to solve the problem, the manual will show you a further procedure – contact to the customer service center or the nearest service center
