Instruction/ maintenance manual of the product OFED Q-Logic
Go to page of 124
Draft IB0056101-00 G .02 Preliminary QLogic HCA and QLogic OFED Sof tware Inst all Guide QLogic OFED V ersion 1.4.
Draft Page ii Preliminary IB0056101-00 G .02 S QLogic HCA and QLogic OFED Software Inst all Guide QLogic OFED V ersion 1.4 Information fu rnished in this manual is belie ved to be accurate and reliable.
Draft QLogic HCA and QLogic OFED Sof tware Inst all Guide QLogic OFED V ersion 1.4 IB0056101-00 G .02 Preliminary Page iii S Replace EM64T with 64-bit Intel Xeon. “Supported Linux Distributions” on p age 5-1 Added more inform ation on VNIC inter face.
Draft Page iv Preliminary IB0056101-00 G .02 S QLogic HCA and QLogic OFED Software Inst all Guide QLogic OFED V ersion 1.4 Sections rearranged, renam ed, and expanded due to multiple install meth ods.
Draft QLogic HCA and QLogic OFED Sof tware Inst all Guide QLogic OFED V ersion 1.4 IB0056101-00 G .02 Preliminary Page v S New Appendix for Write Combining. MTRR set- tings information moved her e. “Write Co mbining” on pag e B-1 Minor wording change to introductio n section.
Draft Page vi Preliminary IB0056101-00 G .02 S QLogic HCA and QLogic OFED Software Inst all Guide QLogic OFED V ersion 1.4 Changed title of T roubleshooting issue.
Draft QLogic HCA and QLogic OFED Sof tware Inst all Guide QLogic OFED V ersion 1.4 IB0056101-00 G .02 Preliminary Page vii S Adapter Setting s section name ch anged. Also added new bullet point “Check the PCIe bus wid th” “Adapter and Other Settin gs” on page 7-1 Updated tables of RPMs.
Draft Page viii Preliminary IB0056101-00 G .02 S QLogic HCA and QLogic OFED Software Inst all Guide QLogic OFED V ersion 1.4 Notes.
Draft IB0056101-00 G .02 Preliminary ix T able of Content s 1 Introduction Who Should Read this Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 How this Guide is Organized . . . . . . . . . . . . . . . . . . . . . . .
Draft x Preliminary IB0056101-00 G .02 QLogic HCA and QLogic OFED Software Inst all Guide QLogic OFED V ersion 1.4 S Hardware Installation for QLE724 0, QLE7280, or QLE7 140 with PCI Express Riser . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Draft IB0056101-00 G .02 Preliminary xi QLogic HCA and QLogic OFED Sof tware Inst all Guide QLogic OFED V ersion 1.4 A Uninstalling So ftware with Rocks o r Platform OCS . . . . . . . . . . . . . . . 5-25 Downgrading RPMs. . . . . . . . . . . . . . . .
Draft xii Preliminary IB0056101-00 G .02 QLogic HCA and QLogic OFED Software Inst all Guide QLogic OFED V ersion 1.4 S ibsrpdm Command Hangs When T wo HCAs are Installed but Only Unit 1 is Connected to the Switch . . . . . . . . . . . . . . . . . . . .
Draft IB0056101-00 G .02 Preliminary xiii QLogic HCA and QLogic OFED Sof tware Inst all Guide QLogic OFED V ersion 1.4 A List of Figures Figure Page 4-1 QLogic QLE7280 with IBA7220 ASIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Draft xiv Preliminary IB0056101-00 G .02 QLogic HCA and QLogic OFED Software Inst all Guide QLogic OFED V ersion 1.4 S Notes.
Draft IB0056101-00 G .02 Preliminary 1-1 1 Introduction This chapter describes the content s, intended au dience, and organization of the QLogic HCA and QLogic OFED Software In stall Guide .
Draft Introduction Overview 1-2 Preliminary IB0056101-00 G .02 S Section 3, S tep-by-S tep Inst allation Checklist , provides a high-level overview of the hardware and sof tware installation procedures.
Draft Introduction Interoperability IB0056101-00 G .02 Preliminary 1-3 A QLogic OFED OpenFabrics software is interoperable with other vendors’ InfiniBand Host Channel Adapters (HCAs) runn ing compatible Ope nFabrics releases.
Draft Introduction Documenta tion 1-4 Preliminary IB0056101-00 G .02 S Document ation The product document ation includes: The QLogic HCA and QLogic OF ED Soft ware Install Guid e The QLogic H.
Draft Introduction Contact Information IB0056101-00 G .02 Preliminary 1-5 A Cont act Information Support Headquart ers QLogic Corporation 4601 Dean Lakes Blvd Shakopee, MN 55379 USA QLogic Web Site www .qlogic.com T echnical Support Web Site support.qlogic.
Draft Introduction Contact Information 1-6 Preliminary IB0056101-00 G .02 S Notes.
Draft IB0056101-00 G .02 Preliminary 2-1 2 Feature Overview This section contain s the features for this release, t he supported QLogic adapter models, supported distributions and kern els, and a list of the sof tware component s. Features The QLogic OFED 1.
Draft Feature Overview Features 2-2 Preliminary IB0056101-00 G .02 S MV APICH and OpenMPI compiled with new er versions of each of four different co mpilers (GCC, PGI, Intel and PathScale) are available. The QLogic InfiniBand Fabric Suite (IFS) is available sep arately for purchase.
Draft Feature Overview Features IB0056101-00 G .02 Preliminary 2-3 A This version of the QLogic OFED software provides support for all of the QLogic HCAs in Ta b l e 2 - 1 . Additional up-to-date information c an be f ound on the QLogic web site, specifically: The high performance computing p age at www .
Draft Feature Overview Software Comp onents 2-4 Preliminary IB0056101-00 G .02 S Sof tware Component s This release includes all o f OFED 1.4 with enhancement s (QLogic OFED 1.4), including a new version of the VNIC tools and driver , and support for the QHT7xxx and QLE7xxx adapters.
Draft IB0056101-00 G .02 Preliminary 3-1 3 S tep-by-S tep Inst allation Checklist This section provides an overview of the hardware and softwar e installa tion procedures. Det ailed steps are foun d in Section 4 “ Hardware Inst allation ” and Section 5 “ Software Inst allation ”.
Draft S tep-by-S tep Installation Checklist Software Inst allation 3-2 Preliminary IB0056101-00 G .02 S Sof tware Inst allation The following step s summarize the basic QLogic OFED 1.4 software inst allation and startup . These steps must be perfor med on each node in the cluster: 1.
Draft S tep-by-S tep Installation Checklist Software Inst allation IB0056101-00 G .02 Preliminary 3-3 A 1 1. Y ou can optimize your system and adapter for the best performance. See “Adapter and Other Settings” o n page 7-1 . Also see the Performance Settings and Management T ips section in the QLogic HCA and QLogic OFED Software Users Guide.
Draft S tep-by-S tep Installation Checklist Software Inst allation 3-4 Preliminary IB0056101-00 G .02 S Notes.
Draft IB0056101-00 G .02 Preliminary 4-1 4 Hardware Inst allation This section lists the req uirements and pro vides instructions for inst alling the QLogic InfiniPath Interconnect adapters.
Draft Hardware Installation Hardware Installa tion Requ irement s 4-2 Preliminary IB0056101-00 G .02 S Installation of the QLE7240, QLE7280, QLE7140, QHT7040, or QHT7 140 in a 1U or 2U chassis requires the use of a riser card. See Figure 4-4 for an illustration of a PCI Express (PCIe) slot in a typical motherboard.
Draft Hardware In stallation Hardware Installa tion Requirements IB0056101-00 G .02 Preliminary 4-3 A Cabling and Switches The cable inst allation uses a st andard InfiniBand (IB) 4X cab le. Any InfiniBand cable that has been qualified by the vendor should work.
Draft Hardware Installation Hardware Installa tion Requ irement s 4-4 Preliminary IB0056101-00 G .02 S Optical Fibre Option The QLogic adapter also support s connection to the switch b y means of optical fibres through optical media converters such as the EMCORE™ QT2400.
Draft Hardware In stallation Safety with Electricity IB0056101-00 G .02 Preliminary 4-5 A Safety with Electricity Observe these guidelines and sa fety prec autions when working around computer hardware and electrical equipment: Locate the power source shutof f for the computer room or lab where you are working.
Draft Hardware Installation Unpacking Information 4-6 Preliminary IB0056101-00 G .02 S The package conten ts for the QLE72 80 adapter are: QLogic QLE7280 Additional short bracket Quick S tart Guide S tandard PCIe risers can be used, typically supplied by your system or motherboard vendor .
Draft Hardware In stallation Unpacking Information IB0056101-00 G .02 Preliminary 4-7 A Figure 4-1. QLogic QLE7280 with IBA7220 ASIC Figure 4-2. QLogic QLE7140 Ca rd with Riser , T op V iew InfiniBand.
Draft Hardware Installation Unpacking Information 4-8 Preliminary IB0056101-00 G .02 S Figure 4-3. QLogic QHT7 040/QHT7140 Full and Low Profile Ca rds with Riser , Top V iew Unp a cking the QLogic Adapter Follow these steps whe n unpacking the QLogic adapter: 1.
Draft Hardware In stallation Hardware In stallation IB0056101-00 G .02 Preliminary 4-9 A Hardware Inst allation This section cont ains hardware inst allation instructions for the QLE72 40, QLE7280, QLE7140, QHT7040, and QHT7140.
Draft Hardware Installation Hardware Installation 4-10 Preliminary IB0056101-00 G .02 S 4. Remove the cover screws and cover plate to e xpose the system’s motherboard. For specific instructions on how to do this, follow the hardware documentation that came with your system.
Draft Hardware In stallation Hardware In stallation IB0056101-00 G .02 Preliminary 4-1 1 A 9. Connect the QLogic adapter and PCIe riser card together , forming the assembly that you will insert into your motherboard. First, visually line up th e adapter slot connector edge with the edge conne ctor of the PCIe riser card (see Figure 4-5 ).
Draft Hardware Installation Hardware Installation 4-12 Preliminary IB0056101-00 G .02 S 13. Insert the riser assembly into the motherboard’ s PCIe slot, ensuring good contact. The QLo gic adapter should now be parallel to th e motherboard and about one inch above it (see Figu re 4-6 ).
Draft Hardware In stallation Hardware In stallation IB0056101-00 G .02 Preliminary 4-13 A T o install the QLogic adapter with an HTX riser card: 1. The BIOS should be already be conf igured properly by the moth erboard manufacturer .
Draft Hardware Installation Hardware Installation 4-14 Preliminary IB0056101-00 G .02 S 7. Remove the QLogic QHT7140 from the anti-st atic bag. 8. Locate the face plate on the connector edge of the card. 9. Connect the QLogic adap ter and HTX rise r card together , forming the assembly that you will insert into your motherboard.
Draft Hardware In stallation Hardware In stallation IB0056101-00 G .02 Preliminary 4-15 A 13. Insert the HT riser assembly into the motherboard’ s HTX slot, ensuring good contact. The QLo gic adapter should now be parallel to th e motherboard and about one inch above it, as shown in Figure 4 -9 .
Draft Hardware Installation Hardware Installation 4-16 Preliminary IB0056101-00 G .02 S Hardware Inst allation for the QHT7140 Without an HTX Riser Installing the QL ogic QHT7140 without an HTX riser card requires a 3U or larger chassis. The card slot connectors on the QHT7140 fit in to the HTX slot in a vertical installation.
Draft Hardware In stallation Switch Configuration and Moni toring IB0056101-00 G .02 Preliminary 4-17 A 8. Insert the card by pressing firmly a nd evenly on the top of the horizont al bracket and the top rear corner of the card simult aneously . The card should insert evenly into the slot.
Draft Hardware Installation Completing the Installation 4-18 Preliminary IB0056101-00 G .02 S The QLE7240, QLE7280, QLE7 140, QHT7040, and QHT7140 adapters are all cabled the same way . T o install the InfiniBand cables: 1. Check that you have removed the protect or plugs from the cable connector ends.
Draft IB0056101-00 G .02 Preliminary 5-1 5 Sof tware Inst allation This section provides instructions for in stalling QLogic OFED 1.4, which includes QLogic InfiniPath and the OpenFabrics so ftware.
Draft Software Inst allation Cluster Setup 5-2 Preliminary IB0056101-00 G .02 S Distribution Identifiers Distribution identifiers for this relea se are listed in the ta ble below .
Draft Software Inst allation Compiler Support IB0056101-00 G .02 Preliminary 5-3 A Compiler Support QLogic MPI support s use of a number of compilers. These include: GNU gcc 3.3. x , 3.4. x , 4.0, 4.1, 4.2.x, and 4.3.x comp iler suites PathScale Compiler Suite 3.
Draft Software Inst allation Choose th e Appropri ate Download Files 5-4 Preliminary IB0056101-00 G .02 S Some operating system packages are required for OpenFabrics; they are listed in T able 5-3 . There are also OS p ackage requirement s for some specific comp onents in QLogic OFED.
Draft Software Inst allation Choose the Appropriate Download Files IB0056101-00 G .02 Preliminary 5-5 A The components of th e QLogic OFED 1.4 release are available pa ckaged as separate d ownloads: QLogicIB-Basic with the T ext User Interface Installer QLogic OFED 1.
Draft Software Inst allation Choose th e Appropri ate Download Files 5-6 Preliminary IB0056101-00 G .02 S QLogic OFED 1.4 RPM Set Includes: QLogic OFED 1.
Draft Software Inst allation Install QLogicIB-Basic with the Installer T ool IB0056101-00 G .02 Preliminary 5-7 A Inst all QLogicIB-Basic with the Inst aller T ool The Installer to ol has a T ext User Interfac e (TUI) that af fords easy inst allation of the software.
Draft Software Inst allation Install QLogicIB-Basic with the Installer T ool 5-8 Preliminary IB0056101-00 G .02 S 2. After unp acking the .tgz file, change directory to: cd <QLogicIB-Basic>.<version> 3. Become root, then type: # ./INSTALL (Note that if you need 32-bit supp ort on 64 bit OSs, invoke the inst aller with .
Draft Software Inst allation Install QLogicIB-Basic with the Installer T ool IB0056101-00 G .02 Preliminary 5-9 A 4. T ype 1 , which will display the scree n for software inst allation. The next screen shows the packag es to select for inst allation: QLogic Inc.
Draft Software Inst allation Install QLogicIB-Basic with the Installer T ool 5-10 Preliminary IB0056101-00 G .02 S The next screen contain s these choices: QLogic Inc. IB Install (4.4.1.0.8 release) Menu Please Select Install Action (screen 2 of 3): 0) MVAPICH/PSM (gcc) [ Install ][Available] 1.
Draft Software Inst allation Install QLogicIB-Basic with the Installer T ool IB0056101-00 G .02 Preliminary 5-1 1 A Open SM ( 1 ) should only be inst alled on one node in the cluster where it will be used. If desired, T ype 1 for Ope n SM. 7. Finally , type p to start the installation.
Draft Software Inst allation Install QLogicIB-Basic with the Installer T ool 5-12 Preliminary IB0056101-00 G .02 S ./INSTALL -C list s all the available comp onents, which include: ib_stack, truescale.
Draft Software Inst allation About rpm Inst allation IB0056101-00 G .02 Preliminary 5-13 A About rpm Inst allation Linux distributions of QLogic OFED (Inf iniPath and OpenFa brics) software can be installed from binary RPMs. RPM is a Li nux p ackaging and installa tion tool used by Red Hat, SUSE, and CentOS.
Draft Software Inst allation Using rpm to Inst all Infini Path and OpenFabr ics 5-14 Preliminary IB0056101-00 G .02 S RPMs contain config files. Y our current config files will not be o verwritten when new RPMs are inst alled. New config files will cont ain the suf fix .
Draft Software Inst allation Using rpm to Install InfiniPath and OpenFabrics IB0056101-00 G .02 Preliminary 5-15 A Become root, then: # cd QLogicOFED<version>-<distribution-x86_64 (Another way is to put the RPMs in a directory that is accessible (e.
Draft Software Inst allation Using rpm to Inst all Infini Path and OpenFabr ics 5-16 Preliminary IB0056101-00 G .02 S Note that you need to inst all the InfiniPath-MPI/32bit/mpi-f rontend-*.rpm even if you do not plan to use any other 32-bit RPMs. Howeve r , you can add the other 32bit subdirectories to the rpm command, if you need 32-bit support.
Draft Software Inst allation Install QLogic OFED User-l evel Software with the rpm Command IB0056101-00 G .02 Preliminary 5-17 A OpenFabrics/32bit/ OpenFabrics-Static/ OpenFabrics-Static/32bit/ OpenFa.
Draft Software Inst allation Install QLogic OFED User-l evel Software with the rpm Command 5-18 Preliminary IB0056101-00 G .02 S The tar command creates a directory based on the tar file name and places the RPMs and other files in this directo ry . 2.
Draft Software Inst allation Rebuilding or Reinstalling the kernel-ib Driver with rpm Af ter a Ke rnel Upgrade IB0056101-00 G .02 Preliminary 5-19 A The complete RPM directories for this down load are organized as follows: InfiniPath_license.txt,LEGAL.
Draft Software Inst allation Install QLogic OFED Using Rocks 5-20 Preliminary IB0056101-00 G .02 S Inst all QLogic OFED Using Rocks Rocks is a distribution designed f or managing clusters from the San Dieg o Supercomputer center (SDSC). Rocks is a way to manage the kickst art automated inst allation method created by Red Hat.
Draft Software Inst allation Install QLogic OFED Using Ro cks IB0056101-00 G .02 Preliminary 5-21 A Boot, Core, OS Ro ll DVD Note that you may also need updates; look fo r the latest files with the service-pack prefix. Make sure you downloaded the .iso images correctly; verify by checking the md 5 checksum from the web site.
Draft Software Inst allation Install QLogic OFED Usin g a Platform OCS Kit 5-22 Preliminary IB0056101-00 G .02 S If you have a burned a CD version of InfiniPath roll from the .iso image: # mount /mnt/cdrom # rocks-dist --install copyroll # umount /mnt/cdrom # cd /home/install # rocks-dist dist Y ou can use this method if you download the .
Draft Software Inst allation Install Additional Sof tware IB0056101-00 G .02 Preliminary 5-23 A See also the current version of the F ast Fabric Users Guide, available with the software or from the QLogic download p age. Inst all Additional Sof tware This section cont ains information about a dditional third-party sof tware inst allation.
Draft Software Inst allation Removing Software Packages 5-24 Preliminary IB0056101-00 G .02 S Documentation is fo und in: /usr/share/man /usr/share/doc/infinipath /usr/share/doc/mpich-infinipath Note that license information only is found in usr/share/doc/infinipath.
Draft Software Inst allation Removing Sof twa re Packages IB0056101-00 G .02 Preliminary 5-25 A For both InfiniPath and OpenFabrics, QLogic recommends that you remove all the package s at the same time. 1. Use the script ofed_uninstal l.sh to uninstall the OFED software.
Draft Software Inst allation Removing Software Packages 5-26 Preliminary IB0056101-00 G .02 S Notes.
Draft IB0056101-00 G .02 Preliminary 6-1 6 Configuring Drivers and Services This section provides instructions for configuring and using the drivers and services available with QLogic OFED 1.
Draft Configuring Drivers and Services OpenFabrics Drivers and Servic es Configuration and S tartup 6-2 Preliminary IB0056101-00 G .02 S Configuring the IPoIB Network Interface The following instructions show you how to manually configur e your OpenFabrics IPoIB network interface.
Draft Configuring Drivers and Services OpenFabrics Drivers an d Services Configuration an d S tartup IB0056101-00 G .02 Preliminary 6-3 A The output of the ping command will be similar to the following, with a line for each host already configured a nd connected: WARNING: pinging broadcast address PING 10.
Draft Configuring Drivers and Services OpenFabrics Drivers and Servic es Configuration and S tartup 6-4 Preliminary IB0056101-00 G .02 S Y ou can stop opensmd again like this: # /etc/init.d/opensmd stop If you want to pass any argu ments to t he OpenSM program, modify the following file, and add the argument s to the OPTIONS variable: /etc/init.
Draft Configuring Drivers and Services OpenFabrics Drivers an d Services Configuration an d S tartup IB0056101-00 G .02 Preliminary 6-5 A In the output, look for lines similar to the se: GUID: 0002c90200402c04 ID: LSI Storage Systems SRP Driver 200400a0b8114527 service entries: 1 service[ 0]: 200400a0b8114527 / SRP.
Draft Configuring Drivers and Services OpenFabrics Drivers and Servic es Configuration and S tartup 6-6 Preliminary IB0056101-00 G .02 S 5. Y ou can look for the newly created devices in the /proc/parti tions file.
Draft Configuring Drivers and Services OpenFabrics Drivers an d Services Configuration an d S tartup IB0056101-00 G .02 Preliminary 6-7 A their IOCGUID, IOCSTRING , etc.
Draft Configuring Drivers and Services OpenFabrics Drivers and Servic es Configuration and S tartup 6-8 Preliminary IB0056101-00 G .02 S HCA No = 0, HCA = mlx4_0, Po rt = 2, Port GUID = 0x0002c9030000.
Draft Configuring Drivers and Services OpenFabrics Drivers an d Services Configuration an d S tartup IB0056101-00 G .02 Preliminary 6-9 A 2. Create the VNIC interfaces using the configuration file: /etc/infiniband/qlgc_vnic.cfg. Look at the qlgc_vnic.
Draft Configuring Drivers and Services OpenFabrics Drivers and Servic es Configuration and S tartup 6-10 Preliminary IB0056101-00 G .02 S b. Format 2: Defining an IOC using the IOCST RING . Defining the IOC using the IOCSTRING allows VIO hardwa re to be hot-swapped in and out of a specific slot.
Draft Configuring Drivers and Services OpenFabrics Drivers an d Services Configuration an d S tartup IB0056101-00 G .02 Preliminary 6-1 1 A c. Format 3: S tarting VNIC using DGID.
Draft Configuring Drivers and Services OpenFabrics Drivers and Servic es Configuration and S tartup 6-12 Preliminary IB0056101-00 G .02 S 5. S tart the QLogic VNIC driver and the QLo gic VNIC interfaces.
Draft Configuring Drivers and Services OpenFabrics Drivers an d Services Configuration an d S tartup IB0056101-00 G .02 Preliminary 6-13 A If you want to restart the QLogic VNIC interfaces, run the following command: # /etc/init.
Draft Configuring Drivers and Services Other Configuration: Changing th e MTU Size 6-14 Preliminary IB0056101-00 G .02 S 2. Check that you have a /etc/dat.c onf file. It should be inst alled by the dapl- RPM. The file dat.conf cont ains a list of inte rface adapters supported by uDAPL service providers.
Draft Configuring Drivers and Services Managing the InfiniPath Driver IB0056101-00 G .02 Preliminary 6-15 A Managing the InfiniPath Driver The st artup script for ib_ipa th is inst alled automatically a s p art of the sof tware inst allation, and normally does not need to be changed.
Draft Configuring Drivers and Services Managing the Infini Path Driver 6-16 Preliminary IB0056101-00 G .02 S The device files are : /dev/ipath /dev/ipath0, /dev/ipath1, ... The numbered device files allow access to a specific InfiniPath unit. See the ib_ipath man p age for more det ails.
Draft Configuring Drivers and Services Further Inform ation on Configuring and Load ing Drivers IB0056101-00 G .02 Preliminary 6-17 A The sequence of commands to rest art the driver are as follows. # /etc/init.d/opensmd stop # /etc/init.d/ openibd stop .
Draft Configuring Drivers and Services Further Information on Configurin g and Loading Drivers 6-18 Preliminary IB0056101-00 G .02 S Notes.
Draft IB0056101-00 G .02 Preliminary 7-1 7 Inst allation V erification and Additional Settings This section provides instructions for verifying that the software has been properly installed and that the drivers are load ed, and the fabric is active and ready to use.
Draft Installati on V erification and Additional Settings Customer Acceptance Utility 7-2 Preliminary IB0056101-00 G .02 S Make sure that write combining is en abled. The x86 Page Attribute T able (P A T) mechanism that allocates write-combining (WC) mapping s for the PIO buffers has been added and is now the default.
Draft Installation V erification and Ad ditional Settings Customer Acceptance Utility IB0056101-00 G .02 Preliminary 7-3 A 3. Gathers and analyzes system c onfiguration from the nodes. 4. Gathers and analyzes RPMs insta lled on the nodes. Missing RPMs can be found this way .
Draft Installati on V erification and Additional Settings Customer Acceptance Utility 7-4 Preliminary IB0056101-00 G .02 S In most cases of failure, the script suggest s recommended a ctions. Please see the ipath_checkout man pag e for more information and updates.
Draft IB0056101-00 G .02 Preliminary A-1 A Inst allation T roubleshooting The following sections cont ain informat ion about issues that may occur during installation. Some of this material is repeated in the T roubleshooting appendix of the QLogic HCA and QLogic OF ED Sof tware User Guide .
Draft Installati on T roubleshooting BIOS Settings A-2 Preliminary IB0056101-00 G .02 S BIOS Settings This section covers issues related to BIOS settings. Y ou can check and adjust BIOS settings using the BIOS Setup utility . For specific instructions, follow the hardware documentation that came with your system.
Draft Installation T roubleshooting Software Inst allation Issues IB0056101-00 G .02 Preliminary A-3 A If these RPMs have not been installed, you will see error messages like this when installing InfiniPath: Building and installing InfiniPath modules for 2.
Draft Installati on T roubleshooting Software Inst a llation Issues A-4 Preliminary IB0056101-00 G .02 S Resolving Conflict s Occasionally , conflicts may arise when trying to inst all "on top of" an existing set of files that may come from a differen t set of RPMs.
Draft Installation T roubleshooting Configuration Issues IB0056101-00 G .02 Preliminary A-5 A In older distributions, such as RHEL4, the 32-bit glibc is cont ained in the libgcc RPM. The RPM name will b e similar to this: libgcc-<version>.EL4.i386.
Draft Installati on T roubleshooting Configuration Issues A-6 Preliminary IB0056101-00 G .02 S When ifconfig or ifup are run, the error will look similar to this (assuming ipath_ether was used for eth.
Draft IB0056101-00 G .02 Preliminary B-1 B W rite Combining Introduction Write comb ining improves write bandwidth to the QLogic chip by writing mu ltiple words in a single bus transaction (t ypically 64 bytes). W rite combining applies only to x86_64 systems.
Draft Write Combining P A T and Write Combining B-2 Preliminary IB0056101-00 G .02 S . . If you do not see any of these messages on your console, but suspect this problem, check the /var/log/messages file. Some systems suppress driver load messages but still output them to the log file.
Draft Write Combining MTRR Mapping and Write Combining IB0056101-00 G .02 Preliminary B-3 A For systems with very large amount s of memory (32GB or mor e), it may also be necessary to adjust the BIOS setting for the PCI ho le granularity to 2GB.
Draft Write Combining MTRR Mapping and Write Combining B-4 Preliminary IB0056101-00 G .02 S Notes.
Draft IB0056101-00 G .02 Preliminary C-1 C Configuration Files T able C-1 contains descriptions of the conf iguration and configuratio n template files used by the InfiniPath and OpenFabrics sof tware. T able C-1. Co nfiguration Files Configuration File Na me Description /etc/infiniband/qlgc_vnic.
Draft Configuration Files C-2 Preliminary IB0056101-00 G .02 S /etc/sysconfig/network/ifcfg- <NAME> Network configuration file for network inter- faces When used for VNIC configuration, <NAME> is in the form eioc X, where X is the device number .
Draft IB0056101-00 G .02 Preliminary D-1 D Package Descriptions The following sections cont ain detailed descriptions of the p ackages for the InfiniPath and OpenFabrics sof tware.
Draft Package Descriptions InfiniPath RPM V ersion Numbers and Identi fiers D-2 Preliminary IB0056101-00 G .02 S Although QLogic recommen ds installing all RPMs on all nodes, not all InfiniPath software is required on all nodes.
Draft Package Descr iptions Documentation RPMs IB0056101-00 G .02 Preliminary D-3 A Document ation RPMs InfiniPath RPMs T able D-1. Doc umentation/RPMs RPM Name Front End Compute Development infinipath-doc- xxx _yyy .noarch.rpm InfiniPath man p ages Optional Optional Optional mpi-doc- xxx_yyy .
Draft Package Descriptions OpenFabrics RPMs D-4 Preliminary IB0056101-00 G .02 S OpenFabrics RPMs For ease of inst allation, QLogic reco mm ends that all of the OpenFabrics RPMs listed in T able D-5 be installed on all nodes. V ersion numbers are indicated by xxx .
Draft Package Descr iptions OpenFabrics RPMs IB0056101-00 G .02 Preliminary D-5 A dapl- xxx .x86_64.rpm uDAPL 2.0.15 support Optional for OpenFabrics dapl-utils- xxx .x86_64.rpm uDAPL support Optional for OpenFabrics ibsim- xxx .x86_64.rpm InfiniBand Fabric Simulator Optional for OpenFabrics ibutils- xxx .
Draft Package Descriptions OpenFabrics RPMs D-6 Preliminary IB0056101-00 G .02 S libibverbs- xxx .x86_64.rpm Library that allows usersp ace processes to use In finiBand verbs as described in the InfiniBan d Architecture Specification . This library includes direct hardware access for fast path o perations.
Draft Package Descr iptions OpenFabrics RPMs IB0056101-00 G .02 Preliminary D-7 A The development RPMs in T able D-6 are only needed on the nodes where OFED programs are compiled. qlgc_vnic_daemon- xxx .x86_64.rpm Used with VNIC ULP service Optional for OpenFabrics qlvnictools- xxx .
Draft Package Descriptions OpenFabrics RPMs D-8 Preliminary IB0056101-00 G .02 S The opensm pa ckage in Ta b l e D - 7 should be installed only on the node that will be used as a subnet manage r . libibcm-devel- xxx .x86_64.rpm Development f iles for the libibcm library Optional for OpenFabric libibcommon-devel- xxx .
Draft Package Descr iptions Other HCAs IB0056101-00 G .02 Preliminary D-9 A Other HCAs The packages in T able D-9 should be in stalled only if other HCAs are used. The development RPMs in Ta b l e D - 1 0 are only needed on the nodes where programs are compiled T able D-8.
Draft Package Descriptions Other MPIs D-10 Preliminary IB0056101-00 G .02 S Other MPIs The packages in T able D-1 1 should be installed if ot her MPI implement ations are desired.RPMs that in clude yyy in the name are supplied by QLogic; others a re from OFED.
Draft Package Descr iptions Other MPIs IB0056101-00 G .02 Preliminary D-1 1 A openmpi_gcc- xxx .x86_64.rpm Open MPI compiled with GNU Optional openmpi_gcc_qlc- xxx . yyy. x86_64.rpm Open MPI compiled with GNU for PSM support Optional openmpi_intel_qlc- xxx .
Draft Package Descriptions Other MPIs D-12 Preliminary IB0056101-00 G .02 S Notes.
Draft IB0056101-00 G .02 Preliminary Index-1 Index A ACPI 4-4, A-2 Adapter , see HCA B BIOS configuring 4-4 settings A-2 settings to fix MTRR issues B-2 C -c 7-3 Cables supported 4-3 Compiler support .
Draft QLogic HCA and QLogic OFED Software Install Guide QLogic OFED V ersion 1.4 Index-2 Preliminary IB0056101-00 G .02 S Hardware requirements 4-1 HCA form factors 4-2 other HCAs-Devel/RPMs D-9 model.
Draft IB0056101-00 G .02 Preliminary Index-3 QLogic HCA and QLogic OFED Sof tware Inst all Guide QLogic OFED V ersion 1.4 A Lockable memory error A-5 Lustre, installing 5-23 M Model numbers for HCAs 2.
Draft QLogic HCA and QLogic OFED Software Install Guide QLogic OFED V ersion 1.4 Index-4 Preliminary IB0056101-00 G .02 S using to inst all InfiniPath and OpenFabrics 5-14 version numbers and identifi.
Draft.
Draft Preliminary © 2005-2009 QLogic Corporation. S pecifications are subject to change without notic e. All rights reserved worldwide . QLogic and the QLogic logo are registered trademarks of QLogic Corporatio n. QLA, SANsurf er, InfiniPath, and SilverS torm are trademarks or registered trade marks of QLogic Corporation.
An important point after buying a device Q-Logic OFED (or even before the purchase) is to read its user manual. We should do this for several simple reasons:
If you have not bought Q-Logic OFED yet, this is a good time to familiarize yourself with the basic data on the product. First of all view first pages of the manual, you can find above. You should find there the most important technical data Q-Logic OFED - thus you can check whether the hardware meets your expectations. When delving into next pages of the user manual, Q-Logic OFED you will learn all the available features of the product, as well as information on its operation. The information that you get Q-Logic OFED will certainly help you make a decision on the purchase.
If you already are a holder of Q-Logic OFED, but have not read the manual yet, you should do it for the reasons described above. You will learn then if you properly used the available features, and whether you have not made any mistakes, which can shorten the lifetime Q-Logic OFED.
However, one of the most important roles played by the user manual is to help in solving problems with Q-Logic OFED. Almost always you will find there Troubleshooting, which are the most frequently occurring failures and malfunctions of the device Q-Logic OFED along with tips on how to solve them. Even if you fail to solve the problem, the manual will show you a further procedure – contact to the customer service center or the nearest service center