HP Scalable Visualization Array Version 2.
© Copyright 2006, 2007 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Table of Contents About This Document.......................................................................................................11 1 Intended Audience.............................................................................................................................11 2 Document Organization.....................................................................................................................11 3 Typographic Conventions......................................................
.3.1 Set Up Control for an Application..........................................................................................34 4.3.2 Launch an Application in Interactive Mode...........................................................................35 4.3.3 Launch an Application in Batch Mode....................................................................................36 4.3.4 Run an Application Using HP RGS.........................................................................................
6.3.7 Launch Script...........................................................................................................................62 Glossary............................................................................................................................65 Index.................................................................................................................................
List of Figures 1-1 1-2 1-3 2-1 3-1 3-2 6-1 6-2 6-3 System View of a Computing Environment with Integrated SVA...............................................13 Standalone SVA Data Flow............................................................................................................14 Software Support for Application Development and Use............................................................16 SVA Data Flow Overview...............................................................................
List of Tables 3-1 3-2 3-3 3-4 3-5 6-1 Operating System and Driver Components..................................................................................29 HP XC System Components Relevant to SVA Operation.............................................................30 HP SVA System Software..............................................................................................................30 Third Party System Software..........................................................................
About This Document The SVA User's Guide introduces the components of the HP Scalable Visualization Array (SVA). The SVA product has hardware and software components that together make up the HP high performance visualization cluster. This document provides a high level understanding of SVA components.
[] The contents are optional in syntax. If the contents are a list separated by a pipe ( | ), you must choose one of the items. The contents are required in syntax. If the contents are a list separated by a pipe ( | ), you must choose one of the items. The preceding element can be repeated an arbitrary number of times. Indicates the continuation of a code example. Separates items in a list of choices.
1 Introduction This chapter gives an overview of the HP Scalable Visualization Array (SVA). It describes how the SVA works within the context of overall HP cluster solutions. It also discusses attributes of the SVA that make it a powerful tool for running data intensive graphics applications. The SVA is a scalable visualization solution that brings the power of parallel computing to bear on many demanding visualization challenges.
is also available for display remotely, but limited to single or two-tile output from a single graphics card. The SVA serves as a key unit in an integrated computing environment that displays the results of generated data in locations where scientists and engineers can most effectively carry out analyses individually or collaboratively. 1.2 SVA Clusters This section gives a high-level description of a standalone SVA, that is, an HP Cluster Platform system built to include visualization nodes.
Final images can also be transmitted to a remote workstation display over a network external to the cluster. This lets users interact with applications running on the cluster from their offices. Optionally, you can use HP Remote Graphics Software (RGS) or VirtualGL to accomplish this more easily. See Chapter 6 for more information on both these packages. Figure 1-2 also shows a master application node communicating with the other visualization nodes over the SI.
1.4.2 Flexibility One of the most powerful attributes of the SVA is its flexibility, which makes it possible to apply the SVA effectively to a wide range of technical problems. This flexibility derives from the architectural characteristics of the SVA.
full advantage of the various toolkits and libraries. The SVA uses standards such as OpenGL, Linux, InfiniBand, and Gigabit Ethernet for portability and interoperability. The HP Parallel Compositing Library that ships with SVA can help application developers accomplish parallel rendering. See the SVA Parallel Compositing Reference Guide. To achieve maximum performance scaling when running on the SVA, an application must be parallel and distributed.
2 SVA Architecture This chapter gives a detailed look at the architecture of the HP Scalable Visualization Array (SVA). It compares the SVA to other clusters and describes the flow of data within the cluster. 2.1 SVA as a Cluster It is important to understand the cluster characteristics of the SVA. These characteristics have implications for how SVA functions. They also affect how applications take advantage of cluster features to achieve graphical performance and display goals. 2.1.
Thus, the SVA provides the HP Cluster Platform with a visualization component for those applications that require visualization in addition to computation. The following sections describe the components that make up an HP Cluster Platform, followed by those tasks and components that are unique to an SVA. 2.2.1 Components of the HP Cluster Platform Because the SVA is an extension of the HP Cluster Platform, you can begin by understanding its base components without any visualization nodes.
Remote images. Integrate an application user interface. images. The output can be a complete display or one tile of an aggregate display. The SVA also supports the transmission of a complete image to a system external to the cluster over an external network for remote viewing; for example, to an office workstation outside the lab. A node with a port connected to the external network is recommended.
• • • • Multiple displays with different resolutions. Use of a variable number of display and render nodes to solve the computational and rendering requirements of an application. Bounded configuration designed for a single user. Larger, modular, expandable systems designed for one or more concurrent users. See Chapter 3 for more information on the physical configurations of the SVA. 2.3 SVA Operation This section describes a common way data flows through an SVA. 2.3.
features extracted based on application settings. The application then caches the results of the analysis or rendering to display an animation of the time steps. Although parallel visualization is a relatively new approach, some file access patterns that applications use include the following: • Master portion of the application reads data from files and distributes data to visualization nodes using the SI. • Visualization nodes all read data from the same files.
3 SVA Hardware and Software This chapter provides information on the hardware and software that make up the SVA. It is a useful reference for anyone involved in managing the SVA. It is also useful for anyone who wants to understand the hardware that makes up the SVA and the software that is installed on it. The SVA combines commodity hardware components with software that include the following: • A cluster of Intel EM64T or AMD-64 Opteron HP workstations and servers as visualization nodes.
Platform system, the visualization nodes can use a high speed system interconnect to load data from an HP SFS. The two SVA physical configurations are built using one or more of three types of cluster building blocks. Each building block uses a single rack. Utility Visualization Block (UVB) Utility Building Block (UBB) Visualization Building Block (VBB) Base utility unit of a Bounded Configuration.
Figure 3-1 Sample SVA Bounded Configuration Base Rack (UVB) GigE Display Devices External Node 3.3 Modular Packaging Configuration A Modular Packaging Configuration is built from the UBB and VBB rack systems. It has the following components as summarized in Chapter 2: • Render and Display nodes. Workstations: xw8200, xw8400, xw9300, xw9400, or servers: DL140 G3 or DL145 G3 . • Head node. DL380 G4 or a DL385 G4 server. • • Optional KVM.
The management switches are collected together in one rack. SVA nodes connect to branch switches in the Administrative Network. SVA nodes do not connect to the console branch. Nodes connect to the switches according to the Cluster Platform Administrative Network connections for HP XC. Display and render node types are typically grouped together. 3.
Figure 3-2 Software Hierarchy in the SVA Applications Launch Visualization Libraries (optional) OpenGL Initialize Cleanup X Servers HP XC Linux SVA Software Utilities Allocate Cluster Nodes and Displays 3.6.1 Linux Operating System The SVA software is layered on top of HP XC System Software Version 3.2, a clustering Linux distribution compatible with Red Hat Enterprise Linux Advanced Server V4.0 Update 3. The kernel version is V2.6.9–x. See Section 3.6.2.
• • Naming each of the nodes in the cluster and determining which nodes are up and running. Serializing application use of the cluster. For more information on HP XC, consult the HP XC documentation set at the following Web site: http://docs.hp.com/en/highperfcomp.html Table 3-2 summarizes the software components that are provided by the HP XC operating system that relate to the SVA.
Table 3-4 Third Party System Software Software Description OpenGL Primary interface programmers use to create images. http://www.opengl.org/ OpenGL Utility library (GLU) Contains routines that build on the lower level OpenGL library to perform such tasks as setting up matrices for specific viewing orientations and projections, performing polygon tessellation, and rendering surfaces.
4 Quick Start This chapter lets you quickly try some of the sample applications on the SVA Kit. Details on using scripts are provided in other chapters of this HP SVA User's Guide and other documents in the HP SVA Documentation Library. 4.1 Typical Uses of SVA SVA has three primary usage scenarios as described in detail in Chapter 6. • • • A workstation application that is launched remotely to use only a single node in the SVA.
TIP: See the HP SVA System Administration Guide for detailed information on how to define Display Surfaces, including a recommended incremental series of steps for configuring SVA for your displays. • Verify and possibly modify supported resolutions, display modelines, and refresh rates. This step is likely to be required for stereo displays and more exotic mono displays. Typical desktop display devices (monitors and flat panels) are supported by default by SVA.
• • In the case of third-party applications, for example, ParaView and EnSight, there is a separate user interface whose location is determined by setting the DISPLAY environment variable before you run the application. In the specific case of the SVA sva_chromium_dmx.sh script only: Use the -i option or set the DISPLAY environment variable to specify the node from which you want to provide input.
• • • You need to substitute the name of the site-specific input node for YOUR_INPUT_NODE. The city application is shipped with the SVA kit and is already on your PATH. It is a good application for seeing that the image is properly aligned among the individual tiles of a multi-tile display. It is possible that you do not see any images on the input node's monitor; it could appear blank. This is because the graphics card only supports one resolution at a time.
% sva_remote.sh -I (Note that problems sometimes occur running the script or logging into the Linux GUI. Check your login file for incompatible settings.) 3. HP RGS displays the name of the node it is connecting to on the cluster. Make a note of the name. 4. Immediately start the RGS Receiver application on your desktop. It's helpful to have an icon available for this. 5. Enter the name of the connected sender node into the Receiver window. Click on the Go button in the Receiver window. 6.
5 Setting Up and Running a Visualization Session This chapter explains how to run visualization applications on the SVA. A visualization session relies primarily on HP XC utilities to do the underlying work; however, you can avoid manually using the underlying utilities by means of job launch scripts and associated templates provided by the SVA kit. For details on HP XC utilities, see the HP XC system documentation. 5.
5.2 Running an Application Using Scripts Typically, you encapsulate the various commands needed to run applications using a script file. This speeds the process of running the application, given the likelihood that this is a task you repeat. The installation of the SVA Software Kit provides several general purpose script templates. These templates are the starting points for creating scripts to launch your own application.
• sva_startx.sh This script takes a list of nodes and allocates them as part of an SVA job. Once the nodes are allocated, an X server starts on each node using the specified (or default) tile geometry. You are then left with a shell prompt, which you can use to start other programs or job steps. This is a fully functional script on your PATH. The sva_startx.sh script is located in the /opt/sva/bin directory. • sva_paraview.sh This script is specifically optimized to launch the ParaView application2.
4. Surface you choose for the job. See the SVA System Administration Guide for detailed information on the Configuration Data Files and Display Surfaces. Terminate: Stops the visualization session cleanly. Also stops the services that were started by the script; for example, X Servers and DMX. 5.2.
You can start, stop, and restart an application to make it easier to test and debug. IMPORTANT: You must be able to view the SVA Display Surface because the DMX Console provides limited visual feedback. You also need to be able to interact with your application as it runs, for example, by using a KVM. Ways to do this are described in Section 4.3.1. Creating an interactive session in this way lets you take advantage of your multi-tile display for other applications.
Once you are logged into the cluster, use a terminal window and start one of the other SVA launch scripts, for example sva_paraview.sh with the --local option. Alternatively, you follow a similar process if you use VirtualGL and TurboVNC rather than HP RGS for remote viewing: % sva_vgltvnc.sh -I Once you are logged into the cluster and have the VNC desktop connected, you can run X applications as normal from a terminal window.
The following command uses a launch script with the same Display Surface as a mono device: % sva_chromium_dmx.sh -d STEREO_DISPLAY_1 "/usr/X11R6/lib/xscreensaver/atlantis" The following SVA launch scripts support the --stereo option. • • • • sva_amiravr.sh sva_chromium_dmx.sh sva_paraview.sh sva_job_template.sh CAUTION: The stereo capabilities in SVA best support a single class of mono display devices and a single class of stereo display devices.
5.7.2 Use the Framelock Script Function If you write or modify job launch scripts, there is an SVA scripting function (found in svainit) to enable framelock on a Display Surface: svaEnableFrameLock For an example of how to use this function, see /opt/sva/samples/sva_job_template.sh. See the SVA Visualization System Software Reference Guide for more information on all the scripting functions, including svaEnableFrameLock. 5.7.
6 Application Examples This chapter describes the steps to start several representative applications that vary in their structure and requirements: • • • A workstation application that is launched remotely to use only a single node in the SVA. See Section 6.1. An application that uses render and display capabilities of the SVA (for example, ParaView). See Section 6.2. A workstation application that uses Chromium software and DMX to display on multiple tiles using the SVA. See Section 6.3.
This example also assumes that you use HP RGS to provide the remote viewing capability. Alternatively, you could use VirtualGL and TurboVNC for this purpose. These are open source packages whose web link is www.virtualgl.org. The goal of this example is to make the application run on the SVA while maintaining control remotely from a desktop that is outside the cluster. This desktop is remote relative to the SVA although you may consider it your local workstation.
SVA as determined by the cluster node running the RGS Sender if connected to a display device. Figure 6-1 shows the relationships among the various processes that run when you launch visualization jobs. There are four processes that must run when a remote visualization session begins. • • • • The X Server. RGS Sender on the SVA RGS-capable node. RGS Receiver on your local desktop. Your visualization application.
for this example, assuming the display node has the RGS Sender software installed, an external NIC, and uses a single graphics card to output one or two tiles. Because the RGS Sender routes the display output to your local desktop, its display device is the one you typically use. Display output can appear simultaneously on the display device of the SVA if you specify a Display Surface when you start the launch script.
This command specifies that the SVA_DS_1_2 Display Surface be used. If you omit the –d option, the script automatically allocates a visualization node capable of using RGS as the remote node. The script draws the node from the pool of render nodes. If there are no render nodes available, then the script chooses a node from the pool of available display nodes or the head node. The window immediately displays the external name of the display node running the atlantis application.
6. Open a terminal window in the desktop environment and enter the following command: % /usr/X11R6/lib/xscreensaver/atlantis -count 20 The atlantis application display begins. 7. Exit the application to stop the application only. You can then restart the application using the same application command or another command, including a command with a different application. Cluster resources remain allocated.
the cluster_config process, the system administrator configures the SVA to identify remote-capable nodes. 2. 3. Your system manager installed the necessary launch scripts for these packages. Your system manager opened the required firewall ports on remote-capable nodes on SVA. This may already be done as part of the SVA installation. See the HP SVA System Administration Guide for information on these steps. 6.1.3.
7. Open a terminal window using MB3 in the Linux Desktop (gnome) that appears. If you get a desktop that is not the SVA cluster default (gnome), you may have an old ~/.vnc/xstartup file. Delete this to get the gnome Desktop. 8. X applications can be run as normal from a terminal window. OpenGL applications have to be run with the vglrun command. See the VirtualGL website for command options, which may be needed for certain applications.
4. In the Category pane, click on Session. Connect to the remote-capable cluster node using PuTTY. Enter the node's name as provided in the output from Step 2. Click on Open in the PuTTY window. At this point, the SSH tunnel is set up. When you connect to port 5901 on the local host, the data will be tunneled to port 5901 on the remote-capable cluster node. 5. Start the TurboVNC client on your desktop and connect using localhost:1.
6.2.2 ParaView Overview ParaView is an open source, multiplatform, extensible application designed for visualizing large datasets. This scalable application runs on single-processor workstations as well as on large parallel supercomputers. ParaView features include: • • • • • • • • • • • Runs parallel on distributed and shared memory systems using MPI. These include workstation clusters, visualization systems, large servers, supercomputers, and so on.
Host while its display is pushed back to your local desktop. HP recommends that you use a Display Surface that uses a display node as its Execution Host. The Execution Host is defined for each Display Surface. You specify the Display Surface by name when launching a visualization job. The Execution Host for a Display Surface is the default location for running an application.
Tip: You can use a SLURM srun command to do this. 2. 3. 4. 5. Launch X Servers on all the allocated nodes. Launch the ParaView Client on the Execution Host. When launching the Client, set the DISPLAY environmental variable to your local desktop in order to push the Client display to that machine. Alternatively, you could use sva_paraview.sh with the --local option. See Section 5.4 for more information. You also need to set the ParaView launch command option to listen mode.
Your site administrator must define multi-tile Display Surfaces using the Display Surface Configuration Tool. The Display Surface Configuration Tool also can list all the named Display Surfaces for the cluster. Entering specific Display Surfaces to the script to access the display resources of the cluster. Because this example routes the display output to your local desktop, its display device is the one you use to manipulate any image.
Although Chromium has several configuration files that you typically need to edit, the SVA launch script eliminates this need by using configuration data from the SVA Configuration Data Files. A link to the Chromium documentation is available from the SVA Documentation Library on the HP XC Documentation CD. 6.3.3 Distributed Multi-Head X (DMX) Xdmx is a proxy X Server that provides multi-head support for multiple displays attached to different machines (each of which is running a typical X Server).
IMPORTANT: Note that the you may lose the video display; however, the keyboard and mouse continue to work such that you can actually control the multi-tile image. This is because the multi-tile display device may be using a resolution that is unsupported by the KVM or RKM. You should be able to use the arrow keys to move among the various cluster nodes, including the head node. Figure 6-3 shows the relationships among the processes that run when you launch a visualization job.
If you choose to store data locally, you can copy the data file to the display node after the application starts. This ensures that you access a node allocated to your job. Tip: Consider running the launch script interactively if you plan to use local disk access to the data. When run in interactive mode, the script allocates cluster resources first. You can then copy the data file to the allocated display node before launching the visualization application. See Section 6.3.
You begin by logging in to the SVA using a terminal window. The following command runs the atlantis application on the FULL_DISPLAY Display Surface using the DMX-Chromium launch script. Note the use of the application-specific command line parameter: -count 20. Launch scripts can start from any cluster node, for example, the head or a login node. % sva_chromium_dmx.
Glossary Administrative Network Connects all nodes in the cluster. In an HP XC compute cluster, this consists of two branches: the Administrative Network and the Console Network. This private local Ethernet network runs TCP/IP. The Administrative Network is Gigabit Ethernet (GigE); the Console Network is 10/100 BaseT. Because the visualization nodes do not support console functions, visualization nodes are not connected to a console branch.
modular packing configuration This SVA configuration has two or more racks as needed to contain from four to ninety-five workstations or servers, along with a server head node. This configuration is based on HP Cluster Platform building blocks, namely the Visualization Building Block (VBB) and the Utility Building Block (UBB). It can be exclusively visualization nodes or be combined with compute nodes as part of an integrated HP Cluster Platform system.
Index A H Admin/service node, 20 Administrative network, 20, 26, 27 Architecture of SVA, 19 Head node, 20 use in visualization job, 43 HP Mlib, 31 B I Beowulf cluster, 19 Bounded configuration, 25 Interactive session how to run, 42 provide input to, 60 C Chromium, 30 Compilers on kit, 31 Compute cluster components of, 20 Compute node, 20 Configuration data files hierarchy of, 39 job settings, 39 overview, 39 site, 39 user, 39 D Data flow within SVA, 22 Debugger on kit, 31 Development tools, 31 Diag
UVB, 26 R Remote Graphics Software (HP), 30 Render node, 21 in SVA, 21 RGS example use, 48 launched via script, 50 launching of process, 49 RGS Display flow of control for, 49 RGS node route display to local desktop from, 43 Run application, 40 S Sample program types of, 47 Scripts use for job launch, 40 used for launching applications, 42 used for stereo applications, 44 Serial application example use, 59 Site Configuration file, 39 SLURM use in job launch, 41 Stereo display use by job script, 44 SVA arc
