Red Hat Enterprise Linux 5 Cluster Suite Overview Red Hat Cluster Suite for Red Hat Enterprise Linux 5 Edition 5
Red Hat Enterprise Linux 5 Cluster Suite Overview Red Hat Cluster Suite for Red Hat Enterprise Linux 5 Edition 5
Legal Notice Co pyright © 20 14 Red Hat, Inc. This do cument is licensed by Red Hat under the Creative Co mmo ns Attributio n-ShareAlike 3.0 Unpo rted License. If yo u distribute this do cument, o r a mo dified versio n o f it, yo u must pro vide attributio n to Red Hat, Inc. and pro vide a link to the o riginal. If the do cument is mo dified, all Red Hat trademarks must be remo ved.
T able of Cont ent s T able of Contents .Int . .roduct . . . . . .ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . . . . 1. Do c ument Co nventio ns 2 1.1. Typ o g rap hic Co nventio ns 3 1.2. Pull-q uo te Co nventio ns 4 1.3. No tes and Warning s 5 2 . Feed b ac k 5 . .hapt C . . . .er . .1. .. Red . . . . Hat . . . .Clust . . . . er . . .Suit . . . e. .O. verview . . . . . . . .
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Introduction This document provides a high-level overview of Red Hat Cluster Suite for Red Hat Enterprise Linux 5 and is is organized as follows: Chapter 1, Red Hat Cluster Suite Overview Chapter 2, Red Hat Cluster Suite Component Summary Although the information in this document is an overview, you should have advanced working knowledge of Red Hat Enterprise Linux and understand the concepts of server computing to gain a good comprehension of the infor
Int roduct ion 1.1. T ypographic Convent ions Four typographic conventions are used to call attention to specific words and phrases. These conventions, and the circumstances they apply to, are as follows. Mono-spaced Bold Used to highlight system input, including shell commands, file names and paths. Also used to highlight keys and key combinations.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Whether mono-spaced bold or proportional bold, the addition of italics indicates replaceable or variable text. Italics denotes text you do not input literally or displayed text that changes depending on circumstance. For example: To connect to a remote machine using ssh, type ssh username@domain.name at a shell prompt. If the remote machine is example.com and your username on that machine is john, type ssh john@example.com.
Int roduct ion 1.3. Not es and Warnings Finally, we use three visual styles to draw attention to information that might otherwise be overlooked. Note Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should have no negative consequences, but you might miss out on a trick that makes your life easier.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Chapter 1. Red Hat Cluster Suite Overview Clustered systems provide reliability, scalability, and availability to critical production services. Using Red Hat Cluster Suite, you can create a cluster to suit your needs for performance, high availability, load balancing, scalability, file sharing, and economy. This chapter provides an overview of Red Hat Cluster Suite components and functions, and consists of the following sections: Section 1.
Chapt er 1 . Red Hat Clust er Suit e O verview you can match the number of nodes according to load requirements. If a node in a load-balancing cluster becomes inoperative, the load-balancing software detects the failure and redirects requests to other cluster nodes. Node failures in a load-balancing cluster are not visible from clients outside the cluster. Red Hat Cluster Suite provides load-balancing through LVS (Linux Virtual Server).
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Global Network Block D evice (GNBD ) — An ancillary component of GFS/GFS2 that exports block-level storage to Ethernet. This is an economical way to make block-level storage available to GFS/GFS2. Note Only single site clusters are fully supported at this time. Clusters spread across multiple physical locations are not formally supported. For more details and to discuss multi-site clusters, please speak to your Red Hat sales or support representative.
Chapt er 1 . Red Hat Clust er Suit e O verview The Red Hat Cluster Suite cluster infrastructure provides the basic functions for a group of computers (called nodes or members) to work together as a cluster. Once a cluster is formed using the cluster infrastructure, you can use other Red Hat Cluster Suite components to suit your clustering needs (for example, setting up a cluster for sharing files on a GFS file system or setting up service failover).
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.2. C MAN /D LM O verview 1.3.2. Lock Management Lock management is a common cluster-infrastructure service that provides a mechanism for other cluster infrastructure components to synchronize their access to shared resources. In a Red Hat cluster, D LM (D istributed Lock Manager) is the lock manager.
Chapt er 1 . Red Hat Clust er Suit e O verview Power fencing — A fencing method that uses a power controller to power off an inoperable node. Two types of power fencing are available: external and integrated. External power fencing powers off a node via a power controller (for example an API or a WTI power controller) that is external to the node.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.4 . Fib re C h an n el Swit ch Fen cin g Examp le Specifying a fencing method consists of editing a cluster configuration file to assign a fencingmethod name, the fencing agent, and the fencing device for each node in the cluster. The way in which a fencing method is specified depends on if a node has either dual power supplies or multiple paths to storage.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.5.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.6 . Fen cin g a N o d e wit h D u al Fib re C h an n el C o n n ect io n s You can configure a node with one fencing method or multiple fencing methods. When you configure a node for one fencing method, that is the only fencing method available for fencing that node.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.7. C C S O verview Other cluster components (for example, CMAN) access configuration information from the configuration file through CCS (refer to Figure 1.7, “ CCS Overview” ).
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.8. Accessin g C o n f ig u rat io n In f o rmat io n The cluster configuration file (/etc/cluster/cluster.conf) is an XML file that describes the following cluster characteristics: Cluster name — D isplays the cluster name, cluster configuration file revision level, and basic fence timing properties used when a node joins a cluster or is fenced from the cluster.
Chapt er 1 . Red Hat Clust er Suit e O verview service. A high-availability cluster service can fail over from one cluster node to another with no apparent interruption to cluster clients. Cluster-service failover can occur if a cluster node fails or if a cluster system administrator moves the service from one cluster node to another (for example, for a planned outage of a cluster node). To create a high-availability service, you must configure it in the cluster configuration file.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.9 . Failo ver D o main s Figure 1.10, “ Web Server Cluster Service Example” shows an example of a high-availability cluster service that is a web server named " content-webserver" . It is running in cluster node B and is in a failover domain that consists of nodes A, B, and D .
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.10. Web Server C lu st er Service Examp le Clients access the cluster service through the IP address 10.10.10.201, enabling interaction with the web server application, httpd-content. The httpd-content application uses the gfs-content-webserver file system. If node B were to fail, the content-webserver cluster service would fail over to node D . If node D were not available or also failed, the service would fail over to node A.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview GFS/GFS2 is a native file system that interfaces directly with the Linux kernel file system interface (VFS layer). When implemented as a cluster file system, GFS/GFS2 employs distributed metadata and multiple journals. Red Hat supports the use of GFS/GFS2 file systems only as implemented in Red Hat Cluster Suite. Note Although a GFS/GFS2 file system can be implemented in a standalone system or as part of a cluster configuration, for the RHEL 5.
Chapt er 1 . Red Hat Clust er Suit e O verview The Cluster Logical Volume Manager (CLVM) provides a cluster-wide version of LVM2. CLVM provides the same capabilities as LVM2 on a single node, but makes the volumes available to all nodes in a Red Hat cluster. The logical volumes created with CLVM make logical volumes available to all nodes in a cluster. The key component in CLVM is clvmd.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.11. C LVM O verview You can configure CLVM using the same commands as LVM2, using the LVM graphical user interface (refer to Figure 1.12, “ LVM Graphical User Interface” ), or using the storage configuration function of the C o n g a cluster configuration graphical user interface (refer to Figure 1.13, “ Conga LVM Graphical User Interface” ) . Figure 1.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.12.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.13.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.14 . C reat in g Lo g ical Vo lu mes 1.7. Global Net work Block Device Global Network Block D evice (GNBD ) provides block-device access to Red Hat GFS over TCP/IP. GNBD is similar in concept to NBD ; however, GNBD is GFS-specific and tuned solely for use with GFS. GNBD is useful when the need for more robust technologies — Fibre Channel or single-initiator SCSI — are not necessary or are cost-prohibitive.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.15. G N B D O verview 1.8. Linux Virt ual Server Linux Virtual Server (LVS) is a set of integrated software components for balancing the IP load across a set of real servers. LVS runs on a pair of equally configured computers: one that is an active LVS router and one that is a backup LVS router. The active LVS router serves two roles: To balance the load across the real servers. To check the integrity of the services on each real server.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.16 . C o mp o n en t s o f a R u n n in g LVS C lu st er The pulse daemon runs on both the active and passive LVS routers. On the backup LVS router, pulse sends a heartbeat to the public interface of the active router to make sure the active LVS router is properly functioning. On the active LVS router, pulse starts the lvs daemon and responds to heartbeat queries from the backup LVS router.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Synchronize the data across the real servers. Add a third layer to the topology for shared data access. The first option is preferred for servers that do not allow large numbers of users to upload or change data on the real servers. If the real servers allow large numbers of users to modify data, such as an ecommerce website, adding a third layer is preferable. There are many ways to synchronize data among real servers.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.17. T wo - T ier LVS T o p o lo g y Service requests arriving at an LVS router are addressed to a virtual IP address or VIP. This is a publicly-routable address that the administrator of the site associates with a fully-qualified domain name, such as www.example.com, and which is assigned to one or more virtual servers [1] .
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Also, the active LVS router dynamically monitors the overall health of the specific services on the real servers through simple send/expect scripts. To aid in detecting the health of services that require dynamic data, such as HTTPS or SSL, you can also call external executables. If a service on a real server malfunctions, the active LVS router stops sending jobs to that server until it returns to normal operation.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.18. T h ree- T ier LVS T o p o lo g y This topology is suited well for busy FTP servers, where accessible data is stored on a central, highly available server and accessed by each real server via an exported NFS directory or Samba share. This topology is also recommended for websites that access a central, high-availability database for transactions.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Figure 1.19, “ LVS Implemented with NAT Routing” , illustrates LVS using NAT routing to move requests between the Internet and a private network. Fig u re 1.19 . LVS Imp lemen t ed wit h N AT R o u t in g In the example, there are two NICs in the active LVS router. The NIC for the Internet has a real IP address on eth0 and has a floating IP address aliased to eth0:1.
Chapt er 1 . Red Hat Clust er Suit e O verview 1 .8 .3.2 . Dire ct Ro ut ing D irect routing provides increased performance benefits compared to NAT routing. D irect routing allows the real servers to process and route packets directly to a requesting user rather than passing outgoing packets through the LVS router. D irect routing reduces the possibility of network performance issues by relegating the job of the LVS router to processing incoming packets only. Fig u re 1.20.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview correct IP/MAC address combination receives the packet. The IP/MAC associations are stored in an ARP cache, which is cleared periodically (usually every 15 minutes) and refilled with IP/MAC associations.
Chapt er 1 . Red Hat Clust er Suit e O verview Because of its efficiency and ease-of-use, administrators of LVS should use firewall marks instead of persistence whenever possible for grouping connections. However, you should still add persistence to the virtual servers in conjunction with firewall marks to ensure the clients are reconnected to the same server for an adequate period of time. 1.9.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview You can populate the database of one lu ci instance from another lu ciinstance. That capability provides a means of replicating a lu ci server instance and provides an efficient upgrade and testing path. When you install an instance of lu ci, its database is empty. However, you can import part or all of a lu ci database from an existing lu ci server when deploying a new lu ci server. Each lu ci instance has one user at initial installation — admin.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.22.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.23. lu ci st o rag e T ab 1.9.2. Clust er Administ rat ion GUI This section provides an overview of the system-config-cluster cluster administration graphical user interface (GUI) available with Red Hat Cluster Suite. The GUI is for use with the cluster infrastructure and the high-availability service management components (refer to Section 1.3, “ Cluster Infrastructure” and Section 1.4, “ High-availability Service Management” ).
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.24 . C lu st er C o n f ig u rat io n T o o l The C lu st er C o n f ig u rat io n T o o l represents cluster configuration components in the configuration file (/etc/cluster/cluster.conf) with a hierarchical graphical display in the left panel. A triangle icon to the left of a component name indicates that the component has one or more subordinate components assigned to it.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fence Devices — D isplays fence devices. Fence devices are represented as subordinate elements under Fence Devices. Using configuration buttons at the bottom of the right frame (below Properties), you can add fence devices, delete fence devices, and edit fence-device properties. Fence devices must be defined before you can configure fencing (with the Manage Fencing For This Node button) for each node.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.25. C lu st er St at u s T o o l The nodes and services displayed in the C lu st er St at u s T o o l are determined by the cluster configuration file (/etc/cluster/cluster.conf). You can use the C lu st er St at u s T o o l to enable, disable, restart, or relocate a high-availability service. 1.9.3.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview T ab le 1.1. C o mman d Lin e T o o ls C o mman d Lin e Tool U sed Wit h Pu rp o se ccs_tool — Cluster Configuration System Tool Cluster Infrastructure ccs_tool is a program for making online updates to the cluster configuration file. It provides the capability to create and modify cluster infrastructure components (for example, creating a cluster, adding and removing a node). For more information about this tool, refer to the ccs_tool(8) man page.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.26 . T h e Welco me Pan el The following sections provide a brief description of the Piran h a C o n f ig u rat io n T o o l configuration pages. 1.10.1. CONTROL/MONITORING The CONTROL/MONITORING Panel displays runtime status. It displays the status of the pulse daemon, the LVS routing table, and the LVS-spawned nanny processes.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.27. T h e CONTROL/MONITORING Pan el Auto update Enables the status display to be updated automatically at a user-configurable interval set in the Update frequency in seconds text box (the default value is 10 seconds). It is not recommended that you set the automatic update to an interval less than 10 seconds. D oing so may make it difficult to reconfigure the Auto update interval because the page will update too frequently.
Chapt er 1 . Red Hat Clust er Suit e O verview The GLOBAL SETTINGS panel is where the LVS administrator defines the networking details for the primary LVS router's public and private network interfaces. Fig u re 1.28. T h e GLOBAL SETTINGS Pan el The top half of this panel sets up the primary LVS router's public and private network interfaces. Primary server public IP The publicly routable real IP address for the primary LVS node.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview NAT Router IP The private floating IP in this text field. This floating IP should be used as the gateway for the real servers. NAT Router netmask If the NAT router's floating IP needs a particular netmask, select it from drop-down list. NAT Router device D efines the device name of the network interface for the floating IP address, such as eth1:1. 1.10.3.
Chapt er 1 . Red Hat Clust er Suit e O verview Redundant server public IP The public real IP address for the backup LVS router. Redundant server private IP The backup router's private real IP address. The rest of the panel is for configuring the heartbeat channel, which is used by the backup node to monitor the primary node for failure. Heartbeat Interval (seconds) Sets the number of seconds between heartbeats — the interval that the backup node will check the functional status of the primary LVS node.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.30. T h e VIRTUAL SERVERS Pan el Each server displayed in the VIRTUAL SERVERS panel can be configured on subsequent screens or subsections. To add a service, click the ADD button. To remove a service, select it by clicking the radio button next to the virtual server and click the DELETE button. To enable or disable a virtual server in the table click its radio button and click the (DE)ACTIVATE button.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.31. T h e VIRTUAL SERVERS Su b sect io n Name A descriptive name to identify the virtual server. This name is not the hostname for the machine, so make it descriptive and easily identifiable. You can even reference the protocol used by the virtual server, such as HTTP. Application port The port number through which the service application will listen. Pro t o co l Provides a choice of UD P or TCP, in a drop-down menu.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Firewall Mark For entering a firewall mark integer value when bundling multi-port protocols or creating a multi-port virtual server for separate, but related protocols. Device The name of the network device to which you want the floating IP address defined in the Virtual IP Address field to bind. You should alias the public floating IP address to the Ethernet interface connected to the public network.
Chapt er 1 . Red Hat Clust er Suit e O verview Fig u re 1.32. T h e REAL SERVER Su b sect io n Click the ADD button to add a new server. To delete an existing server, select the radio button beside it and click the DELETE button. Click the EDIT button to load the EDIT REAL SERVER panel, as seen in Figure 1.33, “ The REAL SERVER Configuration Panel” .
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fig u re 1.33. T h e REAL SERVER C o n f ig u rat io n Pan el This panel consists of three entry fields: Name A descriptive name for the real server. Note This name is not the hostname for the machine, so make it descriptive and easily identifiable. Address The real server's IP address. Since the listening port is already specified for the associated virtual server, do not add a port number.
Chapt er 1 . Red Hat Clust er Suit e O verview The value can be arbitrary, but treat it as a ratio in relation to other real servers. 1 .1 0 .4 .3. EDIT MONITORING SCRIPTS Subse ct io n Click on the MONITORING SCRIPTS link at the top of the page. The EDIT MONITORING SCRIPTS subsection allows the administrator to specify a send/expect string sequence to verify that the service for the virtual server is functional on each real server.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Note If an external program is entered in the Sending Program field, then the Send field is ignored. Send A string for the nanny daemon to send to each real server in this field. By default the send field is completed for HTTP. You can alter this value depending on your needs. If you leave this field blank, the nanny daemon attempts to open the port and assume the service is running if it succeeds.
Chapt er 2 . Red Hat Clust er Suit e Component Summary Chapter 2. Red Hat Cluster Suite Component Summary This chapter provides a summary of Red Hat Cluster Suite components and consists of the following sections: Section 2.1, “ Cluster Components” Section 2.2, “ Man Pages” Section 2.3, “ Compatible Hardware” 2.1. Clust er Component s Table 2.1, “ Red Hat Cluster Suite Software Subsystem Components” summarizes Red Hat Cluster Suite components. T ab le 2.1.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fu n ct io n Cluster Manager (CMAN) C o mp o n en t s D escrip t io n cluster.conf This is the cluster configuration file. The full path is /etc/cluster/cluster.conf . The kernel module for CMAN. cman.ko cman_tool dlm_controld Resource Group Manager (rgmanager) gfs_controld D aemon started by cman init script to manage gfs in kernel; not used by user.
Chapt er 2 . Red Hat Clust er Suit e Component Summary Fu n ct io n C o mp o n en t s D escrip t io n fence_vixel fence_rsa Fence agent for Vixel Fibre Channel switch. Fence agent for SANBox2 Fibre Channel switch. Fence agent for HP ILO interfaces (formerly fence_rib). I/O Fencing agent for IBM RSA II. fence_gnbd Fence agent used with GNBD storage. fence_scsi I/O fencing agent for SCSI persistent reservations. Fence agent used with Egenera BladeFrame system. Fence agent for manual interaction.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Fu n ct io n C o mp o n en t s D escrip t io n gnbd_import Command to import and manage GNBD s on a GNBD client. A server daemon that allows a node to export local storage over the network. This is the controlling process which starts all other daemons related to LVS routers. At boot time, the daemon is started by the /etc/rc.d/init.d/pulse script. It then reads the configuration file /etc/sysconfig/ha/lvs.cf .
Chapt er 2 . Red Hat Clust er Suit e Component Summary Fu n ct io n C o mp o n en t s D escrip t io n lvs.cf mkqdisk This is the LVS configuration file. The full path for the file is /etc/sysconfig/ha/lvs.cf . D irectly or indirectly, all daemons get their configuration information from this file. This is the Web-based tool for monitoring, configuring, and administering LVS. This is the default tool to maintain the /etc/sysconfig/ha/lvs.cf LVS configuration file.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview fence_drac (8) - fencing agent for D ell Remote Access Card fence_egenera (8) - I/O Fencing agent for the Egenera BladeFrame fence_gnbd (8) - I/O Fencing agent for GNBD -based GFS clusters fence_ilo (8) - I/O Fencing agent for HP Integrated Lights Out card fence_ipmilan (8) - I/O Fencing agent for machines controlled by IPMI over LAN fence_manual (8) - program run by fenced as a part of manual I/O Fencing fence_mcdata (8) - I/O Fencing agent for McD ata
Chapt er 2 . Red Hat Clust er Suit e Component Summary lvm (8) - LVM2 tools lvm.
Red Hat Ent erprise Linux 5 Clust er Suit e O verview Revision History R evisio n 5.11- 6 Version for 5.11 GA release Mo n Sep 8 2014 St even Levin e R evisio n 5.11- 4 Mo n Ju n 30 2014 Beta release of Red Hat Enterprise Linux 5.11 St even Levin e R evisio n 5.10- 4 T u e O ct 1 2013 Final release of Red Hat Enterprise Linux 5.10 Jo h n H a R evisio n 5.10- 3 Fri Ju l 12 2013 Beta release of Red Hat Enterprise Linux 5.10 Jo h n H a R evisio n 5.
Revision Hist ory co mman d lin e t o o ls t ab le, C o mman d Lin e Ad min ist rat io n T o o ls co mp at ib le h ard ware - cluster components, Compatible Hardware Conga - overview, Conga C o n g a o verview, C o n g a F f eed b ack, Feed b ack G G FS/G FS2 f ile syst em maximu m siz e, R ed H at G lo b al File Syst em I in t ro d u ct io n , In t ro d u ct io n - other Red Hat Enterprise Linux documents, Introduction L LVS - direct routing - requirements, hardware, D irect Routing - requirements, ne
Red Hat Ent erprise Linux 5 Clust er Suit e O verview - VIRTUAL SERVER subsection, VIRTUAL SERVERS - Firewall Mark , The VIRTUAL SERVER Subsection - Persistence , The VIRTUAL SERVER Subsection - Scheduling , The VIRTUAL SERVER Subsection - Virtual IP Address , The VIRTUAL SERVER Subsection - VIRTUAL SERVERS, VIRTUAL SERVERS R R ed H at C lu st er Su it e - components, Cluster Components T t ab le - cluster components, Cluster Components - command line tools, Command Line Administration Tools 64
