SWsoft, Inc. OpenVZ User's Guide Version 2.7.
ISBN: N/A SWsoft Inc 13755 Sunrise Valley Drive Suite 325 Herndon, VA 20171 USA Tel: +1 (703) 815 5670 Fax: +1 (703) 815 5675 Copyright © 2005 by SWsoft, Inc. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, V1.0 or later (the latest version is presently available at http://www.opencontent.org/openpub/). Distribution of substantively modified versions of this document is prohibited without the explicit permission of the copyright holder.
Contents Preface 7 About This Guide .........................................................................................................................................7 Who Should Read This Guide ......................................................................................................................7 Organization of This Guide ..........................................................................................................................8 Documentation Conventions.........
Contents 4 Configuring Virtual Private Server..................................................................................................34 Starting, Stopping, Restarting, and Querying Status of Virtual Private Server ..........................................37 Listing Virtual Private Servers....................................................................................................................39 Deleting Virtual Private Server............................................................
Contents Reference 5 80 Configuring OpenVZ..................................................................................................................................81 Matrix of OpenVZ Configuration Files ...........................................................................................81 Managing OpenVZ Scripts..............................................................................................................87 OpenVZ Command Line Interface ....................................
Table of Figures Figure 1: OpenVZ Technology .................................................................................................15 Figure 2: Fedora Core Installation - Choosing System Type.................................................23 Figure 3: Fedora Core Installation - Choosing Manual Partitioning....................................24 Figure 4: Fedora Core Installation - Disk Druid.....................................................................
CHAPTER 1 Preface In This Chapter About This Guide.................................................................................................................. 7 Who Should Read This Guide............................................................................................... 7 Organization of This Guide................................................................................................... 8 Documentation Conventions................................................................
Preface 8 Organization of This Guide Chapter 2, OpenVZ Philosophy, is a must-read chapter that helps you grasp the general principles of OpenVZ operation. It provides an outline of OpenVZ architecture, of the way OpenVZ stores and uses configuration information, of the things you as administrator are supposed to perform, and the common way to perform them.
Preface 9 Typographical Conventions The following kinds of formatting in the text identify special information. Formatting convention Type of Information Example Special Bold Items you must select, such as menu options, command buttons, or items in a list. Go to the QoS tab. Titles of chapters, subsections. Read the Basic Administration chapter. sections, and Italics Used to emphasize the importance of a point or to introduce a term. Such servers are called Hardware Nodes.
Preface 10 General Conventions Be aware of the following conventions used in this book. Chapters in this guide are divided into sections, which, in turn, are subdivided into subsections. For example, Documentation Conventions is a section, and General Conventions is a subsection. When following steps or using examples, be sure to type double-quotes ("), left singlequotes (`), and right single-quotes (') exactly as shown. The key referred to as RETURN is labeled ENTER on some keyboards.
CHAPTER 2 OpenVZ Philosophy In This Chapter About OpenVZ Software ...................................................................................................... 11 Distinctive Features of OpenVZ ........................................................................................... 12 Main Principles of OpenVZ Operation ................................................................................. 15 Hardware Node Availability Considerations ..............................................
OpenVZ Philosophy 12 OpenVZ Applications OpenVZ provides a comprehensive solution for Hosting Service Providers allowing them to: Have hundreds of customers with their individual full-featured virtual private servers (Virtual Private Servers) sharing a single physical server; Provide each customer with a guaranteed Quality of Service; Transparently move customers and their environments between servers, without any manual reconfiguration.
OpenVZ Philosophy 13 OS Virtualization From the point of view of applications and Virtual Private Server users, each VPS is an independent system. This independency is provided by a virtualization layer in the kernel of the host OS. Note that only an infinitesimal part of the CPU resources is spent on virtualization (around 1-2%). The main features of the virtualization layer implemented in OpenVZ are the following: VPS looks like a normal Linux system.
OpenVZ Philosophy 14 Resource Management OpenVZ Resource Management controls the amount of resources available to Virtual Private Servers. The controlled resources include such parameters as CPU power, disk space, a set of memory-related parameters.
OpenVZ Philosophy 15 Main Principles of OpenVZ Operation Basics of OpenVZ Technology user Application Software root Application Software user user Application OpenVZ Software root Application Software root user root user user Application Software Virtual Private Server user user Virtual Private Server user user Virtual Private Server Virtual Private Server root Physical Server (Hardware Node) #3 user Virtual Private Server Virtual Private Server root Physical Server (Hardware Node) #2 V
OpenVZ Philosophy 16 OpenVZ is installed in such a way that you will be able to boot your computer either with OpenVZ support or without it. This support is presented as “OpenVZ” in your boot loader and shown as OpenVZ Layer in the figure above. However, at this point you are not yet able to create Virtual Private Servers.
OpenVZ Philosophy 17 Understanding Templates A template is a VPS building block. An OS template is a set of packages needed to operate a VPS. Templates are usually created right on your Hardware Node; all you need is template tools (vzpkg) and template metadata. Template metadata Template metadata are information about a particular OS template.
OpenVZ Philosophy 18 Understanding Licenses The OpenVZ software consists of the OpenVZ kernel and user-level tools, which are licensed by means of two different open source licenses. The OpenVZ kernel is based on the Linux kernel, distributed under the GPL terms, and is licensed under GNU GPL version 2. The license text can be found at http://openvz.org/documentation/licenses/gnu-gpl. The user-level tools (vzctl, vzquota, and vzpkg) are licensed under the terms of the QPL license.
OpenVZ Philosophy 19 Hardware Node Availability Considerations Hardware Node availability is more critical than the availability of a typical PC server. Since it runs multiple Virtual Private Servers providing a number of critical services, Hardware Node outage might be very costly. Hardware Node outage can be as disastrous as the simultaneous outage of a number of servers running critical services.
CHAPTER 3 Installation and Preliminary Operations The current chapter provides exhaustive information on the process of installing and deploying your OpenVZ system including the pre-requisites and the stages you shall pass. In This Chapter Installation Requirements...................................................................................................... 20 Installing and Configuring Host Operating System on Hardware Node............................... 23 Installing OpenVZ Software ........
Installation and Preliminary Operations 21 Hardware Compatibility The Hardware Node requirements for the standard 32-bit edition of OpenVZ are the following: IBM PC-compatible computer; Intel Celeron, Pentium II, Pentium III, Pentium 4, Xeon, or AMD Athlon CPU; At least 128 MB of RAM; Hard drive(s) with at least 4 GB of free disk space; Network card (either Intel EtherExpress100 (i82557-, i82558- or i82559-based) or 3Com (3c905 or 3c905B or 3c595) or RTL8139-based are recommended).
Installation and Preliminary Operations 22 Network Requirements The network pre-requisites enlisted in this subsection will help you avoid delays and problems with making OpenVZ for Linux up and running.
Installation and Preliminary Operations 23 Installing and Configuring Host Operating System on Hardware Node This section explains how to install Fedora Core 4 on the Hardware Node and how to configure it for OpenVZ. If you are using another distribution, please consult the corresponding installation guides about the installation specifics. Choosing System Type Please follow the instructions from your Installation Guide when installing the OS on your Hardware Node.
Installation and Preliminary Operations 24 Disk Partitioning On the Disk Partitioning Setup screen, select Manual partition with Disk Druid. Do not choose automatic partitioning since this type of partitioning will create a disk layout intended for systems running multiple services. In case of OpenVZ, all your services shall run inside Virtual Private Servers.
Installation and Preliminary Operations 25 It is suggested to use the ext3 file system for the /vz partition. This partition is used for holding all data of the Virtual Private Servers existing on the Hardware Node. Allocate as much disk space as possible to this partition. It is not recommended to use the reiserfs file system as it is proved to be less stable than the ext3, and stability is of paramount importance for OpenVZ-based computers. The root partition will host the operating system files.
Installation and Preliminary Operations 26 Finishing OS Installation After the proper partitioning of your hard drive(s), proceed in accordance with your OS Installation Guide. While on the Network Configuration screen, you should ensure the correctness of the Hardware Node’s IP address, host name, DNS, and default gateway information. If you are using DHCP, make sure that it is properly configured. If necessary, consult your network administrator. On the Firewall Configuration screen, choose No firewall.
Installation and Preliminary Operations 27 Installing OpenVZ Software Downloading and Installing OpenVZ Kernel First of all, you should download the kernel binary RPM from http://openvz.org/download/kernel/. You need only one kernel RPM, so please choose the appropriate kernel binary depending on your hardware: If there is more than one CPU available on your Hardware Node (or a CPU with hyperthreading), select the vzkernel-smp RPM.
Installation and Preliminary Operations 28 # On Hardware Node we generally need # packet forwarding enabled and proxy arp disabled net.ipv4.ip_forward = 1 net.ipv4.conf.default.proxy_arp = 0 # Enables source route verification net.ipv4.conf.all.rp_filter = 1 # Enables the magic-sysrq key kernel.sysrq = 1 # TCP Explict Congestion Notification #net.ipv4.tcp_ecn = 0 # we do not want all our interfaces to send redirects net.ipv4.conf.default.send_redirects = 1 net.ipv4.conf.all.
Installation and Preliminary Operations 29 Downloading and Installing OpenVZ Packages After you have successfully installed and booted the OpenVZ kernel, you can proceed with installing the user-level tools for OpenVZ. You should install the following OpenVZ packages: vzctl: this package is used to perform different tasks on the OpenVZ Virtual Private Servers (create, destroy, start, stop, set parameters etc.). vzquota: this package is used to manage the VPS quotas.
Installation and Preliminary Operations 30 You can also use one of the already pre-cached OS templates available at http://openvz.org/download/template/cache/ for the VPS creation. To this effect, you should download the corresponding OS template and place it to the /vz/template/cache directory on the Node.
CHAPTER 4 Operations on Virtual Private Servers This chapter describes how to perform day-to-day operations on separate Virtual Private Servers taken in their wholeness. Note: We assume that you have successfully installed, configured, and deployed your OpenVZ system. In case you have not, please turn to Chapter 3 providing detailed information on all these operations. In This Chapter Creating and Configuring New Virtual Private Server .........................................................
Operations on Virtual Private Servers 32 Choosing Virtual Private Server ID Every Virtual Private Server has a numeric ID, also known as VPS ID, associated with it. The ID is a 32-bit integer number beginning with zero and unique for a given Hardware Node. When choosing an ID for your Virtual Private Server, please follow the simple guidelines below: ID 0 is used for the Hardware Node itself. You cannot and should not try to create a Virtual Private Server with ID 0.
Operations on Virtual Private Servers 33 Choosing OS Template Next, you shall decide on which OS template you want to base the new VPS. There might be several OS templates installed on the Hardware Node; use the vzpkgls command to find out the templates installed on your system: # vzpkgls fedora-core-3 fedora-core-4 centos-4 Creating Virtual Private Server After the VPS ID and the installed OS template have been chosen, you can create the VPS private area with the vzctl create command.
Operations on Virtual Private Servers 34 In principle, now you are ready to start your newly created Virtual Private Server. However, typically you need to set its network IP address, host name, DNS server address and root password before starting the Virtual Private Server for the first time. Please see the next subsection for information on how to perform these tasks.
Operations on Virtual Private Servers 35 Setting Network Parameters In order to be accessible from the network, a Virtual Private Server shall be assigned a correct IP address and host name; DNS server addresses shall also be configured. The session below illustrates setting the Virtual Private Server 101 network parameters: # vzctl set 101 --hostname test101.my.org --save Hostname for VPS set: test101.my.org Saved parameters for VPS 101 # vzctl set 101 --ipadd 10.0.186.1 --save Adding IP address(es): 10.
Operations on Virtual Private Servers 36 Setting root Password for VPS By default, the root account is locked in a newly created VPS, and you cannot log in. In order to log in to the VPS, it is necessary to create a user account inside the Virtual Private Server and set a password for this account or unlock the root account.
Operations on Virtual Private Servers 37 Starting, Stopping, Restarting, and Querying Status of Virtual Private Server When a Virtual Private Server is created, it may be started up and shut down like an ordinary computer. To start Virtual Private Server 101, use the following command: # vzctl start 101 Starting VPS ... VPS is mounted Adding IP address(es): 10.0.186.101 Hostname for VPS 101 set: test.my.org VPS start in progress...
Operations on Virtual Private Servers 38 vzctl has a two-minute timeout for the VPS shutdown scripts to be executed. If the VPS is not stopped in two minutes, the system forcibly kills all the processes in the Virtual Private Server. The Virtual Private Server will be stopped in any case, even if it is seriously damaged. To avoid waiting for two minutes in case of a Virtual Private Server that is known to be corrupt, you may use the --fast switch: # vzctl stop 101 --fast Stopping VPS ...
Operations on Virtual Private Servers 39 Listing Virtual Private Servers Very often you may want to get an overview of the Virtual Private Servers existing on the given Hardware Node and to get additional information about them - their IP addresses, hostnames, current resource consumption, etc. In the most general case, you may get a list of all VPSs by issuing the following command: # vzlist -a VPSID 101 102 103 NPROC 8 7 - STATUS running running stopped IP_ADDR 10.101.66.1 10.101.66.159 10.101.66.
Operations on Virtual Private Servers 40 Deleting Virtual Private Server You can delete a Virtual Private Server that is not needed anymore with the vzctl destroy VPS_ID command. This command removes the Virtual Private Server private area completely and renames the VPS configuration file and action scripts by appending the .destroyed suffix to them. A running VPS cannot be destroyed with the vzctl destroy command.
Operations on Virtual Private Servers 41 Running Commands in Virtual Private Server Usually, a Virtual Private Server administrator logs in to the VPS via network and executes any commands in the VPS as on any other Linux box. However, you might need to execute commands inside Virtual Private Servers bypassing the normal login sequence.
CHAPTER 5 Managing Templates A template is basically a set of packages from some Linux distribution used to populate a VPS. An OS template consists of system programs, libraries, and scripts needed to boot up and run the system (VPS), as well as some very basic applications and utilities. Applications like a compiler and an SQL server are usually not included into an OS template. In This Chapter Template Lifecycle ........................................................................................
Managing Templates List of packages this template comprises Locations of (network) package repositories Scripts needed to be executed on various stages of template installation Public GPG key(s) needed to check signatures of packages Additional OpenVZ-specific packages 43 In order to operate with a template, you should first create its metadata (available from http://openvz.org/download/template/metadata/).
Managing Templates 44 Listing Templates The vzpkgls utility allows you to list the templates installed on the Hardware Node. They may be already used or not used by certain VPSs: # vzpkgls fedora-core-4 centos-4 As you see, the fedora-core-4 and centos 4 templates are available on the Hardware Node. Note that some of them might not be cached yet.
Managing Templates 45 Working with VPS If you need to update an already existing VPS with the newer packages available from distribution repositories or install some packages that are not part of the template, use the vzyum command, which is a simple yum wrapper. For example, to update the VPS with ID 123, run: # vzyum 123 update This will find, download, and install all the available updates. As you may have noticed, a lot of applications are not installed with an OS template.
CHAPTER 6 Managing Resources The main goal of resource control in OpenVZ is to provide Service Level Management or Quality of Service (QoS) for Virtual Private Servers. Correctly configured resource control settings prevent serious impacts resulting from the resource over-usage (accidental or malicious) of any Virtual Private Server on the other Virtual Private Servers.
Managing Resources System This group of parameters defines various aspects of using system memory, TCP sockets, IP packets and like parameters by different VPSs.
Managing Resources 48 What are Disk Quotas? Disk quotas enable system administrators to control the size of Linux file systems by limiting the amount of disk space and the number of inodes a Virtual Private Server can use. These quotas are known as per-VPS quotas or first-level quotas in OpenVZ. In addition, OpenVZ enables the Virtual Private Sever administrator to limit disk space and the number of inodes that individual users and groups in that VPS can use.
Managing Resources 49 Turning On and Off Per-VPS Disk Quotas The parameter that defines whether to use first-level disk quotas is DISK_QUOTA in the OpenVZ global configuration file (/etc/sysconfig/vz). By setting it to “no”, you will disable OpenVZ quotas completely. This parameter can be specified in the Virtual Private Server configuration file (/etc/sysconfig/vz-scripts/vpsid.conf) as well. In this case its value will take precedence of the one specified in the global configuration file.
Managing Resources 50 Setting Up Per-VPS Disk Quota Parameters Three parameters determine how much disk space and inodes a Virtual Private Server can use. These parameters are specified in the Virtual Private Server configuration file: DISKSPACE Total size of disk space that can be consumed by the Virtual Private Server in 1Kb blocks.
Managing Resources 51 Turning On and Off Second-Level Quotas for Virtual Private Server The parameter that controls the second-level disk quotas is QUOTAUGIDLIMIT in the VPS configuration file. By default, the value of this parameter is zero and this corresponds to disabled per-user/group quotas.
Managing Resources 52 Setting Up Second-Level Disk Quota Parameters In order to work with disk quotas inside a VPS, you should have standard quota tools installed: # vzctl exec 101 rpm -q quota quota-3.12-5 This command shows that the quota package is installed into the Virtual Private Server. Use the utilities from this package (as is prescribed in your Linux manual) to set OpenVZ secondlevel quotas for the given VPS.
Managing Resources 53 Checking Quota Status As the Hardware Node system administrator, you can check the quota status for any Virtual Private Server with the vzquota stat and vzquota show commands. The first command reports the status from the kernel and shall be used for running Virtual Private Servers. The second command reports the status from the quota file (located at /var/vzquota/quota.vpsid) and shall be used for stopped Virtual Private Servers. Both commands have the same output format.
Managing Resources cpuunits This is a positive integer number that determines the minimal guaranteed V share of the CPU time the corresponding Virtual Private Server will receive. cpulimit This is a positive number indicating the CPU time in per cent the V corresponding VPS is not allowed to exceed. 54 The OpenVZ CPU resource control utilities allow you to guarantee any Virtual Private Server the amount of CPU time this Virtual Private Server receives.
Managing Resources 55 Managing System Parameters The resources a Virtual Private Server may allocate are defined by the system resource control parameters. These parameters can be subdivided into the following categories: primary, secondary, and auxiliary parameters. The primary parameters are the start point for creating a Virtual Private Server configuration from scratch. The secondary parameters are dependent on the primary ones and are calculated from them according to a set of constraints.
Managing Resources tcprcvbuf The total size of receive buffers for TCP sockets, i.e. the amount of V kernel memory allocated for the data received from the remote side, but not read by the local application yet. othersockbuf The total size of UNIX-domain socket buffers, UDP, and other V datagram protocol send buffers. dgramrcvbuf The total size of receive buffers of UDP and other datagram protocols. V oomguarpages The out-of-memory guarantee, in pages (one page is 4 Kb).
Managing Resources 57 Monitoring System Resources Consumption It is possible to check the system resource control parameters statistics from within a Virtual Private Server. The primary use of these statistics is to understand what particular resource has limits preventing an application to start. Moreover, these statistics report the current and maximal resources consumption for the running Virtual Private Server. This information can be obtained from the /proc/user_beancounters file.
Managing Resources # for res in tcpsendbuf tcprcvbuf unixsockbuf sockrcvbuf kmemsize; \ do echo;echo "$res usage for all VEs, in MB:";cat \ /proc/user_beancounters |grep $res|sed "s/[[:digit:]]\+://g" \ |awk 'BEGIN{ cur=max=lim=0; } { cur+=$2; max+=$3;lim+=$5 } \ END {print "held:",cur/1024/1024, "max:", max/1024/1024, \ "limit:", lim/1024/1024}'; done 58
Managing Resources 59 Monitoring Memory Consumption You can monitor a number of memory parameters for the whole Hardware Node and for particular Virtual Private Servers with the help of the vzmemcheck utility. For example: # vzmemcheck -v Output values in % veid LowMem LowMem RAM MemSwap MemSwap Alloc Alloc Alloc util commit util util commit util commit limit 101 0.19 1.93 1.23 0.34 1.38 0.42 1.38 4.94 1 0.27 8.69 1.94 0.49 7.19 1.59 2.05 56.
Managing Resources 60 The union of RAM and swap space is the main computer resource determining the amount of memory available to applications. If the total size of memory used by applications exceeds the RAM size, the Linux kernel moves some data to swap and loads it back when the application needs it. More frequently used data tends to stay in RAM, less frequently used data spends more time in swap. Swap-in and swap-out activity reduces the system performance to some extent.
Managing Resources 61 2 Using OpenVZ specialized utilities for preparing configuration files in their entirety. The tasks these utilities perform are described in the following subsections of this section. 3 The direct creating and editing of the corresponding configuration file (/etc/sysconfig/vz-scripts/VPS_ID.conf). This can be performed either with the help of any text editor. The instructions on how to edit VPS configuration files directly are provided in the four preceding sections.
Managing Resources 62 Validating Virtual Private Server Configuration The system resource control parameters have complex interdependencies. Violation of these interdependencies can be catastrophic for the Virtual Private Server. In order to ensure that a Virtual Private Server does not break them, it is important to validate the VPS configuration file before creating VPSs on its basis. The typical validation scenario is shown below: # vzcfgvalidate /etc/sysconfig/vz-scripts/101.conf Error: kmemsize.
CHAPTER 7 Advanced Tasks In This Chapter Determining VPS ID by Process ID...................................................................................... 64 Changing System Time from VPS ........................................................................................ 64 Obtaining Hardware Node ID from Inside Virtual Private Server........................................ 65 Accessing Devices from Inside Virtual Private Server .........................................................
Advanced Tasks 64 Determining VPS ID by Process ID Each process is identified by a unique PID (process identifier), which is the entry of that process in the kernel's process table. For example, when you start Apache, it is assigned a process ID. This PID is then used to monitor and control this program.The PID is always a positive integer. In OpenVZ you can use the vzpid (retrieve process ID) utility to print the Virtual Private Server ID the process with the given id belongs to.
Advanced Tasks 65 root@vps101's password: Last login: Mon Oct 28 23:25:58 2002 from 10.100.40.18 [root@vps101 root]# date Mon Oct 28 23:31:57 EST 2002 [root@vps101 root]# date 10291300 Tue Oct 29 13:00:00 EST 2002 [root@vps101 root]# date Tue Oct 29 13:00:02 EST 2002 [root@vps101 root]# logout Connection to ve101 closed. # date Tue Oct 29 13:01:31 EST 2002 The command session above shows the way to change the system time from Virtual Private Server 101.
Advanced Tasks 66 Accessing Devices from Inside Virtual Private Server It is possible to grant a Virtual Private Server read, write, or read/write access to a character or block device. This might be necessary, for example, for Oracle database software if you want to employ its ability to work with raw disk partitions. In most cases, providing access to the file system hierarchy for a Virtual Private Server is achieved by using bind mounts.
Advanced Tasks 67 p primary partition (1-4) p Partition number (1-4): 2 First cylinder (256-2231, default 256): Using default value 256 Last cylinder or +size or +sizeM or +sizeK \ (256-2231, default 2231): +100M Command (m for help): p Disk /dev/sdb: 255 heads, 63 sectors, 2231 cylinders Units = cylinders of 16065 * 512 bytes Device Boot /dev/sdb1 * /dev/sdb2 Start 1 256 End 255 268 Blocks 2048256 104422+ Id 83 83 System Linux Linux Command (m for help): w After the new partition table has been wr
Advanced Tasks 68 Moving Network Adapter to Virtual Private Server By default, all the VPSs on a Node are connected among themselves and with the Node by means of a virtual network adapter called venet0. Still, there is a possibility for a VPS to directly access a physical network adapter (for example, eth1). In this case the adapter becomes inaccessible to the Hardware Node itself.
Advanced Tasks 69 Enabling VPN for VPS Virtual Private Network (VPN) is a technology which allows you to establish a secure network connection even over an insecure public network. Setting up a VPN for a separate VPS is possible via the TUN/TAP device.
Advanced Tasks 70 Loading iptables Modules to Hardware Node To have certain iptables modules loaded on the Hardware Node startup, you should provide their names as the value of the IPTABLES_MODULES parameter in the /etc/sysconfig/iptables-config file.
Advanced Tasks 71 Rebooting Virtual Private Server When you issue the reboot command at your Linux box console, the command makes the reboot system call with argument ‘restart’, which is passed to the computer BIOS. The Linux kernel then reboots the computer. For obvious reasons this system call is blocked inside Virtual Private Servers: no Virtual Private Server can access BIOS directly; otherwise, a reboot inside a VPS would reboot the whole Hardware Node.
CHAPTER 8 Troubleshooting This chapter provides the information about those problems that may occur during your work with OpenVZ and suggests the ways to solve them. In This Chapter General Considerations ......................................................................................................... 73 Kernel Troubleshooting ........................................................................................................ 75 Problems With VPS Management ................................
Troubleshooting 73 General Considerations The general issues to take into consideration when troubleshooting your OpenVZ system are listed below. You should read them carefully before trying to solve more specific problems. You should always remember where you are located now in your terminal. Check it periodically using the pwd, hostname, ifconfig, cat /proc/vz/veinfo commands.
Troubleshooting 74 Nmap run completed -- 1 IP address (1 host up) scanned in 169 seconds to check if any ports are open that should normally be closed. That could however be a problem to remove a rootkit from a VPS and make sure it is 100% removed. If you're not sure, create a new VPS for that customer and migrate her data there. Check the /var/log/ directory on the Hardware Node to find out what is happening on the system.
Troubleshooting 75 Kernel Troubleshooting Using ALT+SYSRQ Keyboard Sequences Press ALT+SYSRQ+H (3 keys simultaneously) and check what's printed at the HN console, for example: SysRq : HELP : loglevel0-8 reBoot tErm Full kIll saK showMem Nice powerOff showPc unRaw Sync showTasks Unmount This output shows you what ALT+SYSRQ sequences you may use for performing this or that command. The capital letters in the command names identify the sequence.
Troubleshooting 76 Saving Kernel Fault (OOPS) You can use the following command to check for the kernel messages that should be decoded and reported to OpenVZ developers: grep -E "Call Trace|Code" /var/log/messages* Then you should find these lines in the correspondent log file and figure out what kernel was booted when the oops occurred. Search backward for the "Linux" string, look for strings like that: May 23 16:55:00 ts13 Linux version 2.6.8-022stab026.1 (root@kern26x.build.sw.ru) (gcc version 3.3.
Troubleshooting 77 Finding Kernel Function That Caused D Process State If there are too many processes in the D state and you can't find out what is happening, issue the following command: # objdump -Dr /boot/vmlinux-`uname -r` >/tmp/kernel.
Troubleshooting 78 Failure to Start VPS An attempt to start a Virtual Private Server fails. Solution 1 If there is a message on the system console: parameters missing, and the list of missed parameters follows the message, set these parameters using the vzctl set --save command (see Configuring Virtual Private Server on page 34 for instructions). Try to start the VPS once again. Solution 2 If there is a message on the system console: Address already in use, issue the cat /proc/vz/veinfo command.
Troubleshooting 79 Solution 2 Make sure the routing to the Virtual Private Server is properly configured. Virtual Private Servers can use the default router for your network, or you may configure the Hardware Node as rooter for its VPSs. Failure to Log In to VPS The Virtual Private Server starts successfully, but you cannot log in. Solution 1 You are trying to connect via SSH, but access is denied. Probably you have not set the password of the root user yet or there is no such user.
CHAPTER 9 Reference In order to make OpenVZ successfully accomplish its tasks you need to understand how to configure OpenVZ correctly. This section explains what configuration parameters OpenVZ has and how they affect its behavior. In This Chapter Configuring OpenVZ ............................................................................................................ 81 OpenVZ Command Line Interface........................................................................................
Configuring OpenVZ In order to make OpenVZ successfully accomplish its tasks you need to understand how to configure OpenVZ correctly. This chapter explains what configuration parameters OpenVZ has and how they affect its behavior. Matrix of OpenVZ Configuration Files There are a number of files responsible for the OpenVZ system configuration. These files are located in the /etc directory on the Hardware Node; a list of these files is given below: /etc/sysconfig/vz OpenVZ global configuration file.
Reference 82 Global OpenVZ Configuration File OpenVZ keeps its system wide configuration parameters in the /etc/sysconfig/vz configuration file. This file is in shell format. Keep in mind that OpenVZ scripts source this file – thus, shell commands in this file will cause system to execute them under root account. Parameters in this file are presented in the form PARAMETER=”value”.
Reference 83 Disk quota parameters allow you to control the disk usage by the Virtual Private Servers: Parameter Description Default value DISK_QUOTA DISK_QUOTA defines whether to turn on disk quota for yes Virtual Private Servers. If set to “no” then disk space and inodes accounting will be disabled. VZFASTBOOT If set to "no", disk quota is reinitialized for each VPS when no the Hardware Node is booted after an incorrect shutdown, which results in a very long booting time.
Reference 84 Supplementary parameters define other OpenVZ settings: Parameter Description Default value VZWDOG Defines whether the vzwdog module is loaded on OpenVZ no startup. This module is responsible for catching messages from the kernel. It is needed in case you configure the serial Monitor Node for OpenVZ. VPS Configuration File Each Virtual Private Server has its own configuration file, which is stored in the /etc/sysconfig/vz-scripts directory and has a name like vpsid.conf.
Reference 85 Resource management parameters control the amount of resources a VPS can consume. They are described in the Managing Resources chapter in detail; here is only a list of parameters allowed in VPS configuration file. All resource management parameters can be subdivided into the general, disk, and system categories for your convenience. Any parameter can be set with the vzctl set command and the corresponding option name (in the lower case, e.g. --kmemsize for KMEMSIZE, etc.).
Reference 86 AVNUMPROC Number of processes expected to run in the Virtual 0…NUMPROC Private Server on average. This is informational parameter used by utilities like vzcfgvalidate in order to ensure configuration correctness. NUMTCPSOCK Number of TCP sockets (PF_INET family, 40…500 SOCK_STREAM type). This parameter limits the number of TCP connections and, thus, the number of clients the server application can handle in parallel. NUMOTHERSOCK Number of socket other than TCP.
Reference 87 NUMFILE Number of files opened by all VPS processes. 512…8192 NUMFLOCK Number of file locks created by all VPS processes. 50…200 – 60…220 NUMPTY Number of pseudo-terminals. For example, ssh 4…64 session, screen, xterm application consumes pseudo-terminal resource. NUMSIGINFO Number of siginfo structures (essentially this 256…512 parameter limits size of signal delivery queue). DCACHESIZE Total size of dentry and inode structures locked 184320…3932160in memory.
Reference 88 Overview Along with OpenVZ configuration files responsible for the OpenVZ system configuration, there are a number of OpenVZ scripts allowing you to customize the VPS behaviour in different ways. These are the following scripts: Script Name Description /etc/sysconfig/vzscripts/VPS_ID.action VPS private action scripts. These scripts allow to run userdefined actions on particular events. Currently defined actions are start, stop, mount, umount.
Reference 89 The normal order of executing action scripts is shown in the figure below. The mount and umount scripts run in the context of the Host OS rather than in the Virtual Private Server.
Reference 90 Action scripts are located in the same directory as VPS configuration files and have names like vpsid.action. The global scripts are named vps.mount and vps.umount, and the other scripts have the corresponding VPS ID as part of their name. As the names of the global scripts are fixed, they are called when any VPS is started or stopped.
OpenVZ Command Line Interface OpenVZ is shipped with a number of command line tools. This chapter documents the utilities, which are supported in OpenVZ. For every utility, all available command-line options and switches are described. Matrix of OpenVZ Command Line Utilities The table below contains the full list of OpenVZ command-line utilities. General utilities are intended for performing day-to-day maintenance tasks: vzctl Utility to control Virtual Private Servers.
Reference 92 vzctl vzctl is the primary tool for Virtual Private Server management. To use it, you have to log in to the Hardware Node as the root user.
Reference 93 vzctl create This command is used to create a new Virtual Private Server. It has the following syntax: vzctl create vpsid [--ostemplate name] [--config name] [--private path] [--root path] With this command, you can create Virtual Private Servers. Virtual Private Server ID vpsid is required for this command and shall be unique for the Hardware Node. Note: Virtual Private Server IDs from 1 to 100 are reserved for internal OpenVZ needs.
Reference 94 vzctl destroy The syntax of this command is: vzctl destroy vpsid This command is used to delete a Virtual Private Server, which is no longer needed. It physically removes all the files located in VPS private area (specified as VE_PRIVATE variable in the VPS configuration file) and renames the VPS configuration file in /etc/sysconfig/vz-scripts/ from vpsid.conf to vpsid.conf.destroyed. It also renames VPS action scripts if any in a similar manner.
Reference 95 You should use action scripts (mount/umount and start/stop) if you would like to carry out some actions upon VPS startup/shutdown. The vzctl restart vpsid command consecutively performs the stopping and starting of the corresponding VPS. The vzctl status vpsid command shows current VPS state.
Reference --noatime yes|no Sets the noatime flag (do not update inode access times) on the VPS file system. The default is yes for a Class 1 VPS, and no otherwise. --devnodes device:r|w|rw|none Lets the VPS access the specified devices in the specified mode - read-only, write-only, or read-write or denies any access. E.g.: --devnodes hda1:rw The device must be present in the VPS /dev directory, otherwise, a new device is automatically created.
Reference 97 Resource management settings control the amount of resources a VPS can consume. If the setting has bar:lim after it than this setting requires specifying both barrier and limit values separated by colons. --applyconfig name This option lets you set the resource parameters for the VPS not one by one, but by reading them from the VPS sample configuration file.
Reference --othersockbuf bar:lim Total size in bytes of UNIX-domain socket buffers, UDP and other datagram protocol send buffers. --dgramrcvbuf bar:lim Total size in bytes of receive buffers of UDP and other datagram protocols. --oomguarpages bar:lim Out-of-memory guarantee, in 4 Kb pages. Any VPS process will not be killed even in case of heavy memory shortage if the current memory consumption (including both physical memory and swap) does not reach the oomguarpages barrier.
Reference --diskspace bar:lim Total size of disk space consumed by VPS, in 1 Kb blocks. When the space used by a Virtual Private Server hits the barrier, the VPS can allocate additional disk space up to the limit during grace period specified by the --quotatime setting. --diskinodes bar:lim Total number of disk inodes (files, directories, symbolic links) a Virtual Private Server can allocate.
Reference 100 vzctl exec, vzctl exec2, and vzctl enter These commands are used to run arbitrary commands inside a Virtual Private Server being authenticated as root on the Hardware Node. The syntax of these commands is as follows: vzctl exec|exec2 vpsid command vzctl enter vpsid where command is a string to be executed in the Virtual Private Server. If command is specified as “-” then the commands for execution will be read from the standard input until the end of file or “exit” is encountered.
Reference 101 -o parameter[.specifier] This option is used to display only particular information about the VPSs. The parameters and their specifiers that can be used after the -o option are listed in the following subsection. To display a number of parameters in a single output, they should be separated with commas, as is shown in the synopsis above. -s, --sort Sort the VPSs in the list by the specified parameter. If "-" is given before the name of the parameter, the sorting order is reversed.
Reference kmemsize .m, .b, KMEMSIZE Size of unswappable kernel memory (in bytes), allocated for internal kernel structures of the processes of a particular VPS. Typical amounts of kernel memory are 16…50 Kb per process. LOCKEDP Memory not allowed to be swapped out (locked with the mlock() system call), in 4Kb pages. PRIVVMP Size in 4 Kb pages of private (or potentially private) memory, allocated by Virtual Private Server applications.
Reference numpty .m, .b, NPTY Number of pseudo-terminals. For example, ssh session, screen, xterm application consumes pseudo-terminal resource. NSIGINFO Number of siginfo structures (essentially this parameter limits size of signal delivery queue). TCPSNDB Total size (in bytes) of send buffers for TCP sockets – amount of kernel memory allocated for data sent from an application to a TCP socket, but not acknowledged by the remote side yet.
Reference 104 cpulimit none CPULIM This is a positive number indicating the CPU time in per cent the corresponding VPS is not allowed to exceed. cpuunits none CPUUNI Allowed CPU power. This is a positive integer number, which determines the minimal guaranteed share of the CPU the Virtual Private Server will receive. You may estimate this share as ((VPS CPUUNITS)/(Sum of CPU UNITS across all busy Virtual Private Servers))*100%.
Reference 105 OpenVZ quota works on a file system sub-tree or area. If this area has additional file systems mounted to its subdirectories quota will not follow this mount points. When you initialize quota, you specify the file system sub-tree starting point for the quota. Quota keeps its current usage and settings for a Virtual Private Server in the /var/vzquota/quota.vpsid file. Quota file has a special flag, which indicates whether the file is “dirty”.
Reference -e, --block-exptime time 106 Required. Expiration time for excess of the block soft limit. Time can be specified in two formats: dd:hh:mm:ss For example: 30 - 30 seconds; 12:00 12 minutes; 20:15:11:00 - 20 days, 15 hours, 11 minutes xxA, where A - h/H(hour); d/D(day); w/W(week); m/M(month); y/Y(year). For instance: 7D - 7 days; 01w - 1 week; 3m – 3 months -i, --inode-softlimit num Required. Inodes soft limit – amount of inodes allowed for the Virtual Private Server to create.
Reference 107 vzquota on and vzquota off These commands are used to turn quota on and off. Their syntax is as follows: vzquota [options] on vpsid [command-options] vzquota [options] off vpsid [-f] [-c quota_file] vzquota off turns the quota off for the file system tree specified in quota file given with an optional –c switch. If this switch is omitted, the default /var/vzquota/quota.vpsid file is used.
Reference -b, -B, -e, -i, -I, -n, --block-softlimit num --block-hardlimit num --block-exptime time --inode-softlimit num --inode-hardlimit num --inode-exptime time -c quota_file 108 These options are optional for the vzquota on command. However, at least one of these options or -u, --ugidlimit num must be specified. These options are described in the vzquota init subsection. Optional. Specifies the file to write output of counted disk space and inodes as well as limits.
Reference 109 vzquota stat and vzquota show These commands are used for querying quota statistics. The syntax is as below: vzquota [options] show vpsid [-t] [-f] [-c quota_file] vzquota [options] stat vpsid [-t] [-c quota_file] The difference between the vzquota stat and vzquota show commands is that the first one reports usage from the kernel while the second one reports usage as written in the quota file. However, by default vzquota stat updates the file with the last kernel statistics.
Reference 110 Template Management Utilities A template is basically a set of packages from some Linux distribution used to populate a VPS. An OS template consists of system programs, libraries, and scripts needed to boot up and run the system (VPS), as well as some very basic applications and utilities. Applications like a compiler and an SQL server are usually not included into an OS template. A template cache is an OS template installed into a VPS and then packed into a gzipped tar archive.
Reference 111 vzpkgcache This utility creates/updates template caches for OS templates. You should run this utility before you can use a newly installed OS template for creating Virtual Private Servers. It has the following syntax: vzpkgcache [-f|--force] [osname ...] vzpkgcache -r|--remove osname [...
Reference 112 Supplementary Tools vzdqcheck This utility counts inodes and disk space used using the same algorithm as OpenVZ quota. It has the following syntax: vzdqcheck [options] path The command traverses directory tree given as the path argument and calculates space occupied by all files and number of inodes. The command does not follow mount points. Options available to the vzdqcheck command are: -h Usage info. -V Vzquota version info. -v Verbose mode. -q Quiet mode.
Reference 113 Quotas must be turned off when the vzdqload utility is working. Mind that only 2nd-level disk quotas are handled by the utilities. vzcpucheck This utility displays the current Hardware Node utilization in terms of allocated CPU units as well as total hardware node CPU units capacity. It has the following syntax: vzcpucheck [-v] Without arguments, the utility prints the sum of CPU units of all running Virtual Private Servers and total Hardware Node capacity.
Reference 114 vzsplit This utility is used to generate a sample VPS configuration file with a set of system resource control parameters. The syntax of this command is as follows: vzsplit [-n num] [-f sample_name] [-s swap_size] This utility is used for dividing Hardware Node into equal parts.
Glossary 115 Glossary Hardware Node (or Node) is a computer where OpenVZ is installed for hosting Virtual Private Servers. HN is an abbreviation of Hardware Node. Host Operating System (or Host OS) is an operating system installed on the Hardware Node. MAC address stands for Media Access Control address, a hardware address that uniquely identifies each Node in a network. The MAC layer interfaces directly with the network media.
Glossary 116 VPS is an abbreviation of Virtual Private Server. VENET device is a virtual networking device, a gateway from a VPS to the external network. Virtual Private Server (or VPS) is a virtual private server, which is functionally identical to an isolated standalone server, with its own IP addresses, processes, files, its own users database, its own configuration files, its own applications, system libraries, and so on. Virtual Private Servers share one Hardware Node and one OS kernel.
Index A About OpenVZ Software • 11 About This Guide • 7 Accessing Devices from Inside Virtual Private Server • 66 Advanced Tasks • 63 Failure to Start VPS • 78 Feedback • 10 Finding Kernel Function That Caused D Process State • 77 Finishing OS Installation • 26 G Basics of OpenVZ Technology • 15 Before you Begin • 31 General Considerations • 73 General Conventions • 10 Global OpenVZ Configuration File • 82 Glossary • 115 C H Changing System Time from VPS • 64 Checking Quota Status • 53 Choosing
Index Matrix of OpenVZ Command Line Utilities • 91 Matrix of OpenVZ Configuration Files • 81 Monitoring Memory Consumption • 59 Monitoring System Resources Consumption • 57 Moving Network Adapter to Virtual Private Server • 68 N Network Requirements • 22 Network Virtualization • 13 O OpenVZ • 13 OpenVZ Action Scripts • 88 OpenVZ Applications • 12 OpenVZ Command Line Interface • 91 OpenVZ Configuration • 18 OpenVZ Philosophy • 11 Operations on Virtual Private Servers • 31 Organization of This Guide • 8 OS
Index W What are Disk Quotas? • 48 What are Resource Control Parameters? • 46 What is OpenVZ • 11 Who Should Read This Guide • 7 119
