CIFS/9000 Server File Locking Interoperability Version 1.03 September, 2001 Updates for version 1.03: * Chapters 5, 6, 7, 8 – CIFS/9000 Open Mode locking added * Appendix B – CIFS/9000 Open Mode locking added Eric Roseme SNSL Advanced Technology Center E0300 Printed in: U.S.A.
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material. Warranty.
Contents Chapter 1 Introduction ......................................................................................................4 Chapter 2 CIFS/9000 Product Overview.............................................................................5 Chapter 3 File Locking Overview.......................................................................................6 Chapter 4 File Locking Implementations ...........................................................................8 4.1 WINDOWS (CIFS) ......
Chapter 1 Introduction A key feature of HP’s CIFS/9000 product is file server consolidation on a UNIX platform with transparent shared file access between Windows and UNIX clients. A major challenge of shared file access is providing secure, fast access for Windows, UNIX, and mixed client environments.
Chapter 2 CIFS/9000 Product Overview CIFS is a file system specification called Common Internet File System, which evolved from Microsoft’s SMB protocol for Windows operating systems. CIFS/9000 is an HP-UX product (based upon Samba version 2.0.6, 2.0.7 in 3/2001, or 2.0.9 in 9/2001) that utilizes the Common Internet File System to provide two major benefits for two platforms.
Chapter 3 File Locking Overview File locking is typically initiated by the file system – either by configuration parameters or API and function calls – to prevent data corruption by more than one process accessing a file while it is open in write mode (either dual writes occurring, or reading a file that has since been changed due to a write – “stale data”). There are two important factors to consider when enabling file locking: first – when to use file locking; second – how to lock the file.
Educating customers about when file locking is required will reduce the possibility of this feature being a lockout for bidding a deal 2. How a file is locked by a file system depends upon the locking features that are invoked (Windows has multiple locking modes), or what platform the file resides upon (Windows or UNIX), or what access method the client is using to open the file.
Chapter 4 File Locking Implementations File systems and pseudo file systems (like PC-NFS) implement autonomous file locking models. These file-locking models are invoked by the particular file system, and do not necessarily interoperate, or even know each other exist. Thus, when multiple client access methods open the same file – even with valid locking procedures for the opening file system – the locks can be ignored, resulting in concurrent file access and likely data corruption.
BYTE RANGE LOCKING is invoked programmatically – on a file that is already open - by the Windows LockFile and LockFileEx APIs when a client application wants to lock a region of a file for exclusive access. LockFile parameters only specify the file name and byte ranges to be locked. Therefore the lock must be exclusive – all other processes are denied access to the specified region of the file. If the region cannot be locked, the API returns immediately – it does not block or wait.
An important detail is that Oplocks are invoked by the file system, not an application API. Therefore, an application can close an oplocked file, but the file system does not relinquish the Oplock. When the Oplock break is issued, the file system then simply closes the file in preparation for the subsequent open by the second process. 4.2 UNIX UNIX utilizes 2 kinds of file locking: Advisory locking Mandatory locking Both lock types are implemented with the same function calls: fcntl and lockf.
NETWORK LOCK MANAGER - like UNIX - uses an advisory locking protocol: an NFS server can obliviously grant a client read and/or write access to a file that has already been locked by another client. The NLM assumes that all processes attempting to access a file will follow locking protocol by calling the locking functions correctly and honoring the lock status of the file. The NLM honors read and/or write locking, and allows for byte-range locks or locking of the entire file.
Chapter 5 CIFS/9000 File Locking Implementation CIFS/9000 mirrors the Windows file locking mechanisms, using the CIFS (SMB) protocol to provide Windows-equivalen t file locking on a UNIX system for Windows clients (homogenous client access). For Windows client access to files residing on the local UNIX file system, CIFS/9000 utilizes: Mandatory Share Mode Locking (Open Mode), Byte-Range Locking (programmatic), Opportunistic Locking (Oplocks).
Mandatory Share Mode (Open Mode) locking applies to Windows clients only. File sharing issues arise when concurrent file access occurs between Windows clients and UNIX clients, PC-NFS clients, or on files that have been NFS-mounted. UNIX processes have no concept of Mandatory Share Mode locks.
benefit is improved security, but in most cases the additional checks will be redundant to the application checks, and the cost will be reduced performance due to the extra overhead of checking for locks on every read and write. CIFS/9000 enables strict locking on a per-share basis in the smb.conf file: [share_name] strict locking = yes The default is “no”. Blocking locks enables the smbd to recognize a timeout period specified on a LockFileEx call.
UNIX/NFS process or PC-NFS application can open the same file for write access without an Oplock Break being sent to the client, resulting in unacceptable risk of data corruption. Even a UNIX/NFS process that adheres to the advisory locking protocol will not send an oplock break. Disabling Oplocks removes the risk of this particular data corruption, but also removes the performance benefit that Oplock file caching provides.
Chapter 6 CIFS/9000 File Locking Interoperability Examples Because the primary benefit of CIFS/9000 is to provide Windows client access to files residing on a UNIX operating system, shared client access examples all include at least one Windows client. The following examples will explain how file locking behaves when more than one client accesses a file via a CIFS/9000 share, both locally and NFS mounted. 6.
Mandatory, Byte Range, and Opportunistic locking are all enabled by default, and should always be enabled for a Windows-only client access environment. They may be explicitly configured on a per-share basis by editing the smb.
6.2 WINDOWS ONLY CLIENT ACCESS – NFS Mounted File System Clients Windows CIFS/9000 File UNIX PC-NFS CIFS/9000 Windows NFS The diagram above shows 2 Windows clients requesting concurrent file access on the CIFS/9000 server to a NFS mounted file system.
Mandatory, Byte Range, and Opportunistic locking are all enabled by default, and should always be enabled for a Windows-only client access environment. They may be explicitly configured on a per-share basis by editing the smb.
6.3 WINDOWS AND UNIX CLIENT ACCESS – Local File System Clients Windows CIFS/9000 File UNIX PC-NFS CIFS/9000 Windows NFS The diagram above shows a Windows client and a UNIX client requesting concurrent file access on the CIFS/9000 server to a local file system. The key issue for this configuration is understanding when the client platform locking schemes can or cannot interoperate. A UNIX process is not aware of Windows Mandatory Share Mode (Open Mode) locking.
(mixed-mode access) does not remove the element of risk from data corruption because the locking interoperability is not comprehensive system-wide 5. Byte Range locking is implemented for both Windows and UNIX access with UNIX advisory byte range locking via the fcntl function. Since UNIX byte range locking is advisory, a UNIX process must be properly coded to participate in the locking protocol. The CIFS/9000 smbd process actually calls the UNIX fcntl function to implement Windows byte range locking.
6.4 WINDOWS AND UNIX CLIENT ACCESS – NFS Mounted File System Clients Windows CIFS/9000 File UNIX PC-NFS CIFS/9000 Windows NFS The diagram above shows a Windows client and a UNIX client requesting concurrent file access on the CIFS/9000 server to a NFS mounted file system. The key issue for this configuration is understanding when the client platform locking schemes can or cannot interoperate.
those same UNIX/NFS processes (mixed-mode access) does not remove the element of risk from data corruption because the locking interoperability is not comprehensive system-wide 6. Byte Range locking is implemented for concurrent Windows and UNIX access with UNIX advisory byte range locking via the fcntl function, and NFS propagates the locks over the NFS mount to the disk file.
6.5 WINDOWS AND UNIX/NFS CLIENT ACCESS – Local and NFS Mounted File Systems Clients Windows CIFS/9000 File UNIX/NFS PC-NFS CIFS/9000 Windows NFS The diagram above shows a Windows client and a UNIX/NFS client requesting concurrent file access on the CIFS/9000 server. The key issue for this configuration is understanding when the client platform locking schemes can or cannot interoperate. A UNIX/NFS process is not aware of Windows Mandatory Share Mode (Open Mode) locking.
(mixed-mode access) does not remove the element of risk from data corruption because the locking interoperability is not comprehensive system-wide 7. Byte Range locking is implemented for both Windows and UNIX/NFS access with UNIX advisory byte range locking via the fcntl function. Since UNIX/NFS byte range locking is advisory, a UNIX process must be properly coded to participate in the locking protocol.
6.6 WINDOWS AND PC-NFS CLIENT ACCESS – Local and NFS Mounted File Systems Clients Windows CIFS/9000 File UNIX PC-NFS CIFS/9000 Windows NFS The diagram above shows a Windows client and a PC-NFS client requesting concurrent file access on the CIFS/9000 server to a local and/or NFS mounted file system.
Windows byte range locks will interoperate with other Windows clients using byte range locks or with UNIX processes that are properly coded to participate in the byte range locking protocol. When both processes correctly participate in the advisory locking protocol, byte range locking is fully effective in a mixed Windows, PC-NFS, and UNIX client environment. Byte range locking should remain enabled8.
6.7 WINDOWS AND CIFS/9000 CLIENT – Local and NFS Mounted File Systems Clients Windows CIFS/9000 File UNIX PC-NFS CIFS/9000 Windows NFS The diagram above shows a Windows client and a CIFS/9000 client requesting concurrent file access on the CIFS/9000 server to a local file system and a NFS mounted file system. The key issue for this configuration is that a CIFS/9000 client has no file locking capability.
locking = yes oplocks = no 29
Chapter 7 CIFS/9000 LOCKING SUMMARY The following table summarizes CIFS/9000 file locking: Mandatory Share Mode (Open Mode) Lock Windows Lock UNIX – no advisory locking Lock UNIX – advisory locking Lock UNIX/NFS – no advisory locking Lock UNIX/NFS – advisory locking Lock PC-NFS Byte Range Locking Lock Windows Lock UNIX – no advisory locking Lock UNIX – advisory locking Lock UNIX/NFS – no advisory locking Lock UNIX/NFS – advisory locking Lock PC-NFS Oplocks Break Oplock for Windows access Break Oplock for
Chapter 8 CIFS/9000 COMPETITION LOCKING SUMMARY CIFS/9000 competitors are comprised of dedicated NAS (Network Attached Storage) file servers and general purpose UNIX/NT integration servers. Competitor file locking data has been gathered from the documentation of the competitor’s products and summarized here. In a future revision of this paper, actual competitor locking test results will be published.
8.1 Network Appliance Network Appliance markets very specific NAS servers that they call “Filers”. Filers are single-purpose front-end “thin servers” used exclusively for storage access. NetApps employs a feature called the “SecureShare” cross-protocol Lock Manager. NetApps literature claims that SecureShare provides comprehensive multi-platform (heterogeneous) file locking capability. NetApps servers are single-purpose file servers running a proprietary OS called Data Ontap.
NetApps SecureShare provides Mandatory Share Mode (open mode) locking for concurrent access with PC-NFS and UNIX/NFS. CIFS/9000 provides this locking feature at the HP-UX September 2001 Application Release. NetApps SecureShare provides byte range locking interoperability even for UNIX/NFS processes that do not participate in the advisory locking protocol. CIFS/9000 has a planned enhancement scheduled to provide the same locking feature.
8.2 EMC Celerra EMC markets its Celerra file server as a NAS front-end to a Symetrix disk array. Celerra is a single-purpose front-end server used exclusively for storage access to Symetrix. It runs a proprietary OS called DART that is incapable of running applications, users, or accessing and interoperating with non -EMC storage devices. Since Celerra supports no users or applications, UNIX-Windows concurrent file access is not an issue.
EMC’s Celerra documentation claims support of oplocks, but makes no mention of UNIX/NFS interoperability. They recommend turning oplocks off when “handling critical data with the need to avoid even slight data loss”, which implies that they do not interoperate with UNIX/NFS. CIFS/9000 has a planned enhancement scheduled to provide UNIX, UNIX/NFS, and PC-NFS oplock support. Empirical data will be added when test equipment becomes available.
8.3 Auspex NeTservices Auspex markets a NAS server called 4Front NetServer 2000. Auspex is an Advanced Server for UNIX OEM customer of AT&T, and their version of Advanced Server is the NeTservices UNIX/Windows consolidation product, running on UNIX on the 4Front NS 2000 server. AT&T Advanced Server is also the OEM source for HP’s AS/9000 product. NeTservices runs on a UNIX operating system, but like NetApps and EMC, appears to be a single-purpose NAS server.
NeTservices provides byte range locking for UNIX/NFS when the advisory locking protocol is adhered to. They make no statement about PC-NFS. CIFS/9000 provides byte range locking for UNIX, UNIX/NFS and PC-NFS. NeTservices provides oplocks support for Windows-only file access. Advanced Server for UNIX oplock support is enabled server-wide, and automatically disables UNIX/NFS byte range lock support, so oplocks can only be enabled for a purely Windows homogeneous client environment.
8.4 Veritas File Server Edition Veritas markets a UNIX/NT integration product called File Server Edition that runs on Sun SPARC Solaris servers. File Server Edition consists of the Veritas File System, Veritas Volume Manager, Veritas Quicklog, and Samba. Veritas File Edition literature does not reference file locking capabilities.
8.
Chapter 9 CIFS/9000 File Locking Interoperability Summary A CIFS/9000 server can provide file storage (and printing) for Windows clients, UNIX clients, NFS and PC-NFS clients, and the CIFS/9000 client. Understanding how the various client platforms invoke file locking for homogenous and heterogeneous (mixed) client access is critical when designing and implementing a NAS or technical server configuration.
protect Windows clients that have locked files from UNIX/NFS and PC-NFS processes that implement the byte range advisory locking protocol. CIFS/9000 will not protect Windows clients from processes that do not adhere to the advisory locking protocol. CIFS/9000 has a planned enhancement to provide this capability. CIFS/9000 provides locking interoperability between Windows clients with Mandatory Share Mode (Open Mode) locks and UNIX/NFS or PC-NFS clients as of the September 2001 HP-UX application Release15.
Appendix A smb.conf Examples A.1 smb.conf for Windows-Only Access # Samba config file created using SWAT # Date: 2000/04/10 12:59:01 # Global parameters [global] workgroup = SNSLATC netbios name = CIFS_SERVER server string = Samba Server security = DOMAIN encrypt passwords = Yes password server = * syslog = 0 log file = /var/opt/samba/log.
A.2 smb.conf for Mixed-Mode Access # Samba config file created using SWAT # Date: 2000/04/10 12:59:01 # Global parameters [global] workgroup = SNSLATC netbios name = CIFS_SERVER server string = Samba Server security = DOMAIN encrypt passwords = Yes password server = * syslog = 0 log file = /var/opt/samba/log.
Appendix B Sales Tool: Locking Technology Examples Hewlett-Packard Sales Force personnel must understand the customer’s operating environment in order to accurately assess their file locking requirements. HP’s competitors have been very effective in misinforming potential customers about the necessity of crossplatform CIFS/NFS file locking. It is critical that the Sales Force personnel determine if a customer needs file locking, and then what level of file locking (if any) is required. B.
B.2 Byte Range Locking Windows and NFS clients invoke byte range locking with API and function calls. These locks are processed in server user space, where the Windows client locking semantics are translated by smbd to UNIX fcntl locks and propagated to the Kernel Lock Manager. The Kernel Lock Manager is the common denominator for the platform locking semantics, which allows the platform locks to interoperate and guarantee data integrity when the advisory protocol is adhered to.
B.3 Mandatory Share Mode (Open Mode) Locks CIFS/NFS Open Mode Locking Old Behavior pre-Sept 2001 NFS CIFS Open Mode Lock Samba HP-UX 11 Byte Range Lock NLM Samba Internal Locking Mechanism Network Lock Manager User space User space Kernel space Kernel space KLM Kernel Lock Manager Open Mode Locks DO NOT Interoperate at KLM! JFS 3.3 File System Previously, CIFS/9000 Mandatory Share Mode (Open Mode) locks were processed and managed by the Samba Internal Locking Mechanism only.
B.4 Mandatory Share Mode (Open Mode) Enhancement CIFS/NFS Open Mode Locking New– HP-UX AR 0901 CIFS NFS Open Mode Lock Samba HP-UX 11 Byte Range Lock NLM Samba Internal Locking Mechanism Network Lock Manager User space User space Kernel space Byte Range Lock Kernel space KLM Kernel Lock Manager Translate Open Mode Locks to Byte Range Locks! JFS 3.
B.5 Competitor’s Claims for NFS Non-Locking Protection Competitors claim to protect their CIFS clients from NFS clients that are not participating in the advisory locking protocol. Their claim is based upon their mechanism of comparing every NFS read and write operation with locking records in the Kernel Lock Manager (or similar lock manager). The KLM will record every outstanding lock that is held by a CIFS or NFS client.
