Open Source Object Storage for Unstructured Data: Ceph on HP ProLiant SL4540 Gen8 Servers

Reference Architecture | Product, solution, or service
The sample reference configuration fits in a single rack, but is scalable in some important ways. The rack reserves space to
configure for further HP ProLiant SL4540 Gen8 Server scaling or other datacenter equipment. It’s relatively simple to source
this configuration to multiple racks by replicating elements of the BOM and distributing monitors/object gateways across
the racks.
Multi-site and multiple-POD can get more complicated than the sample reference architecture shows, as object gateway
and monitor configuration involve hardware selection and failure domains beyond the scope of a single data center.
However, for POC or initial cluster implementation what’s here is a good conceptual start.
Licensing and support
The BOM lists service, support, and licenses for iLO. These are important for a scale-out solution with industry-standard
servers as they provide reliability and management required to operate petabyte scale clusters and beyond. HP iLO provides
the foundation for linking the hardware platform to cluster performance, along with remote hardware management. HP
service and support provide expertise through setup, operation and escalation for issues with HP provided hardware.
Inktank support on the software solution side is also recommended to protect your cluster investment. Inktank Ceph
Enterprise is a subscription combining the most stable version of Ceph for object and block storage, with the Calamari
graphical manager, enhanced integration tools and support services. Inktank also provides expertise and professional
services for your Ceph cluster.
Workload testing
A guiding principle of testing for this paper is to create performance that’s simple to replicate. Although object storage does
not have the same SLA as traditional storage, performance benchmarking shows where bottlenecks are and where
resources could be allocated for solution requirements.
Object and block data are the chosen focus cases for unstructured data use on Ceph. Traffic to the Ceph object gateway is
using the Swift API, which has the advantage of a similar API and traffic generation test tool for both OpenStack Swift and
Ceph testing. These results help set expectations for performance at a level of load and type of access, which helps scale
planning and matching cluster capabilities to use case.
Ceph provides a native object API with librados and a test tool in rados-bench; while a very useful tool for testing Ceph
cluster performance HP believes customer object storage use cases are more likely to focus around the APIs provided by
the object gateway.
Workload description
Without a recognized standard for object storage benchmarking, baseline data comes from canned IO testing. The test
takes objects of varying sizes and is run to achieve a reliable average performance sample.
Test matrix configuration
The cluster is pre-seeded with 1000 accounts: 100 containers per account and 100 objects per container. This gives a
representation of IO running on a system with used capacity. Runs operate at a fixed number of processes/threads (30)
using three traffic generators for all tests.
Test matrix terminology
• A ‘Suite’ is all tests run for a given access method
• A ‘Pass’ is all types of test at a given object/block size
• A ‘Phase’ is a single type of test at an object size
• A ‘Step’ is any subdivision of a Phase
Object testing
• Test passes are done at 1KB, 16KB, 64KB, 128KB, 512KB, 1MB, 4MB, 16MB, 128MB object sizes
• The account, containers and objects being accessed by the test are not pre-seeded
• Each tested phase has steps at 100% PUTs, 100% GETs, and then 90%/10% GETs to PUTs
• Object count for a pass is chosen so the100% PUT phase lasts at least 30 minutes
• On the MIX step, do PUTs as step #1. Step #2 is GETs to step #1objects and PUTs using new object name prefix
• There are no object DELETEs between sizes
This suite consists of pure write and read load tests, and then a mixed test to represent an active cluster with mostly static
data being read (but some ongoing writes). As load tests are a ‘warmer’ use case, the MIX approximates a file hosting
service load. GETs must be performed after PUTs, so file system cache impact occurs. Although object storage does not
have the same SLA as traditional storage, performance benchmarking still reveals bottlenecks and resource restrictions.
17