HPjmeter 4.2 User's Guide

There are two important rules to remember when modifying programs to improve performance.
These might seem obvious, but in practice they are often forgotten.
• Don't put performance above correctness.
When you modify the code, and especially when you change some of the algorithms, always
take care to preserve program correctness. After you change the code, you'll want to test its
performance. Do not forget to test its correctness. You don't have to perform thorough testing
after each minor change, but it is certainly a good idea to do this after you're done with the
tuning.
• Measure your progress.
Try to keep track of the performance improvements you make. Some of the code changes,
although small, can cause great improvements. On the other hand, extensive changes that
seemed very promising may yield only minor improvements, or improvements that are
offset by a degradation in another part of the application. Remember that good performance
is only one of the factors determining software quality. Some changes (for example, inlining)
may not be worth it if they compromise code readability or flexibility.
Preparing a Benchmark
To test the performance of your application, you need to run it on some input data. Preparing
good data for performance testing can be a difficult task. For many server-type applications this
may involve setting up several machines, and creating an “artificial” load. Constructing the
benchmark itself is probably the single most time-consuming step in application tuning, but
when done correctly, it can be worth the effort.
Generally, you want the benchmark to resemble as much as possible the real load that the
application will face when finally deployed.
There are several mistakes that are often made at this stage. To avoid the most common ones,
remember the following:
• Make the benchmark repeatable; this is key to the systematic approach to performance
tuning.
• Do not create unrealistic situations that could speed up or slow down your application
considerably. For example:
— Do not direct all the traffic to just a few records in the database, which could cause
exceptionally good performance because of caching, or exceptionally bad performance
because of lock contention.
— Do not run all clients from one machine. This might not exercise your application
properly because of the performance bottleneck on the client side.
— Do not use too much invalid data; it may never get to the components of the application
that do the real work.
— Do not use a different environment (like the JVM, or options) for the performance testing
than the one to be used for the real deployment.
• Size your benchmark appropriately:
— If the benchmark runs only briefly, the profile data will contain too much “startup
noise” from the effects caused by class loading, dynamic libraries loading, bytecode
compilation and optimization, and the application initializations (database connections,
threads, and data structures).
— If the benchmark runs for too long, it will be very time consuming to repeat the run,
and you'll be tempted to make several changes to your application between the
consecutive profiling runs, which is not recommended.
• As your application grows in terms of performance, you may need to scale up your
benchmark as well.
60 Profiling Applications