Technical information

ManualsBrandsQ-NOTE ManualsGadgetsQN-7000HX

XAPP1206 v1.1 June 12, 2014 www.xilinx.com 1

United States and other countries. AMBA, AMBA Designer, ARM, ARM1176JZ-S, CoreSight, Cortex, and PrimeCell are trademarks of ARM in the EU and other countries. All

other trademarks are the property of their respective owners.

Summary Xilinx

Zynq

-7000 All Programmable SoC is an architecture that integrates a dual-core ARM

Cortex™-A9 processor, which is widely used in embedded products. Both ARM Cortex-A9

cores have an advanced single instruction, multiple data (SIMD) engine, also known as NEON.

It is specialized for parallel data computation on large data sets. This document explains how to

use NEON to improve software performance and cache efficiency, thus improving NEON

performance generally.

Introduction Generally speaking, a CPU executes instructions and processes data one-by-one. Typically,

high performance is achieved using high clock frequencies, but semiconductor technology

imposes limits on this. Parallel computation is the next strategy typically employed to improve

CPU data processing capability. The SIMD technique allows multiple data to be processed in

one or just a few CPU cycles. NEON is the SIMD implementation in ARM v7A processors.

Effective use of NEON can produce significant software performance improvements.

Document Content Overview

Technical information provided in the this document includes:

Before You Begin: Important Concepts

This document provides the following information you need to be more effective when

optimizing your code:

• Software Optimization Basics

• NEON Basics

Software Performance Optimization Methods

This document describes four ways to optimize software performance with NEON :

• Using Using NEON Optimized Libraries

As Cortex-A9 prevails in embedded designs, many software libraries are optimized for

NEON and have performance improvements. This document lists those libraries which are

frequently used by the community.

• Using Compiler Automatic Vectorization

GCC, the popular open source compiler, can generate NEON instructions with proper

compilation options. However, the C language does not excel at expressing parallel

computations. You might need to modify your C code to add compiler hints. Lab 1 provides

a hands-on example.

• Using NEON Intrinsics

Usually, the compiler handles simple optimizations well (optimizations such as register

allocation, instruction scheduling, etc.). However, you might need to use NEON intrinsics

when the compiler fails to analyze and optimize more complex algorithms. Moreover, some

NEON instructions have no equivalent C expressions, and intrinsics or assembly are the

Application Note: Zynq-7000 AP SoC

XAPP1206 v1.1 June 12, 2014

Boost Software Performance on Zynq-7000

AP SoC with NEON

Author: Haoliang Qin

Summary of content (28 pages)

PAGE 1
Application Note: Zynq-7000 AP SoC Boost Software Performance on Zynq-7000 AP SoC with NEON XAPP1206 v1.1 June 12, 2014 Author: Haoliang Qin Summary Xilinx® Zynq®-7000 All Programmable SoC is an architecture that integrates a dual-core ARM® Cortex™-A9 processor, which is widely used in embedded products. Both ARM Cortex-A9 cores have an advanced single instruction, multiple data (SIMD) engine, also known as NEON. It is specialized for parallel data computation on large data sets.
PAGE 2
Software Examples and Labs only options. So, in some situations, you might need to re-write time-critical code in NEON intrinsics to achieve better performance. Lab 2 provides a hands-on example. • Optimizing NEON Assembler Code In some situations, the compiler might be unable to generate the highest performance binary. When this happens, you must write functions in NEON assembler code, with everything under developer control. Lab 3 provides a hands-on example.
PAGE 3
Before You Begin: Important Concepts Before You Begin: Important Concepts Before addressing specific optimization techniques, it is important to understand some related concepts. The sections below describe: • Software Optimization Basics • NEON Basics Software Optimization Basics In an embedded system, you often optimize for speed, but you can also optimize for battery life, code density, or memory footprint.
PAGE 4
Before You Begin: Important Concepts Figure 1 shows the single core Cortex-A9 processor block diagram.
PAGE 5
Before You Begin: Important Concepts number can be different from the overall performance. The best way to evaluate the performance of complex application code is to measure it in a real system. The standard method for improving software performance is to run the software at high speed for a period of time sufficient for the collection of the performance data. This is accomplished using profiling tools or the system performance monitor embedded in the Cortex-A9 processor.
PAGE 6
Before You Begin: Important Concepts cores, and one private timer (32-bit) for each core. The timers have a pre-scaler to lower the timer clock rate and to gain longer overflow time, at the cost of timer resolution. The actual timer clock can be calculated with the following equation: PERIPHCLK ----------------------------------------------------------------PRESCALERvalue + 1 On the Zynq-7000 AP SoC, the PERIPHCLK is half of CPU clock frequency.
PAGE 7
Before You Begin: Important Concepts Figure 2 compares SIMD parallel add with 32-bit scalar add.
PAGE 8
Before You Begin: Important Concepts Cortex-A9 processor. For floating-point operation, VFP can support both single-precision and double-precision, whereas NEON only supports single-precision. VFP can also support more complex functions, such as square roots, division, and others, but NEON cannot. NEON and VFP share the thirty-two 64-bit registers in hardware. This means that VFP is present in VFPv3-D32 form, which has 32 double-precision floating-point registers.
PAGE 9
Before You Begin: Important Concepts NEON Performance Limit When using NEON to optimize software algorithms, it is important to determine how much performance improvement you can expect. An ARM document, the Cortex-A9 NEON Media Processing Engine Technical Reference Manual [Ref 4], contains the detailed information for each VFP and NEON instruction. Table 2 and Table 3 are subsets of the information provided in this document for quick reference.
PAGE 10
Software Performance Optimization Methods All techniques for programing with NEON listed above are discussed in detail in the following sections. Software Performance Optimization Methods Optimization methods include: • Using NEON Optimized Libraries • Using Compiler Automatic Vectorization • Using NEON Intrinsics Using NEON Optimized Libraries The ARM Cortex A9 processor has seen increasing utilization and is one of the most popular platforms for embedded designs.
PAGE 11
Software Performance Optimization Methods In addition to these open source libraries, NEON is also popular with commercial providers, including those shown in Table 5: Table 5: NEON 3rd Party Ecosystem Provider Project and Description Sasken Communication Technologies H.264, VC1, MPEG-4 Skype On2 VP6 video, SILK audio (v1.08+) Ittiam Systems MPEG-4, MPEG-2, H.263, H.264, WMV9, VC1, DD Aricent MPEG-4, H.263, H.264, WMV9, audio Tata Elxsi H.
PAGE 12
Software Performance Optimization Methods code from standard C or C++. However, in practice, this optimization level cannot always produce binaries faster than -O2. Check the software performance case-by-case. • -Os. Selects optimizations that attempt to minimize the size of the image, even at the expense of speed. (This is not a point of focus in this document.) • -Ofast. Disregards strict standards compliance. '-Ofast' enables all '-O3' optimizations.
PAGE 13
Software Performance Optimization Methods Figure 4 shows the setting optimization flags. X-Ref Target - Figure 4 Figure 4: Setting Optimization Flags The compiler might not always vectorize C language code as expected, so you must ensure that compilers generate appropriate instructions: • Read the disassembly. This is the most straightforward method, but it requires a full understanding of NEON instructions. • Use the compiler option -ftree-vectorizer-verbose=n.
PAGE 14
Software Performance Optimization Methods • Use suitable data types An example of a standard dot product algorithm is used here. The following function calculates the dot product of two float vectors, with each vector having len number of float type elements. float dot_product(float * pa, float * { float sum=0.
PAGE 15
Software Performance Optimization Methods Using the techniques above, you can modify the C source code to the following style to help the compiler do automatic vectorization. float dot_product(float * restrict pa, float * len) { float sum=0.0; unsigned int i; restrict pb, unsigned int for( i = 0; i < ( len & ~3); i++ ) sum += pa[i] *pb[i]; return sum; } GCC also supports the alternative forms __restrict__ and __restrict when not compiling for C99.
PAGE 16
Software Performance Optimization Methods produce a result different from non-NEON optimized code for floating-point numbers. Typically, however, the difference is not significant. When you need to validate the code by comparing the computational result, be aware that the term “equal” for a data type of float or double does not mean exactly the same thing, but the difference is acceptable. Lab 1 1. Create a new project in SDK. 2. Import lab1 source files. 3.
PAGE 17
Software Performance Optimization Methods Table 6 can give developers some basic ideas about NEON types.
PAGE 18
Software Performance Optimization Methods Accessing Two D-registers of a Q-register This can be done using vget_low and vget_high, as shown below: vec64a = vget_low_u32(vec128); // split 128 bit vector vec64b = vget_high_u32(vec128); // into 2x 64 bit vectors Casting NEON Variables Between Different Types NEON intrinsics are strongly typed, and you cannot perform type casts as freely as you can in C language.
PAGE 19
Software Performance Optimization Methods The disadvantage is obvious. First, it is difficult to maintain assembler code. Even though all Cortex-A series processors support NEON instructions, the hardware implementations are different, so instruction timing and movement in the pipeline are different. This means that NEON optimization is processor-dependent. Code running faster on one Cortex-A series processor might not work as well on another Cortex-A series processor.
PAGE 20
Boost NEON Performance by Improving Memory Access Efficiency Therefore, when writing assembly instructions, it is best to distribute memory access instructions with data processing instructions. Boost NEON Performance by Improving Memory Access Efficiency When talking about processor core performance, developers typically make some assumptions that are not always accurate in real-world applications, specifically: • The processor pipeline is optimal and there is no interlock.
PAGE 21
Boost NEON Performance by Improving Memory Access Efficiency Generally, loading and storing multiple instructions can yield better performance than the equivalent multiple load-and-store instructions, especially when cache is not enabled or a memory region is marked as non-cacheable in the translation table. To understand this, you must study the AMBA® specification carefully. Each memory access has overhead on the AXI bus.
PAGE 22
Boost NEON Performance by Improving Memory Access Efficiency A graphical demonstration of VST3 is shown in Figure 5. X-Ref Target - Figure 5 >5 @ [ \ ] [ \ ] [ \ ] [ [ [ [ ' \ \ \ \ ' ] ] ] ] ' 967 ^' ' ' ` >5 @ ; Figure 5: Demonstration of VST3 Operation VLD2 loads two or four registers, de-interleaving even and odd elements.
PAGE 23
Boost NEON Performance by Improving Memory Access Efficiency The algorithm used is still the dot product calculation on two float vectors (length=1024), written in assembler codes. There are two versions: preload optimized, and non-preload optimized with all PLD instructions commented out. .align 4 .global neon_dot_product_vec16_pld .arm neon_dot_product_vec16_pld: pld [r0, #0] pld [r1, #0] pld [r0, #32] pld [r1, #32] vmov.i32 vmov.i32 vmov.i32 vmov.i32 q10, q11, q12, q13, #0 #0 #0 #0 .
PAGE 24
Boost NEON Performance by Improving Memory Access Efficiency this is at line 67 of the source file benchmarking.c. You can see that before each run, the L1 and L2 cache is flushed by function call Xil_DCacheFlush(). Comment out this line to see the hot cache performance. The execution drops to around 2.67 s, demonstrating that cache can improve performance significantly. In this example, because the latency for PLD instructions is much longer than the computation time, not all PLD instructions take effect.
PAGE 25
Boost NEON Performance by Improving Memory Access Efficiency X-Ref Target - Figure 6 0DLQ 0HPRU\ &DFKH [ [ [ [ [ [ ELW $GGUHVV [ 7DJ ,QGH[ 2IIVHW %\WH [ /LQHV ,QGH[ [ [ 'DWD +LW ; Figure 6: Direct Mapped Cache In Figure 6, you can see the 16-byte arrays with starting addresses 0x0, 0x40 and 0x80 share the same cache line.
PAGE 26
Boost NEON Performance by Improving Memory Access Efficiency Fully associative cache can solve this issue. With this solution, any specific location in main memory can be stored in any cache line. However, building such a cache is impractical unless it is very small. The control hardware can be complex and the power consumption high. In practice, N-way set associative cache is more widely used to balance complexity and performance.
PAGE 27
Summary ii < 8; ii++, rresult += N, ra += N) for (ki = 0, rb = &b[ko][jo];ki < 8; ki++, rb += N) for (ji = 0; ji < 8; ji++) rresult[ji] += ra[ki] * rb[ji]; Lab 3 1. Create a new project, import lab 3 source files. 2. Run the standalone application on board. 3. In the code provided, set the matrix size to 512*512, so that the execution does not take too long. Note: Because the focus is on how tiling can improve performance, you do not enable compiler automatic vectorization for NEON in this lab.
PAGE 28
Revision History 7. Cortex™-A Series Programmer's Guide silver.arm.com/download/download.tm?pv=1296010 8. Zynq-7000 All Programmable SoC Technical Reference Manual (UG585) www.xilinx.com/support/documentation/user_guides/ug585-Zynq-7000-TRM.pdf 9. RealView Compilation Tools Compiler Reference Guide, available at infocenter.arm.com. 10. GCC documentation, available at gcc.gnu.org/onlinedocs/gcc/ARM-NEON-Intrinsics.html Revision History Notice of Disclaimer XAPP1206 v1.