MAAS Show and Tell: Performance Engineering

Development
1

The video

Andy gives us a solid video summary of Performance Engineering.

The talking points

There are many learning points in this video:

  • Why Performance Engineering Matters:
    • Cost Savings: Faster applications consume fewer cloud computing resources, leading to lower costs.
    • Environmental Impact: Reduced resource consumption means less power usage and fewer servers.
    • Productivity and Efficiency: Quicker program completion allows for more runs, better convenience, and optimal setups.
    • Time-Critical Environments: Essential for rapid results in scenarios like volcanic ash dispersion simulations to save lives and reduce costs.
    • Learning Opportunity: Understanding underlying hardware and software stack improves knowledge, turning black-box applications into comprehensible systems.
  • Choosing a Performance Objective:
    • The most common objective is the “lowest time to solution” or reducing application runtime.
    • Other objectives include lower memory footprint (for IoT/embedded devices) or minimal power consumption (for solar-powered devices).
  • Single-Node Optimizations (Processor/CPU):
    • CPU Complexity: Modern CPUs are highly complex with multiple cores, caches (L1, L2, L3), and threads (e.g., hyperthreading).
    • Memory Hierarchy: Data transfer speeds vary greatly between CPU caches/registers (fastest, smallest), RAM, flash storage, and disk.
    • Von Neumann Architecture: Highlights the speed mismatch between processing elements and main memory, which caches aim to ease.
    • Cache Stalls/Memory Bottleneck: Occur when data is constantly retrieved from main memory due to inefficient cache usage.
    • Data Locality:
      • Spatial Locality: Accessing neighboring data more often (e.g., contiguous array elements).
      • Temporal Locality: Reusing recently computed values (e.g., keeping newest values in cache).
    • Optimizing for Memory:
      • Know the target architecture (e.g., cache line sizes).
      • Reduce data copying.
      • Improve data locality.
    • Performance Counters (Intel CPUs): Track quantities like bytes read/written and cache hit ratios to identify bottlenecks.
  • Parallelism and Multicore Architectures:
    • Goal: Maximize resource utilization and saturate processing elements for throughput.
    • Worth’s Law: Software often gets slower faster than hardware gets faster due to increased complexity with multicore/multithreaded designs.
    • Challenges:
      • Race Conditions: Multiple threads vying for the same resource, leading to unpredictable outcomes.
      • Software Design: Serial programs are difficult to parallelize effectively.
      • Imbalanced Utilization: One thread doing all the work while others are idle.
      • Synchronization: Difficulty in coordinating threads (joining, ensuring data coherence).
    • Solutions:
      • Libraries/Frameworks: OpenMP for parallelizing loops and supporting reduction operations.
      • Vectorization Instructions (SIMD): Performing one operation on a set of data at once (e.g., AVX).
      • Balanced Data Partitioning: Distributing work equally among threads/processes.
      • Minimize Synchronization: Avoid threads waiting unnecessarily.
      • Locks: Protect contested resources from simultaneous access.
  • Instruction Set Utilization:
    • ISA (Instruction Set Architecture): CPUs implement specific ISAs (e.g., x86, ARM) with evolving extensions.
    • Optimization Goals:
      • Use the most performant instructions.
      • Reduce branching in code to prevent CPU stalls.
      • Use the latest and widest vector instructions (e.g., AVX 512 with ZMM registers).
    • Analyzing Binaries: Disassemble binaries (e.g., using objdump, Intel Advisor) to understand compiler output and identify non-optimal instructions.
    • Helping the Compiler:
      • Compilers can struggle to optimize without clear information (loop bounds, array sizes, reuse patterns).
      • Provide compiler hints (directives, pragmas, specific data types).
      • Break apart loops to separate dependencies and enable vectorization.
      • Review optimization reports to understand why optimizations failed.
      • Recompile for specific target architectures for maximum performance.
  • Multi-Node Optimizations:
    • Build upon single-node optimizations.
    • Message Passing Interface (MPI): Used for inter-process communication across nodes.
    • Considerations: Network topology, interprocess communication patterns, and data distribution.
    • Strategies:
      • Balance work evenly across processes.
      • Minimize data transfer over the network.
      • Use non-blocking MPI operations to allow concurrent computation and communication.
      • Implement hybrid designs for fast on-node and inter-node communication.
      • Pin MPI processes to specific cores for coherent performance.
  • General Tips for Faster Code:
    • Structure and Data Types: Use appropriate data types and structures; pack them for memory optimization.
    • Break Data Dependencies: Help compilers optimize loop execution by splitting loops.
    • Use a Profiler: Identify where the most time is spent in your application (e.g., gprof). Loops are often performance bottlenecks.
    • Understand Optimization Reports: Learn how arrays are accessed and break dependencies to enable vectorization.
    • Know Your Stack and Best Practices: Understand performant ways of coding for specific languages/frameworks (e.g., React, Python, Go).

The longer summary

This video is a presentation on performance engineering by Andy. He begins by highlighting the importance of performance engineering, citing reasons such as cost savings in cloud computing, environmental benefits, improved productivity, and the ability to operate in time-critical environments. He also emphasizes that performance engineering offers a valuable learning opportunity.

The presentation outlines various aspects of performance optimization, starting with single-node or on-node considerations. This includes understanding CPU architecture, memory hierarchy (caches, RAM, flash storage), and Non-Uniform Memory Access (NUMA) regions. Andy explains the concept of a memory bottleneck, where the CPU is stalled waiting for data from slower memory. To mitigate this, he suggests optimizing data locality (spatial and temporal) and using performance counters to track cache hit ratios.

Andy then discusses parallelism, noting the shift to multi-core architectures and the challenges associated with parallelizing software, such as race conditions, imbalanced resource utilization, and synchronization. Tools like OpenMP are mentioned as aids for parallel programming. Vectorization instructions are also presented as a means to improve data throughput.

Instruction set utilization is covered next, with an emphasis on using the most performant and latest instructions, and minimizing branching in code. Andy advises disassembling binaries to understand how the compiler is optimizing code and referring to architecture manuals. Additionally, he explains how to provide compiler hints to generate more optimal code.

Finally, the presentation moves to multi-node performance engineering, focusing on the Message Passing Interface (MPI) for inter-process communication. Key considerations include network topology, balancing work across processes, minimizing data transfers, and using non-blocking operations. Andy also recommends hybrid designs and pinning MPI processes to specific cores for better performance consistency. The presentation concludes with general tips for improving code performance, such as considering data structures, breaking apart data dependencies, using profilers, understanding optimization reports, and adhering to best practices for specific programming languages and frameworks.