Extreme Java - Concurrency Performance

The "Extreme Java - Concurrency Performance" Course is authored by Dr Heinz Kabutz, the publisher of The Java Specialists' Newsletter. It includes (under license) training material produced by JavaPerformanceTuning.com. It is based on our bestselling Java Specialist Master Concurrency Specialist courses.

The Java Concurrency Course is the only such training officially endorsed by Brian Goetz, and is based on his best-seller book Java Concurrency in Practice. The course was written by Dr Heinz Kabutz, author of The Java Specialists' Newsletter, with contributions by Victor Grazi, author of the Java Concurrent Animated Tutorial.

Introduction

• Welcome to the course
• History of concurrency
• Benefits of threads
• Risks of threads
• Threads are everywhere
• Short Java 7 Primer

Thread Safety

• Introduction to thread safety
• Atomicity
• Sharing objects
• Designing a thread safe class

Building Blocks

• Synchronized collections
• Concurrent collections
• Blocking queues and the producer-consumer pattern
• Deques
• Synchronizers

Task Execution

• The Executor framework
• Executor interface
• Motivation for using Executor
• Decoupling task submission from execution
• Execution policies
• Thread pool structure
• Thread pool benefits
• Memory leaks with ThreadLocal
• Standard ExecutorService configurations
• ThreadPoolExecutor
• Executor lifecycle, state machine
• Shutdown() vs ShutdownNow()
• Finding exploitable parallelism

Cancellation

• Reasons for wanting to cancel a task
• Cooperative vs preemptive cancellation
• Using flags to signal cancellation
• Cancellation policies
• Origins of interruptions
• How does interrupt work?
• Policies in dealing with InterruptedException
• Thread.interrupted() method
• Letting the method throw the exception
• Restoring the interrupt and exiting
• Ignoring the interrupt status
• Saving the interrupt for later
• Reactions of IO libraries to interrupts
• Interrupting locks

Thread Pools

• Homogenous, independent and thread-agnostic tasks
• Sizing thread pools
• Configuring ThreadPoolExecutor
• Thread factories
• Customizing thread pool executor after construction

Fork/Join

• Breaking up work into chunks
• ForkJoinPool and ForkJoinTask
• Work-stealing in ForkJoinPool
• ForkJoinTask state machine
• RecursiveTask vs RecursiveAction
• Parallel Fibonacci Calculator
• Fork/Join vs. Compute
• Parallel merge sort
• Canceling a task
• Visibility guarantees with fork/join
• Use cases of fork/join

Avoiding Liveness Hazards

• The drinking philosophers
• Causing a deadlock amongst philosophers
• Resolving deadlocks
• Discovering deadlocks
• Lock-ordering deadlocks
• Defining a global ordering
• Dynamic lock order deadlocks
• Defining order on dynamic locks
• Checking whether locks are held
• Imposing a natural order
• Deadlock between cooperating objects
• Open calls and alien methods
• Avoiding multiple locks
• Using open calls
• Unit testing for lock ordering deadlocks
• Adding a sleep to cause deadlocks
• Verifying thread deadlocks
• Deadlock analysis with thread dumps
• Livelock

Testing Concurrent Programs

• Testing for correctness
• Checking for data races
• Automatic tooling
• Testing through bulk updates
• Server HotSpot interference
• Testing pitfalls
• Controlling HotSpot and JIT
• Turning off optimizations
• Randomizing bulk operations
• Testing field visibility
• Single updates, with time delays
• Pros and cons of various approaches
• Examples of testing broken code
• Testing for deadlocks
• Testing for performance
• HotSpot tricks
• Statistics
• Concurrent performance Testing

Performance and Scalability

Thinking about performance

• Effects of serial sections and locking
• Performance vs scalability
• How fast vs how much
• Mistakes in traditional performance optimizations
• 2-tier vs multi-tier
• Evaluating performance tradeoffs

Amdahl's and Little's laws

• Formula for Amdahl's Law
• Utilization according to Amdahl
• Maximum useful cores
• Problems with Amdahl's law in practice
• Formula for Little's Law
• Applying Little's Law in practice
• How threading relates to Little's Law

Costs introduced by threads

• Context switching
• Cache invalidation
• Locking and unlocking
• Memory barriers
• Escape analysis and uncontended locks
• Lock elision
• Spinning before actual blocking

Reducing lock contention

• Exclusive locks
• Safety first!
• Narrowing lock scope
• Using ConcurrentHashMap
• Performance comparisons
• Reducing lock granularity
• Lock splitting
• Using CopyOnWrite collections
• Lock striping
• Avoiding "hot fields"
• ReadWriteLock
• Immutable objects
• Atomic fields
• How to monitor CPU utilization
• Reasons why CPUs might not be loaded
• How to find "hot locks"
• Hotspot options for lock performance

Explicit Locks

Lock and ReentrantLock

• Memory visibility semantics
• ReentrantLock implementation
• Using the explicit lock
• Using try-finally
• tryLock and timed locks
• Using try-lock to avoid deadlocks
• Interruptible locking

Performance considerations

• Java 5 vs Java 6 performance
• Throughput on contended locks
• Uncontended performance
• Heavily contended locks

Synchronized vs ReentrantLock

• Memory semantics
• Ease of use
• Prefer synchronized

Read-write locks

• ReadWriteLock interface
• Understanding system to avoid starvation
• ReadWriteLock implementation options

Building Custom Synchronizers

Managing state dependence

• Single-threaded vs multi-threaded
• Structure of blocking state-dependent actions
• Example using bounded queues
• Exceptions on pre-condition fails
• Crude blocking by polling and sleeping
• Introducing condition queues

Using condition queues

• State-dependence
• Condition predicate
• Lock
• Condition queue
• Waking up too soon
• Waiting for a specific timeout
• Conditional waits
• Missed signals
• Explicit condition objects

Atomic Variables and Nonblocking Synchronization

Disadvantages of locking

• Elimination of uncontended intrinsic locks
• Volatile vs locking performance
• Priority inversion

Hardware support

• Optimistic locking
• Compare-and-Swap (CAS)
• Compare-and-Set
• Managing conflicts with CAS
• Simulation of CAS
• Nonblocking counter
• CAS support in the JVM
• Shared cache lines
• Performance advantage of padding
• Using "Unsafe" to access memory directly

Atomic variable classes

• Optimistic locking classes
• Very fast when not too much contention
• Types of atomic classes
• How do atomics work?
• Atomic array classes
• Performance comparisons: Locks vs atomics
• Cost of atomic spin loops

Nonblocking algorithms

• Scalability problems with lock-based algorithms
• Definition of nonblocking and lock-free
• Nonblocking stack
• Doing speculative work
• Performance

Java Memory

Garbage Collection

• Generational Spaces
• Young and Old Space Collectors
• Measuring GC Activity
• Sizing of spaces

References

• Reference Objects
• Object Reachability
• Using References
• WeakHashMap
• SoftHashMap

Java Optimizations

Tuning Process

• Optimization Techniques - Big Gains Quickly
• Specifying the required performance
• System Overview - "The Box"
• Consumers of the CPU
• Microbenchmarking

JIT and HotSpot

• Just-in-Time
• HotSpot

Typical Problem Areas

• Object Creation
• Strings and constants
• Loops
• Benchmarking
• Other Areas