Modern NVIDIA GPUs utilize thousands of small cores executing the same instruction simultaneously. This massive throughput relies directly on the SIMD (or SPMD) concepts detailed in Quinn's architecture chapters.
The book covers several critical areas of parallel computing, including:
Quinn's work is distinguished by its balance between academic rigor and practical application. While many texts focus exclusively on mathematical proofs, this book emphasizes designing and implementing parallel algorithms that are suitable for "real parallel computers". Key theoretical areas covered include: Modern NVIDIA GPUs utilize thousands of small cores
Parallel computing : theory and practice / Michael J. Quinn - NLB
: A theoretical framework for designing parallel algorithms where multiple processors share a single memory. Flynn's Taxonomy While many texts focus exclusively on mathematical proofs,
: Graph-theoretic problems and combinatorial search. The "Theory vs. Practice" Balance
Parallel computing is the simultaneous use of multiple compute resources to solve a computational problem. A problem is broken down into discrete parts that can be solved concurrently. Each part is further broken down into a series of instructions, and instructions from each part execute simultaneously on different CPUs or cores. Why Parallel Computing Matters Modern NVIDIA GPUs utilize thousands of small cores
For shared memory systems, OpenMP utilizes compiler directives to easily parallelize loops and sections of C/C++ or Fortran code.
The book provides a solid grounding in measuring success using metrics like speedup, efficiency, and overhead, using laws such as Amdahl's Law to explain the theoretical limits of parallel performance.