Introduction: Motivation for parallel programming; Performance metrics for parallelprograms such as speedup, utilization, efficiency, scalability; Models of Parallel Computation such as Single-instruction Multiple-Data (SIMD), Multiple-Instruction Multiple-Data (MIMD); Parallel Random Access Memory (PRAM) models such as Concurrent-Read Concurrent-Write (CRCW) and Concurrent-Read ExclusiveWrite(CREW); Basics of pipelining, latency, and throughput; Amdahl’s Law; Sequential consistency; Correctness of parallel programs-Mutual Exclusion, Liveness and Safety conditions.
Introduction to Parallel Hardware: Organization of multi-core and multi-processor machines; shared memory and message passing architecture, coherence.
Programming Shared Memory architectures: Process and Thread management -creation and termination; Inter-process communication mechanisms; Critical sectionand its importance; Synchronization - need for it, and tools for it such as locks and barriers; Relaxed Consistency and memory fences - benefits and pitfalls; Examples ofparallel algorithms and its implementations on shared memory architectures.
Programming Distributed Memory architectures : Introduction to Message Passing Interface (MPI); Synchronous and asynchronous send-receive; Collective communication - Scatter, Gather, Broadcast, Reduce; Examples of parallel algorithms and its implementations on distributed memory architectures. Programming Graphics Processing Units (GPUs) Introduction to GPU architecture; Introduction to CUDA and SIMT computation; Thread blocks; Warps; Divergence in Control Flow and avoiding them; Example programs
Past Offerings
- Offered in Aug-Nov, 2023 by Unnikrishnan C
- Offered in Jan-May, 2022 by Unnikrishnan C
- Offered in Jul-Dec, 2020 by Unnikrishnan
- Offered in Jan-May, 2019 by Sandeep
Course Metadata
| Item | Details |
|---|---|
| Course Title | Parallel Programming |
| Course Code | CS5005 |
| Course Credits | 3-0-0-3 |
| Course Category | PME |
| Proposing Faculty | Sandeep Chandran |
| Approved on | Senate 6 of IIT Palakkad |
| Course prerequisites | Data Structures, Analysis & Design of Algorithms, Computer Organizations |
| Course status | New |