Parallel Programming Principle and Practice

Course Description

The students will get the skills to use some of the best existing parallel programming tools, and be exposed to a number of open research questions. This course will:

  • provide an introduction to parallel computing including parallel computer architectures, analytical modeling of parallel programs, the principles of parallel algorithm design.
  • include material on TBB, OpenMP, CUDA, OpenCL, MPI, MapReduce.

Prerequisite: C language, discrete mathematics, data structure, principles of computer composition, operating system, C++, algorithms

Credits: 2

Instructor: Prof. Hai Jin (hjin@hust.edu.cn)

Associate Prof. Feng Lu (lufeng@hust.edu.cn)

Syllabus

  • Part 1: Principle (Lecture 1-5)

    This section covers the very basics of parallel computing, and is intended for someone who is just becoming acquainted with the subject. It begins with a brief overview, including concepts and terminology associated with parallel computing. The topics of parallel memory architectures and programming models are then explored. These topics are followed by a discussion on a number of issues related to designing parallel programs.

  • Part 2: Typical Issues Solved by Parallel Programming (Lecture 6-10)

    This section concludes with several examples of how to parallelize simple serial programs. Including: threads and shared memory programming with TBB and OpenMP, SIMD programming model and Cuda & OpenCL, programming using the Message Passing Paradigm, parallel computing with MapReduce.

  • Part 3: Parallel Programming Case Study and Assignment (Lecture 11 and Assignment)

    Students in this course are required to complete several assignments, which accounts for 30% of their grade.

Lecture Notes

LEC #
TOPICS
 CLASS HOURS
FILES
VIDEOS
1
 Why Parallel Programming?
2
(PPT)
(VIDEO)
2
Parallel Architecture
2
(PPT)
(VIDEO)
3
Parallel Programming Models
2
(PPT)
(VIDEO)
4
Parallel Programming Methodology
2
(PPT)
(VIDEO)
5
 Parallel Programming Performance
2
(PPT)
(VIDEO)
6
 Shared memory programming and OpenMP
2
(PPT)
(VIDEO)
7
 Thread Programming with TBB
2
(PPT)
(VIDEO)
8
 Programming Using the Message Passing Paradigm
2
(PPT)
(VIDEO)
9
 Introduction to GPGPUs and Cuda Programming Model 2h
2
(PPT)
(VIDEO)
10
 Parallel Computing with MapReduce
2
(PPT)
(VIDEO)
11
 Experiment
2
(PPT)
(VIDEO)

Assignment

Finish three experiments: (PDF)

  • Solve Akari using backtracking
  • Solve Akari using parallelizing backtracking
  • Solve Akari using improved parallelizing backtracking

Grading: The final exam covers 70% while the assignment accounts for 30%.

Reference Books

Principles of Parallel programming
By Yun Calvin Lin, Lawrence Snyder
Pearson/Addison Wesley, 2008.
Heterogeneous Computing with OpenCL
By Benedict Gaster, Lee Howes, David R. Kaeli, Perhaad Mistry
Morgan Kaufmann, 2011.
Introduction to Parallel Computing
By Anath Grama
Addison-Wesley Educational Publishers Inc, 2nd Revised edition, 2003.

Other Resources for Future Study