Computer Science

ECS 158 Programming on Parallel Architectures

ECS 158 PROGRAMMING ON PARALLEL ARCHITECTURES (4) III

Format
Lecture: 3 hours
Discussion: 1 hour

Catalog Description:
Techniques for software development using the shared-memory and message-passing paradigms on parallel architectures and networks of workstations. Locks, barriers, and other techniques for synchronization. Performance issues, such as memory contention and network latency and bandwidth.  Introduction to parallel algorithms

Prerequisites: Course 60; courses 150 and 154B recommended

Credit restrictions, cross listings: None

Summary of course contents

  1. Programming Paradigms
    1. Shared-memory, on multicore and GPU
    2. Message-passing, on networked computers and clusters
    3. SIMD
    4. Software packages: OpenMP, MPI, pthreads, CUDA
  2. Overview of Parallel Architectures
    1. Shared-memory, message-passing and SIMD hardware
    2. Interconnect structures, e.g., bus, crossbar, Omega-net, mesh
    3. Cache coherency
  3. Interprocess Synchronization
  1. Overview of operating systems material: processes, threads, OS-based interprocess synchronization
  2. Synchronization hardware, e.g., test-and-set and fetch-and-add instructions
  3. Locks, barriers
  4. Latency/Bandwidth Issues
    1. Memory/network delays
    2. Latency hiding, e.g., using nonblocking messages
  5. Collective Communication
    1. Broadcast, scatter/gather, all-scatter/all-gather
  6. Overview of Parallel Applications
    1. Sorting, database, prefix scan
    2. Graph algorithms
    3. Scientific computing

The course involves extensive programming assignments on our CSIF-networked machines, including those with suitable GPUs. The shared-memory work is done primarily using OpenMP and CUDA, with the message-passing component using MPI.
Students in the course write sophisticated, systems-level programs according to their own design. Many of the programs involve analysis of tradeoffs between efficient use of the hardware and ease of programming.

Goals:  Students will: (1) learn techniques of software development using the shared-memory and message-passing paradigms on parallel architectures and networks of workstations; and (2) be introduced to parallel algorithms.

Illustrative reading
N. Matloff, Programming on Parallel Machines,2012.Open-source textbook, available at  http://heather.cs.ucdavis.edu/parprocbook

Computer Usage:
Extensive programming assignments on our CSIF networked machines. The shared-memory work is done using Adsmith and the MulSim simulator, and the message-passing component using MPI.

Engineering Design Statement:
Students in the course write sophisticated, systems-level programs according to their own design. Many of the programs involve analysis of tradeoffs between efficient use of the hardware and ease of programming.

ABET Category Content:
Engineering Science: 2 units
Engineering Design: 2 units

GE3: Science & Engineering

Overlap: None

Instructors: F. Gygi and N. Matloff

History:2012.10.26 (N. Matloff): refreshed catalog description, course contents, and reading.  Prior course description by N. Matloff, November 2003

Outcomes

1

X

an ability to apply knowledge of mathematics, science, computing, and engineering

2

X

an ability to design and conduct experiments, as well as to analyze and interpret data

3

X

an ability to design, implement, and evaluate a system, process, component, or program to meet desired needs, within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability

4

 

an ability to function on multi-disciplinary teams

5

 

an ability to identify, formulate, and solve computer science and engineering problems  and define the computing requirements appropriate to their solutions

6

 

an understanding of professional, ethical, legal, security and social issues and responsibilities

7

 

an ability to communicate effectively with a range of audiences

8

 

the broad education necessary to understand the impact of computer science and engineering solutions in a global and societal context

9

 

a recognition of the need for, and an ability to engage in life-long learning

10

X

knowledge of contemporary issues

11

 

an ability to use current techniques, skills, and tools necessary for computing and engineering practice

12

X

an ability to apply mathematical foundations, algorithmic principles, and computer science and engineering theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices

13

X

an ability to apply design and development principles in the construction of software systems or computer systems of varying complexity

border