ECS 173 IMAGE PROCESSING AND ANALYSIS (4 units)
Lecture: 3 hours
Discussion: 1 hour
Techniques for automated extraction of high-level information from images generated by cameras, three-dimensional surface sensors, and medical devices. Typical applications include detection of objects in various types of images and describing populations of biological specimens as they appear in medical imagery.
Prerequisite: (MAT 067 C- or better or MAT 022A C- or better); (ECS 060 or ECS 032B or ECS 036C)
Credit restrictions/cross listings: None
Summary of course contents
The course will be taught in three modules covering the fundamentals of automated analysis of photographic, three-dimensional surface, and volumetric imaging data.
- Analysis of 2D Images (photographs)
- Low-level information extraction: Frequency-domain representation of images, edge and corner detection, and texture analysis will be discussed in detail. Specific algorithms to be discussed include derivative of Gaussian edge detection, Harris corner detection, and texton estimation.
- Image features: Students will learn various constructions for extracting relevant features from images. Students will learn about filter bank approaches, geometric and illumination invariants, and linear subspace decompositions of image patches. Identification of salient surface patches will also be discussed.
- Object recognition: Students will receive an overview of methods for describing and detecting objects in 2D images. Appearance-based methods based on principal components analysis, convolutional image filters, and raw image classification will be described. Shape-based object detection based on constellations of object parts, local edge features, and alignment to prototype shapes will be presented.
- Analysis of 3D surface imagery
- 3D surface parameterization and representation: Representation of 3D surfaces based on points, parametric surface models, patches, and geons will be discussed. The effects of noise, partial occlusion, and sensor artifacts on these surface descriptions will also be described.
- Automated model building from surface data: Semi-automated and fully-automated procedures for building complete 3D models from collections of partial 3D sensor data sets will be presented. Semi-automated techniques based on landmark placement will be discussed. Fully-automated techniques based on local surface descriptors, constrained data collection, and global surface descriptors will be shown as well.
- Object recognition in 3D data: Detection of objects in 3D surface data sets will be discussed. The effects of partial occlusion and clutter will be discussed. Techniques based on alignment, local surface descriptors, and machine learning approaches will be described.
- Analysis of volumetric images
- Low-level processing: Students will learn approaches to correct for image artifacts found in computed tomography (CT), magnetic resonance (MR), and functional MR images. Bias field correction, blowout, and scattering artifacts will be discussed.
- Description and modeling of biological shapes: Methods for mathematically describing biological objects found in volumetric images will be presented. Representation of 3D solids will be discussed, and computational anatomy approaches to representation of populations of shapes will be presented.
- Localization and detection of objects: Principles and algorithms for localizing biological shapes in volumetric images will be presented. Shape-model-based techniques for estimating the location of constrained, expected objects such as the brain will be discussed. Low-level detection of amorphous, unexpected objects such as tumors and calcifications will also be presented.
Three programming projects will be assigned to allow students to implement algorithms for processing each of the 3 image modalities. (1) 2D images: Students will study and implement a selected algorithm for low-level image processing or multiple-view geometry estimation; (2) 3D surface data: Students will implement a selected algorithm for detection of 3D objects in radar image data; (3) Volumetric images: Students will implement an algorithm for modeling or detection of shapes in medical images.
Goals: Students should exit the course conversant in a range of state-of-the-art methods for extracting useful information from images. Students should also be able to write computer programs that extract such information.
Terry S. Yoo (Editor), Insight into Images: Principles and Practice for Segmentation, Registration, and Image Analysis, AK Peters Publishers, 2004.
Three programming projects will be assigned to allow students to implement algorithms for processing each of the 3 image modalities.
- 2D images: Students will study and implement a selected algorithm for low-level image processing or multiple-view geometry estimation
- 3D surface data: Students will implement a selected algorithm for detection of 3D objects in radar image data
- Volumetric images: Students will implement an algorithm for modeling or detection of shapes in medical images
The programming projects for this class are chosen to enhance the lecture material in the course.
Engineering Design Statement:
The individual student taking this class will design and document a set of software modules for automated analysis of a variety of image data that includes the well-established algorithms described in the course outline. Examinations will include questions based on design components of the course.
ABET Category Content:
Engineering Science: 0 unit
Engineering Design: 0 unit
GE3: Science & Engineering
Overlap: Volumetric images are also treated in ECS 177, but the emphasis there is on the visualization rather than automated extraction of high-level data. Two-dimensional image processing is treated in greater mathematical depth in EEC 206 and EEC 208, graduate courses which are currently taught infrequently. A graduate reading course, ECS 289H, covers these subjects in greater depth for a graduate audience.
Instructors: N. Amenta and O. Carmichael
History: Updated 9.7.2018 (CSUGA): Prerequisites updated to include new lower division ECS series courses. Reviewed by O. Carmichael (2012.10.25): Changed abbreviated name to “Image Processing.” Updated summary of course contents to include current lecture material. Added “Goals.” Prior course description from N. Amenta (March 2008).
|1||X||an ability to apply knowledge of mathematics, science, computing, and engineering|
|2||an ability to design and conduct experiments, as well as to analyze and interpret data|
|3||X||an ability to design, implement, and evaluate a system, process, component, or program to meet desired needs, within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability|
|4||an ability to function on multi-disciplinary teams|
|5||an ability to identify, formulate, and solve computer science and engineering problems and define the computing requirements appropriate to their solutions|
|6||an understanding of professional, ethical, legal, security and social issues and responsibilities|
|7||X||an ability to communicate effectively with a range of audiences|
|8||X||the broad education necessary to understand the impact of computer science and engineering solutions in a global and societal context|
|9||X||a recognition of the need for, and an ability to engage in life-long learning|
|10||knowledge of contemporary issues|
|11||X||an ability to use current techniques, skills, and tools necessary for computing and engineering practice|
|12||an ability to apply mathematical foundations, algorithmic principles, and computer science and engineering theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices|
|13||X||an ability to apply design and development principles in the construction of software systems or computer systems of varying complexity|