The objective of this course is to convey the basic issues in computer
vision and major approaches that address them. After completing the
course, the students may expect to have the knowledge needed to read
and understand the more advanced topics and current research
literature, and the ability to solve basic industrial
"vision" problems. The course will have plenty of hands-on experience
building exercises. This course is NOT designed to be a
"cookbook" course that gives just a survey of the methods needed in
"practice" and will it not cover "commercial" systems in any detail.
Vision is a very broad topic and a this single course is a beginning,
not an end.
Students must be able to program in C++.
CS 145 or equivalent - Data Structures, good programming skills.
CS306 (OO programing in C++). recommend
Ability to convert vague problem descriptions and informatl specifications into computer algorithms.
Recommended: CS330 Software Engineering
Basic Mathematics - Good facility with calculus and analytical solid
geometry is essential. If you have not used them for several years,
you must be prepared to spend some time to review them (and the
assignments may take more than the alloted time). Undergraduate
knowledge of linear algebra, matrix theory and elementary probability
theory will also be helpful.
The course will have numerous programming assignments. Assignments
can be done on UCCS machines or on your own. Assignments can be done
in either windows of linux. Web cameras will be availible (Depending
on enrollment either for the semester or on a daily basis).
Assignments and class handouts will be available here.
All "software related" assignments in this class is to be done on a two
person team basis. Each team assignment will be given a total number of
points. Based on the teams reports, each team will achieve a total number of
points. These points will be shared, not necessarily equally, among team
You are bound by the rules of
CU-Colorado Springs Academic Honor Code Except for sharing
information between members of the same team, any collaboration,
plagiarism or any other offense listed therein is consider cheating
and will be referred to the disciplinary committee. Even though
assignments are team assignment, sharing materials or even detailed
discussions of solution techniques, between teams, is considering
CHEATING. Make sure you protect your files, if someone copies from
you, you are both responsible. If in assignments you "borrow" code
from places other than our text or openCV, even small segments of
code, you need to give proper acknowledgement in your code. I reserve
the right to deny any student from being graded under the homework
weighted option, independent of the handling of their case by the
There will be two grading schema, with your grade being the maximum of the two.
Grading system 1:
Under this opinion who have an A- or better before the
final are guaranteed that the final exam cannot lower their grade (even if they get a zero).
Grading system 2:
Class participation (2 grades, 6%each)
Each assignment/exam will be graded on a curve, and for each assignment
letter grades will be assigned (F=0 thru A+=4.33). A curve per assignment
will be used for all students (with zero scores not affecting
the curve). In general each curves will have a mean between B- and B+,
depending on my assessment of the overall performance. Programs will have
point values which reflect their relative weight. Class participation grade
will be based on my assessment of the students voluntary contribution, plus
their response to questions I ask them during class, with the possible grades
F, C,B,B+,A-,A. Interactions with the instructor outside of class, such as
in office hours, is also considered "class" participation.
There will be a number of chances for extra credit which is added to your
final letter grade with 300pts extra credit = 1 letter grade. Extra credit
does not impact curving. I do not offer individuals extra credit, if I offer it, it is offered to the entire class.
Teams will be self organizing, with the limitation that any two students
CANNOT be on more than 2 teams together. Teams will be limited to groups of
2 unless I approve otherwise. Teams should submit a group report as well as
a shorter individual report for each team member. The Individual reports
should include a breakdown (by %) of who did what and should get what
fraction of the credit.
WORKLOAD and HOMEWORK
I expect students to devote an average of 7-10 hours per week on the
work for this class. Working on theory assignments, the expetation is
an average student will finish 75-80% of it in that time. For some
student the time needed to totally finish is unbounded. For the
software assignements the average student should, if they work in a
team and the team works effectively, finish approximately 80% of the
assignments. If, however, the team does not work effectively the time
required could double or triple. Debugging badly understood/designed
code could can make an assignment take unlimited amounts of time (like
100's of hours).
If an assignment is given multiple weeks, do NOT wait until the last few days
to start. Don't count on putting in 20 hours in 3 days, because you might
just need to ask me a few questions.
Following is a list of topics expected to be covered, in anticipated order,
and with expected time to be spent on them. This list is intended to be
only indicative, the actual topics, the order and the time may vary
somewhat depending on various factors including student interests
Introduction (1 week)
Background, requirements and issues, human vision.
For Week of 1/26 read chapters 1,2. Download and read about
Intro to OpenCV (1 class)
Notes and manual
Image formation (1 week)
Radiometry and color, (Ch 4 and 6)
Image segmentation (2 weeks)
Edge and line finding, region segmentation
Personal assignment #1. Due Feb 2. Questions from the text.
Using a digital video camera, take your picture and setup a
class web page with your picture (In a later assignment you
will be doing some filtering of it from the intel openCV
package, and eventually extending their face recognition
system. These images will be the class "DB".) Include the
URL in your "writeup".
Software assignment #1 (part 1, 80 pts). Due Feb 9.
Using OpenCV, process your "picture" (i.e. photo from last week) to produce:
A blurred facial image, 3 levels of blurring.
A detected/localize face on all 4 (orig + 3 blurred images)
and edge image with 3 different levels of "edgeiness", applied to
all 4 images (i.e. 12 different images).
3 different levels of "segmented" image
Write a openCV program that loads an image, blurs it, adds
random noise, detects the face and computes the errors of the
detection/localization. Run it for a range of parameters and
report the errors. To facilitate comparison you should use both
your picture and my picture (from my web page). You should
explain how you define and computed the error in
Your team will turn in your results by posting them on a web-page and
emailing me the URL.
Note: people have asked if you can use Hawk. The answer is yes but I
strongly suggest you try to get the Compiler-version going in your
environment. The next assignment is video based and it is hard to do
it in Hawk.
Personal (Theory) assignment #2 ( 50 pts). From the Text.
Due Feb 16.
Software assignment #2 (teams allowed). 200 points. Due Mar 1.
Choose an application (e.g. hand-recognition, or part inspection) not
already implemented in openCV (e.g. face detection is not allowed) and
do the following:
Do some image "normalization"/filtering to reduce noise improve dynamic range (may be before of after color work below).
Develop a color-based algorithm to approximately locate the object
then apply a "correlation-based" template matching to locate features for your object (normalized correlation is allowed)
Test the system with at least 20 different "examples" of the input
with varying illumination. If possible define and measure the "error"
as a function of illumination variations.
and display/save the results.
generate a "report" on the results of your experiment
30pts extra credit for a near-real-time implementation (say < 150ms per frame) that does this on video streams.
You are encourage to find a paper in the literature related to your
algorithm and see if you can implement the key ideas of that paper. Include proper citations in your report.
If you don't have an application in mind and you would like to help
out on our research we can have teams working on the same application
(detection and location of fingers/creases on the hand).
Software assignment #3 (teams allowed). 200 points. Due Mar 10.
Implement Union-find based Connected Components. Output a list of regions with basic properties (area, Bbox, average gray value, gray variance) and a "colored" image showing the results. Test as basis for background-subtraction using 2 different "scenes" with 3 different thresholds.
Implement Quasi-connected Components. Output a list of regions with basic properties (area, Bbox, average gray value, gray variance) and a "colored" image showing the results. Test as basis for background-subtraction using 2 different "scenes" with 3 different thresholds.
Implement basic Component-Morphology allowing pruning of components based on properties. If practical allow pruning before the "relabeling" phase so only the pruned objects are left in the image.
Make one of CC or QCC fast/efficient and in the style of opencv.
Use both CC and QCC to detect the color targets you used in software assignment #2 and compare with the results of the segmentation you did there.
5pts Extend your union-find to compute properties (like area, Bbox, average, variance) as it does the union steps instead of computing them afterward.
5pts Compare QCC with morphology applied to basic thresholding and CC.
10pts Extend QCC to have "temporal" overlap, using a previous parent image with labels which effects the label generation, i.e. keep the same label if the component in this time frame overlaps component in previous time frame.
10pts Make a multi-channel (RGB) QCC with 3 low and 3 high thresholds.
15pts Make your QCC use multiple scales and explain how it works across scales.
As before prepare a web-page with the result (and email me the code).
Personal (Theory) assignment #3 ( 80 pts). From the Text.
Due April 12.
15.7 Extra Credit (10 pts)
Software assignment #4 (teams allowed). 2 parts, 100 points each. Part One due April 28, Part 2 due May 5.
Using OpenCV implement both a Kalman Filter and
Condensation tracking syntehtic data where you define the object,
motion and noise models (algebraically) and simulate the data then the
tracking and then compute formal error and performance measures.
Test simple motion (velocity+acceleration model) with noise data drawn from a two different "unimodal" (at most one guassian) distributions,
Test model with N>=3 targets both moving with vel+accel.
As before prepare a web-page with the result (and email me the code).
Your report (and code comments) should discuss the models used why/How Kalman/Condensation applies.
Your report should discuss how you addressed the data association problem.
Your grade depends on good design/implementation, use of the library
and explination, Not in choosing a model for which
Kalman/Condensenation tracking works well.
Part 2. Extend your solution to Part 1 to include a "parametric"
motion involving multiple constained "parts" (e.g. a simple human
model or multiple targets contrained by connections)..
Your report (and code comments) should discuss the models and how you
implemented it. Discuss its physical plausabiity. Discuss why/how
Kalman/Condensation applies, in particular how you developed your
dynamic motion model matrixices.
Your report should discuss how you addressed the data association
Extra Credit (50 pts) Due by May 10.
Using OpenCV, choose either the Kalman Filter or
Condensation tracking, and apply it to tracking on real video data.
You also need to "segment" to find the data and have some way to
May 17 4:30-7PM, EAS 105
Format/length same as midterm (half-open, half-closed)
Will Cover include material from midterm(Chapters 1,2,5,6,7,8)
but will have more emphais on material from after midterm Ch 3,9,10,12,14,15,16,17,18,22), "Math" type questions will come only from 3,9,12,14,15,16,17.