CS 584 Computer Vision

Course Objective

1. Computer Vision : A Mordern Approach, D. Forsyth and J. Ponce, Prentice-Hall, 2001
2. Papers, manuals and web documents as assigned.

1. Computer Vision, D. Ballard and C. Brown, Prentice-Hall, 1982. and its avaible online (for free)
2. Computer and Robot Vision vol. 2, R. Haralick and L. Shapiro, Addison-Wesley, 1992
3. Robot Vision, B.K.P. Horn, MIT Press, 1986
4. Three-Dimensional Computer Vision, O. Faugeras, MIT Press, 1993
5. A Guided Tour of Computer Vision, V. S. Nalwa, Addison Wesley, 1993
6. Vision, David Marr, Freeman, 1982

1. Students must be able to program in C++.
   CS 145 or equivalent - Data Structures, good programming skills.
   CS306 (OO programing in C++). recommend
2. Ability to convert vague problem descriptions and informatl specifications into computer algorithms. Recommended: CS330 Software Engineering
   
3. Basic Mathematics - Good facility with calculus and analytical solid geometry is essential. If you have not used them for several years, you must be prepared to spend some time to review them (and the assignments may take more than the alloted time). Undergraduate knowledge of linear algebra, matrix theory and elementary probability theory will also be helpful.

All "software related" assignments in this class is to be done on a two person team basis. Each team assignment will be given a total number of points. Based on the teams reports, each team will achieve a total number of points. These points will be shared, not necessarily equally, among team members.

You are bound by the rules of CU-Colorado Springs Academic Honor Code Except for sharing information between members of the same team, any collaboration, plagiarism or any other offense listed therein is consider cheating and will be referred to the disciplinary committee. Even though assignments are team assignment, sharing materials or even detailed discussions of solution techniques, between teams, is considering CHEATING. Make sure you protect your files, if someone copies from you, you are both responsible. If in assignments you "borrow" code from places other than our text or openCV, even small segments of code, you need to give proper acknowledgement in your code. I reserve the right to deny any student from being graded under the homework weighted option, independent of the handling of their case by the disciplinary committee.

Each assignment/exam will be graded on a curve, and for each assignment letter grades will be assigned (F=0 thru A+=4.33). A curve per assignment will be used for all students (with zero scores not affecting the curve). In general each curves will have a mean between B- and B+, depending on my assessment of the overall performance. Programs will have point values which reflect their relative weight. Class participation grade will be based on my assessment of the students voluntary contribution, plus their response to questions I ask them during class, with the possible grades F, C,B,B+,A-,A. Interactions with the instructor outside of class, such as in office hours, is also considered "class" participation.

There will be a number of chances for extra credit which is added to your final letter grade with 300pts extra credit = 1 letter grade. Extra credit does not impact curving. I do not offer individuals extra credit, if I offer it, it is offered to the entire class.

Teams will be self organizing, with the limitation that any two students CANNOT be on more than 2 teams together. Teams will be limited to groups of 2 unless I approve otherwise. Teams should submit a group report as well as a shorter individual report for each team member. The Individual reports should include a breakdown (by %) of who did what and should get what fraction of the credit.

If an assignment is given multiple weeks, do NOT wait until the last few days to start. Don't count on putting in 20 hours in 3 days, because you might just need to ask me a few questions.

Course Outline

Assignments

1. Personal assignment #1. Due Feb 2. Questions from the text.
   • 1.2
   • 1.5
   • 2.1
   • 2.4
   • 2.7
   Using a digital video camera, take your picture and setup a class web page with your picture (In a later assignment you will be doing some filtering of it from the intel openCV package, and eventually extending their face recognition system. These images will be the class "DB".) Include the URL in your "writeup".

   ---

   
   
2. Software assignment #1 (part 1, 80 pts). Due Feb 9. Using OpenCV, process your "picture" (i.e. photo from last week) to produce:
   1. A blurred facial image, 3 levels of blurring.
   2. A detected/localize face on all 4 (orig + 3 blurred images)
   3. and edge image with 3 different levels of "edgeiness", applied to all 4 images (i.e. 12 different images).
   4. 3 different levels of "segmented" image
   5. Write a openCV program that loads an image, blurs it, adds random noise, detects the face and computes the errors of the detection/localization. Run it for a range of parameters and report the errors. To facilitate comparison you should use both your picture and my picture (from my web page). You should explain how you define and computed the error in detection/localization.
   Your team will turn in your results by posting them on a web-page and emailing me the URL.
   Note: people have asked if you can use Hawk. The answer is yes but I strongly suggest you try to get the Compiler-version going in your environment. The next assignment is video based and it is hard to do it in Hawk.

   ---

   
   
3. Personal (Theory) assignment #2 ( 50 pts). From the Text. Due Feb 16.
   • 4.2
   • 4.6
   • 4.10
   • 6.2
   • 6.6

   ---

   
   
4. Software assignment #2 (teams allowed). 200 points. Due Mar 1.
   Choose an application (e.g. hand-recognition, or part inspection) not already implemented in openCV (e.g. face detection is not allowed) and do the following:
   1. Do some image "normalization"/filtering to reduce noise improve dynamic range (may be before of after color work below).
   2. Develop a color-based algorithm to approximately locate the object
   3. then apply a "correlation-based" template matching to locate features for your object (normalized correlation is allowed)
   4. Test the system with at least 20 different "examples" of the input with varying illumination. If possible define and measure the "error" as a function of illumination variations.
   5. and display/save the results.
   6. generate a "report" on the results of your experiment
   7. 30pts extra credit for a near-real-time implementation (say < 150ms per frame) that does this on video streams.
   You are encourage to find a paper in the literature related to your algorithm and see if you can implement the key ideas of that paper. Include proper citations in your report.

   If you don't have an application in mind and you would like to help out on our research we can have teams working on the same application (detection and location of fingers/creases on the hand).

   ---

   
   
5. Software assignment #3 (teams allowed). 200 points. Due Mar 10.
   • Implement Union-find based Connected Components. Output a list of regions with basic properties (area, Bbox, average gray value, gray variance) and a "colored" image showing the results. Test as basis for background-subtraction using 2 different "scenes" with 3 different thresholds.
   • Implement Quasi-connected Components. Output a list of regions with basic properties (area, Bbox, average gray value, gray variance) and a "colored" image showing the results. Test as basis for background-subtraction using 2 different "scenes" with 3 different thresholds.
   • Implement basic Component-Morphology allowing pruning of components based on properties. If practical allow pruning before the "relabeling" phase so only the pruned objects are left in the image.
   • Make one of CC or QCC fast/efficient and in the style of opencv.
   • Use both CC and QCC to detect the color targets you used in software assignment #2 and compare with the results of the segmentation you did there.
   Extra credit:
   • 5pts Extend your union-find to compute properties (like area, Bbox, average, variance) as it does the union steps instead of computing them afterward.
   • 5pts Compare QCC with morphology applied to basic thresholding and CC.
   • 10pts Extend QCC to have "temporal" overlap, using a previous parent image with labels which effects the label generation, i.e. keep the same label if the component in this time frame overlaps component in previous time frame.
   • 10pts Make a multi-channel (RGB) QCC with 3 low and 3 high thresholds.
   • 15pts Make your QCC use multiple scales and explain how it works across scales.
   As before prepare a web-page with the result (and email me the code).

   ---

   
   
6. Personal (Theory) assignment #3 ( 80 pts). From the Text. Due April 12.
   • 15.2
   • 15.3
   • 15.4
   • 16.2
   • 16.3
   • 3.1
   • 3.3
   • 3.8
   • 15.7 Extra Credit (10 pts)

   ---

   
   
7. Software assignment #4 (teams allowed). 2 parts, 100 points each. Part One due April 28, Part 2 due May 5. Part 1:
   • Using OpenCV implement both a Kalman Filter and Condensation tracking syntehtic data where you define the object, motion and noise models (algebraically) and simulate the data then the tracking and then compute formal error and performance measures.
   • Test simple motion (velocity+acceleration model) with noise data drawn from a two different "unimodal" (at most one guassian) distributions,
   • Test model with N>=3 targets both moving with vel+accel.
   As before prepare a web-page with the result (and email me the code).
   Your report (and code comments) should discuss the models used why/How Kalman/Condensation applies.
   Your report should discuss how you addressed the data association problem.
   Your grade depends on good design/implementation, use of the library and explination, Not in choosing a model for which Kalman/Condensenation tracking works well.

   Part 2. Extend your solution to Part 1 to include a "parametric" motion involving multiple constained "parts" (e.g. a simple human model or multiple targets contrained by connections).. Your report (and code comments) should discuss the models and how you implemented it. Discuss its physical plausabiity. Discuss why/how Kalman/Condensation applies, in particular how you developed your dynamic motion model matrixices.
   Your report should discuss how you addressed the data association problem.
   Extra Credit (50 pts) Due by May 10. Using OpenCV, choose either the Kalman Filter or Condensation tracking, and apply it to tracking on real video data. You also need to "segment" to find the data and have some way to

   As before prepare a web-page with the result (and email me the code). For examples of condensation in use see: http://www.stanford.edu/~jbrasket/cs223b/project/ http://www.robots.ox.ac.uk/~misard/condensation.html

   ---

   
   

Final Exam