EECS 556, Class Project, W21 Goals: To provide deeper understanding and hands-on experience with image processing. To learn the importance of reproducible research, the importance of writing clearly to facilitate reproducibility. To provide the opportunity for innovation and investigation in image processing. To help improve project skills: project proposing, independent learning, project execution, and project presentation (both oral and written). There are two general possible approaches to the project. Option A (recommended) combines reproducible research and innovation. In this version you first reproduce the results of an existing paper about a image processing topic, most likely by using available code provided by the authors; then you propose, develop, and investigate your own extensions or variations or alternatives to the method in the paper and compare yours versus theirs. It is not required that your method outperform the method in the paper, but your method, as proposed, should have some rationale for why it might improve upon the results in the paper considered. For this option it acceptable to use the authors' (or others) existing software for the part about reproducing the results in the paper, and then you develop your own algorithm and code for the extension(s). Your extensions can use some existing tools (you need not rewrite the FFT in assembly), but still should have substantial code development of your own at a level appropriate for your background, e.g., students who have had more advanced courses like EECS 559 (aka 598) should strive to leverage that expertise. Option B is to reproduce the results in a contemporary image processing journal publication. Typically this will mean learning about some image processing technique and implementing it (e.g., in Julia or Matlab or Python) and then evaluating it using at least the same evaluation methods used in the publication (which will often involve comparisons one or more previously existing methods). (If the paper compares the proposed method with 10 previous methods you do not have to implement all 10!) If time permits, after you have replicated the paper results you are encouraged to extend the work by either refining the method further based on what you have learned, or by performing additional evaluations or comparisons beyond those reported in the paper. Option B is acceptable *only* for publications where the authors (or others) have NOT already provided all the necessary software online for their paper. Option C is to propose your own image processing method(s) and then develop and implement and investigate it/them. Using some existing software tools is acceptable, but there should be substantial additional work of your own. Option C is appropriate only if you are investigating a new image processing problem where there is no prior work (or at least no code available). Most known image processing problems (like denoising, deblurring, etc.) have lots of available methods, so Option A is typically more appropriate. The project does not have to be based on the applications covered in class, but should involve some type of image processing. Projects that could be done without taking 556 are unacceptable. Feeding data into an uninterpretable black-box deep learning network could be done by a sophomore who knows python and does not involve anything from 556. However, it could be very interesting to combine any of the options above with a deep learning approach where you compare, say, some contemporary model-based image processing approach to a deep learning approach. Deep learning has been used for many image processing applications, like image denoising, image super resolution, image deblurring, inpainting, image registration, solving inverse problems, ... There are many modern methods like ADMMNet and SGD-Net that combine model-based ideas with CNN components and these can be fine topics. All projects should critically assess the design and performance of the system being studied. Be critical and quantitative. Such judgments can be based on theory and/or experimentation. It is more important to assess the system than it is for the system to have outstanding performance. (But avoid testing an overly naive system.) Groups Groups of 4 are strongly preferred, but a couple groups of size 3 or 5 can be approved by Prof. Fessler depending on steady-state enrollment. With 4 students it should be quite feasible to do Option A. "Deliverables" (Your project grade will be based primarily on these items.) To insure steady progress there will be a sequence of deadlines. See the Ch0 notes for the specific dates and times. 1. Form your teams (on-time delivery counts towards proposal score) Try to arrange groups yourself; use Piazza or Discussions in Canvas if needed. If that does not work, then ask me for help. 2. (5%) Project proposal One team member uploads to Canvas a file named like this: unique1-unique2-unique3-unique4-proposal.pdf with the last names in alphabetical order. I will provide templates on overleaf.com and I strongly recommend using them because your documents are going to need to have equations and citations in them and LaTeX is the most suitable format. Overleaf makes LaTeX easy for those who don't know it (the template will help you learn), and is an excellent tool for collaborative technical writing. Consider using overleaf.com to be a requirement unless you discuss with me first and get prior approval for some other collaborative mechanism. Microsoft format files are not acceptable for the proposal, progress report, or final report. The proposal should spell out in reasonable detail the scope of your project. Describe in your own words the method(s) in the paper(s) that you plan to reproduce and how you will implement and evaluate the method(s). It should include a bibliography that lists any other relevant papers that are needed for the project, and should also list any other resources you plan to use such as collections of image data from certain web sites, or code repositories. If the paper being reproduced has "too many" methods or comparisons in it, then clearly describe which are the method(s) and evaluations that your team plans to reproduce. The proposal should indicate how you plan to share the work among the group members. Assuming you are doing Option A, describe the alternative method(s) you plan to develop and how you will compare it/them to the method(s) from the paper. I strongly recommend you use a *private* repo like github.com for the code that your team develops. This is not the sort of thing you do the night before the due date! And you should not consider that you start your project after the proposal is graded. Rather you should invest a lot of thought in your proposal, so that your project will go as smoothly as possible. There is no page limit for the proposal; the more completely you explain the problem goals and algorithms, the more specific I can be in giving feedback, and the more text you have to re-use when writing the final report! Typical length is 4-6 pages of single-spaced text, plus figures and references. See the "proposal guidelines" on the course web page for more details. 3. (8%) Progress report The progress report should build on your proposal document. First incorporate any edits that were requested by me on your proposal, e.g., typos, requests for more clarification about the methods and so on. At the time you submit the progress report you should already have some of the method(s) implemented and some preliminary results. Summarize in the report what steps you have completed and show some evidence of progress, typically with some figures. Summarize any challenges encountered so far, and list what are the remaining steps. A typical progress report is probably a couple pages of text longer than the original proposal, plus some pages of figures and plots. If things are going well, then most of this additional text and figures will be part of your final report too. So this is not "extra work" but rather progress towards the final product. The organization of the progress report should include all the sections in the project proposal, and also have a new section titled "Progress / preliminary results." Most likely your section describing the method will be more detailed here, and your Plans section will be updated. Being an image processing class, one expects to see some processed images in the progress report. One team member uploads to Canvas a file named like: unique1-unique2-unique3-unique4-progress.pdf 4. (5%) Oral presentation [recorded video in W21] 4a. Oral presentation file One team member uploads your "slides" to Canvas in a file named like: unique1-unique2-unique3-unique4-talk.pdf Uploading the file on time is mandatory to be eligible for the prize(s). You are permitted to use .ppt to prepare your recording, but you still must also export the slides as a pdf to upload to Canvas. An overleaf.com template based on "beamer" will be provided, but you are not required to use it. 4b. Oral presentations [recorded video in W21] Record a video presentation describing your project. One team member uploads the recording to Canvas as file unique1-unique2-unique3-unique4-video.mp4 Every team member should contribute to the video (present part of it). W21: the video should be at most 15 minutes The presentation should be targeted at students in the class. That is, it should assume the students in the audience understand the class material, but have not seen the new material that you have developed or investigated. In both your oral presentation and your report, if you cut-and-paste in figures / images that you found on the web, then you must cite the source on the same slide (small font is fine). Exceeding the allotted time will disqualify your team from the prize (and decrease your score). W18: Each group has 15 minutes for its presentation and 5 minutes for questions. Be sure to test your computer/talk in the classroom *before hand* to check it. 5. (20%) Written report One team member uploads to Canvas a file named like: unique1-unique2-unique3-unique4-report.pdf The report should be written for an audience of EECS 556 students. Assume that the students understand the class material, but have not seen the new material that you have investigated. They should be able to learn your topic from your report. The description should be sufficiently detailed that they could easily repeat what you have done. No code should be included with the report itself; instead the algorithms should be described with mathematics and/or with pseudo-code, like in a journal paper. A typical report might be 10-12 pages plus figures, tables, and references, but there is no page limit. Concise and thoughtful summaries of your results are more desirable than a "laundry list" of example images. 6. (2%) Peer review Details later 7. Peer evaluation form In most cases, all members of a group will receive the same project score. However, you must also submit a form (available later) summarizing how the group effort was distributed, and individual grades may be adjusted in the event of significantly inequitable distributions of effort. This will be a simple text file that you will email to the instructor and cc all group members. 8. Individual evaluation form This will be a google form (link available later) that reports confidentially about the effort and contributions of yourself and your teammates. Feedback To obtain feedback and suggestions, each group is encouraged to meet with me during office hours in advance of the proposal deadline, to discuss plans, and after the proposal is graded and returned to get feedback. The GSIs are image processing experts and you should seek their help along the way. Scoring factors include: quantity, quality, degree of difficulty of the work. The quality of your critical judgments and analysis. The depth of understanding displayed. Connectedness to the course. One comment here: projects that convert the data to a binary image very early in the processing tend to score low on course connection and sophistication. Sometimes such projects look like they could have been done by someone who never even took EECS 556. Similarly image "enhancement" methods tend to be very qualitative and hard to assess quantitatively so generally make poor projects.