ECE516 (ECE516H1S): Intelligent Image Processing
Labs and authentic direct mentorship
The most important part of this course is the labs which offer authentic
direct mentorship with a high degree of involvement from the professor and
other leading experts in imaging,
sensing, meta-sensing, and human machine learning.
Our goal for the undergraduates is to help you get into grad school at
MIT (Prof. Mann's alma mater) or Stanford, or to build the skills you need
to found a great startup or be the world's leader in your chosen field,
and for graduate students, finding a great thesis topic.
Lab topics (learn directly from the professor who invented many of these technologies and concepts):
- Fourier transform, wavelet transform, and chirplet transform;
- Machine learning for
computer vision: Radar Vision and LEM neural network (world's first
transform with machine learning built-in);
- AI for humans = Humanistic Intelligence (abbreviated H.I. or H. Int.)... link
- Wearable AI (Wearable AI = H. Int., link)
- VR (Virtual Reality), AR (Augmented Reality), XR (eXtended Reality),
metaverse (metaverse.ieee.org), and
beyond-metaverse = XV (eXtendiVerse).
- Biosignals and biosensing.
In this lab we can build an ultrasound system to image the heart.
Seismocardiographic Signals Using
Polynomial Chirplet Transform...].
- Brain-Computer Interfaces (InteraXon company co-founded by Mann and
- Fluid User Interfaces: Build a musical physiotherapy machine based on
an array of ultrasonic lock-in amplifiers for phase-coherent sonar;
- See and photograph sound waves, radio waves, and light waves using your
- Passive vision:
Many courses on computer vision fail to teach the fundamental concepts of what
sensing is and does. We'll begin with fundamental principles by exploring
first a 1-pixel camera and 1-pixel display, quantigraphic (quantifiable)
sensing, and meta-sensing.
- Begin with fundamentals, e.g. 1-pixel camera and display;
- Quantigraphic sensing: Comparametric Equations;
- Self-driving vehicles, sensing, and meta-sensing;
- Phenomenological augmented reality with Metavision;
- Understanding 3 phase motors and electric vehicles;
- Build your own autonomous e-vehicle...
- Complex-Valued Signal Generators
- Build a signal generator that produces a complex-valued output.
You will fundamentally
understand the difference between positive and negative frequencies
and be able to explain that difference to a 5-year old child!
In later labs you will use this signal generator as the
foundation upon which to build autonomous electric vehicles!
- Phase-coherent detection for active computer vision:
- Active vision systems (sonar, radar, lidar):
Build your own extreme broadband lock-in amplifier;
- Build a sonar vision system for the blind;
- Your final project of your own choosing...
Lab schedule 2023: Thursdays Feb 2, 16 and Mar 2, 16, 30 at 9am
Lab 1. What is a camera?
.Pinhole camera (effect of aperture size),
.Mathematical models tan(arctan())...,
.Lens=optional part of lab.
(easy to make from household items).
.Intro to metaverse (free opensource Blender, Openbrush) and beyond...
.Intro to Gitlab...
Lab 2. (Chapter 4 of text) Comparametric Equations (HDR = High Dynamic Range)
.Photocell experiment or use laptop computer camera, webcam, or the like,
.Comparagrams and comparagraphs,
.Optional: compositing of images; CCRF.
Lab 3. (Chapter 5 of text) Long-exposure photogaphy, CEMENT, Superposimetrics.
.Abakography and Toposculpting (Openbrush, Blender, etc.)
.Fourier series, harmonic analysis in phase space, complex-valued functions in VR/AR/XR...
Lab 4. (Chapter 6 of text)
.Orbits (image stitching),
.Type I, Type II, and Type III panoramas,
.VR, AR, XR 'Vironment maps.
.Metaverse and beyond...
Lab 5. Active vision, radar, sonar, etc.
.SWIM, sonar (audio) with external microphone or speaker.
Optional additional topics (depending on student interest):
- Machine learning, polyphase machine learning, LEM, radar, sonar, lidar
- RGB moveillance;
- Self-driving vehicle;
- 3-phase signal generator for smart cars.
- Metaveillance standards: smart car certification
- Metaverse and
- Wearable computing and Intelligent Image Processing: smart vision
Course instructor: Prof. Steve Mann
TAs: Ali and Navid