Gaze-controlled computer

2022 COE Engineering Design Project (NMK05)

---

Faculty Lab Coordinator

Naimul Mefraz Khan

Topic Category

Software Systems

Preamble

Many Canadians suffer from complete physical disability due to debilitating diseases like cerebral palsy. An accessible means to control computer input can greatly enhance their day-to-day life.

Objective

To create a software that can help users control the mouse and (virtual) keyboard of a computer through gaze only.

Partial Specifications

1. Must have accurate and real-time gaze data extracted from webcam only.
2. Must be able to perform at least the basic tasks (click, double click, type).
3. Must demonstrate some successful basic task flows (e.g. open a folder, open the browser, search by typing in the address bar).

Suggested Approach

1. Create a movable mount for the webcam that will put the camera close to the users' eyes.
2. Machine learning-based model to extract gaze from webcam in real-time.
3. Create API to control the mouse and virtual keyboard movement.
4. Connect the two aspects.

Group Responsibilities

Please see below

Student A Responsibilities

Research gaze extraction machine learning models

Student B Responsibilities

Research gaze extraction machine learning models

Student C Responsibilities

Research computer control possibilities through programming

Student D Responsibilities

Research optimal positioning of the webcam and custom mount needed to optimally position it

Course Co-requisites

To ALL EDP Students

Due to COVID-19 pandemic, in the event University is not open for in-class/in-lab activities during the Winter term, your EDP topic specifications, requirements, implementations, and assessment methods will be adjusted by your FLCs at their discretion.