GazeMap: Webcam Eye Tracking and Attention Heatmaps

A CNN trained from scratch turns a laptop webcam into an eye tracker and records where you look on images and video.

problem

Eye trackers need infrared cameras and special hardware. A normal webcam is everywhere, but low resolution, changing light and different eyes make gaze hard to read from it.

solution

MediaPipe finds the eye and mouth corners, a 3D face model gives the head pose, and each eye is cropped and straightened the same way as the MPIIGaze training images. A small CNN trained from scratch predicts a gaze direction per eye, the two eyes are fused, and a 13-dot calibration maps gaze onto the screen. A 16-dot test then shows the accuracy, and the demo records gaze trails and heatmaps on any image or video. Everything runs offline; the trained models download on the first run.

I trained it leave-one-person-out across all 15 MPIIGaze participants: each model learns from 14 people and is tested on the one it never saw. One eye alone gives 6.03° mean angular error. Fusing both eyes brings that to 4.97° and helps all 15 people. With a 9-frame calibration from one day, tested on the other days, it reaches 3.98°.

The live demo measured 1.55° (1.9 cm on screen) after calibration. That is one user on one MacBook in the same session, so read it as a best case.

Where it breaks

With the head turned more than 30° the error rises to 7.28°, and covering the top 30 % of the eye adds 2.39°. Calibration also loses about 45 % of its gain once the head turns more than 20° from where you calibrated. Low light, surprisingly, costs nothing once the image is histogram-equalised.

What it shows

Heatmaps on paintings, reading and visual search show where attention goes first and what gets skipped. On driving footage it shows where the eyes were when something happened on the road. It is a research demo, not a safety system: a webcam gives centimetres, not millimetres.

What I learned

Training longer did not help. Between epochs 5 and 15 the training loss fell about 40 % while the error on the held-out person stayed flat. A wider network was the only change that clearly moved the number. The limit is how different people's eyes look, not how long you train.

Solo project: data pipeline, training, evaluation and demo. Built on the MPIIGaze dataset by Zhang, Sugano, Fritz and Bulling. The evaluation has not been peer reviewed. The same question in VR became my next project, gaze-XR-training.