-
-
Project purpose & Deliverables
-
Our 3D printed headband positioning the camera in front of the subject's eyes. Light added for better camera quality at night.
-
Starting screen showing eye ready to be tracked.
-
Eye tracking calibration
-
Visual eye-tracking popup with motor-skill and reaction time activity
-
Steady path mission
-
Operator dashboard to view results, statistics - with Gemini API chat for quick data summaries
-
3d printed headband design
-
3d printed controller design
-
Circuit schematic
-
Electrical wiring for the controller (includes buzzer, button, joystick)
-
Game Controller (circuitboard with joystick, button, buzzer with custom 3d printed case)
Inspiration
Our team was inspired by the Posner Cueing experiment, which tests how visual cues shift attention using manual or eye-movement reaction times. By simultaneously tracking a child's eye movements and joystick inputs, we can spot subtle lags between seeing and reacting, catch unwanted eye jumps (saccadic intrusions) when they are trying to focus, and measure fine-motor steadiness.
Normally, capturing this kind of objective eye-tracking and motor data requires specialized clinical equipment, long appointments, and referrals. Publicly funded ADHD assessments in parts of Canada can take a year or more. Meanwhile, dyspraxia (developmental coordination disorder) affects about 5–6% of school-aged children, frequently overlaps with ADHD, and often goes unrecognized in schools and clinics.
We wanted to turn a daunting clinical barrier into a quick-and-easy 30-second space-arcade game that kids might actually enjoy playing. This accessible approach supports early prescreening for hard-to-detect conditions like ADHD, dyspraxia, and concussions, giving parents and educators objective data on eye-hand coordination and focus shifts right away.
What it does
BrightEye is a space-arcade game that kids play with a joystick while a camera tracks their eyes. A full run takes about 30 seconds:
- Pilot check-in: Pick a call sign and age group (5-7, 8-10, 11-13, 14-17), note writing hand and glasses, and confirm adult consent.
- Eye calibration: Look at four corners so the tracker can map eye position to the screen.
- Odd-One-Out Corners (15 s): Four stars appear; select the one with the different colour.
- Steady Path (up to 15 s): Steer a ship along a winding trail to collect stars in order while brief red distractors flash off-trail.
Every run records:
- Wrong-corner selections
- Flash-to-gaze time (how quickly the eyes reach a flash)
- Eye-to-hand time
- Off-trail deviation
- Joystick jitter
- Gaze lead (whether the eyes lead or trail the ship along the path)
An Operator view for parents or clinicians displays run history, replays, charts that flag unusually high samples within a run, CSV exports, and hardware diagnostics. It incorporates Gemini API to summarize data quickly and effectively for users.
We were careful with our framing. BrightEye records gameplay measurements from a prototype. It does not diagnose conditions or use screening cutoffs, and it points families to their primary care provider if difficulties persist in everyday life. No camera video is ever recorded.
How we built it
- Controller: An Arduino (Uno, and later the Mega 2560) with an analog joystick, an action button, and a piezo buzzer. The firmware debounces the button, calibrates the joystick's centre at startup, applies a deadzone, and streams the controller state as JSON at 50 Hz over USB serial. The buzzer firmware timestamps each tone using the board's microsecond clock.
- Eye tracker: A Raspberry Pi 5 with a camera running a Python/Flask service. OpenCV locates the eye using open-source Haar cascade, isolates the pupil as the darkest region, and maps it to screen coordinates based on calibration. By starting with calibration, it fits the live pupil location based on the user's eye movement by mapping to x and y equations. The service also detects blinks and fast eye jumps (saccades), then serves gaze data and a live preview to the game.
- Game and dashboard: Built with React 19, TypeScript, Vite, and Tailwind CSS. The browser reads the Arduino directly via the Web Serial API and polls the Pi for gaze data. USB gamepad and mouse/touch modes let anyone play without our custom hardware.
- Storage: A lightweight Node.js API saves completed runs (a timeline of gaze, ship, and input samples) to PostgreSQL on Tiger Cloud, allowing the operator to reload, replay, or delete any run.
Challenges we ran into
- Timing across three clocks: The Arduino, Raspberry Pi, and browser each keep their own time. Network polling and serial delays blur "eye-to-hand" measurements, so we explicitly documented which delays each metric includes rather than pretending they are exact.
- Camera lag: The default camera driver queued frames and added about 130 ms of delay. Shrinking the buffer to a single frame resolved this.
- Serial congestion: At 50 Hz, buzzer timestamps could get stuck behind joystick packets. The firmware now checks for free buffer space before writing and prioritizes buzzer records.
- Low-cost eye tracking: Blinks, glasses, and ambient lighting can easily break simple pupil detection, so we implemented blink handling, smoothing, and per-player calibration for personalized experience.
- 3D printing: Designing and printing personalized hardware required several iterations to ensure comfortable, precise, and controlled movement for both the camera headband and the gaming controller.
Accomplishments that we're proud of
- Building a complete hardware-to-cloud pipeline in a single weekend: joystick and buzzer, camera eye tracker, browser game, operator dashboard, and saved replays.
- Honest measurement design. Every metric clearly states what it captures and what can skew it, avoiding overreaching diagnostic claims.
- Creating a game kids actually want to finish, with a full run taking only 30 seconds.
- Broad accessibility. It works on any laptop via mouse mode, while the custom hardware unlocks real gaze and hand-tracking data.
What we learned
- Real-time systems are largely about timestamps; knowing where a timestamp originates matters just as much as the number itself.
- How to stream microcontroller data directly into a web app using Web Serial, and how to keep a serial protocol simple enough that hardware and front-end development can happen in parallel.
- In health tech, how you frame results matters as much as the underlying technology. Overclaiming is easy and potentially harmful, so we intentionally focused on describing what we measured rather than drawing clinical conclusions.
What's next for BrightEye
- Implementing a hardware sync signal and a microphone so the buzzer test can be measured against when the sound actually plays.
- Moving to a stronger eye tracker (infrared camera, more robust pupil model) and a custom PCB controller housed in a dedicated kiosk enclosure.
- Designing an adjustable camera mount that comfortably fits a wider variety of head shapes and sizes.
Built With
- arduino
- flask
- json
- lucide
- node.js
- numpy
- opencv
- python
- raspberry-pi
- react.js
- sql
- tailwindcss
- tigercloud
- typescript
- vite
