GestureCanvas
Making hand tracking usable as an input method rather than a demo, with no installation and no hardware beyond a webcam.
- Role
- Sole engineer
- Context
- Personal project
- Stack
- React · TypeScript · Vite
- Status
- Not deployed
Not currently deployed. The source is the best look at this one.
Real-time hand tracking through MediaPipe, interpreted by a gesture engine and rendered to an HTML5 canvas. The modules are split by responsibility — tracking, gesture interpretation, canvas rendering — because the hard part is the boundary between them.
Three engines, not one component
MediaPipe integration, gesture interpretation and canvas rendering live in separate modules with custom hooks bridging model inference into React's lifecycle. Collapsing them into one component is the obvious first version and the reason that version becomes unmaintainable.
Latency against smoothing
Raw landmark coordinates jitter, so strokes look shaky. Smoothing fixes the line and adds lag, and past a certain point the cursor stops feeling attached to the hand. The useful range is narrower than it first appears.
- React
- TypeScript
- Vite
- MediaPipe