Dish Detection Vision Bot
Project Overview
An automated roommate accountability system that uses computer vision to detect when dishes are left in the sink and identify who left them — then logs and displays the results on a web dashboard. The system combines a custom-trained YOLOv8/NCNN dish detection model with OpenCV face recognition, running on a Raspberry Pi 4 with two cameras.
How It Works
- Dish Camera (Camera 0) watches the sink. A custom YOLOv8/NCNN model detects dishes in real time.
- When a dish is detected, the system assigns it a unique tracking ID and begins a 3-second face-capture window.
- Face Camera (Camera 4) runs OpenCV LBPH face recognition simultaneously.
- After the dish has been stationary for 15 seconds, the system associates it with the closest recognized face from the capture window.
- The event (person name, timestamp, dish snapshot, face snapshot) is saved to a SQLite database.
- A Flask web dashboard serves live dual-camera feeds and a paginated event history.
Hardware
Bill of Materials
| # | Item | Description | Qty |
|---|---|---|---|
| 1 | Raspberry Pi 4 | Main compute board running all software | 1 |
| 2 | Logitech Brio | 4K webcam used as the face recognition camera | 1 |
| 3 | Small USB Camera | Compact USB camera for dish detection at the sink | 1 |
Software Architecture
The application is built around a unified Flask server that runs both camera pipelines concurrently.
| Script | Description |
|---|---|
app.py | Flask server, dual camera loop, dish-to-face association logic |
dish_tracker.py | YOLOv8/NCNN inference, IoU-based tracking, stationary detection |
face_recognizer.py | OpenCV LBPH face detection and recognition |
database.py | SQLite event storage and query operations |
media_cleanup.py | Auto-deletes snapshots older than 7 days |
Face Recognition
The face recognition pipeline uses OpenCV's LBPH (Local Binary Pattern Histograms) algorithm with Haar Cascade detection. Before deployment, face images are captured per person using capture_faces_opencv.py and the model is trained with train_faces_opencv.py, producing face_recognizer.xml. 8–10 photos per person are recommended for reliable accuracy.
During operation, the face camera runs recognition every 3rd frame. Faces are labeled with a green bounding box if recognized or red if unknown.
Web Interface
The Flask dashboard is accessible on the local network at http://dish-bot.local:5000.
| URL | Page |
|---|---|
/ | Live dual-camera dashboard |
/history | Detection history with pagination |
/api/recent_events | Last 10 events (JSON) |
/api/daily_stats/YYYY-MM-DD | Daily dish count by person (JSON) |
Dish Tracking Color Key
| Color | Meaning |
|---|---|
| Blue | Dish is moving |
| Yellow | Dish is stationary, waiting |
| Orange | Dish stationary 15s+ — ready for association |
| Green | Dish associated with a person |
Future Plans
- Daily leaderboard sent to GroupMe at 10pm
- Manual override commands via GroupMe
- Dish removal tracking (when person comes back to wash)
- Achievement system and streaks
- Export detection history to CSV