Sudoku OCR
Point it at a photo of a Sudoku grid and it detects, reads, solves, and hands the answer back — with a full computer-vision pipeline written from scratch, no OpenCV involved.
Started in 2023 · rewritten solo from a school project
See it solve a real photo
A short recording of the app end-to-end — load a photo, correct the detected grid or a digit if needed, and get the solution back.
From a blurry photo to a solved grid.
Sudoku OCR is a C++/Qt6 desktop app that takes a photo of a Sudoku — however skewed, shadowed or slightly blurry — and turns it into a solved puzzle overlaid back onto the original picture. It started as a group project in C at EPITA; I later rewrote the whole thing in modern C++, replaced the GTK3 interface with Qt6, and kept improving it since.
Every computer-vision step — grayscale conversion, bilateral filtering, Canny edge detection, the Hough transform, perspective correction — is hand-written, with no OpenCV or other vision library involved. Digit recognition runs on a small fully-connected neural network built on Neurocore, a deep-learning library I co-wrote with a friend, and the finished grid is solved with a constraint-based backtracking search.
Cleaning up a real photo, not a scan
The image is grayscaled, denoised with a bilateral filter, opened (dilate then erode) to clear speckle, and Gaussian-blurred. Binarization then picks between two adaptive-threshold strategies based on the image's measured pixel dispersion, so a crisp flatbed scan and a grainy, unevenly lit phone photo both end up cleanly black-and-white.
Finding the grid without a library to ask
Canny edge detection is implemented from its individual stages — gradients, non-maximum suppression, double thresholding, hysteresis — and feeds a hand-written Hough transform whose vote threshold is nudged up or down automatically until a plausible number of lines shows up. A flood fill isolates the largest connected group of edge pixels, and the four corners are found with a homemade algorithm that repeatedly picks the point farthest from the corners already found, rather than relying on a generic bounding box.
A dedicated squareness check and a fitness score double-check that the detected quadrilateral is actually square-ish and mostly outlined in black before it's trusted.
Straightening the grid, isolating each digit
A projective transform is built from the four detected corners and used to warp the grid into a straight 9×9 square, which is then split into individual cells. Empty cells are flagged by their ink ratio; for the rest, every connected component in the cell is scored by a fitness function that rewards both size and closeness to the cell's center, so a stray grid-line fragment never gets mistaken for a digit, before each digit is centered and resized for the network.
Reading digits with Neurocore
Each isolated digit is classified by a small fully-connected network — dropout layers, a choice of activation functions, trained with the Adam optimizer — built on Neurocore, a deep-learning library I co-wrote with a friend. Training splits batches across threads: each thread trains its own copy of the network, and the resulting gradients are merged back before the next step. The trained weights are saved to, and loaded from, a plain binary file at runtime.
Backtracking, then handing the photo back
Once the 81 digits are known, a constraint-based backtracking solver — precomputing which digits are still possible per row, column and 3×3 box — fills in the rest. The solution is then rendered back through the inverse of the original perspective transform, so the digits land exactly on the original, unwarped photo instead of a cropped, straightened copy.
Watching the solver think
The exact backtracking algorithm from the C++ solver, re-implemented here and run live in your browser on a sample puzzle — including the moment it backtracks.
Starting from the given clues.
- C++
- Qt6
- Computer Vision
- Neural Networks
- Adam Optimizer
- Multithreading