Detecting squares in OpenCV when a diagonal splits the contour: rebuild the box from perpendicular edges

Make OpenCV square detection robust when a diagonal breaks contours: use polygon approximation and perpendicular edges to reconstruct the box. Works on rotations.

Detecting squares with clean boundaries is straightforward; detecting the same squares when there’s a diagonal drawn inside them is where a typical contour-based pipeline often fails. If your edge map contains that inner diagonal, the contour you get may no longer represent the outer square. Here is a practical way to make such detections robust, staying within a familiar OpenCV workflow.

Problem overview

You need to find the contours of boxes. Some boxes have a diagonal drawn inside. A standard pipeline based on edge detection and contour filtering fails to detect the box that contains the diagonal. The boxes may be rotated.

Baseline pipeline that misses the diagonal box

The following code performs a Laplacian edge detection, extracts contours, filters by area, and uses cv2.minAreaRect to validate candidate boxes by size. The logic is sound, but it struggles when a diagonal splits the shape.

def pull_cells(mat, trace: bool = False):
    if trace:
        mat = cv2.imread(mat)
    new_w = 710
    new_h = 710
    mat = cv2.resize(mat, (new_w, new_h))
    gray = cv2.cvtColor(mat, cv2.COLOR_BGR2GRAY)
    blur = cv2.GaussianBlur(gray, (5, 5), 0)
    lap = cv2.Laplacian(blur, cv2.CV_8U, ksize=5)
    panel = np.ones_like(mat) * 255
    curves, hier = cv2.findContours(lap, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
    print(f"Number of Contours: {len(curves)}")
    tiles = []
    for shp in curves:
        patch = cv2.contourArea(shp)
        if patch < 1900:
            continue
        rr = cv2.minAreaRect(shp)
        pts = cv2.boxPoints(rr)
        pts = np.int32(pts)
        ww, hh = rr[1]
        if 50 < min(ww, hh) < 80 and 50 < max(ww, hh) < 80:
            tiles.append(pts)
            cv2.drawContours(mat, [pts], 0, (0, 255, 0), 2)
            cv2.drawContours(panel, [pts], 0, (0, 0, 255), 2)
    cv2.imshow('lap', lap)
    cv2.imshow('Detected Blocks', mat)
    cv2.imshow('Detected Blocks (White Background)', panel)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

Why it fails

After Laplacian, every prominent line becomes an edge, including the diagonal. The contour extraction can then produce shapes that follow the diagonal, effectively splitting the square into two triangles. A min-area rectangle fitted to such a contour is not guaranteed to reflect the outer square’s geometry and may fail the size check. In short, the inner diagonal changes the connected components so the target square isn’t represented by a single clean contour anymore.

Solution: reconstruct the square from perpendicular axes

Keep the same pre-processing and contour extraction. Instead of immediately fitting a rotated rectangle, approximate each contour as a polygon and look for two consecutive edges that are roughly perpendicular and whose lengths match your side constraints. When you find such a pair, reconstruct the missing fourth corner, building the square even if the contour only gave you a triangle due to the diagonal. This approach uses only geometry extracted from the contour itself.

You can also consider verifying that the newly constructed edges are supported by features in the edge map to avoid false positives. Another alternative mentioned in practice is using cv2.minAreaRect to also retrieve the angle and points; the angle typically needs the usual handling in OpenCV.

Revised detection with axis-based reconstruction

This version keeps the same pipeline up to contours, then switches to polygonal approximation and square reconstruction via perpendicular vectors.

def pull_cells_axes(mat, trace: bool = False):
    if trace:
        mat = cv2.imread(mat)
    new_w = 710
    new_h = 710
    mat = cv2.resize(mat, (new_w, new_h))
    gray = cv2.cvtColor(mat, cv2.COLOR_BGR2GRAY)
    blur = cv2.GaussianBlur(gray, (5, 5), 0)
    lap = cv2.Laplacian(blur, cv2.CV_8U, ksize=5)
    panel = np.ones_like(mat) * 255
    curves, hier = cv2.findContours(lap, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
    print(f"Number of Contours: {len(curves)}")
    tiles = []
    for shp in curves:
        patch = cv2.contourArea(shp)
        if patch < 1900:
            continue
        eps = 0.02 * cv2.arcLength(shp, True)
        poly = cv2.approxPolyDP(shp, eps, True)
        for k in range(len(poly)):
            a = poly[k][0]
            b = poly[(k + 1) % len(poly)][0]
            c = poly[(k + 2) % len(poly)][0]
            v1 = b - a
            v2 = c - b
            d1 = np.linalg.norm(v1)
            d2 = np.linalg.norm(v2)
            if 50 < d1 < 80 and 50 < d2 < 80:
                cosang = np.dot(v1, v2) / (d1 * d2)
                ang = np.arccos(cosang) * 180 / np.pi
                if 80 < ang < 100:
                    print(f"Found square with side lengths {d1:.2f} and {d2:.2f} and angle {ang:.2f}")
                    t4 = c - v1
                    t4 = a + v2
                    t4 = t4.astype(int)
                    quad = np.array([a, b, c, t4])
                    tiles.append(quad)
                    cv2.drawContours(mat, [quad], 0, (0, 255, 0), 2)
                    cv2.drawContours(panel, [quad], 0, (0, 0, 255), 2)
                    break
    print(f"Number of Blocks: {len(tiles)}")
    cv2.imshow('lap', lap)
    cv2.imshow('Detected Blocks', mat)
    cv2.imshow('Detected Blocks (White Background)', panel)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

Sample output

Number of Contours: 3982
Found square with side lengths 63.20 and 68.03 and angle 87.15
Found square with side lengths 68.12 and 65.19 and angle 91.03
Found square with side lengths 63.63 and 67.74 and angle 90.92
Found square with side lengths 66.76 and 65.80 and angle 89.75
Number of Blocks: 4

Why this matters

Real-world inputs rarely stay clean. A single line inside a shape can derail contour-based heuristics that assume a solid boundary. By switching the decision criterion from “does the contour look like a square’s rectangle?” to “do we observe two perpendicular, properly sized edges that imply a square?”, you recover robustness to internal strokes and rotations while reusing the same edge and contour extraction.

Takeaways

If your contour fails because of internal diagonals, don’t fight the edge map. Work with it by extracting local geometric cues from the polygonal approximation. Two adjacent edges of similar length and near 90 degrees are sufficient to reconstruct the missing corner and restore the square. Keep your pre-processing, adapt the detection rule, and add optional verification against your edge map when you need to tighten precision.