Integritas — Real-Time Structural Crack Inspection & Health Monitoring Studio

A unified deep learning + IoT pipeline for concrete and masonry structural inspection, combining:

1. AI Crack Detection — A trained UNet-VGG16 convolutional neural network that detects surface cracks in real-time video and high-resolution photos.
2. Structural Health Monitoring (SHM) — Live MPU-6050 6-axis IMU telemetry (tilt, vibration, shock, temperature) streamed from an ESP32 microcontroller, with a master alert engine, 30-minute rolling history, and configurable threshold alarms.

Integritas/
├── model_unet_vgg_16_best.pt          # Trained UNet-VGG16 weights (Git LFS)
├── unet_model.py                      # Standalone PyTorch UNet16 architecture & weight loader
├── check_model.py                     # Model diagnostic audit, parameter verification & benchmark
├── predict.py                         # Core inference engine & CLI (tiled/full modes, metrics, panels)
├── quick_test.py                      # Simple 1-step demo script
├── web_app.py                         # Flask web dashboard (crack AI + MPU SHM + ESP32 bridge)
├── live_camera.py                     # Native desktop OpenCV camera inspector
├── requirements.txt                   # Python dependencies (pip install -r requirements.txt)
├── model_audit_report.json            # Automated audit report from check_model.py
│
├── templates/
│   └── index.html                     # Full web interface (light blue theme)
├── static/
│   ├── css/
│   │   └── mpu_alert_system.css       # Alert panel & SHM dashboard styles
│   └── js/
│       ├── mpu_alert_engine.js        # Master alert engine (thresholds, debounce, notifications)
│       ├── mpu_alert_ui.js            # Alert panel UI rendering & interaction
│       ├── mpu_history_store.js       # 30-minute rolling telemetry history & statistics
│       └── mpu_simulator.js           # Built-in demo simulator for judge presentations
│
├── esp32_cam_mpu6050/                 # Arduino firmware: ESP32-CAM + MPU-6050 (camera + tilt)
│   ├── esp32_cam_mpu6050.ino
│   └── README.md                      # Wiring diagram, FTDI upload, pin mapping
├── esp32_mpu6050/                     # Arduino firmware: ESP32 DevKit + MPU-6050 (tilt only)
│   ├── esp32_mpu6050.ino
│   └── README.md                      # Wiring diagram & upload instructions
│
├── sample_images/                     # Curated test images (cracks, moss, distractors)
├── predictions/                       # Output directory for masks, heatmaps, and panels
└── snapshots/                         # Saved camera snapshots from live_camera.py

• Python 3.10+ (tested on Python 3.12)
• Git LFS (for downloading the 112 MB model weights)

git clone https://github.com/adi-coderr/Integritas.git
cd Integritas

# Create a virtual environment (recommended)
python3 -m venv .venv
source .venv/bin/activate      # macOS / Linux
# .venv\Scripts\activate       # Windows

# Install all required packages
pip install -r requirements.txt

Git LFS should automatically download model_unet_vgg_16_best.pt during clone. If the file is only a pointer (< 1 KB), run:

git lfs pull

Verifies layer weights, parameters, hardware acceleration, and benchmarks inference speed:

python3 check_model.py

Runs inference on sample_images/pure_crack_01.jpg and saves quick_test_output.jpg:

python3 quick_test.py

python3 web_app.py

Open http://localhost:5050 in your browser. The dashboard has two major sections:

• Live Footage AI Stream: The UNet-VGG16 model continuously analyzes live video from your camera in real time (~190ms / ~5.3 FPS on Apple Silicon Metal GPU).
  • Real-time crack segmentation overlays, boundary contours, and live bounding boxes.
  • Live HUD verdict: ⚠️ CRACK DETECTED (with damage %, confidence, and severity color) vs 🟢 SOUND SURFACE.
  • Audio alert chime when cracks enter the frame.
  • Cycle live view modes: Overlay, Dual Side-by-Side, Heatmap, and Binary Mask.
  • Live timer HUD and shutter flash effect on capture.
• 📸 Take Picture & Deep Scan:
  • Captures full-resolution uncompressed photo from the camera.
  • Executes multi-tile overlapping deep scan with sub-millimeter flaw localization.
  • Populates the Pinpointed Flaw Locations & Structural Severity Inventory table (coordinates, pixel area, severity rating).
  • Interactive Before / After Split Reveal Slider to compare raw photo vs AI detected flaws.
  • Export and download annotated inspection report.

• Master Alert Banner: Live system status (NOMINAL / WARNING / CRITICAL) with color-coded severity.
• Real-Time Gauges: Dual tilt gauges (Pitch & Roll), acceleration, gyroscope, vibration RMS, and temperature.
• 30-Minute Rolling History: Charts and statistics for all telemetry axes.
• Master Alert Engine:
  • Configurable warning & critical thresholds for: tilt, shock, vibration, rotation speed, freefall, temperature, and connection loss.
  • Anti-false-alarm hysteresis with consecutive-sample debounce.
  • 10-second cooldown per alert type.
  • Audible Web Audio chimes (warning tone vs critical tone).
  • Browser push notifications for critical alerts.
  • Optional webhook URL for external alert delivery.
• Connection Modes (connect the ESP32 any way you like):
  • USB Serial Cable — plug in and select the COM / serial port.
  • Wi-Fi Direct Push — ESP32 pushes data to http://<your-laptop-ip>:5050/api/mpu/feed (no ESP32 IP needed!).
  • Wi-Fi Pull — dashboard polls http://<esp32-ip>/telemetry.
  • Port 3000 Bridge — a built-in HTTP bridge on port 3000 for direct ESP32 compatibility.
  • Demo Simulator — built-in physics-based simulator for testing and judge presentations (no hardware needed).
• Tare Calibration: Zero the current orientation as the reference level.
• Test Shock: Trigger a simulated high-G shock to test the alert pipeline.
• PDF Report: One-click printable summary report.
• Presentation Mode: Clean fullscreen view for projector demos.

If you prefer a native desktop window with OpenCV:

python3 live_camera.py

• Real-time overlay directly on your screen.
• Hotkeys: [SPACE] to freeze, [c] for CLAHE contrast, [m] to cycle modes, [s] to save snapshot, [q] to quit.

Two firmware options are included depending on your board:

See esp32_mpu6050/README.md for wiring and upload instructions.

See esp32_cam_mpu6050/README.md for wiring, FTDI adapter upload, and camera-specific pin conflicts.

Important: ESP32-CAM uses GPIO 13 (SDA) and GPIO 14 (SCL) because the default I2C pins (21, 22) are used by the OV2640 camera!

The recommended method is Wi-Fi Direct Push — the ESP32 pushes telemetry directly to your laptop. Set your laptop's IP in the .ino sketch:

const char *pushServerHost = "10.169.71.17"; // Your laptop's LAN IP
const int   pushServerPort = 5050;
const bool  enableDirectPush = true;

No need to know the ESP32's IP address. Just power it on and data flows automatically.

Run batch inference on individual images or entire directories:

# Run on sample images using high-resolution tiled mode
python3 predict.py --input sample_images/ --mode tiled --threshold 0.50

# Run on a single photo
python3 predict.py --input /path/to/my_wall.jpg --output-dir my_results/

# Fast mode (resizes image to 448x448)
python3 predict.py --input sample_images/ --mode full

• <filename>_comparison.jpg: 3-panel composite: [Original Photo] | [Probability Heatmap] | [Crack Overlay + HUD]
• <filename>_overlay.jpg: High-res original photo with highlighted crack boundaries
• <filename>_mask.png: Binary 0/255 segmentation mask
• batch_summary.json: Machine-readable metrics (confidence, area %, pixel count)

from predict import CrackPredictor

# Initialize predictor (auto-detects MPS / CUDA / CPU)
predictor = CrackPredictor("model_unet_vgg_16_best.pt")

# Run inference
result = predictor.predict("sample_images/pure_crack_01.jpg", mode="tiled", threshold=0.5)

print("Crack detected:", result["crack_detected"])
print(f"Affected surface: {result['crack_percentage']}%")
print(f"Peak confidence: {result['peak_probability'] * 100:.1f}%")
print(f"Inference latency: {result['inference_time_ms']} ms")

# Binary mask (numpy uint8 array, 0 or 255)
mask = result["binary_mask"]

import torch
from PIL import Image
import numpy as np
from unet_model import load_unet16_model, get_default_device

device = get_default_device()
model, meta = load_unet16_model("model_unet_vgg_16_best.pt", device=device)

# Prepare 448x448 image tensor
img = Image.open("sample_images/pure_crack_01.jpg").convert("RGB").resize((448, 448))
arr = np.array(img).astype(np.float32) / 255.0
arr = (arr - np.array([0.485, 0.456, 0.406])) / np.array([0.229, 0.224, 0.225])
tensor = torch.from_numpy(arr).permute(2, 0, 1).unsqueeze(0).float().to(device)

with torch.no_grad():
    logits = model(tensor)
    probabilities = torch.sigmoid(logits)[0, 0].cpu().numpy()

# probabilities is a (448, 448) float array with values in [0.0, 1.0]

• tiled (Recommended for High-Res Photos): Slides a 448×448 window across the image with 25% overlap, runs inference on each patch, and smooths overlapping probabilities. Prevents thin hairline cracks from being blurred or erased by downscaling.
• full (Fastest): Resizes the entire image directly to 448×448 in a single forward pass, then resizes the mask back to original dimensions. Ideal for fast previews or video frames.

This project was built for the TechTrix competition.