Defect Vision

A full-stack demo built for the Intel Semiconductor Solutions Challenge 2026 - Problem A: Small-Sample Learning for Defect Classification. It wraps a fine-tuned EfficientNet-B2 classifier (trained on a class-balanced, heavily-augmented small-sample dataset) in a FastAPI inference service and a minimal React front end, so the model can be exercised interactively instead of read off a Jupyter notebook.

Live UI: https://defect-vision-mu.vercel.app (frontend only, see note below) Model weights: https://huggingface.co/Sehastrajit/defect-vision-efficientnet-b2

The frontend is deployed on Vercel; the inference API is a PyTorch/FastAPI service that needs a real process (not a serverless function), so it isn't publicly hosted. Run the backend locally (below) and the deployed UI

• or localhost:5173 - will talk to it at http://localhost:8000 out of the box. To make the hosted UI fully live for other people, deploy src/app/backend to any always-on host (Render, Fly.io, a GPU box, etc.) and point VITE_API_URL at it. The trained checkpoint itself is published standalone on Hugging Face (link above) with a usage snippet, for anyone who just wants the weights.

Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries. This is an independent student project built for the challenge and is not an Intel product.

Why this problem is hard

Semiconductor defect classification directly gates yield and time-to-market, but production defect data is almost always scarce and imbalanced - a handful of examples for some defect types, none at all for others in a given batch. The challenge asks for a model that learns fast from limited data the way a human inspector would, and stays reliable when defect classes are wildly imbalanced, while running at production-relevant speed (~1s/image, up to ~1500×2500px input, target ~85% accuracy).

Result

Metric	Target	Achieved
Overall classification accuracy	~85%	95.6% (test) / 97.5% (val)
Inference latency	~1s / image	~40–500ms / image (GPU / CPU)
Defect classes	8 + background	8 defect classes + "no defect" (9-way softmax)

Full per-class classification report (held-out test set, 360 images)

Test Loss : 0.6153  |  Test Accuracy : 0.9556

              precision    recall  f1-score   support

     defect1     0.9773    0.9556    0.9663        45
     defect2     0.9375    1.0000    0.9677        45
     defect3     1.0000    1.0000    1.0000        45
     defect4     1.0000    1.0000    1.0000        45
     defect5     0.9130    0.9333    0.9231        45
     defect8     0.8837    0.8444    0.8636        45
     defect9     0.9556    0.9556    0.9556        45
    defect10     0.9773    0.9556    0.9663        45
    new_good     0.0000    0.0000    0.0000         0

    accuracy                         0.9556       360
   macro avg     0.8494    0.8494    0.8492       360
weighted avg     0.9555    0.9556    0.9553       360

new_good has zero held-out samples in this dataset revision - the 9th output neuron is reserved for future "no defect found" imagery so the model doesn't need retraining/re-architecting to add it later.

Screenshots

Classify	Live prediction

Model insights

About / problem statement

How it works

┌────────────────────┐      image (multipart)      ┌───────────────────────────┐
│   React + Vite UI   │ ───────────────────────────▶ │        FastAPI service     │
│  Classify · Insights│                              │  EfficientNet-B2 (PyTorch) │
│      · About        │ ◀─────────────────────────── │  class probs + latency     │
└────────────────────┘      JSON prediction          └───────────────────────────┘
                                                                  │
                                                                  ▼
                                                   src/dataset/output_balanced/
                                                   best_model.pth, evaluation plots,
                                                   split_summary.json

1. Ingest - a gray-scale wafer/die image (up to ~1500×2500px) is uploaded or picked from a held-out sample gallery.
2. Preprocess - resized to 260×260 (EfficientNet-B2's native resolution) and ImageNet-normalized.
3. Backbone - EfficientNet-B2 (~9.2M params, ImageNet-pretrained, fine-tuned end-to-end).
4. Head - Dropout(0.4) → Linear(1408→512) → SiLU → Dropout(0.3) → Linear(512→9).
5. Output - softmax over 9 classes, returned with per-class probabilities and measured inference time.

Handling class imbalance with few samples

• Class-balanced dataset construction - equal train/val/test counts per class (210/45/45) via augmentation, rather than naive minority oversampling or loss reweighting, so the model never learns a majority-class prior.
• Aggressive augmentation - random crop, flips, rotation, perspective warp, and color jitter multiply the small per-class sample count without duplicating exact pixels.
• Label smoothing (0.1) on cross-entropy keeps the model from over-committing on visually similar defect types.
• OneCycleLR + early stopping (patience 7) for fast, stable convergence on limited data - the run in this repo converged and early-stopped at epoch 16.

Full training script: src/app/h1.ipynb.

Project structure

intel/
├── README.md
├── Problem A_ Small Sample Learning for Defect Classification (1).pdf   # official problem brief
├── docs/
│   ├── banner.svg
│   └── screenshots/
├── src/
│   ├── app/
│   │   ├── h1.ipynb                 # training pipeline (EfficientNet-B2 fine-tune)
│   │   ├── backend/                 # FastAPI inference service
│   │   │   ├── app/
│   │   │   │   ├── main.py          # routes: /predict, /metrics, /insights, /samples
│   │   │   │   └── model.py         # model definition + checkpoint loading
│   │   │   └── requirements.txt
│   │   └── frontend/                # React + TypeScript + Tailwind UI
│   │       └── src/
│   │           ├── components/      # ClassifyPage, InsightsPage, AboutPage, ...
│   │           ├── api/client.ts
│   │           └── constants.ts     # class → color/label mapping
│   └── dataset/
│       └── output_balanced/
│           ├── best_model.pth               # trained checkpoint (val acc 97.5%)
│           ├── classification_report.txt
│           ├── confusion_matrix_{val,test}.png
│           ├── training_curves.png
│           ├── training_history.json
│           ├── split_summary.json
│           └── split_data/{train,val,test}/<class>/*.png
└── deploy/
    ├── hf-space/    # Docker Space packaging for the FastAPI backend (needs HF PRO to host)
    └── hf-model/    # Model card + LICENSE pushed to the standalone HF model repo

Running it locally

1. Backend (FastAPI + PyTorch)

cd src/app/backend
pip install -r requirements.txt
uvicorn app.main:app --host 127.0.0.1 --port 8000

Uses CUDA automatically if available (torch.cuda.is_available()), otherwise falls back to CPU. The model checkpoint is loaded from src/dataset/output_balanced/best_model.pth relative to the repo - no extra config needed.

Health check: curl http://127.0.0.1:8000/api/health

2. Frontend (React + Vite)

cd src/app/frontend
npm install
cp .env.example .env.local   # VITE_API_URL, defaults to http://localhost:8000
npm run dev

Open the printed local URL (default http://localhost:5173).

API reference

Method	Route	Description
GET	/api/health	Service + device status, checkpoint val accuracy/epoch
GET	/api/classes	Class list + train/val/test split summary
GET	/api/metrics	Classification report, training history, split summary
GET	/api/samples?per_class=N	Random held-out test filenames per class, for the demo gallery
GET	/api/samples/{class}/{filename}	Raw sample image bytes
GET	/api/insights/{name}	Evaluation plot PNG - training-curves, confusion-matrix-val, confusion-matrix-test
POST	/api/predict	Multipart image upload → predicted class, full probability breakdown, inference time

Example:

curl -F "file=@wafer_sample.png" http://127.0.0.1:8000/api/predict

{
  "predicted_class": "defect1",
  "confidence": 0.843,
  "probabilities": { "defect1": 0.843, "defect9": 0.040, "...": 0.0 },
  "top3": [{"class_name": "defect1", "probability": 0.843}, "..."],
  "inference_ms": 38.2,
  "image_size": { "width": 1166, "height": 692 },
  "filename": "wafer_sample.png"
}

Tech stack

• Model: PyTorch 2.4, torchvision EfficientNet-B2 (ImageNet-pretrained backbone, custom head)
• Backend: FastAPI, Uvicorn
• Frontend: React 19, TypeScript, Vite, Tailwind CSS
• Training hardware: NVIDIA RTX 3060 12GB, mixed precision (fp16 AMP), gradient accumulation

Deliverables checklist (per the problem brief)

• Application that detects and classifies defects from gray-scale images
• Evaluation plots - confusion matrices (val/test) and training curves, served live from the API
• ~85% target accuracy - achieved 95.6% test / 97.5% val
• Demonstrates learning speed - training curves show convergence by ~epoch 10, early-stopped at 16
• Documentation of approach, assumptions, class-imbalance handling, and hardware used (this README + About tab)