Decision Gap
A defect label says what the pattern resembles, but not why it occurred or what the engineer should do next.
Back to Projects
WaferCopilot — GenAI Smart Manufacturing Assistant
Semiconductor Smart Manufacturing Assistant Based on Generative AI
A human-in-the-loop decision workflow that connects wafer defect classification, explainability, SOP-grounded diagnostics, and simulation before engineers approve process adjustments.
View on GitHubProblem
Classification Identifies the Defect, Not the Next Action
Traditional defect classifiers can recognize an abnormal wafer pattern, but they do not automatically explain its process context, identify a root cause, or recommend a safe response. Engineers still need to reconcile visual evidence with equipment logs and SOPs, then judge parameter changes through experience and trial and error.
Fragmented Manufacturing Context
Wafer maps, machine signals, SOPs, and engineering knowledge are disconnected during diagnosis.
High-Cost Trial-and-Error
Testing an unvalidated parameter change on production equipment creates risk and makes expert judgment difficult to transfer.
Proposed Solution
One Decision Workflow From Detection to Engineer-Approved Action
01
Explainable Defect Recognition
Use ResNet-18 to classify eight wafer defect patterns and Grad-CAM to show which regions influenced the prediction.
02
Context-Grounded Diagnosis
Combine simulated machine logs with hybrid retrieval so the LLM can ground root-cause analysis and recommendations in relevant SOP content.
03
Pre-Action Simulation
When an engineer asks to test concrete parameters for Center, Donut, or Random defects, route them to the matching rule-based simulator before considering further action.
04
Human-in-the-Loop Integration
The proposed fab workflow places WaferCopilot between FDC and engineer review; APC would execute only an adjustment approved by the engineer.
System Architecture
One Decision Path, Two Levels of Architecture
The first level shows what the prototype actually runs. The second shows how that decision support is proposed to sit between existing fab systems and engineering judgment.
Prototype Workflow
Four Stages to a Diagnosis, Then Validation Only When Needed
The primary path always produces a traceable diagnosis. Digital-twin validation is a conditional branch triggered by an engineer's simulation request.
- Step 01
Input
Wafer Map
An engineer uploads a PNG or JPG wafer map through the Streamlit interface.
Input
WM-811K-style wafer image
- Step 02
Visual Analysis
Deep Vision Model
ResNet-18 classifies one of eight defect patterns, while Grad-CAM shows the regions behind the prediction.
Output
Defect class, confidence, and heatmap
- Step 03
Manufacturing Context
Mock Production Data
The prediction selects a simulated lot history, route, batch yield, and FDC-style machine log.
Output
Lot history and machine anomaly context
- Step 04
Grounded Reasoning
LLM + RAG Diagnosis
The LLM combines visual evidence, production context, and retrieved SOP knowledge into a cited report.
Primary Result
Root cause, cited SOP, and recommended action
Conditional branch
Only when an engineer requests a parameter simulation
Step 05Conditional Validation
Digital Twin
For Center, Donut, or Random defects, the matching rule-based simulator can estimate yield and risk for requested parameters.
Optional Result
Tested parameters, predicted yield, and risk feedback
My Contribution
Integrated the vision output, simulated production context, RAG diagnosis, conditional digital-twin tools, and the Streamlit conversation interface into one decision-support workflow.
Prototype Evidence
The Workflow Produces a Reviewable Diagnosis, Not Just a Class Label
This Streamlit prototype screen shows an Edge-Loc example: Grad-CAM exposes the model's focus area while the diagnosis panel turns the result into a structured, conversational artifact for engineering review.
- Grad-CAM focus area
- Structured defect summary
- Conversational follow-up
Proposed Fab Integration
Support the Existing Decision Chain Without Replacing It
This is a conceptual deployment boundary. The current prototype is not connected to live FDC or APC systems.
Existing System
FDC
Supplies anomaly signals and equipment data.
Decision-Support Layer
WaferCopilot
Performs defect analysis, context reasoning, and SOP-grounded recommendations.
Decision Authority
Engineer
Reviews the evidence and decides whether to approve the recommendation.
Existing System
APC
Executes only an engineer-approved parameter adjustment.
AI recommends · Engineer decides · APC executes
Implementation & Outcomes
A Prototype That Tests the Full Decision Path
Explainable Visual Analysis
The prototype uses the WM-811K open dataset, ResNet-18 transfer learning, and Grad-CAM. The lightweight model reached 99.2% accuracy after 20 epochs in the team's benchmark while retaining a visual explanation for each prediction.
Traceable Context Reasoning
A synthetic knowledge base, simulated machine logs, and hybrid retrieval combine deterministic defect-section matching with semantic search. In the demonstration scenarios, log context narrowed a broad SOP checklist into a specific root-cause hypothesis and action.
Conditional Digital-Twin Validation
A Factory Pattern routes Center defects to a CMP simulator, Donut defects to an Etch simulator, and Random defects to an environmental simulator. The other five defect classes return a non-simulatable response instead of inventing a yield estimate.
Enterprise-Aware Integration
The proposed deployment sits above existing FDC and APC systems instead of replacing them. It keeps engineers in control and supports both cloud and local LLM options to address performance and fab data-security tradeoffs.
- Integrated Decision Workflow
- 3 Modules
- Visual, cognitive, and validation layers
- Wafer Map Classification Accuracy
- 99.2%
- ResNet-18 after 20 epochs on the WM-811K benchmark
- Engineer Approval
- Human-in-the-Loop
- Proposed operating model; live APC integration remains future work
Limitations
What Remains to Be Validated
WaferCopilot remains a competition prototype built with the WM-811K public dataset, simulated machine logs, a synthetic knowledge base, and a rule-based digital twin surrogate. The 99.2% figure measures the classification module on that benchmark, not end-to-end fab performance. Real adoption would require representative production data, live FDC/APC integration, validation across tools and process conditions, stronger simulation fidelity, access control, and security review.