Computer Vision for Quality Inspection: A Practical Guide

Automated visual inspection is one of the few AI applications with an unambiguous business case: it runs continuously, it does not get tired at the end of a shift, and its decisions are consistent in a way human inspection is not. It is also one of the applications where the gap between a working prototype and a working installation is widest.

The reason is that almost none of the difficulty is in the model. It is in optics, class imbalance, label disagreement, cycle time and integration, and those are determined long before anyone trains anything.

Start with the physics, not the model

More vision projects are decided by lighting than by architecture. If the defect is not reliably visible in the image, no model recovers it, and the amount of engineering effort spent compensating for a poor optical setup is almost always greater than the cost of fixing the setup.

Before any modelling work, establish:

• Illumination that makes the defect separable. Diffuse, coaxial, darkfield, backlight and structured light each reveal different defect classes. Surface scratches and dimensional deviations often need entirely different setups.
• Resolution against the smallest defect that matters. A useful rule of thumb is at least three to five pixels across the minimum feature you must detect, below that, detection becomes unreliable regardless of model.
• Consistent part presentation. Variation in position, rotation and standoff distance is variation the model has to learn to ignore, and every bit of it costs you training data.
• Environmental stability. Ambient light changing between day and night shift is a classic cause of models that work in commissioning and degrade a month later.

An afternoon with a lighting engineer routinely removes more error than a week of model tuning.

The class imbalance problem is the real problem

A good production line is, by definition, one that rarely produces defects. That is excellent operationally and awkward statistically: you may be looking for an event that occurs in one part in ten thousand.

Practical consequences:

• Accuracy is meaningless. A model that declares everything good is 99.99% accurate and worthless. Track recall per defect class, and false-reject rate separately.
• You may not have enough examples of each defect class to train a supervised classifier at all, which pushes you toward anomaly detection, where you model normality and flag deviation, rather than classification.
• Rare classes need deliberate collection. That often means keeping physical defect samples and imaging them, or synthesising defects, rather than waiting for production to supply them.

Your labels disagree, and you need to know by how much

Before treating historical inspection records as ground truth, run an agreement study: have several experienced inspectors independently classify the same set of parts and measure how often they agree with each other.

The result is frequently uncomfortable, agreement in the 70–85% range is common for subjective surface defects. That number matters enormously, because it is effectively the ceiling on what a supervised model can be measured against. If your inspectors agree 80% of the time, a model scoring 85% against one inspector's labels may be performing better than the label set suggests.

It also forces a valuable conversation about what the standard actually is, which frequently improves the human process regardless of whether the AI ships.

Cycle time is a hard constraint, not a target

The system has to produce a decision before the part reaches the reject mechanism. That budget includes image acquisition, transfer, preprocessing, inference and the signal back to the controller, not just the model's forward pass.

This is what usually forces inference to the edge. A round trip to a cloud endpoint introduces latency and, more importantly, a dependency: when the network degrades, the line either stops or runs uninspected. Neither is acceptable, so inference generally belongs on hardware at the cell, with the cloud used for training, monitoring and model distribution rather than for the decision itself.

Integration is where the value is realised or lost

A vision system that displays results on its own screen has automated looking, not inspecting. The output has to reach the systems that already run the plant, and that happens at several distinct tempos:

OPC-UA is the usual route for structured exchange with MES and SCADA, with MQTT commonly used for higher-frequency streaming to analytics. Older installations may need middleware or a custom interface, and scoping that work is regularly the longest single item in the plan.

Plan for drift before go-live

Vision models degrade in ways that are specific and predictable. Suppliers change material finish. A camera drifts out of focus or accumulates dust. A lamp ages and its colour temperature shifts. A new product variant arrives that nobody mentioned.

What a maintainable installation has:

• Monitoring on the input distribution, not only on the reject rate
• A sample of production images retained for periodic revalidation
• A defined procedure for adding a new defect class or product variant
• Someone named as owner of model performance after handover

A workable sequence

1. Agree the defect taxonomy and run an inspector agreement study
2. Fix optics and lighting; validate the defect is visible before modelling
3. Collect a representative image set, including deliberately sourced rare defects
4. Establish the operating point: required recall per class, tolerable false-reject rate
5. Model, evaluate against the operating point, iterate
6. Deploy to edge hardware, integrate to PLC and MES, run in parallel with human inspection
7. Compare against the human baseline over a defined period before switching over
8. Hand over with monitoring, retraining procedure and a named owner

The parallel-run step is the one most often cut for schedule reasons and the one most worth defending. It is the only evidence that will convince a plant manager, and it is how you find the failure modes that a test set never contained.