AI Visual Inspection For Defect Detection in Manufacturing (2022)

“Why should I care about a cool new technology until it’s solving any of my problems?”This was the exact conversation I had with the executive of a water purification plant over a warm cup of coffee.

What started as idle chit chat ended up evolving into an insightful and meaningful discussion about AI technologies for defect detection and quality control. This conversation significantly changed my vision and approach.

The goal of this article is to describe how to implement AI-based visual inspection for defect detection. It’s not about the coolness of artificial intelligence—no, it’s about practical solutions to specific problems.

Image credit

Concept of AI-Powered Visual Inspection

Artificial intelligence in manufacturing is a trendy term. When it comes to describing AI-based defect detection solutions, it’s often about some kind of visual inspection technology that bases on deep learning and computer vision.

AI Visual Inspection For Defect Detection in Manufacturing (1)

What Is Deep Learning in a Visual Inspection?

Deep learning is an aspect of machine learning technology that’s powered by artificial neural networks. The operating principle of deep learning technology is teaching machines to learn by example. By providing a neural network with labeled examples of specific types of data, it’s possible to extract common patterns between those examples, and then transform it into a math equation. This helps to classify future pieces of information.

With visual inspection technology, integration of deep learning algorithms allows differentiating parts, anomalies, and characters, which imitate a human visual inspection while running a computerized system.

So, what does this mean exactly? Let’s use an example.

If you were to create a visual inspection software for automotive manufacturing, you should develop a deep learning-based algorithm and train it with examples of defects it must detect. With enough data, the neural network will eventually detect defects without any additional instructions.

Deep learning-based visual inspection systems are good at detecting defects that are complex in nature. They not only address complex surfaces and cosmetic flaws—but also generalize and conceptualize the parts’ surfaces.

How to Integrate AI Visual Inspection System

1. State the Problem

Visual inspection development often starts with a business and technical analysis. The goal here is to determine what kind of defects the system should detect.

(Video) Mitutoyo AI Inspect - Visual Inspection

Other important questions to ask include:

  • What is the visual inspection system environment?
  • Should the inspection be real-time or deferred?
  • How thoroughly should the visual inspection system detect defects, and should it distinguish them by type?
  • Is there any existing software that integrates the visual inspection feature, or does it require a development from scratch?
  • How should the system notify the user(s) about detected defects?
  • Should the visual inspection system record defects detection statistics?
  • And the key question: Does data for deep learning model development exist, including images of “good” and “bad” products and the different types of defects?

Data science engineers choose the optimal technical solution and flow to proceed based on the answers they receive.

AI Visual Inspection For Defect Detection in Manufacturing (2)

Read also:

AI Application Development Guide For Business Owners

2. Gather & Prepare Data

Data science engineers must gather and prepare data required to train a future model before deep learning model development starts. For manufacturing processes, it’s important to implement IoT data analytics. When talking about visual inspection models, the data is often video records, where images processed by a visual inspection model include video frames. There are several options for data gathering, but the most common are:

  1. Taking an existing video record provided by a client
  2. Taking open-source video records applicable for defined purposes
  3. Gathering data from scratch according to deep learning model requirements

The most important parameters here are the video record’s quality. Higher quality data will lead to more accurate results.

Once we gather the data, we prepare it for modeling, clean it, check it for anomalies, and ensure its relevance.

3. Develop Deep Learning Model

The selection of a deep learning model development approach depends on the complexity of a task, required delivery time, and budget limitations. There are several approaches:

3.1 Using a deep learning model development service (e.g: Google Cloud ML Engine, Amazon ML, etc.)

This type of approach makes sense when requirements for defect detection features are in line with templates provided by a given service. These services can save both time and budget as there is no need to develop models from scratch. You just have to upload data and set model options according to the relevant tasks.

What’s the catch? These types of models are not customizable. Models’ capabilities are limited to options provided by a given service.

(Video) AI Inspection: Machine Learning / Computer Vision for Visual Defect Detection

3.2 Using pre-trained models

A pre-trained model is an already created deep learning model that accomplishes tasks similar to the one we want to perform. We do not have to build a model from scratch as it uses a trained model based on our data.

A pre-trained model may not 100% comply with all of our tasks, but it offers significant time and cost savings. Using models, previously trained on large datasets, lets us customize these solutions according to our problem.

3.3 Deep Learning model development from scratch

This method is ideal for complex and secure visual inspection systems. The approach may be time and effort-intensive, but the results are worth it.

When developing custom visual inspection models, data scientists use one or several computer vision algorithms. These include image classification, object detection, and instance segmentation.

Many factors influence the choice of a deep learning algorithm(s). These include:

  • Business goals
  • Size of objects/defects
  • Lighting conditions
  • Number of products to inspect
  • Types of defects
  • Resolution of images

An example of defect categories:

AI Visual Inspection For Defect Detection in Manufacturing (3)

source: NECAM

Let’s say that we’re developing a visual inspection model for quality assessment in buildings. The main focus is to detect defects on the walls. An extensive dataset is necessary to obtain accurate visual inspection results, as the defect categories might be incredibly diverse, from peeling paint and mold to wall cracks. The optimal approach here would be to develop an instance segmentation-based model from scratch. A pre-trained model approach is also viable in some cases.

Another example is a visual inspection for pharmaceutical manufacturing, where you want to differentiate air bubbles from particles in products like highly viscous parental solutions. The presence of bubbles is the only defect category here, so the required dataset will not be as extensive as in the example above. The optimal deep learning model development approach might be to use a model development service over developing one from scratch.

4. Train & Evaluate

The next step after developing the visual inspection model is to train it. In this stage, data scientists validate and evaluate the performance and result accuracy of the model. A test dataset is useful here. For a visual inspection system, it may be a set of video records that are either outdated or similar to ones we want to process after deployment.

(Video) How to Deploy AI and Deep Learning for Quality Inspection 3-31-2020

5. Deploy & Improve

When deploying a visual inspection model, it’s important to consider how software and hardware system architectures correspond to a model capacity.

Software

The structure of visual inspection-powered software bases itself on the combination of web solutions for data transmission and a Python framework for neural network processing.

The key parameter here is data storage. There are three common ways to store data: on a local server, a cloud streaming service, or on serverless architecture.

A visual inspection system involves the storage of video records. The choice of a data storage solution often depends on a deep learning model functionality. For example, if a visual inspection system uses a large dataset, the optimal selection may be a cloud streaming service.

Hardware

Depending on the industry and automation processes, devices required to integrate visual inspection system may include:

  • Camera. The key camera option is real-time video streaming. Some examples include IP and CCTV.
  • Gateway. Both dedicated hardware appliances and software programs work well for a visual inspection system.
  • CPU / GPU. If real-time results are necessary, a GPU would be the better choice than a CPU, as the former boasts a faster processing speed when it comes to image-based deep learning models. It is possible to optimize a CPU for operating the visual inspection model, but not for training. An example of an optimal GPU might be the Jetson Nano.
  • Photometer (optional). Depending on the lighting conditions of the visual inspection system environment, photometers may be required.
  • Colorimeter (optional). When detecting color and luminance in light sources, imaging colorimeters have consistently high spatial resolutions, allowing for detailed visual inspections.
  • Thermographic camera (optional). In case of automated inspection of steam/water pipelines and facilities it is a good idea to have thermographic camera data. Thermographic camera data provides valuable information for heat/steam/water leakage detection. Thermal camera data is also useful for heat insulation inspection.
  • Drones (optional). Nowadays it is hard to imagine automated inspection of hard-to-reach areas without drones: building internals, gas pipelines, tanker visual inspection, rocket/shuttle inspection. Drones may be equipped with high resolution cameras that can do real-time defect detection.

Deep learning models are open to improvement after deployment. A deep learning approach can increase the accuracy of the neural network through the iterative gathering of new data and model re-training. The result is a “smarter” visual inspection model that learns through increasing the amount of data during operation.

AI Visual Inspection For Defect Detection in Manufacturing (4)

Read also:

Using Explainable AI in Decision-Making Applications

Visual Inspection Use Cases

Healthcare

In the fight against COVID-19, most airports and border crossings can now check passengers for signs of the disease.

(Video) Improve Production Quality with Visual Inspection AI

Baidu, the large Chinese tech company, developed a large-scale visual inspection system based on AI. The system consists of computer vision-based cameras and infrared sensors that predict the temperatures of passengers. Now the technology, operational in Beijing’s Qinghe Railway Station, can screen up to 200 people per minute. The AI algorithm detects anyone who has a temperature above 37.3 degrees.

Another real-life case is the deep learning-based system developed by the Alibaba company. The system can detect the coronavirus in chest CT scans with 96% accuracy. With access to data from 5,000 COVID-19 cases, the system performs the test in 20 seconds. Moreover, it can differentiate between ordinary viral pneumonia and the coronavirus.

Airline

According to Boeing, 70% of the $2.6 trillion aerospace services market is now dedicated to quality and maintenance. In 2018, Airbus introduced a new automated, drone-based aircraft inspection system that accelerates and facilitates visual inspections. This development reduces aircraft downtime while simultaneously increasing the quality of inspection reports.

Automotive

Toyota has recently agreed to a $1.3 billion settlement due to a defect that caused cars to accelerate even when drivers attempted to slow down, resulting in 6 deaths in the U.S. Using the cognitive capabilities of visual inspection systems like Cognex ViDi, automotive manufacturers can analyze and identify quality issues much more accurately and resolve them before they occur.

Computer Equipment Manufacturing

The demand for smaller circuit board designs is growing. Fujitsu Laboratories has been spearheading the development of AI-enabled recognition systems for the electronics industry. They report significant progress in quality, cost, and delivery.

Textile

The implementation of automated visual inspection, along with a deep learning approach, can now detect issues of texture, weaving, stitching, and color matching.

For example, Datacolor’s AI system can consider historical data of past visual inspections to create custom tolerances that match more closely to the samples.

We will conclude with a quotation from the general manager we mentioned earlier: “It makes no difference to me whether the suggested technology is the best, but I do care how well it’s going to solve my problems.”

Solar Panels

Solar panels are known to suffer from dust and microcracks. Automated inspection of solar panels during manufacturing as well as before and after installation is a good idea to prevent shipment of malfunctioning solar panels and quick detection of damaged panels on your solar farm. For example, DJI Enterprise uses drones for solar panels inspection.

Pipeline Inspection

Gas and oil pipelines are known to have a huge length. The latest data from 2014 gives a total of slightly less than 2,175,000 miles (3,500,000 km) of pipeline in 120 countries of the world. Gas and oil leakages may lead to a huge harm to nature by chemical pollution, explosions and conflagrations.

Satellite and drone inspection with the help of computer vision techniques is a good tool for early detection and localization of a gas/oil leakage. Recently, DroneDeploy reported that they mapped about 180 miles of pipelines.

AI Visual Inspection: Key Takeaways

  1. Concept
    Al visual inspection bases itself on traditional computer vision
    methods and human vision.
  2. Choice
    Deep learning model development approach depends on the task, delivery time, and budget limits.
  3. Algorithm
    Deep learning algorithms detect defects by imitating a human analysis while running a computerized system.
  4. Architecture
    Software & hardware should correspond to deep learning model capacity.
  5. Main question
    When initiating a visual inspection, the main question is “What defects should the system detect?”
  6. Improvements
    After deployment, deep learning model becomes “smarter” through data accumulation.

AI Visual Inspection For Defect Detection in Manufacturing (5)

FAQs

What is AI based visual inspection? ›

AI-powered visual inspection uses computer vision AI to analyze machinery, production processes, inventory levels, and workplaces to ensure safe, efficient, and effective business processes.

What is visual inspection in manufacturing? ›

A visual inspection is an inspection of an asset made using only the naked eye. This kind of inspection does not necessarily require any special equipment, but it does require special training so that the inspector knows what to look for as they visually review the asset.

How accurate is human visual inspection? ›

Unfortunately, according to the FMEA (Modal Analysis of Failures and Effects) inspection process guidelines, the "human visual inspection" is effective, reliable in only 80% of cases based on the observation of multiple factors such as different points of sight or operators, reduced cycle time, visual fatigue and ...

What is defect detection in manufacturing? ›

Defect detection during manufacturing processes is a vital step to ensure product quality. The timely detection of faults or defects and taking appropriate actions are essential to reduce operational and quality-related costs.

What tools are used in visual inspection? ›

These include digital cameras, borescopes, videoscopes, video and digital microscopes, and closed-circuit surveillance television. Remote visual inspection (RVI) tools like drones and robots allow inspectors to collect visual data without the need to be physically present in the site being evaluated.

What are the steps in visual inspection? ›

Generally, here are 5 simple steps to effectively perform visual inspections:
  1. Step #1 Clearly define defect criteria. ...
  2. Step #2 Standardize inspection performance. ...
  3. Step #3 Analyze visual defects. ...
  4. Step #4 Communicate improvement measures. ...
  5. Step #5 Use mobile-ready checklists. ...
  6. Take a snapshot of day-to-day issues.
27 Jun 2022

How AI can be used in quality control? ›

Fast identification of material at risk. Thanks to computer vision and deep learning, AI-powered systems can quickly spot a problem in any part of the production line monitored by cameras or sensors. In fact, AI-based solutions can increase defect detection rates by up to 90%, compared to human inspection rates.

What is visual inspection standard? ›

Policy: Visual Inspection Standards (VIS) provide relaxations from the surface condition (normally flawless) implied by engineering drawings. Only those relaxations (called imperfections) allowed by this VIS, or the engineering drawing are permitted on parts to which this VIS is assigned.

How reliable is 100% inspection? ›

Effectiveness of 100% Inspection:

Juran, one of the giants of Quality, has written that, based on his studies performed on Inspector accuracy, 100% inspection is about 87% effective. Juran gave the following formula to calculate the Accuracy of an Inspector.

How effective is visual inspection in manufacturing? ›

Despite all these reviews, 100% visual inspection by even well-trained and experienced production inspectors has been shown to be only about 80–85% effective. It is not humanly possible to visually inspect and remove 100% of occurring defects even in the best of conditions.

What are the factors affecting visual inspection? ›

These are the inspector's visual acuity, the work-place lighting conditions, the time available for inspection and the provision of feedback or knowledge of results to the inspector.

How do you calculate DDE? ›

Formula: The formula for Defect Detection Efficiency (DDE) is: DDE = (Number of defects injected and detected in a phase/ Total number of defects injected in that phase) * 100. In this formula, the unit of the outcome is percentage (%).

What type of test is visual inspection? ›

Visual Inspection (VI), or visual testing (VT), is the oldest and most basic method of inspection. In its simplest form, visual inspection is the process of examining a component or piece of equipment using one's naked eye to look for flaws. Optical aids such as illuminators, mirrors, borescopes, etc.

What are the 4 benefits of inspection checklist? ›

Benefits of Adopting EH&S Inspection Checklist
  • Easier Audit. A safety audit checklist allows you to conduct a much easier examination of all possible hazards in the workplace. ...
  • Tasks Completion. ...
  • Keep Employees Organized. ...
  • Make Employees Accountable. ...
  • Improve Communication. ...
  • Guarantee Confidence.
6 May 2022

What are the limitations of visual inspection? ›

Disadvantages of Visual Inspection Techniques:

Surface indications only. Generally only able to detect large flaws. Possible misinterpretation of flaws.

How many stages are used by visual inspection? ›

Visual inspection can be done at three stages;

Before welding, During welding and. After welding.

What is visual inspection data analysis? ›

In behavior analysis, visual inspection of graphic information is the standard by which data are evaluated. Efforts to supplement visual inspection using inferential statistical procedures to assess intervention effects (e.g., analysis of variance or time-series analysis) have met with opposition.

What are the types of inspection? ›

Types of inspection and methods of quality control inspections differ for a reason so it's good to know the details.
  • Sample Checking.
  • Pre-Production Inspections (PPI)
  • During Production Inspection (DPI)
  • Pre-Shipment Inspection (PSI)
  • Piece by Piece Inspection (or Sorting Inspection)
  • Metal Detection.
17 Sept 2020

What is the method of inspection? ›

methods are visual, microscopy, liquid or dye penetrant inspection, mag- netic particle inspection, eddy current testing, x-ray or radiographic test- ing, and ultrasonic testing.

Why is visual testing important? ›

Visual Testing, sometimes called automated UI testing or visual regression testing verifies software from a purely visual standpoint. It verifies the aesthetic accuracy of everything end-users see and interact with.

What is visual inspection report? ›

A Visual Inspection Report (VIR) is an in-depth visual inspection of the fuse board and all accessories, such as sockets and switches, to identify any visible signs of defects, damage, or deterioration. A VIR is an annual inspection that is required 12 months after an Electrical Installation Condition Report.

Does AI computer vision use images as training data? ›

Computer vision can be only utilized only with image processing through machine learning.

What is the role of artificial intelligence in implementing TQM in an organization? ›

Artificial intelligence makes quality management a less time-consuming process by creating a holistic approach towards critical challenges such as identifying the potential causes of production problems, determining the loopholes that could lead to failures, and much more.

What is the meaning of computer vision? ›

Computer vision is a field of artificial intelligence (AI) that enables computers and systems to derive meaningful information from digital images, videos and other visual inputs — and take actions or make recommendations based on that information.

Which standard is used for basic requirement of visual inspection? ›

The general principles standard is the overall standard for visual testing generally and is not specific to any product.

Why visual testing is important in NDT? ›

Visual Testing (VT) is based on the inspection for flaws that are visible to the naked eye and is the most commonly used NDT method across all industries. It allows for a feasible and fast control of quality at every step of the fabrication or maintenance process.

What is the another name of visual examination? ›

Noun. The act of directing one's gaze towards something or someone. look. gaze.

What is visual inspection method? ›

Visual Inspection (VI), or visual testing (VT), is the oldest and most basic method of inspection. In its simplest form, visual inspection is the process of examining a component or piece of equipment using one's naked eye to look for flaws. Optical aids such as illuminators, mirrors, borescopes, etc.

What exactly AI means? ›

artificial intelligence (AI), the ability of a digital computer or computer-controlled robot to perform tasks commonly associated with intelligent beings.

What is visual inspection system? ›

Visual inspection is the image-based inspection of parts where a camera scans the part under test for both failures and quality defects. Automated inspection and defect detection are critical for high-throughput quality control in production systems.

What is visual inspection data analysis? ›

In behavior analysis, visual inspection of graphic information is the standard by which data are evaluated. Efforts to supplement visual inspection using inferential statistical procedures to assess intervention effects (e.g., analysis of variance or time-series analysis) have met with opposition.

What are the benefits of visual inspection? ›

A well-achieved visual inspection provides a reduction in the amount of repairs to the weld, leading to greater production of the other non-destructive tests and therefore decreasing the cost of production.

How many stages are used by visual inspection? ›

Visual inspection can be done at three stages;

Before welding, During welding and. After welding.

What is the purpose of inspection? ›

Inspection means careful evaluation. The primary objective of inspections is meeting customer requirements and preventing defective products from being distributed. It is undeniable that some of the product defects cannot be checked or fixed at the final production stage.

What are the 4 types of AI? ›

According to the current system of classification, there are four primary AI types: reactive, limited memory, theory of mind, and self-aware.

Why is AI so important? ›

Simply put, AI allows organizations to make better decisions, improving core business processes by increasing both the speed and accuracy of strategic decision-making processes.

What is an example of AI? ›

Several examples of artificial intelligence impact our lives today. These include FaceID on iPhones, the search algorithm on Google, and the recommendation algorithm on Netflix. You'll also find other examples of how AI is in use today on social media, digital assistants like Alexa, and ride-hailing apps such as Uber.

What is machine vision in AI? ›

Machine vision uses the latest AI technologies to give industrial equipment the ability to see and analyze tasks in smart manufacturing, quality control, and worker safety.

What is visual inspection standard? ›

Policy: Visual Inspection Standards (VIS) provide relaxations from the surface condition (normally flawless) implied by engineering drawings. Only those relaxations (called imperfections) allowed by this VIS, or the engineering drawing are permitted on parts to which this VIS is assigned.

What are the types of inspection? ›

Types of inspection and methods of quality control inspections differ for a reason so it's good to know the details.
  • Sample Checking.
  • Pre-Production Inspections (PPI)
  • During Production Inspection (DPI)
  • Pre-Shipment Inspection (PSI)
  • Piece by Piece Inspection (or Sorting Inspection)
  • Metal Detection.
17 Sept 2020

What are the limitations of visual inspection? ›

Disadvantages of Visual Inspection Techniques:

Surface indications only. Generally only able to detect large flaws. Possible misinterpretation of flaws.

What is the another name of visual examination? ›

Noun. The act of directing one's gaze towards something or someone. look. gaze.

What is visual and dimensional inspection? ›

Dimensional Inspection, also called Dimensional Metrology, assesses the geometric characteristics of machined parts and products to verify their compliance with design specifications. Dimensional inspection services ensure the accuracy of the product features that can affect reliability and functionality.

Videos

1. Artificial Intelligence Inspection V3 AI Visual Inspection For Defect Detection Environment Setup
(Raj Experts)
2. Manufacturing AI: Computer Vision for Defect Detection, Process Tracking, PPE Compliance, and more.
(Chooch )
3. AI Visual Inspection for Defect Detection
(MobiDev)
4. Deep Learning Object Detection - AI Visual Inspection for Manufacturers
(Musashi AI North America)
5. WiseEye: AI-based Textile Material Inspection System
(The Hong Kong Polytechnic University)
6. Detecting defects and anomalies through AI: Box Defect Detection
(Konica Minolta Global R&D - Digital Services)

Top Articles

You might also like

Latest Posts

Article information

Author: Sen. Ignacio Ratke

Last Updated: 10/31/2022

Views: 6214

Rating: 4.6 / 5 (76 voted)

Reviews: 83% of readers found this page helpful

Author information

Name: Sen. Ignacio Ratke

Birthday: 1999-05-27

Address: Apt. 171 8116 Bailey Via, Roberthaven, GA 58289

Phone: +2585395768220

Job: Lead Liaison

Hobby: Lockpicking, LARPing, Lego building, Lapidary, Macrame, Book restoration, Bodybuilding

Introduction: My name is Sen. Ignacio Ratke, I am a adventurous, zealous, outstanding, agreeable, precious, excited, gifted person who loves writing and wants to share my knowledge and understanding with you.