Virtual Sorting: Eliminating Manual Sorting Activities by Using Visual Digital Twins

Imagine a fully digital quality inspection process where you can inspect parts even after they have left your premises. The emergence of new technology has made this possible. In this article, we will discuss using product digital twins in manufacturing quality control – virtual sorting. Read further to learn how you can eliminate physical sorting activities completely by performing quality inspection in a software environment, and cut significant costs.

1. What are Digital Twins?
2. What is Virtual Sorting?
3. Virtual Sorting from an Operational Perspective
4. Virtual Sorting from a Strategic Perspective
5. Prerequisites for Enabling Virtual Sorting – and Common Challenges
6. Enabling Virtual Sorting with the Multi-Camera Technology
7. Get Started with Virtual Sorting

What are Digital Twins?

A digital twin, by definition, is a virtual representation of a process or physical object that serves as a digital counterpart. As a concept, digital twins have been known in manufacturing for the last two decades.

However, their practical applications became possible and started to spread only during the last decade with the emergence of enabling technologies. In 2017, Gartner predicted that half of the large industrial companies will be using digital twins by 2021, resulting in a 10% improvement in efficiency by replacing physical practices. And they were not wrong. Today, digital twins enable new ways of creating operational efficiency in multiple domains across industries and processes – particularly in automotive.

Digital twins are categorized based on what physical entity they model such as "product digital twins", "production digital twins", and "performance digital twins". While production and performance digital twins are more common in application, product digital twins hold significant potential for creating operational efficiency.

A key area where product digital twins can be used is manufacturing quality control. They can create a return on investment by eliminating reliance on manual labor, physical processes, and even the physical products themselves. One such application of product digital twins for manufacturing quality control is Virtual Sorting through. 

What is Virtual Sorting?

Virtual Sorting refers to the digital quality inspection of manufactured parts in a software environment either for new or existing inspection points by using each part’s digital twin – without the need for the physical presence of the parts—to conduct internal and external sorting activities fully digitally.

Imagine the following scenario: You observed a decrease in process capability, which might cause quality issues with the true location of a hole. However, the hole’s location is not included in the current inspection specifications. Therefore, you now need to start a sorting activity and inspect the hole’s location for parts that have already been manufactured and, possibly, shipped.

Now, imagine that you could just open a software, define the hole’s location as a new inspection point, and immediately get inspection results in the software with the S/Ns of the non-conforming parts listed. After that, the only thing you need to do is to pass the list of S/Ns for the parts to be picked and scrapped.

And how about external sorting requests from your customers on the other side of the world? For example, a certain feature that has not been included in the inspection requirements is now causing problems at the assembly stage for your OEM customer. Without relying on third-party sorting agreements and waiting for the sorting activity to be conducted, you could immediately run the required inspections digitally on your database of product digital twins and share the list of non-conforming parts with your customer to be scrapped.

By eliminating the need for physical processes and parts, and by fully digitizing the sorting process, Virtual Sorting enables new levels of operational efficiency and flexibility as well as strategic advantages that have not bee possible before.

Virtual Sorting from an Operational Perspective

Conventionally, sorting activities are bound by physical parts and physical processes. Each part needs to be inspected one-by-one through some sort of physical process such as using a gauge, a checking fixture – or through visual inspection in case of presence checks for child components such as nuts and bolts. 

As such, sorting activities are compulsory burdens for manufacturers in terms of costs, resources, and operational complexity. Even though established sorting processes and third-party sorting agreements exist; the required manpower, time, and the need to inspect each part one-by-one put a constraint on reducing the associated costs no matter how well these processes are streamlined. 

Virtual Sorting enables manufacturers to break through such constraints imposed vy physical processes. It completely eliminates the need for accessing and moving around the physical parts to conduct inspections, and the need for dedicated man-hours to inspect parts one-by-one.

Moreover, since it is a fully virtual process, it enables quality specialists to conduct the sorting activity remotely for parts that are already shipped to global customers with facilities spread around the world – cutting not only internal but also external sorting and batch recall costs.

Considering that an OEM-requested sorting activity by conventional means can cost around USD 100k-300k depending on the part complexity and production volume, Virtual Sorting helps manufacturers significantly cut costs arising from poor quality.

Similarly, it provides almost a total payback on the investment for the quality inspection system in one sorting request – hence, providing a significantly higher return on investment over the total lifecycle of the quality inspection system in comparison to conventional systems. 

Virtual Sorting from a Strategic Perspective

Virtual Sorting creates value not only in terms of operational efficiency but also strategically in the long-term. The complexity of parts, operations, and supply chains is increasing—and so are the pressures on price competitiveness, cost control, and quality performance. In response to these challenges posed by the evolving industry landscape, it becomes critical that manufacturers can:

1. Have greater control over the production quality throughout the entire production lifecycle
   
   
2. Respond to quality issues and take corrective actions immediately regardless of the facility location and production lifecycle

By removing the barriers posed by physical parts and processes, Virtual Sorting allows manufacturers to retain effective control on parts produced even after the parts leave their premises. Moreover, they can immediately respond to quality concerns and complaints that arise after the delivery of the parts – improving reputation and gaining customers’ trust while cutting costs.

Prerequisites and Challenges for enabling Virtual Sorting

For a quality inspection system to provide the Virtual Sorting capability, there are two key requirements in principle: 

1. The system should capture data from the whole part i.e. it should observe the whole part so that the data can be treated as a product digital twin
   
   
2. The system should be able to process each manufactured part to create a database of digital twins for each part.

Even though quality inspection solutions saw significant evolution in terms of digitalization throughout the past decade, the underlying technologies have not been able to satisfy these criteria at the same time. 

For conventional in-line vision inspection systems such as robotic or fixed sensor systems, the challenge has been that these systems only collect data from the selected, critical inspection points but not the whole part. The technological reason for this is that these systems need to triangulate each inspection point one by one in order to conduct measurements. Therefore, these systems cannot collect data from the whole part to create a product digital twin that could be re-programmed for new inspection points. 

On the contrary to robotic and fixed sensor systems, laser scanners can generate highly granular point-cloud data for the whole part. However, these systems require a significant amount of time to conduct inspections. This makes them too slow to inspect each manufactured part in-line within the production Takt time in a high-volume production setting – thus, failing to generate data for each manufactured part. The point-cloud data they generate also requires further treatment to be used for GD&T measurements or presence check.

For these reasons, a new paradigm of visual inspection technology had to emerge to make the Virtual Sorting capability possible—and that is multi-camera inspection technology.

Enabling Virtual Sorting with the Multi-Camera Technology

Multi-camera technology is a photography-based inspection technology that is driven by powerful machine vision and photogrammetry algorithms. It typically consists of 30 to 90 fixed, digital cameras that are located around an inspection chamber (image 1). 

 Image 1: Fixed, digital cameras located around the inspection chamber, observing the whole part or large sections of it

Unlike fixed or robotized sensor systems, the cameras do not focus on a single feature at a time but observe the whole part or large sections of the part at once (image 2). Each collected image by each camera provides views of multiple features at a time. Together, they provide a complete view of the whole part which then can be treated as a digital twin to define new inspection points and conduct new inspections. The physical part is only needed for the initial image capturing and it has no further use to conduct inspections once the images are collected – hence completely digitizing the inspection process, moving it into a 100% software environment. 

 Image 2: Each camera observes the part from different angles, altogether providing a complete view of the part

All collected data is stored in a database, coupled with each part’s serial number. In case of new inspection requirements at any point of the part’s lifetime—as in the case for sorting requests—the operator simply defines the new inspection points on the CAD model of the part (video 1).

Video 1: Defining new inspection points on the CAD model before initiating the inspection sequence

Once the new inspection specifications are defined and the inspection program is set, powerful machine vision algorithms detect the inspection points on the product digital twin, which is formed from the images. Finally, advanced photogrammetry algorithms conduct the desired measurements directly on these images (image 3) for each manufactured part. 

Image 3: Machine vision algorithms detect the features on the image and photogrammetry algorithms execute the measurements directly on the images collected

As such, multi-camera technology enables manufacturers to retrospectively inspect parts and conduct the sorting activity in a fully virtual environment. 

Get started with Virtual Sorting – with Mapvision 100% x 100% In-Line Inspection

Equipped with a patented multi-camera technology, Mapvision 100% x 100% In-Line Inspection systems provide Virtual Sorting capability for even the most complex and demanding parts such as underbodies, EV battery modules, and cross car beams. 

Discover more about our solution here, or get in touch with us to learn more about how Virtual Sorting can help you eliminate sorting costs and how much return on investment it can provide.