Dr. Brian Kang, Head of AI & SW Development at Koh Young Research America Sarah Chi, Marketing Specialist at Koh Young Technology.
AI in the Factory of Future
Artificial intelligence (AI) is a multidisciplinary field of science. The goal of AI is to make machines “Smarter.” Historical applications of this goal include natural language processing and translation, visual perception, pattern recognition, and decision making. The number and complexity of applications have been quickly growing across a variety of industries. Out of all the advancements researchers have made, the current driver of the AI inflection point is thanks to major advances in ‘big data analysis by deep learning.’ These advances have been limited exclusively to the human world until now, but there are far-reaching applications within the manufacturing sector too. By using the right combination of AI technologies, manufacturers can boost their efficiency, improve flexibility, speed up processes, and even achieve self-optimizing processes at production sites. The SMT (Surface Mount Technology) industry is no exception. The SMT industry, currently facing a chronic shortage of skilled labor, can utilize and realize the complete automation of the factory with the help of AI technologies. While leading organizations such as IPC and SMTA (Surface Mount Technology Association) are trying to address the challenge with education programs and training initiatives, it is still not enough due to lack of consideration of realities. Equipment providers like Koh Young are enabling the Smart Factory of the Future by adopting AI to generate “knowledge” from “experience.”
On the other hand, machine-to-machine (M2M) communications, guided in part by Industry 4.0, are quickly changing the manufacturing process by aggregating process data such as first pass yield and throughput. Koh Young is strengthening its large-scale datasets using the highest quality data captured through its industry-leading 3D metrology expertise, then the AI algorithms necessary for smarter manufacturing processes are trained by utilizing the datasets. As part of this mission, Koh Young is dedicated to adapting AI technologies to inspection technologies spanning a multitude of applications. Indeed, Koh Young Technology is enabling the Smart Factory of the Future by adopting AI to provide Advanced Solutions as Smart Supporter as shown in picture 1.
Value of Accurate 3D Data
Data is the most crucial element of the success of the AI solutions. Deep learning effectiveness is linked to the quality and quantity of the input data to address many different requirements from many fields. The use of AI to provide smarter inspection systems has been desired by every inspection provider. However, it has been difficult to realize due to the limitations of 2D imaging, which was the de facto standard for the past 25 years. Not only is it difficult for 2D Automated Optical Inspection (AOI) systems to identify defects on curved and reflective solder joints, but 2D AOI systems cannot generate reliable data. Every aspect of 2D inspection relies on 2D features like contrast; thus, it is extremely challenging to correlate with the quantitative measurement of 3D objects, and it is still a challenge to realize when the 3D measurement data is unreliable.
Koh Young, however, measures the 3D information of the components and solder joints. The Koh Young inspection machines (SPI and AOI) offer the most valuable and reliable 3D measurement data in addition to 2D images, so they become the most reliable “sensor” on the line (picture 2). The validity of 3D data is guaranteed as the company uses 3D technology for all component types to extract the exact body dimensions, unlike other systems that use “blob detection”, “estimation”, or “interpolation”, which may be susceptible to external factors like board warpage and component proximity. The combination of measurement and process data piles collected from its SPI and AOI systems, as well as from printers and mounters, allows Koh Young to deliver advanced AI features with dependable “big data.” Indeed, the quality of data is more important than the quantity of data required to create effective and reliable AI solutions with high value proposition.
Small Data, Big Challenges
Unlike other industries where a model can be learned from a sufficiently well-balanced and big enough dataset, the SMT manufacturing industry is facing big challenges from having too little data. When AI-based solutions are deployed in the real-life setting, we face such challenges as below:
- Strict Objective – Reducing the number of false calls while not losing any of the real NGs has been a major challenge.
- Imbalanced Dataset – In the modern production environment, most of the data available for training are good samples (positive examples). It is difficult to acquire real NG (No Good) samples to provide a counter example for training the model. In an extreme case, it is necessary to train the model using only good or positive samples.
- Catastrophic Forgetting – Similar to the problem above, we are observing a new type of component day by day which is not in the training dataset. The model needs to be trained and applied without losing previously learned information.
- Maintaining Training Dataset – It is difficult to afford the dataset if the turnaround time or speed of the training is too long or slow. In the most ideal situation, the new model should be trained with a fixed amount dataset, so the training time and dataset can be manageable and finished within the expected schedule.
- Adaptable Solution Required – The integration of the AI solution should not be a tight integration with the existing pipeline, so it can be adapted and applied for use in various other cases. Over the many years, Koh Young has been working on various areas to address these challenges by leveraging the world’s best analytical approaches for the inspection and the state of art AI technologies. Minimize human resource by AI based Autonomous Solution Koh Young is focused on developing AI applications for preparation, production, optimization, and maintenance, while improving system performance as shown in picture 3.
From practical solutions like Koh Young Auto Programming (KAP) and Koh Young Process Optimizer (KPO) to improvements in measurement quality and inspection accuracy, Koh Young has been utilizing AI to meet the rising difficulty in the SMT industry through its KSMART solution as shown in picture 4.
Measurement getting ever more accurate
So how does Koh Young use AI? It all begins with solving inspection challenges of SMT assemblies. The solder and components on finished boards have many specular surfaces, which will reflect some light back to camera, while creating strong inter-reflection with other lighting reflections. Since some of the reflected light does not reach the camera, they generate false signals which may cause measurement value errors. This specular reflection issue is becoming more troublesome, in relation to increasing board density and decreased component spacing. Koh Young uses AI to prevent measurement errors by incorporating learning in the product. The hybrid fusion of an analytic approach utilizing a mathematical measurement model and AI used for learning abnormal symptoms from the combination of good and bad measurement data, allows Koh Young to detect and eliminate abnormal measurements, which reduces false calls and escapes as illustrated in picture 5. Through the hybrid fusion approach, the measurement accuracy only gets better against many different challenges.
Easy Changeover
Another area where Koh Young has been proactively applying AI is with AOI programming. With the help of deep learning methods using true 3D data, the assignment of components on a PCB (Printed Circuit Board) is gradually becoming autonomous. The Koh Young AI-powered Auto Programming (KAP) system proposes the recommended inspection conditions based on 3D measured data, which not only simplifies inspection condition programming, but also makes it faster and more consistent with the best mapping conditions. Picture 6 describes the general process. KAP reduces job preparation by up to 70 percent, which makes it an ideal solution for high-mix, low-volume, or time-sensitive applications. Moreover, users can build up a User-KAP Library in addition to KY-KAP library for green site while verifying the result of KAP with accuracy rate (confidence level) and origin of the mapped inspection conditions. Going further, Koh Young will apply AI for automatic inspection condition tuning, while incrementally learning new packages at new sites.
Reliable Production (Smart Review)
The most common issue that the industry is facing in the field is that the current inspection pipeline judges NGs with strict criteria to not miss any real NGs. While this helps avoid misclassification of true NGs, it results in many false positives like judging a component as an NG even if correct. This requires field engineers to manually verify the true NGs, which leads to additional costs of time, money, and resources. Koh Young has been working on Smart Review solution to overcome this issue by rule-based auto-classification and incremental learning AI engine, which eventually gets transformed into the Auto Judgement system as shown in picture 7.
The rule-based auto-classification supports reviewer’s decision by rule-based auto classification with 3D measurement data and reduces unintended human error (block defects) and controls the machine in real-time (configurable stop). As shown in picture 8, KY uses 2D&3D information intelligently to automatically classify the defect type to minimize human intervention.
The Smart Review with incremental learning reevaluates the initial inspection results at the Review Station by collecting operator’s feedback as a second level filter. Due to this double-checking process, it can judge whether a component is a false call or a true NG with high accuracy and low inference time with minimal training dataset at the customer’s site.
Improve yield and process optimization
In Surface Mount Assembly (SMA) line, the setup and diagnosis of the printer and mounter machines solely depend on human knowledge and expertise. Tens of operators and engineers are needed to 1) ensure a correct printer setup, 2) diagnose different printing problems (e.g., PCB support issue, squeegee blade defect), 3) initialize the printer machine with adequate parameters (e.g., printing speed, pressure, etc.), and 4) diagnose the mounter mechanical failures and analyze the root cause of the problems. KSMART supports some of these tasks by collecting SPI and AOI data and conducting basic statistical analysis in real-time, serving as a monitoring tool. Koh Young Process Optimizer (KPO), which uses AI algorithms to control and optimize printing and mounting operations, as shown in picture 10, is different from any other monitoring tools.
KPO is Koh Young’s smart factory solution driven by AI to control and optimize the printing and mounting operations. KPO heavily relies on accurate 3D measurement data and error detection from SPI and AOI machines, which sets the stage for smart factory solutions.
The KPO printing solution includes three interlinking modules that exercise complex algorithms to develop closed-loop print process recommendations, which are namely Printer Diagnosis, Printer Advisor, and Printer Optimizer. The enhanced AI engine actively optimizes the printing process by combining real-time printing and SPI measurement data. Printer Advisor automatically performs DOEs (Design of Experiment) designed to perform a detailed SPI result analysis using advanced diagnostic algorithms and noise filtering models, and then recommends the ideal print parameters. Printer Diagnosis uses multiple anomaly detection algorithms to actively optimize the print process and further reduce false calls. The final module, called Printer Optimizer, uses the Koh Young adaptive learning engine to generate models and fine-tune process parameters. While each module provides inherent standalone process benefits, the combined power of the three modules ensures the highest process reliability and production flexibility while reducing dedicated resources and expertise.
The KPO mounting solution includes three modules called Mounter Diagnosis, Mounter Optimizer (Feedback), and Mounter Optimizer (Feedforward). KPO Mounter Diagnosis module identifies the mounter internal issues through placement offset patterns obtained from KPO AOI machines. It studies and analyzes the offset distribution of the major mounter elements and differentiates these offsets from the component offsets measured by the pre-reflow AOI system. The system will automatically identify mechanical and software mounter failures, as well as the root cause(s) while automatically notifying technicians and engineers about mounter issues in real-time during production. The main purpose of KPO Mounter Feedback module is allowing us to zero out the average value of component placement offsets by utilizing highly accurate measurement-based inspection data from Koh Young AOI machines. The combination of the Mounter Diagnosis and Feedback modules will provide an intelligent close loop solution which cannot adapt any new change once the parameters have been fixed. On the other hand, the KPO Mounter Feedforward module proactively provides the optimal offset values to the mounter for each component, which helps to minimize defects after the reflow soldering process.
The ultimate step will be optimization of reflow oven by utilizing all measurement data across SPI, pre-AOI and post-AOI, so the entire manufacturing process can be optimized from end to end starting from the volume of solder to the temperature profile of the reflow oven. The KPO reflow oven solution is currently being investigated and under development with our reflow oven partners.
Conclusion
Artificial Intelligence and its associated benefits will help advance the manufacturing industry confront challenges, like the lack of skilled workforce and rising costs. However, the fundamental challenges caused by small data problems such as imbalanced dataset, maintaining training dataset, and catastrophic forgetting should be addressed to meet the zero escape and flexible solution requirement. Koh Young is focusing on using an AI-based solution as the primary vehicle to enable the future of electronics manufacturing, while addressing these issues with state-of-the-art technologies and the best measurement data from our machines.
As the industry’s technology & market leader, Koh Young set a long-term vision to expand process capabilities and enhance factory performance by utilizing our next-generation AI solutions, based on the core 3D measurement technology. To realize our vision, Koh Young continues to aggressively invest in our R&D power and has established five additional R&D centers across the globe. Our intelligent force in R&D is collaborating universally to generate a quantum leap in technological leadership and competitiveness. Koh Young’s expertise in AI solutions will pave the way into new markets and go beyond the SMT industry.
