智慧檢測技術與工業自動化
- 1. Copyright 2015 ITRI工業技術研究院機密資料 禁止複製、轉載、外流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE 智慧檢測技術與工業自動化 量測技術發展中心 儀器與感測技術發展組 周森益 2016.06.16
- 2. Copyright 2016 ITRI 工業技術研究院 前言 • 面對新經濟模式，自動化生產/檢測，所遭遇 的全新挑戰。 • 當所有資料全部保存到雲端，我們如何看待 這些資料？
- 3. Copyright 2016 ITRI 工業技術研究院 A Self-Learning Measurement System with Intelligent Software Meet industrial needs: • In-Situ inspection: dynamic environment • Quick response: valid measurement data and prediction mechanism • Flexible manufacture • User experience data analysis 1. False alarm 2. Miss detection 3. Unsuitable and manual coefficients setting from sensors 1. Fault detection decreased 2. Feasible or optimized coefficients settings from sensors 3. Self-diagnosis of maintenance and lifecycle time of sensors Increase measures’ accuracy & precision Fault detection Sensor 1 Sensor 2 Sensor … Sensor K Optical modules Hardware modules Measurement software Measurement System AS IS TO BE Sensor 1 Sensor 2 Sensor … Sensor K Optical modules Hardware modules Measurement System output Measurement software Intelligent Software Self-diagnosis Self-learning output 3
- 4. Copyright 2016 ITRI 工業技術研究院 roller steel CCD Lens CCD Lens AOI system (upper) AOI system (lower) Server User A User B Quality control router Slitting process Top view Wide steel Three median cut steels Side view Slitting process 高速相機 自製光源 待測物 檢測系統規格: 量測範圍: 400mm x133mm 影像解析: 3K x 1K 缺陷檢測解析度: 150 μm 線上檢測速度: 96 m/min 前段執行鋼條切割 與壓延製程 後段執行AOI檢測 瑕疵種類: 刮痕 汙點 鑿痕 油汙 Smart FAB - Top Quality Steel Strip Production Line 應用在消費性產品的鋼帶分條生產線，導入線上全檢目的： 1. 快速診斷不良品問題，可即刻修正提高生產良率。 2. 所有鋼捲皆有檢測資料記錄報告，提供下游客戶更完整的品質保障與投料參考。 4
- 5. Copyright 2016 ITRI 工業技術研究院 檢測結果軟體頁面 生產線與檢測系統 鋼捲切割分條後產品 1.檢測資料上傳工廠ERP系統，紀錄包含料 號、鋼捲編號、刀序、鋼條寬度等。 2.檢測報表呈現單一鋼捲大範圍檢視，可檢 測出連續型瑕疵，用以改善製程。 面臨挑戰: (產品少量多樣檢測需求) 不同的原物料與製程，鋼捲具有不同的表 面粗糙度與色澤，例如酸洗製程、壓延製 程、鋼板材質與厚度等，將提高缺陷檢出 率的困難度。 5 已建置之生產線上量測系統
- 6. Copyright 2016 ITRI 工業技術研究院 6 智能學習架構系統 higher detection/recognition rate and higher readability report Training phase Testing phase Steel images Defect region (GO/NG) Feature selection • Area • Gray value • Roundness • … Training feature extraction Training SVM model Testing feature extraction Classifying Classified results Unknown images Traditional AOI Learning orientated classification Feature correlation & normalized AI, machine learning SVM Type I: data points Type II : data points Training model self learning (linear, non-linear) Type III : data points Unknown pattern Predict Type II Quasi-Linear type Non Linear type 瑕疵分類SVM
- 7. Copyright 2016 ITRI 工業技術研究院 Intelligent System Input layer Hidden layer Output layer X1 X2 XN Ɵk f Ɵk f Ɵk f Hk HNk Ɵj f Ɵj f Y1 Yj YN0 … … … illumination Defect features Steel type/width Inspection coefficients Auto ROI adjusting Steel process Oil Non-oil PO CR PO CR Calendering Calendering 8
- 8. Copyright 2016 ITRI 工業技術研究院 • 提高影像缺陷檢出率之機器學習 - 瑕疵判斷導入機器學習(Machine Learning)機制 提高檢出率(80% ↑)、瑕疵分類製程回饋 • 檢測系統智能化控制與預測 - 檢測系統關鍵參數適應性調整，包括光源亮度、相機參數、處理資料、瑕疵 分類演算法等。 - 系統自我診斷與預測功能，包括光源光衰判斷、系統維護保養、系統組件壽 命偵測等。 因應彈性製造進行智能調整及提升機台稼動率 發展機器學習及決策系統 相機模組 料號不同、材質不同、 系統適應性調整 光學模組 待測物 工廠生產資訊 9
- 9. Copyright 2016 ITRI 工業技術研究院 Magic T-Box - Windows Phone/Tablet MMI Testing “Magic T-Box” is a software and hardware integrated solution for Windows tablets and Windows phones manufacturing functional tests. Industrial Technology Research Institute and Microsoft collaborate to design the “Magic T-Box” solution to enable Microsoft OEM and ODM partners to build high quality Windows device efficiently. 1.More than nineteen testing items : System Info, Battery, Brightness, Display, Front Camera, Rear Camera, Accelerometer, Gyrometer, ALS, Compass, GPS, Touch, Removable Device, Wifi, Bluetooth, SIM Card, Speaker and mic, Headset and mic,…,etc. 2.Integrated production record data will be analyzed in the cloud.( 1. Measured results were upload to database in cloud (Azure). 2. Intelligent algorithm start to analysis all measured data (Big Data) and show fit results automatically.) Human Inspection Automatic Magic T-Box Testing results upload to cloud 10
- 10. Copyright 2016 ITRI 工業技術研究院 Incoming Quality Control (IQC) Raw material input In Process Quality Control (IPQC) Produce (manuf./ assembly) Out-Going Quality Control (OQC) Final Test TP / Exterior 後相機組裝 前相機組裝 排線整理 絕緣膠片黏貼 機殼外觀檢查 酒精擦拭接觸點 LCD Others HDMI Mic & SpeakerDisplay Removable Device Play Video Camera Microsoft’s ODM / OEM (目前現況) 11
- 11. Copyright 2016 ITRI 工業技術研究院 CCD Motor MTF Target Auto Focus Camera Resolution AOI Color Temperature QR Code / Position AOI : Point / Line / Corner Chromaticity / Brightness SNR Audio Jack Holding Fixture A. Measure Size : 4 ~ 12 inch B. Adaptive fixture for all customized device ITRI CMS Magic T-Box Jitter / Backward / Input Separation 12
- 12. Copyright 2016 ITRI 工業技術研究院 Headset Good qualityPoor quality but acceptable signal Sample A Sample B Sample A：穩定度較差，測試5次內會有1~2次fail的機會 Sample B：穩定度很好，測試都會過 Sample A的音訊以頻譜分析可以發現在極低頻段有明顯的噪 訊，此噪訊不一定會被人耳聽到。Sample B的音訊在頻譜分 析可以觀察到除了訊號外幾乎沒有噪訊。
- 13. Copyright 2016 ITRI 工業技術研究院 Touchscreen 部分Touchscreen的Report Data有Offset及Jitter的現象 人工測試時會不知不覺地修正Device的錯誤 jitter Offset 斷線 透過精確地數值分析可確實地過濾不良品
- 14. Copyright 2016 ITRI 工業技術研究院 19 Testing Items Cycle Time : < 2 min. More than 100 output data once How do you think about it ? Upload data to the cloud, then do analysis. + 15 ITRI CMS Magic T-Box ITRI CMS Magic Testing Box (T-Box)
- 15. Copyright 2016 ITRI 工業技術研究院 Fault Detection and Classification Machine Unstably or Another Product Line ? Real-time Monitoring Run to Run Control Predict Model Measured Value Predict Value Variance Out of limits of the recipe? Update Model or Data Clean and maintain Adaptive Model for 少量多樣 Predictive Maintenance(PdM) Maintenance records Sensor Data + Machine Learning Prediction Model input output Benefit : 1. decrease downtime event 2. clear what happened 3. predict element’s life time Not Regular maintenance User experience independent 16 建立機器本身狀態的自我偵測分析能力
- 16. 𝑛𝑒𝑡𝑗 = (𝑤1𝑗∗ 𝑥1 + 𝑤2𝑗 ∗ 𝑥2 + 𝑤3𝑗 ∗ 𝑥3) − 𝜃𝑗 17 Prediction Model 𝜔𝑖𝑗 Supervised Learning Training Dataset Input X predicted Y camera frame rate motor speed PLC/FPGA response rate optical illumination air inflow of pneumatic cylinder Maintenance records utilization rate Predictive Maintenance(PdM)Run by Run Control … … 𝜃𝑗 𝑛𝑒𝑡𝑗 f 𝑤1𝑗 𝑤𝑖𝑗 𝑥1 𝑥2 𝑥3 𝑤2𝑗 𝑤3𝑗 𝑦1 𝑦2 𝑦𝑗 = 𝑓(𝑛𝑒𝑡 𝑗) 𝑦1 𝜃𝑗 𝑛𝑒𝑡𝑗 f 𝜃𝑗 𝑛𝑒𝑡𝑗 f 𝑥1 𝑥2 𝑥3 𝑥4 𝑥5 𝑥6 𝑥7 Y : dead percentage Neural Network Test Data 1. summation function 2. activity function 3. transfer function Display Color Distance Camera IMF Sound Frequency Indicators
- 17. SNR : 48.8 dB Standard Wav. Time (ms) Frequency The Same Input, but output A or B or C. It’s the sign that 1) DUT Normal 2) DUT Abnormal 3) Diff. DUT. But how to know in inspect-process ? A B Input Run 2 Run and FDC C SNR : 47.7 dB Diff. DUT
- 18. Run 2 Run and FDC 43.6 dB 42.4 dB 45.3 dB 41.9 dB 47.2 dB 48.0 dB 46.8 dB 1.5 dB 2.6 dB0.5 dB 1.1 dB Predict Model Input 47.6 dB In Run 2 Run process, Predict Model will not be changed and just modify weight of Cluster B. 43.6 dB 42.4 dB 45.3 dB 41.9 dB 47.2 dB 48.0 dB 46.8 dB 1.5 dB 2.6 dB0.5 dB 1.1 dB Predict Model 47.6 dB Cluster A Cluster B Cluster C Cluster B Cluster C Cluster ADUT Normal
- 19. Run 2 Run and FDC 43.6 dB 42.4 dB 45.3 dB 41.9 dB 47.2 dB 48.0 dB 46.8 dB 1.5 dB 2.6 dB0.5 dB 1.1 dB Predict Model Input 37.0 dB In Run 2 Run process, Predict Model will be changed and add New Cluster D. 43.6 dB 42.4 dB 45.3 dB 41.9 dB 47.2 dB 48.0 dB 46.8 dB 1.5 dB 2.6 dB0.5 dB 1.1 dB Predict Model Cluster B37.0 dB Cluster D Cluster A Cluster B Cluster C Cluster C Cluster A Diff. DUT
- 20. Run 2 Run and FDC 43.6 dB 42.4 dB 45.3 dB 41.9 dB 47.2 dB 48.0 dB 46.8 dB 1.5 dB 2.6 dB0.5 dB 1.1 dB Predict Model Input 2.0 dB In Run 2 Run process, Predict Model will not be changed and know something wrong. 43.6 dB 42.4 dB 45.3 dB 41.9 dB 47.2 dB 48.0 dB 46.8 dB 1.5 dB 2.6 dB0.5 dB 1.1 dB Predict Model Cluster B Cluster A Cluster B Cluster C Cluster C Cluster A 2.0 dB DUT Abnormal
- 21. SNR : 48.8 dB Standard Wav. Time (ms) Frequency The Same Input, but sometimes output A, other times output B. It’s the sign that MIC in mainboard will be broken. But how to know in inspect-process ? A B Input Example for Predictive Maintenance
- 22. Example for Predictive Maintenance Maintenance records Broken 1 : Normal 5 times, Abnormal 15 times, Diff. DUT 2 times. Broken 2 : Normal 2 times, Abnormal 7 times, Diff. DUT 1 time. . . . Broken N : Normal 10 times, Abnormal 27 times, Diff. DUT 4 times. Input 2.0 dB Abnormal Abnormal : 6 times Normal : 2 times Diff. DUT : 1 time Broken percentage = 99 % Predict Model : 𝜔𝑖𝑗 𝑛𝑒𝑡𝑗 = (𝑤1𝑗∗ 𝑥1 + 𝑤2𝑗 ∗ 𝑥2 + 𝑤3𝑗 ∗ 𝑥3) − 𝜃𝑗 𝑦𝑗 = 𝑓(𝑛𝑒𝑡 𝑗) 1. summation function 2. activity function 3. transfer function 𝜃𝑗 𝑛𝑒𝑡𝑗 f 𝑤𝑖𝑗 𝑦1 𝜃𝑗 𝑛𝑒𝑡𝑗 f 𝑥1 𝑥2 𝑥3 𝑥4 𝑥5 𝑥6 𝑥7 Y : dead percentage
- 23. Copyright 2016 ITRI 工業技術研究院 Combining Machine Learning and Optimization in Supply Chain Analytics Demand Forecasting Challenges: • Predicting demand for items that have never been sold before • Estimating the lowest cost (Price/Time/…). Techniques: • Clustering • Machine Learning models for regression. Cost Optimization Challenges: • Structure of demand forecast • Demand of each device is depend on cost computing -> exponential number of variables Techniques: • Novel reformulation of cost optimization problem. • Creation of efficient alg. 1. All combinations were recorded in cloud(Train) 2. Modeling(Train) Machine Learning Prediction Model Expected Label 建立生產線最佳化的數據分析能力 24
- 24. Copyright 2016 ITRI 工業技術研究院 未來技術主軸 • 佈建智能化檢測設備 • 檢測設備與生產線的緊密結合 • 上/下游供應商品質資料串連
- 25. Copyright 2016 ITRI 工業技術研究院 謝謝您的參與和聆聽! 26 感謝技術團隊的支援： 陳心怡 劉曉薇 吳潔瑜 藍于瀅 陳義昌 李漢文 林俊仁 劉定坤 卓嘉弘 王柏愷 陳泳昌