PoC for transforming manufacturing assembly with application of Computer Vision
The client
An established organisation from South America - Mexico, dealing with manufacturing assembly lines wanted to develop proof of concept to automate the QC with a computer vision system enabled by Deep Learning which is integrated with sensors and cameras.
Vision system modules to develop
1. Classification of the products/components: Based on labeled image dataset we develop classifier model which can categorise images to corresponding label. E.g. A trained classifier model which can label good vs bad connector images.
2. Localising and identifying the product within the frame: Locate the part or component in image or visible area. E.g. Locate the assembly component on electric circuit board.
3. Object detection and object counter: Detect and count number of components in the frame
4. Detect detection: Detect and classify damages/defects like cracks, contamination, or burr on a component.
5. Segmentation: Segment defects or different parts of a machine. E.g. Segmentation of assembly component on electric circuit board.
6. OCR - Optical character recognition: Read codes mentioned on products, images of parts and extract necessary text.
7. UI: Functional User Interface (UI) consisting all the vision modules.
The approach, challenges and solutions
After scoping and considering the business goals in mind, all the vision modules were decided to be developed by various deep learning models trained for each QC task.
Data engineering and ML development challenges
1. Image Classification module:
The chief requirement for any deep learning vision solution is adequate amount of quality images.
However, unlike the academic curated image datasets, the client could provide very limited amount of apt images that can be used to train deep learning models.
We tried to solve the problem on all the phases involved. Firstly we tackled issue of low number images with data synthesizing and data augmentation techniques. At model training stage we employed best available deep neural net architectures to have near to remarkable 99% accuracy.
Since the client had large set of products portfolio, we tested the our approach on numerous distinct machine components.
2. Image localisation, detection and segmentation:
Similar to classification here as well it was difficult to obtain adequate number of images. Furthermore, these modules require something called image annotations.
There are many off the shelf solutions available for image annotations. However, due to some technical and domain expertise requirements, annotation task was not straight forward and required skilful manual annotations.
Not only we managed to solve this issue, we also achieved SOTA performance on the custom trained deep learning models.
3. OCR:
It is probably considered the easiest module to develop. In most of the cases a readymade solution is applicable.
But extracting text from documents and reading them from images which are in real world manufacturing assembly conditions seem like completely different problem.
For example images that are disproportionately lit, texts written on curved surfaces or are not visually traceable because of shiny finished metal component. This challenge too was dealt successfully by our dedicated team with best possible accuracy and response times.
Delivery and progress monitoring:
Each and every phase of the project and the deliverables were well documented and explained throughly to the client at very beginning of the project.
At TotemX we highly regard ethical practices and even though the client was not very technical, we made sure to explain all the work done in the simplistic manner. All the progress and issues were addressed in the timely manner, i.e. weekly/bi-weekly meetings.