How Deep Learning is Used to Increase the Quality Control of Wood Chips by Classification Author Brian Mock, President Event Capture Systems, 4700 Lebanon Road, Suite C. Mint Hill, NC 28227 Phone (919)349-9001, [email protected]Abstract The greatest expense to fully integrated papermills are the raw material wood chips that make up the slurry that later becomes paper at the reel of the machine. Random and off-line sampling of the chips provides less than .01% classification of the material. Chips that are off-specification in size, wrong species or contaminated with bark and other foreign matter can greatly impact the subsequent paper making process. Additionally, vendors or internal suppliers that sell or provide these chips to the mill cannot be held to any penalty or feedback loop for supplying chips outside set quality standards. This paper discusses how camera-based imaging with deep learning (artificial intelligence) is used to provide real-time classification of wood chips on the conveyer line. Introduction This paper will review the basic concepts of deep learning under the backdrop of AI (artificial intelligence). Several questions will be addressed including - What is this wood chip classifier doing or not doing in terms of computer intelligence? - Are there existing published or open source algorithms that have already solved this problem? - How hard of a problem is this to solve regardless of existing algorithms? - How accurate is the output data? How can the output data be improved? - What are some examples of deep learning that does not follow the requirements of this solution? Sample data will be shown on generic tests to illustrate how the process has been developed. Overview An online and real-time camera-based inspection system positioned to monitor the wood chips from an incoming source can provide a breakdown of key wood chip parameters per time. Based on the time of inspection, the resulting classification can be tied back to the supplier (external or internal). The mill can then benchmark the actual chip quality parameter against set quality standards to determine corrective actions to provide optimum outcome of this raw material as it becomes paper. Additionally, wood chip suppliers can be financially penalized for supplying a quality level that is less than predetermined standards. The resulting financial benefit for online and real-time chip inspection have several key impacts on the paper production facility including: (1) The overall cost of the raw material may be reduced by financially penalizing vendor supplying low quality wood chips. (2) The overall quality of incoming wood chips may be increased while maintaining the same price point. (3) There may be more incentive for vendors to provide better wood chip quality. (4) Feedback may be provided directly to internal chip processes to correct and/or improve production processes. (5) By providing better wood chips the overall quality of the paper is improved and the capacity of the paper machines may be increased. (6) Less paper waste resulting from producing less off specification product. (7) Less production downtime may result by reducing the number of paper breaks. (8) And, paper production may be increased by increasing paper machine speed.
Transcript
How Deep Learning is Used to Increase the Quality Control of Wood Chips by Classification
Author
Brian Mock, President Event Capture Systems, 4700 Lebanon Road, Suite C. Mint Hill, NC 28227 Phone (919)349-9001, [email protected]
Abstract
The greatest expense to fully integrated papermills are the raw material wood chips that make up the slurry that later becomes paper at the reel of the machine. Random and off-line sampling of the chips provides less than .01% classification of the material. Chips that are off-specification in size, wrong species or contaminated with bark and other foreign matter can greatly impact the subsequent paper making process. Additionally, vendors or internal suppliers that sell or provide these chips to the mill cannot be held to any penalty or feedback loop for supplying chips outside set quality standards. This paper discusses how camera-based imaging with deep learning (artificial intelligence) is used to provide real-time classification of wood chips on the conveyer line.
Introduction
This paper will review the basic concepts of deep learning under the backdrop of AI (artificial intelligence). Several questions will be addressed including
- What is this wood chip classifier doing or not doing in terms of computer intelligence? - Are there existing published or open source algorithms that have already solved this problem? - How hard of a problem is this to solve regardless of existing algorithms? - How accurate is the output data? How can the output data be improved? - What are some examples of deep learning that does not follow the requirements of this solution?
Sample data will be shown on generic tests to illustrate how the process has been developed.
Overview
An online and real-time camera-based inspection system positioned to monitor the wood chips from an incoming source can provide a breakdown of key wood chip parameters per time. Based on the time of inspection, the resulting classification can be tied back to the supplier (external or internal). The mill can then benchmark the actual chip quality parameter against set quality standards to determine corrective actions to provide optimum outcome of this raw material as it becomes paper. Additionally, wood chip suppliers can be financially penalized for supplying a quality level that is less than predetermined standards. The resulting financial benefit for online and real-time chip inspection have several key impacts on the paper production facility including: (1) The overall cost of the raw material may be reduced by financially penalizing vendor supplying low quality wood chips. (2) The overall quality of incoming wood chips may be increased while maintaining the same price point. (3) There may be more incentive for vendors to provide better wood chip quality. (4) Feedback may be provided directly to internal chip processes to correct and/or improve production processes. (5) By providing better wood chips the overall quality of the paper is improved and the capacity of the paper machines may be increased. (6) Less paper waste resulting from producing less off specification product. (7) Less production downtime may result by reducing the number of paper breaks. (8) And, paper production may be increased by increasing paper machine speed.
What is Artificial Intelligence?
Artificial intelligence (AI) is the overall bucket that contains the disciplines of machine learning and the sub-discipline of deep learning. AI was first created in the 1950’s with much promise to solve many of the world’s problems via a computer that could think like a human. These pioneers envisioned the computer to work in patterns of neural networks in the same manner as the human brain.
Illustration 1 Timeline of Artificial Intelligence
The foundation of AI, machine learning and deep learning stem from the concept of biological neurons. Neurons are interconnected in multiple layers where each neuron represents a set of binary inputs. These inputs are multiplied by an individual weight value (the synapse strength to each nearby neuron) and combined to create a threshold sum of 1 or 0 (fire the signal or not). The term deep learning derives from the design of convolution neural networks (CNN’s) where these artificial neuron groups are fully connected and able to break down difficult classification requirements (labeling) into millions and millions of smaller and overlapping data sets.
Illustration 2 Simple Biological Neuron Group
Illustration 3 Artificial Neuron Group
Common uses of deep learning
1. Object detection (car, face, animal) 2. Object tracking 3. Face recognition 4. Image restoration 5. 3D reconstruction (monocular and stereo 6. Natural language processing (translation) 7. Novelty and art
a. colorization b. adding sound c. style transfer
A Deeper Dive into the Engine for Chip Inspection
In the wood chip classification task, we want to train a model that will take an image as input and output an image where each pixel is labeled as either: background, bark, chip, or branch. This task is known as semantic segmentation. In addition, it would be even more useful to have a unique label for each instance of each object class. This is known as instance segmentation. Object detection is closely related to instance segmentation, but each instance of each class is coarsely labeled with a rectangle, instead of per pixel bitmaps.
In machine learning parlance, semantic segmentation is nothing more than a form of multi-class classification, which is performed for every pixel in the image. In some problems, you may just want to predict a single label for an entire image. For example - is this a picture of a cat, dog, or bicycle? In a binary classification task, we would only have two labels (on/off or face/not-face), perhaps for the entire image. We need not restrict ourselves to images, as the same approaches can be used for other features or attributes, although CNN’s (computation neural networks) are typically used for data in the form of continuous signals, such as spatial images, temporal audio, or spatial-temporal video.
Model
The model consists of a lightweight and more or less human readable json file describing the computation graph structure of the architecture and typically a much larger binary file with weights:
Training/learning
All the examples mentioned so far are forms of supervised learning. When we train a neural network in this paradigm, we typically start with some random weights (this is somewhat of an oversimplification). At this moment we can run inference (or prediction) with a forward pass of the model (essentially a computation graph), but the output will probably not be useful. We train the model to” learn” weights so that the output will lead to something useful. For this step, we must prepare a training set which consists of input images and corresponding labels. In our case the labels are per pixel class categories. The labeled data is created by a human. The process of training is summarized below:
1. Feed random batches of labeled samples, or mini batches of images to the network
2. Run a forward pass to get the output (per pixel class labels)
3. Compare the output with the labeled data - calculate the loss (training error)
4. Tweak the weights to minimize the loss (in deep learning this is achieved through a process called back propagation, which allows us to optimize any loss function that is differentiable)
5. Repeat 1-4 until the loss function converges (see illustration 7)
Illustration 7 Model Loss
Illustration 7 shows the typical loss-vs-time behavior for a network that has converged. Both the training and validation loss is plotted. The validation data is a separate set of labeled images that is used to monitor how well the learned model generalizes to other images. Images in the validation set are not explicitly used to update model weights during training (i.e. no images from the validation set are included in the mini batch) but can be used indirectly to validate weight choices. After training has converged and the process has terminated, the model will be evaluated on a diverse test set to further measure how well it generalizes to unseen samples.
Depending on the complexity of the network and the amount of training data, this process often takes hours or days, and typically runs on one or more graphics processing units (GPUS) (often using hardware and software developed by the NVIDIA Corporation, Santa Clara, CA), or sometimes in distributed scalable cloud environments.
Inference/prediction
When we have trained the model we can use it on random unseen images to generate our predictions in the form of label masks, where each pixel is assigned a label with a one-to-one mapping to our classes (this is a numeric index from [0...N-1], where N is the number of classes in our set (often including a background label).
This inference step is much faster, as it requires only a single forward pass through our network (or computation graph).
Chip Labeling and Sample Output Data
A fundamental challenge to chip instance segmentation or object detection (both approaches can be used to measure chip size) in typical chip images is the high chip density and resulting occlusion . A common use of deep learning is license plate recognition. In this case, the program can understand within the image how to isolate a single plate or a group of plates. This is an example semantic segmentation. In the case of a moving bed of chips the entire scene is the same object (wood chips). Simply determining the outlines of the respective chips is a fundamental problem.
Illustration 8 Papermill Chip Conveyer Line
Consider the illustrations below comparing sparse scenes to dense scenes for cars and wood chips. The sparse scene for cars is better suited for determining the outline of each subject to label if the car was a SUV, full size sedan, mid-size and compact. Additional filters may be able to determine make, model and color with acceptable degrees of accuracy. However, in a dense scene the first step of segmentation can contribute to a completely failed model as shown in illustration 10. No correct classification of the chips can be done if the deep learning subject boundaries do not match the actual real chip boundaries.
Illustration 9 Sparse vs Dense Scenes
Illustration 10 Failed segmented image on wood chips
An example of a labeled chip image is shown below. It also highlights some incorrect labeling of the images by the human labeler. These inaccuracies will contribute to inconsistencies in the quality of how the deep learning engine assigns per pixel label bitmaps, or bounding boxes, to the chips, from which we derive our size estimates. . An example set of classifiers might include
- Over-sized chips in width and height - Over-sized chips in width - Under-sized chips - Other aspect ratio driven classifications - Sticks - Other foreign matter - Bark - Water - Wood Species - Other custom created classifications
Illustration 6 Chip Labeling
The resulting classifications can be displayed real-time to operations as well as grouped in a time-based graph. An example is shown below that would represent a distribution of classifications over a specific time period.
Illustration 7 Classification Distribution per time period
Conclusion
Advances in computer processing combined with research in deep learning provides a firm foundation to develop a custom-made tool to determine a level of chip quality. Chip inspection is a challenging endeavor due to the complexity of the image and many “black box” deployments of deep learning do not apply to this application. The success of this tool is completely dependent on the quality of the truth data. Once this set of labeling is mastered the subsequent device to determine a reliable quality index for incoming chip streams can be developed. This is also a tool that can improve in accuracy over time.
Gateway to the Future
How Deep Learning is Used to Increase the Quality
Control of Wood Chips by Classification
• Brian Mock
• President
Gateway to the Future
Raw Material to the Paper Making Process Wood Chips
• Cost to the mill• Type of current classification and sampling rate• Feedback to internal suppliers• Feedback to external suppliers• No effective mechanism to provide real-time
classification• Time from chip to paper
Gateway to the Future
• What is AI (Artificial Intelligence)• Is this the same as Deep Learning and/or
Machine Learning?• What is this wood chip classifier doing or
not doing in terms of computer intelligence?
• Why do we need AI to solve this problem?
• Are there existing open source algorithms that have already solved this problem?
• How hard of a problem is this to solve regardless of existing algorithms?
• How accurate is the output data? • How can the output data be improved? • What are some examples of deep
learning that does not follow the requirements of this solution?
Topics to Discuss
Gateway to the Future
Application of Symantec Segmentation using deep convoluted
Neural networks in a supervised learning framework
Gateway to the Future
• The beginning• Bust to boom – why?• Hardware• Storage• Narrow AI• Learning methods• Computer vision• Neural Networks
Inherent Problems- Corrections that create more issues- The output is burdened by the sum of all the errors of each unrelated
programing step (net sum of all failures)- No backward propagation (one way only)
Gateway to the Future
Human knowledgeInputOutput
AI has been around for40 years and the architecturewas understood them – pastopportunities lacked the computerhorsepower to navigate the computational requirements
Gateway to the Future
Layered Neural Networks (Deep Learning)
Connections are made with weights that have rewards and penalties to match the model Human knowledge (ground truth)New connections are madeNew weights are assignedReverse propagation (two way communication)- How well did I do?- Lots of failure
Gateway to the Future
Truth Data
Framework and Network /Architecture(TensorFlow)
Extremely high processing requirements
Model
Field unitCreates output
Using the model(low processing requirements)
Learning Inference/Prediction (output)
Gateway to the Future
Common Uses for Deep Learning
1. Colorization of Black and White Images
2. Adding Sounds to Silent Movies
3. Automatic Machine Translation
4. Object Classification in Photographs
5. Automatic Handwriting Generation
6. Character Text Generation
7. Image Caption Generation
8. Automatic Game Playing
Gateway to the Future
License Plate Recognition
Gateway to the Future
Gateway to the Future
Gateway to the Future
What are we looking for?
SizeColor
Classifiers
Gateway to the Future
What are we looking for?
Segmentation
SizeColor
Challenge
Classifiers
Gateway to the Future
A quick deep dive into the engine
Gateway to the Future
• Model• Training and Learning• Inference and Prediction• How can this be improved?
Gateway to the Future
Gateway to the Future
Do you see any errors?
Gateway to the Future
Output
Classification per time unitTraced back to external and internal vendors
Not Acceptable Acceptable
Foreign Matter
Over thick
Over length
Bark
Gateway to the Future
Results
1. The overall cost of the raw material may be reduced by financially penalizing vendor supplying low quality wood
2. The overall quality of incoming wood chips may be increased while maintaining the same price point
3. There may be more incentive for vendors to provide better wood chip quality4. Feedback may be provided directly to internal chip processes to correct and/or
improve production processes 5. By providing better wood chips the overall quality of the paper is improved and the
capacity of the paper machines may be increased6. Less paper waste resulting from producing less off specification product 7. Less production downtime may result by reducing the number of paper breaks 8. Paper production may be increased by increasing paper machine speed
• What is AI (Artificial Intelligence)• Is this the same as Deep Learning and/or Machine Learning?• What is this wood chip classifier doing or not doing in terms of computer intelligence? • Why do we need AI to solve this problem? • Are there existing open source algorithms that have already solved this problem? • How hard of a problem is this to solve regardless of existing algorithms? • How accurate is the output data? • How can the output data be improved? • What are some examples of deep learning that does not follow the requirements of this
