lindt:candybox:how_learning_works [Inspectron docs]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== How learning works ======

This short article is made to explan steps needed to create recognition recipe.

For AI(or neural networks) these steps are more or less the same for any [[golosary:segmentation|segmentation]] task.

===== Day 1 =====

==== 1. Annotation ====

After all required images are received first thisg to do is to [[golosary:annotation|annotate]] them. Usually this is done by student.

At least 15 images should be manually annotated.

Annotation process is consisting of two phases:
  - selecting contour of the objects
  - assigning a class(fancy name for candy type) to each contour

After the job is done i'm inspecting the images.

{{ :lindt:candybox:1.gif |}}

==== 2. Augmentation ====

Just 15 images is not enough fo AI to understand all concepts of what candy is. So i'm using quite standard trick to extend the data set.
To do that i'm addig small rotation in both directions. Together with "empty" images(with empty blister and truned over candies) this will generate a dataset of about 300 images(black images are masks with candy position and class, windows can't decode them propperly):

{{ :lindt:candybox:2.png?nolink |}}


==== 3. Learning ====

After dataset is prepared the learning process starts. This takes about 6 hours. All this time i need to monitor the results. If AI learning not correctly i need to interrupt it, correct some parameters and arestart learning process.
{{ :lindt:candybox:3.gif?nolink |}}


==== 4. Monitoring ====

Inavoidable part of learning process. As AI learns it tries to find relation between pixels on screen and masks drawn by student.
If concept is understood incorrectly - most likely there is an error at annotation, dataset balance or model needs to be tweeked. If this happens - learning process must be restarted anfter problem is eliminated.

{{ :lindt:candybox:4.gif?nolink |}}

===== Day 2 =====

==== 5. Recipe creation ====

You can think of recipe as of some program that can work with only one type of blister. Recipes are done in separate programs for multiple reasons:
  * stability. if there is a critical error in recipe this will not brake other recipes
  * experiments. recipes can be made differently. this allows to experiment quicly with different models and approaches
  * modularity. recipes can be added or deleted without modification of program itself
Recipe is created from some sort of template. And in initial stage it's quite 'stupid' and can't even find a blister on image.
{{ :lindt:candybox:5.png?nolink |}}


==== 6. Configuration ====

The goal of this step is to establish relation between blister position/angle and candy positions.
After playing around with thansformation matrices i get result similar to this:
{{ :lindt:candybox:5.1.png?nolink |}}

These records are describing where candy is located, what type it is and what its size.
If this 'table' is built correctly i will get something like this:
{{ :lindt:candybox:6.png?nolink |}}

==== 7. Voronoi segmentation ====

When two candies are close to each other or touchng AI thinks that this is one single candy. This creates false-positives. To avoid this Voronoi segmentation is used. It puts virtual borders between candies ensuring that touching candies are separate.

{{:lindt:candybox:9.png?400|}}

==== 8. Color feature classification ====

Each candy is having different amount of different colors. This makes possible to generate a color combination pattern for each candy type for each candy position. But since each candy type may have variations - we are creating a database which contains several patterns for each candy type.

During analysis each candy is compared against database of patterns expected at specified position. If the distance is close enough - the candy is good.

{{:lindt:candybox:10.png?400|}}

==== 9. Validation ====
Now it's time to test the recipe on good images that it's never seen. They should be all green:
{{ :lindt:candybox:7.gif?nolink |}}


==== 10. Coverage checking ====
The last step before it's done. On this step i'm checking how confident AI about it's decisions.
This allows to discover if there is possible problems with recognition on the line.
{{ :lindt:candybox:8.gif?nolink |}}