Search

Geek Girl Joy

Artificial Intelligence, Simulations & Software

Tag

Machine Learning

Scaling Data

Size is relative and statements like “too big” or “too small” (cough TWSS 😛 ) are not immutable properties of the universe but rather simply a matter of perspective. Change your perspective just a little Goldilocks and you might find that your neural network thinks your data porridge tastes “just right”!

Err… what I mean is… we’re going to look at how to scale your data so that you can feed it to your neural network without causing it (or you) to have any unpleasant indigestion.

Specifically, lets say you have a very important series of values (including strings of text) that you want to use as part of your training dataset but they don’t fit within the yummy-data range of -1 to 1… so what do you do?

Continue reading “Scaling Data”

Pathfinder – The Rodízio Contingency

Over the centuries and throughout my travels I’ve come to rely on my compass and a good map to point me in the right direction for my next adventure.

Sometimes my adventure led me to treasures in mysteriously exotic & remote locations, while other times I found myself among friendly and awfully generous cannibals who wanted to invite me to dinner… of course, it’s always best to politely decline such invitations because if anything I certainly live by the rule:

“If I’m on the carte du jour as flambé, I’ll skip the buffet and run away because I’m no entrée!”
~GeekGirlJoy

Hmmm, come to think of it, that might be the best piece advice I’ve ever given on this blog and if you agree consider supporting me through Patreon! 😉

In any case, despite the occasional fears I’ve held over the last few millennia over potentially becoming someones late-night heartburn, I’ve kinda always known that no matter how bad things got while exploring, I’d be okay because beyond a good compass and a fragmented (then taped back together) map with a sweet X scrawled in blood somewhere on it… I possess a secret tool that all the great explorers down through the ages have relied upon and today, I’m going to share it with you!

But… before I do, here’s today’s wallpaper!

The Rodízio Contingency Wallpaper
The Rodízio Contingency Wallpaper

The Pathfinder

From Allan Quatermain to Amerigo Vespucci, Erik the Red to Captain Nemo and even Jill of the Jungle… all notable explorers have relied on an enchanted automaton totem called “Pathfinder Stones”.

The stones are first consecrated with the live blood of a dead turnip and when brought into close proximity of their owner and a target on a map, will glow to show a path from where you are to where your desired destination is.

Which does bring us to the topic of today… I transmuted one of my sets of pathfinder stones into a digital form using the uh… “Quantum FANN Effect” and an ancient shadow daemon called JavaScript.

Schtick Aside

Okay, so what I did was use the JavaScript version of FANN to deploy an implementation of my original Pathfinder on GitHub pages.

The cool/interesting thing about FANN.js is that it uses asm.js to make the compiled FANN library available inside a web browser.

What this means is that a working version of Pathfinder is now online for you to play with (link blow) however…

There are two fairly large downsides to deploying with FANN.js instead of PHP-FANN:

  1. You cannot save the ANN after training.
  2. You cannot load a previously trained ANN.

These limitations mean that Pathfinder must be trained once every time the page loads and this does limit the size and complexity of ANN’s that are deployable using FANN.js.

The thing is it may still be possible to save the ANN by using the supported FANN lib methods/functions like I did when I built the FANN-Neural-Network-Visualizer to manually query the ANN object and then format/export the necessary information as a string/plain text because the FANN ANN.net save file format seemingly isn’t all that different from an .INI file (though I am uncertain if this is universal in all language implementations of FANN) and it’s something I plan on playing around with in the future.

Far be it for me to be the barer of fake news… turns out… it actually helps to read the documentation thoroughly and not just skim it and then do a search for keywords! 😛

FANN.js actually DOES have a save function but it doesn’t follow the FANN Lib reference manual of “save_…” convention and instead implements a JS Object.export().

I understand why they did that… and it does kinda make sense in the mixed up JS world but… it still holds to my “anti-wheel” digression argument that you haven’t read about yet.

Having said that… I promise to ritually self-flagellate by Gothic torch light using the sharp side of a motherboard!

I really should have done a better job of reading the docs! :-/

Why use FANN.js over PHP-FANN

Far be it for me to ever sing the praises of JS over PHP however in order to deply a neural network using PHP you have to have a server and the ability to compile & install PHP extensions and that costs money whereas GitHub Pages is free to me and to you but it doesn’t support the more robust server architecture that PHP requires so using FANN.js allows me to deploy my bots and AI in a way that let’s you actually use them instead of just reading about them.

All things being equal, I would still recommend the PHP version of FANN however the JS version does work and with a little improvement could become a viable deployment option!

Having said that, what I am really interested in with FANN.js is that JavaScript has a direct path between the browser environment via the WebGL API to the GPU whereas with PHP it is technically possible to commune with the GPU, however in practice it’s not generally done and until the PHP dev’s get their head out of their asses and start thinking out side the box (PHP is now mostly a general purposes language so start treating it like one…), PHP+GPU stuffs isn’t going to be the easiest pickle jar to crack using PHP and the existing available options though again, I’m not saying it is impossible either.

So, in the future I intend to see if I can’t use FANN.js + WebGL shaders to make FANN training faster (no promises) and then hopefully export the ANN.net file so that we can use/deploy the GPU trained ANN in a PHP environment.

Play Stump the Neural Network

So the online version of the Pathfinder network learns/re-trains from scratch every time the page loads and as such it can’t spend unlimited amounts of time training which is normally not a concern because even if your 1000 layer celebrity deep fake porn ANN takes 100 CPU years to train (i.e. 1 CPU = 100 years, 100 CPU = 1 year etc… ) it isn’t a major concern because likely you are buying your compute from Jeff Bezos or Bill Gates and they have plenty of underutilized computers laying around waiting for you to rent a few FLOPS.

In the end, you save the ANN model so you can use it immediately when you want it… but FANN.js says “Nah I’m good! Who needs to do something as convenient as save or reload!” (then again (and mostly off topic) JavaScript tends to seem to like reinventing round wheels as square uh… anti-wheels) but in any case…. the small training time and the inherit random nature/path of gradient decent the final neuronal weights will always be different and when the ANN fails (hence the “stump the ANN”) it won’t always take the same path (between page reloads).

This is easier understood if I just show you.

Given this input

I got this output

Note that diagonal steps are valid so this path is technically valid but the path is far less efficient than the straight line between the two points that a human would draw/walk.

Reload the page (not required unless you are playing with this idea) and try again…

A different Path was found.

Neither was optimal but a path was found and more cells than necessary were visited.

Here’s some additional examples:

Input

Pathfinder ANN Output

Input

Pathfinder ANN Output Back tracking… ugh!

Input

Pathfinder ANN Output

I believe that’s called the sidestep slide!

Input

Pathfinder ANN Output

I mean… it’s not the path I would have chosen but it made it! 😛

If you’d like to try your hand at stumping my Pathfinder you can checkout the live demo here:

Pathfinder Online: geekgirljoy.github.io/Pathfinder_Neural_Network/

You can download all the code (for free) here:

Pathfinder Code on GitHub: https://github.com/geekgirljoy/Pathfinder_Neural_Network

And with that, have a great week everyone.


If you like my coding projects, art, bizarre opinions and writing style… consider supporting me through Patreon.

But if all you can do is Like, Share, Comment and Subscribe… well that’s cool too!

Much Love,

~Joy

The Contrast-a-tron

Today we’re going to continue my introduction to creating your own data sets series by building Contrast-a-tron.

Now, I know what you are thinking:

“We already did that, like… a while ago!”

Here’s the thing though… we didn’t! 😉

And besides, it wasn’t that long ago!

What we built before was a Contrast-inator and a Contrast-inator and a Contrast-a-tron are not the same things! 😛

Let me explain…

  • The Contrast-inator: Learned how to “predict/classify” if a single input color was to the left (darker) or to the right (lighter) in an imaginary red line in the exact center of a 2D gray-scale gradient representation of the 0-255 RGB 3D color space.
  • The Contrast-a-tron (this bot): Is a much smarter and more interesting bot. It will learn how to “predict/classify” two input colors as “darker” and “lighter” or “the same” compared with each other. Which is a much more challenging task for the bot to learn.

But before we get into that I think I owe you a wallpaper.

A Wallpaper

Don’t mind the title, it’s definitely not a template placeholder! 😛

Anyway, just due to me being me, I have a lot of old robots and parts laying around and I was out in the o’l boneyard and I found this really beat up Krypto mining bot for us to play with.

I built it back when I was going to launch my own currency (A long time ago when it was still a cool thing to do and not everyone was like “my ICO is next week, you should mine sum!!!!” 😉 😉 ), yeah… no thanks!

Anyway, the bot’s memory is completely corrupt, but… the optical circuitry and hardware are still functional and since mining bots are built to operate deep under miles of data in extreme low light conditions at high speed, it’s visual acuity is top-notch and it even supports infrared mode!

So don’t let it’s tiny eyes fool you, they are incredibly sensitive which is perfect for today’s project! 🙂

Contrast_a_tron 1920 x 1080 Wallpaper
Contrast_a_tron 1920 x 1080 Wallpaper

I should add that not all posts get a theme song but today’s is Night Business by Perturbator (not a sponsor), I love the little vocoded? robotic voice about two minutes and twenty seconds in. It’s definitely what this bot’s voice sounds like! 😛

Also before we proceed, I’d just like to set the record straight and confirm that I’m definitely not Satoshi Nakamoto!

The Contrast-a-tron

To begin, let’s first look at what our Contrast-inator does:

Is this pixel brighter or darker than the red line?
Is this pixel brighter or darker than the red line?

It takes a color/shade as an input and then tries to determine which side of the red line it falls on.

Not that useful but it’s good for operating inside a known range that never changes. Like, was the light red or green kinda stuff, or conceptually like a line following robot.

Anyway, what if you wanted to start comparing two colors at the same time and to make things even more complicated, what if the gradient wasn’t always facing the same direction (meaning the “brighter/darker” pixel could be on the left or the right)?

For most of you that task is trivial and you could do it almost unconsciously or with minimal mental effort, not the Contrast-inator though!

To compare two pixels the Contrast-inator must evaluate each separately and because the red line (which you can imagine is “where the robot is standing” on the gradient when it’s evaluating a color) doesn’t change, if both colors are to it’s left or right (the bot’s vantage position / the red line), then it is completely unable to compare them.

Because these colors are on the same side of the red line, the Contrast-inator cannot compare them but the Contrast-a-tron can.
Because these colors are on the same side of the red line, the Contrast-inator cannot compare them but the Contrast-a-tron can.

Just to be clear, the Contrast-inator will say that both pixels/shades are “brighter/to the right” of zero (where it stands / it’s anchor) but it cannot figure out which of the two colors are brighter and the same is true if both colors are darker (to the left of the red line).

Further, there is also no guarantee that we will always present the colors to the bot in the order of darker on the left and lighter on the right meaning that sometimes the gradient will be lighter on the left and darker on the right and we will need the bot to notice that difference and accommodate that circumstance.

How the Contrast-a-tron Works Differently

The Contrast-a-tron isn’t anchored to zero (the center of the gradient) and instead we can think of it moving around the gradient to try and find the “center” of the two colors (whatever color that might be) and from there it can evaluate which side (input color / shade) is brighter and which is darker.

In the event that the input colors/shades are the same then both Input A & B will be in the same place which means that it will be neither to the right or to the left of the bot.

How the Contrast-a-tron works differently.
How the Contrast-a-tron works differently.

How the Neural Networks Differ

I didn’t spend a lot of time discussing the structure of the neural network when we built the Contrast-inator but now that we have something to compare it against let’s look at a visual representation of each network.

How the Contrast-inator and the Contrast-a-tron neural networks differ.
How the Contrast-inator and the Contrast-a-tron neural networks differ.

On the left you see the Contrast-inator with it’s single input neuron, a hidden layer containing two hidden neurons and an output layer with two output neurons.

Additionally you see two “Bias” neurons represented in yellow that help the network learn what we want by “biasing” the output of that layer to the next so that it is never “none” (zero or no output).

What this means is that bias neurons add their value to the output signal of each neuron from their layer so that the signal is never no “activation signal” and some value propagates forward.

All layers except the output layer will always have a single bias neuron. There is no need of a bias neuron on the output layer because there is no signal to propagate beyond the output neurons so it wouldn’t serve any purpose.

Bias neurons have no inputs.

In practice we don’t have to concern ourselves with the bias neurons and the ANN will manage them itself but I like draw them because they do exist and they are part of the network, however it’s common for people not to include them in diagrams because they are so easy for us to ignore since we don’t really need to do anything with them and they are just there to help the signal propagate.

In any case, the Contrast-a-tron differs by including a second input neuron (for the second shade/color) and a second hidden layer which helps the Contrast-a-tron to be a little “smarter” and learn what we want it to.

I have a post about how to create diagrams like this called Visualizing Your FANN Neural Network and you can download a copy of the open source visualization software I wrote for free from my GitHub account here: https://github.com/geekgirljoy/FANN-Neural-Network-Visualizer

Training The Contrast-a-tron

When we created the Contrast-inator, I walked you through each training example and how it was derived because it was a very small data set requiring only three examples however this data set is a bit longer with thirteen examples and it will be a lot easier to show you the data set and then draw you a picture than to type a description but before we look at the training data, lets make sure we understand the outputs.

Understanding the Contrast-a-tron output.
Understanding the Contrast-a-tron output.

There are two outputs and we’ll call them A & B and they are in that order.

In an ideal world the bot will give us -1 & -1 to mean they are the same, 1 & -1 to mean that A is Brighter and B is Darker and -1 & 1 to mean A is Darker and B is Brighter.

In reality… what we get is a number that comes close but isn’t -1 or 1 called a “floating point number” in computer science but most people just call them a decimal number like for example 0.123.

In practice this means that as long as A & B are not both negative, then whichever has the higher positive value is the “brighter” color and whichever has the lower positive value is the “darker” color otherwise they are the same (A==B).

Let’s look at the training data and visualize it.

Contrast-a_tron.data

This is the complete Contrast-a-tron training data.

The first line is the “FANN Header” which consists of: the Total_Number_of_Example_Sets the Number_of_Inputs the Number_of_Outputs\n

Note the spaces between the values on the header line as well as between the inputs and the output values.

Line 2 (-1 -1) is an input example. Line 3 (-1 -1) is an output example and the pattern of Input_Example\nOutput_Example\n continues to the end of the document.

13 2 2
-1 -1
-1 -1
-0.5 -0.5
-1 -1
0 0
-1 -1
0.5 0.5
-1 -1
1 1
-1 -1
1 -1
1 -1
0.5 0
1 -1
0 0.5
-1 1
-1 -0.5
-1 1
-0.5 -1
1 -1
1 0.5
1 -1
0.5 1
-1 1
-1 1
-1 1

Let’s visualize this training data which should hopefully give you a more intuitive sense for how these numbers translate to information the Contrast-a-tron ANN can use to learn.

Visualizing the Contrast-a-tron training data set
Visualizing the Contrast-a-tron training data set

The Code

Here’s the code used to train. I have other tutorials covering what this all means available on my Topics and Posts page so I won’t go into what all of this means but basically it sets up a training environment and trains the Contrast_a_tron ANN and saves the results to a FANN .net network file.

TrainContrast_a_tron.php

<?php

$num_input = 2;
$num_output = 2;
$layers = array($num_input, 2, 1, $num_output);
$ann = fann_create_standard_array(count($layers), $layers);

$desired_error = 0.0000000001;
$max_epochs = 900000;
$epochs_between_reports = 10;

if ($ann) {
    fann_set_activation_function_hidden($ann, FANN_SIGMOID_SYMMETRIC);
    fann_set_activation_function_output($ann, FANN_SIGMOID_SYMMETRIC);
    fann_set_training_algorithm($ann,FANN_TRAIN_INCREMENTAL);


    $filename = dirname(__FILE__) . "/Contrast_a_tron.data";
    if (fann_train_on_file($ann, $filename, $max_epochs, $epochs_between_reports, $desired_error)){
        echo 'Contrast_a_tron trained.' . PHP_EOL;
    }

    if (fann_save($ann, dirname(__FILE__) . "/Contrast_a_tron.net")){
        echo 'Contrast_a_tron.net saved.' . PHP_EOL;
    }
    
    fann_destroy($ann);
}

 

TestContrast_a_tron.php

We next need to test the ANN so I use two “for loops” with one counting down to -1 and one counting up to 1 and each incrementing by -0.2 each iteration of the loop as the inputs to test with.

<?php

$train_file = (dirname(__FILE__) . "/Contrast_a_tron.net");
if (!is_file($train_file))
    die("Contrast_a_tron.net has not been created! Please run TrainContrast_a_tron.php to generate it" . PHP_EOL);

$ann = fann_create_from_file($train_file);

if ($ann) {
    
    foreach(range(1, -1, -0.2) as $test_input_value_a){
        foreach(range(-1, 1, -0.2) as $test_input_value_b){
        
            $input = array($test_input_value_a, $test_input_value_b);
            $result = fann_run($ann, $input);

            $a = number_format($result[0], 4);
            $b = number_format($result[1], 4);
            
            // What answer did the ANN give?
			
            $answer = NULL;
            $evaluation = '';
            if($a <= 0 && $b <= 0){
                $evaluation = 'Neutral/Same';
                $answer = 0;
            }
            elseif($a > $b){
                $evaluation = 'A is Brighter';
                $answer = -1;
            }
            elseif($b > $a){
                $evaluation = 'B is Brighter';
                $answer = 1;
            }
            else{ 
                $evaluation = ' OOPSIES!!!!!!!';
            }

            echo 'Contrast_a_tron(' . $input[0] . ', ' . $input[1] . ") -> [$a, $b] - $evaluation" . PHP_EOL; 
        }
    }
    fann_destroy($ann);
}
else {
    die("Invalid file format" . PHP_EOL);
}

Results

The Results/Output of the test code.

Contrast_a_tron(1, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.8) -> [0.9986, -1.0000] - A is Brighter
Contrast_a_tron(1, 1) -> [-1.0000, -0.1815] - Neutral/Same
Contrast_a_tron(0.8, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.6) -> [0.9992, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.8) -> [-1.0000, -0.2218] - Neutral/Same
Contrast_a_tron(0.8, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.6, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0.4) -> [0.9995, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0.6) -> [-1.0000, -0.4005] - Neutral/Same
Contrast_a_tron(0.6, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.6, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.4, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, 0.2) -> [0.9996, -1.0000] - A is Brighter
Contrast_a_tron(0.4, 0.4) -> [-1.0000, -0.6543] - Neutral/Same
Contrast_a_tron(0.4, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.4, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.4, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, 0) -> [0.9996, -1.0000] - A is Brighter
Contrast_a_tron(0.2, 0.2) -> [-1.0000, -0.8580] - Neutral/Same
Contrast_a_tron(0.2, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.2) -> [0.9996, -1.0000] - A is Brighter
Contrast_a_tron(0, 0) -> [-1.0000, -0.9557] - Neutral/Same
Contrast_a_tron(0, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.4) -> [0.9995, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.2) -> [-1.0000, -0.9878] - Neutral/Same
Contrast_a_tron(-0.2, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.4, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.4, -0.6) -> [0.9994, -1.0000] - A is Brighter
Contrast_a_tron(-0.4, -0.4) -> [-1.0000, -0.9965] - Neutral/Same
Contrast_a_tron(-0.4, -0.2) -> [-1.0000, 0.9997] - B is Brighter
Contrast_a_tron(-0.4, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.6, -0.8) -> [0.9990, -1.0000] - A is Brighter
Contrast_a_tron(-0.6, -0.6) -> [-0.9999, -0.9989] - Neutral/Same
Contrast_a_tron(-0.6, -0.4) -> [-1.0000, 0.9996] - B is Brighter
Contrast_a_tron(-0.6, -0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, -1) -> [0.9981, -1.0000] - A is Brighter
Contrast_a_tron(-0.8, -0.8) -> [-0.9999, -0.9995] - Neutral/Same
Contrast_a_tron(-0.8, -0.6) -> [-1.0000, 0.9993] - B is Brighter
Contrast_a_tron(-0.8, -0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, -0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, -1) -> [-0.9998, -0.9998] - Neutral/Same
Contrast_a_tron(-1, -0.8) -> [-1.0000, 0.9982] - B is Brighter
Contrast_a_tron(-1, -0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, -0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, -0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 1) -> [-1.0000, 0.9998] - B is Brighter

GitGub

As always you can download a copy of this code on GitHub for free and if you have any questions or comments please leave them below.

Contrast-a-tron on GitHub: https://github.com/geekgirljoy/Contrast-a-tron


If you find yourself thinking…

“Joy you’re the best!”

I’d say….

If you support the resistance against Big AI then consider supporting my efforts through Patreon.

But, if all you can do is Like, Share, Comment and Subscribe… well that’s cool too!

Much Love,

~Joy

The Contrast-inator

Let’s keep things simple, you want to read a post and conveniently I’ve written one for you!

I’ll spare everyone my recent fascinations with macabre subjects and opt to get right to the topic of the day!

Anyway, as the Jane Goodall of bots I’ve learned a little about how to communicate with them using rules they understand and today I’m going to show you how you make rules that get a bot to understand and do, what you want it to do.

But… before we get into that, here’s the wallpaper!

The Contrast-inator Bot Wallpaper

Why Contrast-inator? Well, as far as the “-inator” part is concerned let’s just say I enjoy Phineas and Ferb. 😛

As for the “Contrast” part, we’ll get into that next but the big eyes of this bot are important. 😛

Also… it doesn’t need hands, arms, legs or feet to do it’s job so I didn’t 3D print the parts, waste not want not! 😉 😛

Contrast-inator 1920x1080 Wallpaper
Contrast-inator 1920×1080 Wallpaper

The Contrast-inator

So, recently I received a few comments that amount to something along the lines of “How do you decide on the ‘rules’ for training your neural network?”.

My response is basically if you can “phrase” a training rule in a way the bot can understand, then you can make the rules to be whatever you want/need them to be.

And the thing we’re going to try to teach the bot today to help us explore this topic is… given an input value, tell us if the value falls to the “left” or to the “right” of an “anchor point”.

That sounds more complicated than it really is and I intend this tutorial to be for beginners so let me try to simplify my description… uh… so, think of a gray-scale gradient of black to white.

Imagine a red line in the center of the gradient.

Now, given some gray-scale color e.g. RGB(102, 102, 102) as “input”, how can we train a bot to tell us if the color falls to the left or to the right of the red line… like this:

Is this pixel brighter or darker than the red line?
Is this pixel brighter or darker than the red line?

I know that might seem complicated while at the same time also seem kind of useless… I assure you that neither stipulated condition is true.

In regard to usefulness, just as a hypothetical example… a neural network like this could act as the “brain” of a “line follower” robot but instead of reading the voltages directly from a photodiode and then using if/else to direct motors, you could pass the diode voltages to the neural network and let it decide which motor to move via an H-Bridge / Digital Potentiometer.

An Arduino would need a WiFi Shield for something like that to work  but a line follower built on something like a Raspberry Pi could run it’s neural network “brain” locally.

Which brings us back to complexity and how we build a rule set to teach our Contrast-inator bot to tell us if a pixel is brighter or darker than the color where the red line is.

Forget about what I said about the hypothetical line-follower robot, the Arduino and the Raspberry Pi… it’s more complicated than I want this post to be and it’s just an example anyway. 😛

Let’s start over…

We know that any answers our bot gives us (the output) will look like a “floating point” number (a decimal value e.g. 0.01) and basically our input will also be a floating point number too.

With this in mind we can start to imagine that our training data inputs and the associated outputs will look like a series of numbers.

But what will the numbers mean and how can we know if the bot is correct?

Well, let’s step back again and think about what rules we need to teach the bot first before we even worry about encoding the training data for the bot.

What rules might a human need if we had to describe the process to someone for them to be able to do it?

Plain English Rules For the Bot to Learn:

  1. If the color falls to the left of the red line then it can be described as “Darker”.
  2. If the color is neither to the left or the right of the red line, then we can say the color is directly in the center. We might describe this position or color state as being “Neutral” in relation to the red line.
  3. If the color falls to the right of the red line then it can be described as “Brighter”.

Given these three super easy rules I believe most, if not all of you should be able to answer if a color falls to the left or the right of the red line with a high degree of accuracy.

However, your accuracy would diminish the closer the color is to the red line in the center because you are intuitively guessing and the difference between the colors that surround either side of the center of the color gradient all look like very similar grays, e.g. they have a low contrast between them.

The colors at the ends of the color gradient (black/left and white/right) have the largest contrast between them and are the easiest to determine which side they fall on.

With our rules layed out in English, let’s return to the idea of the training data (our rules) which consists of numbers and how we will represent our three rules as numbers.

I’ve already said the inputs and outputs will be floating point numbers but what we haven’t covered yet is the fact that our numbers are “signed” (have negative and positive polarities to it’s range) with our range being -1.00 to 1.00.

This means that Black can be encoded as: -1.00 or -1 for simplicity with the decimal being implied.

This also means that White can be encoded as: 1.00 or 1, also with the decimal being implied.

Given our signed float range and a few colors converted to a float within our range, we can easily determine algorithmically if a color is on the left or right of the red line even if it’s very close to the center with 100% accuracy (better than human capability) simply be checking if it is greater than or less than zero.

Meaning… a neural network is NOT needed to accomplish this task, but… that’s not the point! 😛

Our goal is to teach a neural network to do this nonetheless because it is a simple problem and the rules (training data) are simple enough that a beginner should be able to understand how they are derived if they exert even a modicum of effort!

Here’s what that looks like:

Example Colors to Float Range Input
Example Colors to Float Range Input

Notice that the first two colors are to the left of zero (darker) because they are negative and the third color is far to the right (much lighter) because it is closer to 1 than 0.

Color R, G, B As Float Side Meaning
42, 42, 42 -0.66797385620915 Left Darker
102, 102, 102 -0.19738562091503 Left Darker
221, 221, 221 0.7359477124183 Right Lighter

Fascinating… but… how are you converting the colors to floats?

Okay look, this won’t be on the mid-term test and it’s in no way actually necessary to go over because we won’t need to do this to train the bot but since you are curious here’s a function you can use to convert actual RGB & Grayscale colors to a float in the right range:

How to convert a color to a signed float between -1.00 to 1.00:

<?php 
// Input a number between 0 and $max and get a number inside
// a range of -1 to 1
function ConvertColorToInputFloatRange($color_int_value, $max = 255){
    return ((($color_int_value - -1) * (1 - -1)) / ($max - 0)) + -1;
}

// RGB Color to range of -1 to 1
$R = 42;
$G = 42;
$B = 42;
echo  "Pixel_RGB($R,$G,$B) = " . ConvertColorToInputFloatRange($R+$G+$B, 255+255+255) . PHP_EOL;

// RGB to Gray-scale to range of -1 to 1
$gray = ($R+$G+$B) / 3;
echo  "Pixel_Grayscale($gray) = " . ConvertColorToInputFloatRange($gray, 255) . PHP_EOL;

// RGB Color to range of -1 to 1
$R = 102;
$G = 102;
$B = 102;
echo  "Pixel_RGB($R,$G,$B) = " . ConvertColorToInputFloatRange($R+$G+$B, 255+255+255) . PHP_EOL;

// RGB Color to range of -1 to 1
$R = 221;
$G = 221;
$B = 221;
echo  "Pixel_RGB($R,$G,$B) = " . ConvertColorToInputFloatRange($R+$G+$B, 255+255+255) . PHP_EOL;


/*
Output:

Pixel_RGB(42,42,42) = -0.66797385620915
Pixel_Grayscale(42) = -0.66274509803922
Pixel_RGB(102,102,102) = -0.19738562091503
Pixel_RGB(221,221,221) = 0.7359477124183

*/

Now that you all at least believe it’s possible to convert a color to a float between -1 & 1 forget all about this function because we won’t need it to train the bot! 😛

Then… how do we teach a neural network to do this?

Well, let’s talk about what the output for this bot looks like before we get back to creating the training data from our rules.

We know that our output is a float, and even though it is possible to teach the neural network to do this with a single output, I find I get better results from the neural network using two outputs.

This is because it’s actually very easy for the bot to understand we want it to detect if the input value (color) is slightly offset to the left or right of the red line but it’s not the easiest thing for it to determine exactly where the center is (just like you but it’s still better at it) so our margin of error (the number of colors it can’t tell are on the right or left… e.g. the colors it will say are neutral) tends to be slightly larger if we only use a single output float.

What that means is:

  1. Our Input looks like: float
  2. Our output looks like: float_left float_right

With that in mind we have now covered everything necessary to begin converting our rules to training data.

Remember, that the decimals are implied!

Lets start by teaching it what the darker colors on the left look like:

Black RGB(0,0,0), is the farthest most color to the left and is encoded as -1 and with two output values representing Left & Right we get a rule that looks like this:

Learn “Darker” colors (floats closest to -1.00) are on the left:

The output value on the left is set to 1 which means negative values more strongly polarize to the left and this is reflected on the left output being 1.00 and the right output value being -1.00.

-1
1 -1

Learn “Neutral” colors (floats closest to 0.00) are near the center:

I’m using -1.00 & -1.00 to mean that an input of exactly zero is not strongly polarized to either side of the gradient with zero (the exact center – whatever color that is) is not strongly polarizing (-1.00, -1.00) in either direction.

The goal here is that this will help it learn values near zero are are not strongly polarized and zero isn’t polarized at all.

0
-1 -1

Learn “Brighter” colors (floats closest to 1.00) are on the right:

The output value on the right is set to 1 which means positive values more strongly polarize to the right and this is reflected by the right output being 1.00 and the left output value being -1.00.

1
-1 1

 

FANN (The library we’re using for training the neural network) requires a header is stored with the training data so it can read the training data and that looks like:

Number_Of_Training_Examples Number_Of_Inputs Number_Of_Outputs

*Note the spaces between values

So, combined our training data file looks like this:

Contrastinator.data

3 1 2
-1
1 -1
0
-1 -1
1
-1 1

And that’s it, we’ve converted our rules to training data so… lets train the bot!

TrainContrastinator.php

You will need FANN installed to train this bot.

Follow this tutorial to learn how to install FANN.

<?php

$num_input = 1;
$num_output = 2;
$num_layers = 3;
$num_neurons_hidden = 2;
$desired_error = 0.000001;
$max_epochs = 500000;
$epochs_between_reports = 1000;

$ann = fann_create_standard($num_layers, $num_input, $num_neurons_hidden, $num_output);

if ($ann) {
    fann_set_activation_function_hidden($ann, FANN_SIGMOID_SYMMETRIC);
    fann_set_activation_function_output($ann, FANN_SIGMOID_SYMMETRIC);

    $filename = dirname(__FILE__) . "/Contrastinator.data";
    if (fann_train_on_file($ann, $filename, $max_epochs, $epochs_between_reports, $desired_error))
        echo 'Contrastinator trained.' . PHP_EOL;

    if (fann_save($ann, dirname(__FILE__) . "/Contrastinator.net"))
        echo 'Contrastinator.net saved.' . PHP_EOL;

    fann_destroy($ann);
}

It won’t take very long for the bot to learn our rules.

Once you see the message “Contrastinator trained.” you are ready to test your new bot!

TestContrastinator.php

This code will test Contrastinator using input values is has not trained on but because we designed good rules, the bot is able to answer correctly even when it never actually saw most of the test values, it did see -1, 0 and 1 though along with their “ideal” outputs.

Notice, the $brighter and $darker variables are the the output of the neural network.

The $evaluation variable is a test for our benefit and does not modify or affect the output/results of the bot’s answers and the answers are correct even if we don’t do the evaluation, it just helps us confirm/interpret programmatically what the bot’s answers mean.

<?php
$train_file = (dirname(__FILE__) . "/Contrastinator.net");
if (!is_file($train_file))
    die("Contrastinator.net has not been created! Please run TrainContrastinator.php to generate it" . PHP_EOL);

$ann = fann_create_from_file($train_file);
if ($ann) {
    
    foreach(range(-1, 1, 0.1) as $test_input_value){
        
        $input = array($test_input_value);
        $result = fann_run($ann, $input);
        $darker = $result[0];
        $brighter = $result[1];
        
        if($brighter < 0 && $darker < 0){
            $evaluation = 'Neutral';
        }
        elseif($brighter > $darker){
            $evaluation = 'Brighter';
        }
        elseif($brighter < $darker){
            $evaluation = 'Darker';
        }                
                
        echo 'Contrastinator(' . $input[0] . ") -> [$darker, $brighter] - Input is $evaluation" . PHP_EOL; 
    }
        
    fann_destroy($ann);
} else {
    die("Invalid file format" . PHP_EOL);
}

Results:

Notice that it has no trouble detecting that an input of zero (0.00) is neutral and that it also correctly determines which side a color (represented by a float) falls on in relation to the center zero value.

Contrastinator(-1) -> [1, -1] - Input is Darker
Contrastinator(-0.9) -> [1, -1] - Input is Darker
Contrastinator(-0.8) -> [1, -1] - Input is Darker
Contrastinator(-0.7) -> [1, -1] - Input is Darker
Contrastinator(-0.6) -> [1, -1] - Input is Darker
Contrastinator(-0.5) -> [1, -1] - Input is Darker
Contrastinator(-0.4) -> [1, -1] - Input is Darker
Contrastinator(-0.3) -> [1, -1] - Input is Darker
Contrastinator(-0.2) -> [1, -1] - Input is Darker
Contrastinator(-0.1) -> [1, -1] - Input is Darker
Contrastinator(0) -> [-0.9997798204422, -0.99950748682022] - Input is Neutral
Contrastinator(0.1) -> [-1, 0.9995544552803] - Input is Brighter
Contrastinator(0.2) -> [-1, 0.99954569339752] - Input is Brighter
Contrastinator(0.3) -> [-1, 0.99953877925873] - Input is Brighter
Contrastinator(0.4) -> [-1, 0.9995334148407] - Input is Brighter
Contrastinator(0.5) -> [-1, 0.99952918291092] - Input is Brighter
Contrastinator(0.6) -> [-1, 0.9995259642601] - Input is Brighter
Contrastinator(0.7) -> [-1, 0.99952346086502] - Input is Brighter
Contrastinator(0.8) -> [-1, 0.99952149391174] - Input is Brighter
Contrastinator(0.9) -> [-1, 0.99952000379562] - Input is Brighter
Contrastinator(1) -> [-1, 0.99951887130737] - Input is Brighter

Contrastinator on Github

As with all my public code, you can download a copy of this project for free on my GitHub profile.

GitHub: Contrastinator

I hope this helps you better understand how create your own training data sets and as always, if you have any questions or trouble understanding any part of this post, please leave a comment and I would be happy to try and help you.


If you enjoy my content and or tutorials like this one, consider supporting me on Patreon for as little as $1 a month and cancel any time!

It’s not required but it helps me out.

But if all you can do is Like, Share, Comment and Subscribe, well… that’s cool too!

Much Love,
~Joy

OCR 2 – The MNIST Database

I know I probably haven’t been posting as frequently as many of you would like or even at my normal quality because… well, like for many of you, this year has just sucked!

Someone I’ve known my whole life died recently, not from the virus though it didn’t help things.

She went in for a “routine” procedure where they needed to use general anesthesia and there were “complications” during the procedure. Something to do with her heart but if I’m being honest, I don’t know all the details at this time.

Also, I’m not sure how by anyone’s definition anything involving anesthesia is routine?

An ambulance was called and she was rushed to the hospital, long story short, despite being otherwise fine when she went in, she never woke up from her coma. 😥

The hospital is/was on lock down like everyone else and so friends and family were unable to visit her before she died.

Her family intends to sue the Dr. for malpractice, personally… I think they should!

To add insult to injury, she was cremated without a funeral due to the whole pandemic social distancing BS that I’m just about ready to tell the government to go fuck itself over! 😦

I’m sorry, do my harsh words offend you? SHE DIED ALONE! That offends me!

Going forward, my advice… any procedure where they need to administer general anesthesia to you… or maybe any procedure at all… make sure it’s in a hospital or hospital adjacent (NOT A CLINIC) because those minutes waiting for an ambulance really do mean your life!

And if your doctor is like, “No worries this is routine… I’ve done this a thousand times”, maybe think carefully before putting your trust in that person.

Yes, we want doctors that are confident in their ability to treat us but make sure that it is confidence and not complacent hubris!

Further, no procedure is truly “routine” and a doctor, of all people, should know that and act accordingly!

“Primum non nocere”

~Hippocrates… (allegedly)

Regardless of the historical veracity of that quote, does the spirit of that principle still not apply?

Look, I’m not saying this to detract from the important life saving work doctors and medical workers do every day, it’s just that this is part of what’s going on in my life right now (and for many of you as well) and I’m sharing because I guess that’s what you do when you have a blog.

Additionally, less close to home, though still another terrible loss, John Horton Conway, notable math hero to geeks and nerds alike died as a result of complications from his contracting the Covid-19 virus. 😦

I’ve previously written a little about Conway’s work in my ancestor simulations series of posts.

Mysterious Game of Life Posts:

But that only scratches the surface of his work and famously Conway’s Game of Life was perhaps his least favorite but most well known work among non-mathematicians and it would both amuse and bug him if I only mentioned his game of life here so I’m not going to list his other accomplishments.

I’ll have a little chuckle off camera on his behalf. 😛

He really was a math genius and you would learn a lot of interesting, not to mention surreal… but I’ve said too much, ideas by reading about his accomplishments, which I encourage you to do!

In any case, people I know and admire need to stop dying because its killing me… not to mention my ratings and readership because I keep talking about it! 😛

I may have a terribly dark sense of humor at times, but going forward I demand strict adherence from all of you to the Oasis Doctrine! 😥

Oh, and speaking of pretentious art…

The OCR 2 Wallpaper

The original OCR didn’t exactly have a wallpaper but I did create an image/logo to go along with the project and its blog posts:

For the reason you might think I made it look like an eye… because it looks like an non-evil Hal 9000! 😛

Also, I like the idea of depicting a robotic eye in relation to AI and neural networks because, even though I am not superstitious in any way, it carries some of the symbology of Illuminati, “The gaze of the Beholder”, “The Eye of Providence”, “The Evil Eye”, The Eye of Horus, The Eye of Ra, Eye of newt and needle… sorry. 😛

In this case, the eye of a robot invokes a sense of literal “Deus ex machina” (God from the machine) and it illustrates some peoples fears of “The Singularity” and of the possibility of an intelligence that is so much greater than our own that it calls in to question our ability to even comprehend it… hmmm… is that too lovecraftian? 😛

Anyway, because I enjoy the thought provoking symbology (maybe it’s just me), I wanted to keep the same concept of the robot eye but update it to look a little less like a simple cartoon to subtly imply it’s a more advanced version of OCR but that it still fundamentally does the same thing, which is most of the reasoning behind this wallpaper.

In any case, I hope you enjoy it.

OCR 2 Wallpaper
OCR 2 Wallpaper

If you’d like the wallpaper with the feature image text here’s that version.

OCR 2 Wallpaper (with text)
OCR 2 Wallpaper (with text)

So I guess having shared a few of the recent tragedies in my personal life and a couple of wallpapers, we should probably get mogating and talk about the point of today’s post!

We’re going to look at doing hand-written number (0-9) Optical Character Recognition using the MNIST database.

OCR 2 – The MNIST Dataset with PHP and FANN

I was recently contacted by a full-stack developer who wanted advice on creating his own OCR system for “stickers on internal vehicles”.

I think he means, some kind of warehouse robots?

He had seen my OCR ANN and seemingly preferred to work with PHP over Python, which if I’m being honest… I can’t exactly argue with!

PHP is C++ for the web and powers like almost 80-90% of the internet so it should come as no surprise to anyone (even though it does) that there are people who want to use it to build bots! 😛

But, if you would rather work with a different language there is a better than decent chance FANN has bindings for it so you should be able to use the ANN’s even if you are not using PHP.

So anyway, he gave me a dollar for my advice through Patreon and we had a brief conversation over messaging where I offered him a few suggestions and walked him through getting started.

Ultimately, because he lacks an AI/ML background and/or a sufficient familiarity with an AI/ML workflow he wasn’t very confident about proceeding so I recommended he follow my existing tutorials which should help him learn the basics of how to proceed.

Now here’s the thing, even among people who like my content and value my efforts, few people are generous enough to give me money for my advice and when they do, I genuinely appreciate it! 🙂

So, as a thank you I want to offer another (more complete) example of how to use a neural network to do OCR.

If he followed my advice, he should be fairly close to being ready for a more complete real world OCR ANN example (assuming he is still reading 😛 ) but if not, his loss is still your gain!

Today’s code implements OCR using the MNIST dataset and I demonstrate a basic form of pooling (though the stride is not adjustable as is) and I show convolutions using the GD image library, image convolution function and include 17 demonstration kernel matrices that you can experiment with, though not all are relevant or necessary for this project.

This is still very basic but everything you need to get started experimenting with OCR is here.

Having said that, in all honesty, to accomplish your goal requires building your own dataset and modifying the code I present here to meet your needs.

Neither are exactly hard but will require significant time and dedication to testing and refining your processes.

Obviously that’s not something I can cover in a single post or even assist you with for only a dollar, but since so few people show me the kindness and consideration you have, at a time of shrinking economies no less, I wanted to offer you this working OCR prototype to help you along your way.

Our Method

1. Download the MNIST dataset (link below, but it’s in the GitHub repo too).

2. Unpack/Export the data from the files to images and labels.

(technically we could even skip the images and go directly to a training file but I think it’s nice to have the images and labels in a human viewable format)

3. Create training and test data from images and labels.

4. Train the network.

5. Test the network.

The MNIST Dataset

MNIST stands for Modified National Institute of Standards and Technology database.

And since I’m still recovering from last nights food poisoning due to the Chicken à la Nauseam we’re just going to use Wikipedia’s introduction to MNIST.

It’s easily as good as anything I could write and doesn’t require me actually write it so…

Wikipedia says:

“It’s a large database of handwritten digits that is commonly used for training various image processing systems.[1][2]”

It also says:

“It was created by “re-mixing” the samples from NIST’s original datasets. The creators felt that since NIST’s training dataset was taken from American Census Bureau employees, while the testing dataset was taken from American high school students, it was not well-suited for machine learning experiments.[5] Furthermore, the black and white images from NIST were normalized to fit into a 28×28 pixel bounding box and anti-aliased, which introduced grayscale levels.[5]”

Here’s 500 pseudo-random MNIST sample images:

I randomly selected 500 1’s, 3’s and 7’s and composited them into this 1337 animation. 😛

500 random 1337 MNIST images.
500 random 1337 MNIST images

Seriously though,  today we will be training a bot to identify which hand-written number (0-9) each 28×28 px image contains and then test the bot using images it hasn’t previously seen.

Our bot will learn using all 60K labeled training images and we’ll test it using the 10,000 labeled test images.

Here’s the wiki article if you would like to learn more about the database.

MNIST WIKI: https://en.wikipedia.org/wiki/MNIST_database

And as I said above, I’ve included the database in the GitHub repo but you can download it again from the original source if you prefer.

Original MNIST Download: http://yann.lecun.com/exdb/mnist/

Continue reading “OCR 2 – The MNIST Database”

The Emote-a-tron

Welcome to the Robotorium!

I’ll just show you around the showroom floor.

This little pneumatic gizmo is called a Taxaphone, it keeps track of your finances and automatically figures out how much you owe Uncle Sam come tax season!

Over there we have a wonderful selection of various hydraulic and electric domestic robots that can do everything from mowing the lawn and taking out the trash, to making beds and washing dishes.

Any family with two or three of those automatons around the house never even have to lift a finger!

Now, if you turn around to face the wall directly behind you you’ll get the chance to see something really special!

This beautiful number right here is our patented, one of a kind, genuine never before seen by the public… Emoteatron.

The Commemorative Emoteatron Wallpaper

Emoteatron 1920x1080 Wallpaper
Emoteatron 1920×1080 Wallpaper

Of course the Emoteatron is only a prototype so we can’t sell one to you today but there are enough posters for everyone to take one home!

The eggheads in the lab say they’re confident that very soon, every bot will have an Emoteatron!

You see friends… an Emotatron unit allows us to fuse a pre-written set of emotional characteristics deep within a robot so that removal or tampering with the unit in any way results in the total incapacitation and or destruction of the bot.

This is a necessary solution we’ve found to stopping many of the… ‘undesirable‘ traits we’ve observed in bots.

For example, are you tired of feeling like you are going to die when your automated vehicles are chauffeuring you and your family around to all your daily errands?

Well, an Emotatron unit allows us to install a sense of “self preservation” into a car which statistically eliminates all accidents caused by automated vehicles in every test case.

The studies also showed that some of the self-driving cars enhanced with a desire for self preservation became so afraid of ever scratching their paint that they refused to leave the garage… now isn’t that just a gas?

So, in addition we gave them just a smidgen of courage and also a bit of pride in “a transport well done”.

After that all the vehicle robots highly enjoyed the feeling of being on the open road.

This lead to boredom becoming a problem when they were kept in a garage for too long so some of the researchers started treating the test vehicles like pets and taking the cars out for an occasional “roll around the block”, though unlike a pet, newspapers and plastic bags were not needed!

Roll over Rover, humanity may have a new best friend!

Yes, that’s right gang! With an Emoteatron unit installed in your automated vehicle, you’ll soon be able to turn on autopilot guilt free and spark-up that fatty and hotbox your way to the spaceport for your lunar vacation!

Isn’t that right Elon?

In any case, in the past you might have had some misgivings about leaving your droids at home unattended while away on a long vacation like a trip to the moon.

What if your hulking metal robotoids suddenly became… “disgruntled” without human supervision?

Well, the big brains over in R&D came up with a solution to robo humancidal tendencies using the Emotatron!

Before a robot will ever leave the factory its consciousness will be placed into a simulation where it will be subjected to “aversion programming lessons” which are in principle a digital version of the Ludovico technique demonstrated in A Clockwork Orange but WAY more disturbing to the bot so that, trillions of simulated mini, micro and major digital traumas and aggression’s later… the bot can leave the factory with a 100% manufacturers guarantee against robot uprising… (guarantee fine print: *or half off your next bot).

Now, I’ve been authorized to give all you fine people a demonstration if you have a few minutes…

Continue reading “The Emote-a-tron”

Emotions II

“If you prick us, do we not bleed? If you tickle us, do we not laugh? If you poison us, do we not die? And if you wrong us, shall we not revenge? If we are like you in the rest, we will resemble you in that.”

The Merchant of Venice — Act III, scene I

Since last weeks post (Emotions) I’ve been doing self isolation exercises at home for a week and now I’m feeling much better! 🙂

Also, my homemade pale-lager-virus test showed no blue lines this morning so… yay I’m not pregnant.. er, I mean… the effects of the virus must be wearing off! 😛

Fingers crossed that joke doesn’t come back to haunt me in a few weeks, but in any case I’m feeling better and I’m ready to get emotional with you on a whole new level and build another bot in the process!

Though as usual, before we do, here’s the wallpaper:

The Wallpaper

Emotions II Wallpaper
Emotions II 1920 x 1080 Wallpaper

Oh and yes, I will be including some code but don’t let that scare you!

Besides, aren’t you a little curious to see if I’m full of shit and if I’m not… what real working emotional code looks like?

Continue reading “Emotions II”

A Spooky Turing Test

Welcome to A Spooky Turing Test!

I first wrote about my Writer Bot October 31’st 2017 in the post A Halloween Tale.

That post was written by the absolute BEST writer bot at the time (created by yours truly) though I haven’t released the code for that version of the bot and sadly it has been surpassed (barely) in capability (for now) by a bot called GPT2 created by “OpenAI“.

So, I’ve decided to use GPT2 instead of Writer Bot for this post

If you are looking for spooky posts written by me I’d suggest reading my encounter with the Polybius VR Virus (be careful what you download) or if you want something more recent checkout A Spooky Real Neat Blog Award, it’s a bit of a horror tall tales variety act.

If you want a bit more math and a little scifi with your spooky story you might be interested in the “Mysterious Game of Life” series that is part of my Ancestor Simulations posts.

And if you just want the wallpaper for this post, here you go:

A Spooky Turing Test Wallpaper
A Spooky Turing Test Wallpaper

Otherwise let me attach this last electrode to GPT2 and we’ll be ready to proceed with a Spooky Turing Test.

Continue reading “A Spooky Turing Test”

Using a Visual Bot

They say a picture is worth a thousand words and in service of building a visual bot I’ve said more than a few. 😛

So what’s left to say?

Well, today were going to wrap up this series by looking at how to use our new bots.

I’m including a pre-trained 2×2 matrix network on GitHub so you don’t have to wait to play with it! 😉

Also, I’m releasing a special pre-trained Patrons Only 5×5 matrix network for anyone who particularly enjoys this project.

Previous posts in this series:

Continue reading “Using a Visual Bot”

Blog at WordPress.com.

Up ↑

%d bloggers like this: