Today we’re going to continue my introduction to creating your own data sets series by building Contrast-a-tron.

Now, I know what you are thinking:

“We already did that, like… a while ago!”

Here’s the thing though… we didn’t! 😉

And besides, it wasn’t that long ago!

What we built before was a Contrast-inator and a Contrast-inator and a Contrast-a-tron are not the same things! 😛

Let me explain…

  • The Contrast-inator: Learned how to “predict/classify” if a single input color was to the left (darker) or to the right (lighter) in an imaginary red line in the exact center of a 2D gray-scale gradient representation of the 0-255 RGB 3D color space.
  • The Contrast-a-tron (this bot): Is a much smarter and more interesting bot. It will learn how to “predict/classify” two input colors as “darker” and “lighter” or “the same” compared with each other. Which is a much more challenging task for the bot to learn.

But before we get into that I think I owe you a wallpaper.

A Wallpaper

Don’t mind the title, it’s definitely not a template placeholder! 😛

Anyway, just due to me being me, I have a lot of old robots and parts laying around and I was out in the o’l boneyard and I found this really beat up Krypto mining bot for us to play with.

I built it back when I was going to launch my own currency (A long time ago when it was still a cool thing to do and not everyone was like “my ICO is next week, you should mine sum!!!!” 😉 😉 ), yeah… no thanks!

Anyway, the bot’s memory is completely corrupt, but… the optical circuitry and hardware are still functional and since mining bots are built to operate deep under miles of data in extreme low light conditions at high speed, it’s visual acuity is top-notch and it even supports infrared mode!

So don’t let it’s tiny eyes fool you, they are incredibly sensitive which is perfect for today’s project! 🙂

Contrast_a_tron 1920 x 1080 Wallpaper
Contrast_a_tron 1920 x 1080 Wallpaper

I should add that not all posts get a theme song but today’s is Night Business by Perturbator (not a sponsor), I love the little vocoded? robotic voice about two minutes and twenty seconds in. It’s definitely what this bot’s voice sounds like! 😛

Also before we proceed, I’d just like to set the record straight and confirm that I’m definitely not Satoshi Nakamoto!

The Contrast-a-tron

To begin, let’s first look at what our Contrast-inator does:

Is this pixel brighter or darker than the red line?
Is this pixel brighter or darker than the red line?

It takes a color/shade as an input and then tries to determine which side of the red line it falls on.

Not that useful but it’s good for operating inside a known range that never changes. Like, was the light red or green kinda stuff, or conceptually like a line following robot.

Anyway, what if you wanted to start comparing two colors at the same time and to make things even more complicated, what if the gradient wasn’t always facing the same direction (meaning the “brighter/darker” pixel could be on the left or the right)?

For most of you that task is trivial and you could do it almost unconsciously or with minimal mental effort, not the Contrast-inator though!

To compare two pixels the Contrast-inator must evaluate each separately and because the red line (which you can imagine is “where the robot is standing” on the gradient when it’s evaluating a color) doesn’t change, if both colors are to it’s left or right (the bot’s vantage position / the red line), then it is completely unable to compare them.

Because these colors are on the same side of the red line, the Contrast-inator cannot compare them but the Contrast-a-tron can.
Because these colors are on the same side of the red line, the Contrast-inator cannot compare them but the Contrast-a-tron can.

Just to be clear, the Contrast-inator will say that both pixels/shades are “brighter/to the right” of zero (where it stands / it’s anchor) but it cannot figure out which of the two colors are brighter and the same is true if both colors are darker (to the left of the red line).

Further, there is also no guarantee that we will always present the colors to the bot in the order of darker on the left and lighter on the right meaning that sometimes the gradient will be lighter on the left and darker on the right and we will need the bot to notice that difference and accommodate that circumstance.

How the Contrast-a-tron Works Differently

The Contrast-a-tron isn’t anchored to zero (the center of the gradient) and instead we can think of it moving around the gradient to try and find the “center” of the two colors (whatever color that might be) and from there it can evaluate which side (input color / shade) is brighter and which is darker.

In the event that the input colors/shades are the same then both Input A & B will be in the same place which means that it will be neither to the right or to the left of the bot.

How the Contrast-a-tron works differently.
How the Contrast-a-tron works differently.

How the Neural Networks Differ

I didn’t spend a lot of time discussing the structure of the neural network when we built the Contrast-inator but now that we have something to compare it against let’s look at a visual representation of each network.

How the Contrast-inator and the Contrast-a-tron neural networks differ.
How the Contrast-inator and the Contrast-a-tron neural networks differ.

On the left you see the Contrast-inator with it’s single input neuron, a hidden layer containing two hidden neurons and an output layer with two output neurons.

Additionally you see two “Bias” neurons represented in yellow that help the network learn what we want by “biasing” the output of that layer to the next so that it is never “none” (zero or no output).

What this means is that bias neurons add their value to the output signal of each neuron from their layer so that the signal is never no “activation signal” and some value propagates forward.

All layers except the output layer will always have a single bias neuron. There is no need of a bias neuron on the output layer because there is no signal to propagate beyond the output neurons so it wouldn’t serve any purpose.

Bias neurons have no inputs.

In practice we don’t have to concern ourselves with the bias neurons and the ANN will manage them itself but I like draw them because they do exist and they are part of the network, however it’s common for people not to include them in diagrams because they are so easy for us to ignore since we don’t really need to do anything with them and they are just there to help the signal propagate.

In any case, the Contrast-a-tron differs by including a second input neuron (for the second shade/color) and a second hidden layer which helps the Contrast-a-tron to be a little “smarter” and learn what we want it to.

I have a post about how to create diagrams like this called Visualizing Your FANN Neural Network and you can download a copy of the open source visualization software I wrote for free from my GitHub account here: https://github.com/geekgirljoy/FANN-Neural-Network-Visualizer

Training The Contrast-a-tron

When we created the Contrast-inator, I walked you through each training example and how it was derived because it was a very small data set requiring only three examples however this data set is a bit longer with thirteen examples and it will be a lot easier to show you the data set and then draw you a picture than to type a description but before we look at the training data, lets make sure we understand the outputs.

Understanding the Contrast-a-tron output.
Understanding the Contrast-a-tron output.

There are two outputs and we’ll call them A & B and they are in that order.

In an ideal world the bot will give us -1 & -1 to mean they are the same, 1 & -1 to mean that A is Brighter and B is Darker and -1 & 1 to mean A is Darker and B is Brighter.

In reality… what we get is a number that comes close but isn’t -1 or 1 called a “floating point number” in computer science but most people just call them a decimal number like for example 0.123.

In practice this means that as long as A & B are not both negative, then whichever has the higher positive value is the “brighter” color and whichever has the lower positive value is the “darker” color otherwise they are the same (A==B).

Let’s look at the training data and visualize it.

Contrast-a_tron.data

This is the complete Contrast-a-tron training data.

The first line is the “FANN Header” which consists of: the Total_Number_of_Example_Sets the Number_of_Inputs the Number_of_Outputs\n

Note the spaces between the values on the header line as well as between the inputs and the output values.

Line 2 (-1 -1) is an input example. Line 3 (-1 -1) is an output example and the pattern of Input_Example\nOutput_Example\n continues to the end of the document.

13 2 2
-1 -1
-1 -1
-0.5 -0.5
-1 -1
0 0
-1 -1
0.5 0.5
-1 -1
1 1
-1 -1
1 -1
1 -1
0.5 0
1 -1
0 0.5
-1 1
-1 -0.5
-1 1
-0.5 -1
1 -1
1 0.5
1 -1
0.5 1
-1 1
-1 1
-1 1

Let’s visualize this training data which should hopefully give you a more intuitive sense for how these numbers translate to information the Contrast-a-tron ANN can use to learn.

Visualizing the Contrast-a-tron training data set
Visualizing the Contrast-a-tron training data set

The Code

Here’s the code used to train. I have other tutorials covering what this all means available on my Topics and Posts page so I won’t go into what all of this means but basically it sets up a training environment and trains the Contrast_a_tron ANN and saves the results to a FANN .net network file.

TrainContrast_a_tron.php

<?php

$num_input = 2;
$num_output = 2;
$layers = array($num_input, 2, 1, $num_output);
$ann = fann_create_standard_array(count($layers), $layers);

$desired_error = 0.0000000001;
$max_epochs = 900000;
$epochs_between_reports = 10;

if ($ann) {
    fann_set_activation_function_hidden($ann, FANN_SIGMOID_SYMMETRIC);
    fann_set_activation_function_output($ann, FANN_SIGMOID_SYMMETRIC);
    fann_set_training_algorithm($ann,FANN_TRAIN_INCREMENTAL);


    $filename = dirname(__FILE__) . "/Contrast_a_tron.data";
    if (fann_train_on_file($ann, $filename, $max_epochs, $epochs_between_reports, $desired_error)){
        echo 'Contrast_a_tron trained.' . PHP_EOL;
    }

    if (fann_save($ann, dirname(__FILE__) . "/Contrast_a_tron.net")){
        echo 'Contrast_a_tron.net saved.' . PHP_EOL;
    }
    
    fann_destroy($ann);
}

 

TestContrast_a_tron.php

We next need to test the ANN so I use two “for loops” with one counting down to -1 and one counting up to 1 and each incrementing by -0.2 each iteration of the loop as the inputs to test with.

<?php

$train_file = (dirname(__FILE__) . "/Contrast_a_tron.net");
if (!is_file($train_file))
    die("Contrast_a_tron.net has not been created! Please run TrainContrast_a_tron.php to generate it" . PHP_EOL);

$ann = fann_create_from_file($train_file);

if ($ann) {
    
    foreach(range(1, -1, -0.2) as $test_input_value_a){
        foreach(range(-1, 1, -0.2) as $test_input_value_b){
        
            $input = array($test_input_value_a, $test_input_value_b);
            $result = fann_run($ann, $input);

            $a = number_format($result[0], 4);
            $b = number_format($result[1], 4);
            
            // What answer did the ANN give?
			
            $answer = NULL;
            $evaluation = '';
            if($a <= 0 && $b <= 0){
                $evaluation = 'Neutral/Same';
                $answer = 0;
            }
            elseif($a > $b){
                $evaluation = 'A is Brighter';
                $answer = -1;
            }
            elseif($b > $a){
                $evaluation = 'B is Brighter';
                $answer = 1;
            }
            else{ 
                $evaluation = ' OOPSIES!!!!!!!';
            }

            echo 'Contrast_a_tron(' . $input[0] . ', ' . $input[1] . ") -> [$a, $b] - $evaluation" . PHP_EOL; 
        }
    }
    fann_destroy($ann);
}
else {
    die("Invalid file format" . PHP_EOL);
}

Results

The Results/Output of the test code.

Contrast_a_tron(1, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(1, 0.8) -> [0.9986, -1.0000] - A is Brighter
Contrast_a_tron(1, 1) -> [-1.0000, -0.1815] - Neutral/Same
Contrast_a_tron(0.8, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.6) -> [0.9992, -1.0000] - A is Brighter
Contrast_a_tron(0.8, 0.8) -> [-1.0000, -0.2218] - Neutral/Same
Contrast_a_tron(0.8, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.6, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0.4) -> [0.9995, -1.0000] - A is Brighter
Contrast_a_tron(0.6, 0.6) -> [-1.0000, -0.4005] - Neutral/Same
Contrast_a_tron(0.6, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.6, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.4, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, 0) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.4, 0.2) -> [0.9996, -1.0000] - A is Brighter
Contrast_a_tron(0.4, 0.4) -> [-1.0000, -0.6543] - Neutral/Same
Contrast_a_tron(0.4, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.4, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.4, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, -0.2) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0.2, 0) -> [0.9996, -1.0000] - A is Brighter
Contrast_a_tron(0.2, 0.2) -> [-1.0000, -0.8580] - Neutral/Same
Contrast_a_tron(0.2, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0.2, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.4) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(0, -0.2) -> [0.9996, -1.0000] - A is Brighter
Contrast_a_tron(0, 0) -> [-1.0000, -0.9557] - Neutral/Same
Contrast_a_tron(0, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(0, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.6) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.4) -> [0.9995, -1.0000] - A is Brighter
Contrast_a_tron(-0.2, -0.2) -> [-1.0000, -0.9878] - Neutral/Same
Contrast_a_tron(-0.2, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.2, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.4, -0.8) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.4, -0.6) -> [0.9994, -1.0000] - A is Brighter
Contrast_a_tron(-0.4, -0.4) -> [-1.0000, -0.9965] - Neutral/Same
Contrast_a_tron(-0.4, -0.2) -> [-1.0000, 0.9997] - B is Brighter
Contrast_a_tron(-0.4, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.4, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, -1) -> [0.9998, -1.0000] - A is Brighter
Contrast_a_tron(-0.6, -0.8) -> [0.9990, -1.0000] - A is Brighter
Contrast_a_tron(-0.6, -0.6) -> [-0.9999, -0.9989] - Neutral/Same
Contrast_a_tron(-0.6, -0.4) -> [-1.0000, 0.9996] - B is Brighter
Contrast_a_tron(-0.6, -0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.6, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, -1) -> [0.9981, -1.0000] - A is Brighter
Contrast_a_tron(-0.8, -0.8) -> [-0.9999, -0.9995] - Neutral/Same
Contrast_a_tron(-0.8, -0.6) -> [-1.0000, 0.9993] - B is Brighter
Contrast_a_tron(-0.8, -0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, -0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-0.8, 1) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, -1) -> [-0.9998, -0.9998] - Neutral/Same
Contrast_a_tron(-1, -0.8) -> [-1.0000, 0.9982] - B is Brighter
Contrast_a_tron(-1, -0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, -0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, -0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.2) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.4) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.6) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 0.8) -> [-1.0000, 0.9998] - B is Brighter
Contrast_a_tron(-1, 1) -> [-1.0000, 0.9998] - B is Brighter

GitGub

As always you can download a copy of this code on GitHub for free and if you have any questions or comments please leave them below.

Contrast-a-tron on GitHub: https://github.com/geekgirljoy/Contrast-a-tron


If you find yourself thinking…

“Joy you’re the best!”

I’d say….

If you support the resistance against Big AI then consider supporting my efforts through Patreon.

But, if all you can do is Like, Share, Comment and Subscribe… well that’s cool too!

Much Love,

~Joy