Simple Artificial Neural Networks

Simple Artificial Neural Networks (SANN) is a naive Python implementation of an artificial neural network (ANN) that's useful for educational purposes and clarifying the concepts of feed-forward neural networks, backpropagation, neuro-evolution of weights and biases, and genetic algorithms. SANN is not intended for production use or performance-critical applications. Rather, use it for educational, playful or small-scale projects. 😉

See Behind the AI Curtain for a comprehensive exploration of the concepts behind this code. This project's code and assets are hosted on GitHub. The code works with both CPython and MicroPython.

Try examples of this library online via PyScript:

• Backpropagated numeral recognition - a neural network that underwent supervised training will categorise hand written numerals from a corpus of unseen test data. ✍️🎓
• Neuro-evolved snake game - the classic "SNAKE" game, but played by a neural network that underwent unsupervised neuro-evolution. Play with the arrow keys, or click the []🤖 checkbox to toggle the AI autopilot. It's a snAIke. 🐍🧬
• Tanks a lot - both virtual and real-world tank-bots use supervised and unsupervised neural networks, along with a "stupid" hard coded solution, to navigate obstacles in the world. 💥⁉️

Installation 📦

If you're using CPython:

1. Create a virtual environment.
2. pip install sann

For MicroPython, just copy the sann.py file somewhere on your Python path. If space is limited with MicroPython, use make minify to create a minified version of the module.

Usage 💪

☠️☠️☠️ Do not use this code in production or for performance-critical applications. ☠️☠️☠️

SANN is best suited for educational, playful or small-scale projects. Most importantly, please use SANN for fun things that enlarge our world in a humane, compassionate and thoughtful way.

If you use SANN to make something wonderful, consider sharing it with us so we can celebrate it and, with your permission, include it as an example. Mutual respect and being open minded to learning from each other are important aspects of the open-source way of working. Therefore we hope everyone involved in this project will abide by the ethos expressed in our statement about care of community.

Create a neural network ✨

The create_network function takes a list defining the number of nodes in each layer. It returns a representation of a fully connected feed-forward neural network.

This example creates a test network with three layers: an input layer with two nodes, a hidden layer with three nodes, and an output layer with one node.

import sann


my_nn = sann.create_network([2, 3, 1])

The network is expressed as a dictionary with three attributes:

1. structure - a list defining the number of nodes in each layer of the network (i.e. what was passed into the create_network function to create it).
2. fitness - by default set to None, but used during neuro-evolution to indicate the arbitrary fitness score of the network during unsupervised training.
3. layers - a list of layers, with each layer containing a Python dict for each node in the layer. Each dict contains a list of incoming weights and a bias value, all of which are initialised with a random value between -1 and 1. Since the input layer doesn't have associated weights nor bias (because its values are the raw input data), it is ignored.

{
  "structure": [2, 3, 1],
  "fitness": None,
  "layers": [
    # No definition of the input layer needed, because its values are the raw 
    # input data
    [ # Hidden layer. Each hidden node has a bias and
      # two input weights: one each from the nodes in
      # the input layer.
      { # Node 1
        'bias': -0.08932407876323856,
        'weights': [
          0.9318837478301161,
          -0.3259188141579621
        ]
      },
      { # Node 2
        'bias': -0.7449380314648402,
        'weights': [
          -0.15786850474033964,
          -0.9455648956883143
        ]
      },
      { # Node 3
        'bias': 0.5168993191227431,
        'weights': [
          -0.8359684467197377,
          0.09538722516032427
        ]
      }
    ],
    [ # Output layer.
      { # A single node with a bias and input weights
        # from each of the three nodes in the hidden
        # layer.
        'bias': -0.46255520816058215,
        'weights': [
          0.991047585915775,
          -0.9995162202419827,
          0.15538558263904179
        ]
      }
    ],
  ],
}

The representation of the neural network is designed to be JSON serializable.

Training 🎓

Training is the process through which the artificial neural network, created with randomly generated weights and biases, is modified and refined so that it achieves some useful outcome. There are broadly two ways to do this:

• Supervised training: where the network is trained with labelled data. Put simply, given many examples of training input, the network is adjusted so it produces the expected (labelled) output provided by humans. Once trained the network is tested with previously unseen labelled test data to check it correctly produces the expected results to the right level of accuracy. SANN provides backpropagation capabilities as an example of this sort of training.
• Unsupervised training: where the network is trained on data that is NOT labelled. This usually involves a training procedure that measures the accuracy or efficiency of the behaviour of the network in some way, combined with a process of adjustment used to improve the network's outcomes. SANN provides a genetic algorithm based neuro-evolution capability as an example of this sort of training.

SANN provides a means of achieving both types of training in the following ways:

Supervised training (backpropagation)... 🎛️

With supervised training, for the network to be useful it needs to be trained with labelled example data that indicates how inputs relate to outputs. Such data should be expressed as a list of pairs of values: the training input, and the expected values in the output layer. Please see the examples/digit_recognition/train.py file for an example of this process.

Use the train function to do exactly what it says:

trained_nn = sann.train(
    my_nn, training_data, epochs=1000, learning_rate=0.1, log=print
)

The train function takes the initial randomly generated neural network, and the training_data expressed as pairs of input/expected output, as described above. The epochs value (default: 1000) defines the number of times the training data is iterated over. The learning_rate (default: 0.1) defines by how much weights and bias values are changed as errors are corrected. Finally, the optional log argument references a callable used to log messages as the training process unfolds. It defaults to a no-op function with no side-effect if it is not given.

The output of the train function is a representation of the network with the refined weights and bias values.

Once trained, it is usual to check and evaluate the resulting neural network with as-yet unseen test data. The examples/digit_recognition/train.py file contains an example of this (see: evaluate_model). If the neural network is not accurate enough, perhaps consider adjusting the epochs or learning_rate values, and re-train.

Unsupervised training (neuro-evolution) 🧬

Alternatively, and usually because supervised training is not possible due to the context in which the neural network is used, unsupervised training via the evolution of a population of networks is required. This is illustrated in the examples/snaike/train.py file.

Use the evolve function in combination with a fitness function and halting function to run a genetic algorithm that evolves ANN. This is the minimal viable way of using evolve:

evolved_population = sann.evolve(
  layers=[3, 5, 2],
  population_size=1000,
  fitness_function=fitness_function,
  halt_function=halt,
  log=print,
)

The layers defines the structure of the neural networks to evolve. This is the same as the argument passed into create_network. The population_size defines how many networks exist in each generation of the genetic algorithm. More networks in a generation allows for a greater variety of solutions, but is slower to evolve.

The fitness_function should be a callable that takes two arguments: a reference to the ANN whose fitness is being measured and a reference to the current generation (of siblings). This final argument is sometimes needed because the network's fitness may depend on competitive performance between all the members of the current generation. For example, a board game playing network might be assessed by how well it performs when playing all the other networks in its generation. The fitness_function should return a numerical value so the networks can be sorted by fitness. The halt_function should be a callable that also takes two arguments: a reference to the current population of networks, along with an integer representing the current generation number. The halt function simply defines when the genetic algorithm should stop and returns a boolean value where True means stop.

Just like the train function, the optional log argument takes a callable used to log messages as the process of evolution unfolds.

Additional optional arguments (not shown in this example) include the generate_function that should take a list of the current population sorted by fitness, along with the optional fittest_proportion that determines the proportion of the fittest individuals to retain. The mutation_chance, and mutation_amount parameters are used to control the mutation process. The generate_function returns a new unsorted population for the next generation. By default SANN will use the built-in simple_generate function which will be good for most purposes. Finally, the reverse flag indicates if the fittest ANN has the highest (True) or lowest (False) fitness score.

Please see the API documentation for more details.

Use the network 🛠️

Given a representation of a trained or evolved neural network and a list of input values, use the run_network function to retrieve a list of output values caused by passing the inputs through the neural network.

That's it!

import sann


# Load the pre-trained neural network from somewhere.
with open("my_nn.json", "r") as f:
  nn = json.load(f)

# Get the input data from somewhere.
input_data = get_input_data_from_somewhere()

# Gather the result by running input data through the network.
result = sann.run_network(nn, input_data)

# Interpret / react to the result in some meaningful manner.
do_stuff_with(result)

Developer Setup 🧑‍💻

Before continuing, please read the statement about care of community ~ we hope everyone involved in this project will abide by the ethos expressed therein.

1. Clone the repository.
2. Create a virtual environment.
3. pip install -r requirements.txt
4. Run the test suite: make check
5. Educational examples in the examples directory.
6. To build the docs: make docs

See Behind the AI Curtain for more information.

Feedback / Bugs 🗣️

Simply create an issue in the GitHub repository. If you're using this code for something fun, please let me know because I'd love to include more examples from real-world creative explorations.

Thank you! 🤗