Convert image into spikes

README.md

@@ -1,17 +1,62 @@
-NE15-MNIST Database
-===================
-NE15-MNIST contains four sub datasets:    
-**Poissonian:**   
-The code and example for generating Poissonian spikes from MNIST is located in the folder *Poissonian*.   
-**Focal Rank Code Order:**   
-The code and example for generating spikes from MNIST using Focol is located in the folder *Focol*.  
-**DVS recorded flashing MNIST digits:**   
-download from: https://goo.gl/ru0fXP   
-**DVS recorded moving MNIST digits:**     
-download from: http://www2.imse-cnm.csic.es/caviar/MNISTDVS.html  
-
-You are welcome to cite the paper if you use the database.    
-"Benchmarking Spike-Based Visual Recognition: a Dataset and Evaluation",    
-Qian Liu, Garibaldi Pineda Garca, Evangelos Stromatias,Teresa Gotarredona, and Steve Furber   
-
-We invite you to **visit the [Wiki page](https://github.com/NEvision/NE15/wiki)** to find further information.
+# Convert Images to Poissonian Spikes
+
+Convert your image data to a Poisson spike source to be able to use with Spiking Neural Networks.
+
+<div align="center">
+  <table>
+    <tr>
+      <td> <img src="images/pumpkins.jpeg" alt="Pumpkins-RGB" height="120"> </td>
+      <td> &rarr; </td>
+      <td> <img src="images/pumpkins_gray.jpeg" alt="Pumpkins-GrayScale" height="120"> </td>
+      <td> &rarr; </td>
+      <td> <img src="images/spikes_plot_pumpkins.png" alt="Pumpkins-SpikesPlot" height="135"> </td> 
+    </tr>
+  </table>
+</div>  
+
+<i>
+  The parameters below are used when running <a href="convert_image_to_spike_array.py">convert_image_to_spike_array.py</a> in order to turn <a href="https://unsplash.com/photos/KnZDAYgRsz8">pumpkins</a> above into a spike array. 
+  <br> max_freq = 60000 (Hz)
+  <br> on_duration = 10000 (ms)
+  <br> off_duration = 5000 (ms)
+</i>
+
+## Requirements
+I use Python 3.5.2 on Linux, necessary packages are listed below along with their versions for reference.
+* matplotlib (3.0.3)
+* numpy (1.17.3)
+* opencv-python (4.1.1.26)
+
+Run `pip install -r requirements.txt` to install them all.
+
+## Project Files and Their Usage
+```
+images-to-spikes/
+├── convert_image_to_spike_array.py
+├── draw_image.py
+├── images
+│   ├── cross.png
+│   ├── horizontal_line_10x.png
+│   ├── horizontal_lines.png
+│   └── t10k-images-idx3-ubyte__idx_000__lbl_7_.png
+├── poisson_tools.py
+└── util_functions.py
+```
+**[convert_image_to_spike_array.py](convert_image_to_spike_array.py)** is the main file. 
+  - Please see its usage by running it: `python convert_image_to_spike_array.py`
+  - The program will store the output spike array as a _pickle_ under _pickles/_ folder in the same directory after the run. 
+  - If you do not want a _pickle_ at the end, change the parameter inside the file, i.e. `save_as_pickle=False`.
+  - You may use a single image file (extension could be anything _OpenCV_ accepts) or a folder which contains multiple images (extensions need to be _.png_) as input.
+
+**[draw_image.py](draw_image.py)** enables you to draw your own images by adding simple shapes into it via _OpenCV_. For more information please see the file.
+
+**[images](images/)** folder contains three of the images that I generated by using _draw_image.py_, and one example from MNIST dataset (t10k-images-idx3-ubyte__idx_000__lbl_7_.png).
+
+**[poisson_tools.py](poisson_tools.py)** is where the Poisson distribution modelling takes place.
+
+**[util_functions.py](util_functions.py)** includes utility functions of files and images.
+
+## References and Citation
+I only used the Poissonian spikes approach to obtain spike arrays from images in this project. The original project also contains _Focal Rank Code Order_ approach in this sense.
+
+Please refer to the original project's [Wiki page](https://github.com/NEvision/NE15/wiki) for further information.

convert_image_to_spike_array.py

@@ -0,0 +1,64 @@
+import numpy as np
+import cv2
+import sys
+import os
+
+import pylab
+
+from poisson_tools import image_to_poisson_trains
+from util_functions import *
+
+
+
+def img_to_spike_array( img_file_name, save_as_pickle=True ):
+    img = cv2.imread( img_file_name, cv2.IMREAD_GRAYSCALE )
+    if img is not None:
+        height, width = img.shape
+
+        spikes = image_to_poisson_trains( np.array( [img.reshape(height*width)] ), # notice reshape
+                                          height, width,
+                                          max_freq, on_duration, off_duration )
+        pylab.figure()
+        raster_plot_spike( spikes )
+        pylab.show()
+
+        #--- Pickle the spike array for further use -------------------------------------------#
+        if save_as_pickle:
+            img_file_name = img_file_name[ img_file_name.rfind('/')+1 : img_file_name.rfind('.') ]
+            pickle_file = "spike_array_{}".format( img_file_name )
+            pickle_it( spikes, pickle_file )
+    else:
+        print( "Image couldn't be read! -> from file ({}) to ({})".format( img_file_name, img ) )
+
+
+
+if __name__ == '__main__':
+    if len( sys.argv ) != 2 and len( sys.argv ) != 5:
+        print( "Usage:" )
+        print( "\t python  convert_image_to_spike_array.py  <img_file_name>  <max_freq>  <on_duration>  <off_duration>" )
+        print( "or  (with the default values for up to a 32x32 image  {max_freq=1000}  {on_duration=200}  {off_duration=100}):" )
+        print( "\t python  convert_image_to_spike_array.py  <img_file_name>" )
+    else:
+        img_file_name = sys.argv[1]
+
+        if len( sys.argv ) > 2:
+            max_freq = int(sys.argv[2])       # Hz
+            on_duration = int(sys.argv[3])    # ms
+            off_duration = int(sys.argv[4])   # ms
+        else:
+            max_freq = 1000      # Hz
+            on_duration = 200    # ms
+            off_duration = 100   # ms
+
+        print( "max_freq: {}".format( max_freq ) )
+        print( "on_duration: {}".format( on_duration ) )
+        print( "off_duration: {}".format( off_duration ) )
+
+        if os.path.isdir( img_file_name ):
+            import glob2
+            image_list = glob2.glob( os.path.join( img_file_name, "**/*.png" ) )
+            for img in image_list:
+                if os.path.isfile( img ):
+                    img_to_spike_array( img )
+        elif os.path.isfile( img_file_name ):
+            img_to_spike_array( img_file_name )

draw_image.py

@@ -0,0 +1,150 @@
+""" Draw (onto)black(canvas) & white images with basic shapes (i.e. lines, circles, rectangles)
+    via the functions below.
+
+    Functions:
+    - draw_horizontal_lines
+    - draw_vertical_lines
+    - draw_line_with_angle
+    - draw_a_rectangle
+    - draw_a_circle
+
+    Feel free to mix and match them to have multiple shapes in an image
+    by passing the same "img" parameter into the functions.
+"""
+
+import numpy as np
+import cv2
+
+from util_functions import *
+
+
+
+""" @Params:
+
+* width: Width of the image
+* height: Height of the image
+* thickness: Thickness of the line
+* key: Sets where the line would stand in the image.
+**  key options: start | middle | end | random
+* line_count: Only matters if the key equals to 'random', otherwise there would
+  always be 1 line in the image.
+* img: When not supplied, a new image is created, otherwise the passed image is used.
+"""
+def draw_horizontal_lines( width, height, thickness, line_count=1, key='middle', img=None ):
+    if thickness < 1 or thickness >= height:
+        print( "Thickness value is invalid for this image. -> {}".format( thickness ) )
+    else:
+        img = np.zeros( (height, width) ) if img is None else img
+
+        if key == 'random':
+            random_idx = np.random.random_integers( 0, height-thickness, line_count )
+            for i in range( line_count ):
+                cv2.line( img, (0, random_idx[i]), (width-1, random_idx[i]), 255, thickness )
+        else:
+            if key == 'middle':
+                idx = height//2
+            elif key == 'start':
+                idx = thickness//2
+            elif key == 'end':
+                idx = height-thickness//2
+            cv2.line( img, (0, idx), (width-1, idx), 255, thickness )
+
+        return img
+
+
+""" For @Params see above.
+"""
+def draw_vertical_lines( width, height, thickness=1, line_count=1, key='middle', img=None ):
+    if thickness < 1 or thickness >= width:
+        print( "Thickness value is invalid for this image. -> {}".format( thickness ) )
+    else:
+        img = np.zeros( (height, width) ) if img is None else img
+
+        if key == 'random':
+            random_idx = np.random.random_integers( 0, width-thickness, line_count )
+            for i in range( line_count ):
+                cv2.line( img, (random_idx[i], 0), (random_idx[i], height-1), 255, thickness )
+        else:
+            if key == 'middle':
+                idx = width//2
+            elif key == 'start':
+                idx = thickness//2
+            elif key == 'end':
+                idx = width-thickness//2
+            cv2.line( img, (idx, 0), (idx, height-1), 255, thickness )
+
+        return img
+
+
+""" @Params:
+
+* angle: Determines the angle of the line to be drawn.
+    Might be either 45 or 135 degrees.
+
+For the rest of the @Params see above.
+"""
+def draw_line_with_angle( width, height, angle, thickness=1, img=None ):
+    if thickness < 1:
+        print( "Thickness value is invalid for this image. -> {}".format( thickness ) )
+    elif angle not in (45, 135):
+        print( "Angle is not right. {} is not in [45, 135]".format( angle ) )
+    else:
+        img = np.zeros( (height, width) ) if img is None else img
+
+        if angle == 45:
+            cv2.line( img, (0, height-1), (width-1, 0), 255, thickness )
+        elif angle == 135:
+            cv2.line( img, (0, 0), (width-1, height-1), 255, thickness )
+
+        return img
+
+
+""" @Params:
+
+* top_left_pt: Top left point tuple of the rectangle to be drawn, e.g. (10, 12)
+* bottom_right_pt: Bottom right point tuple of the rectangle to be drawn, e.g. (29, 31)
+* thickness: -1 to fill inside the rectangle, otherwise determines the thickness
+    of the rectangle's outer lines.
+
+For the rest of the @Params see above.
+"""
+def draw_a_rectangle( width, height, top_left_pt, bottom_right_pt, thickness=1, img=None ):
+    img = np.zeros( (height, width) ) if img is None else img
+    cv2.rectangle( img, top_left_pt, bottom_right_pt, 255, thickness ) # 255 is the colour, default is white obviously
+    return img
+
+
+""" @Params:
+
+* center_pt: Center point tuple of the circle to be drawn, e.g. (14, 20)
+* radius: Radius of the circle, e.g. 6.
+
+For the rest of the @Params see function draw_a_rectangle.
+"""
+def draw_a_circle( width, height, center_pt, radius, thickness=1, img=None ):
+    img = np.zeros( (height, width) ) if img is None else img
+    cv2.circle( img, center_pt, radius, 255, thickness )
+    return img
+
+
+
+if __name__ == '__main__':
+    # Some usage scenarios
+
+    width = height = 32
+    thickness = 3
+    # img = draw_a_rectangle( width, height, (10, 12), (29, 31), -1 )
+    # img = draw_a_circle( width, height, (14, 20), 6, 2 )
+
+    img = draw_line_with_angle( width, height, 45, thickness )
+    # img = draw_line_with_angle( width, height, 135, thickness, img )
+
+    # img = draw_horizontal_lines( width, height, thickness )
+    # img = draw_horizontal_lines( width, height, 1, 5, 'random' )
+
+    # img = draw_vertical_lines( width, height, thickness, img=img )
+    # img = draw_vertical_lines( width, height, 1, 5, 'random' )
+
+    imshow_opencv( img )
+    # img_name = "new_image_etc.png"
+    # save_img( img_name, img )