Hello Everyone, I hope your doing well.

During this quarantine I became interested in machine learning how this can be used for signal processing.

Today I would like to share with you a tutorial on how to classify emg signals in a raspberry pi using tensorflow/tensorflow lite

In case you are wondering, an EMG signal stands for electromyography signal which measures electrical currents generated by muscles. They are usually to detect abnormalities in medicine but today we will use them to detect gestures.

Requirements:

• Myo-wristband or EMG sensors
• raspberry pi 4 with Raspberry Pi OS
• laptop with Ubuntu 18.04
• tensorflow 2.2
• tensorflow lite interpreter
• Python3.6.9

Setting up:

On your laptop with Ubuntu 18.04:
Let’s start by bringing up the terminal and make a new directory that will contain the whole project.

To avoid breaking system tools or others projects,install and create a virtual environment that will contain all python packages for this project

$sudo apt-get install python3-venv
$python3 -m venv env_project
$source env_project/bin/activate

Once that you have activated the environment, download the latest version of tensorflow using the following commands

$pip3 install --upgrade pip
$pip3 install tensorflow

The most important thing that we will need for this project is the emg sensors, it might be difficult to acquire them for now. Fortunately, thanks to the arm school program, people located in arm cambridge can acquire a thalmic lab myo armband which contains 8 emg sensors

To interface with myo-wristband  we will need to use a python module called open-myo developed by Alvipe and bluepy.

Note: these modules will only work in linux and Raspberry Pi OS with generic Bluetooth LE antennas

Once you have installed bluepy download the following zip from here , unzip it and copy the files to your current directory. The original files can be found in https://github.com/Alvipe/Open-Myo

Make a dir called dataSet and run recording.py to test the connection with the myo-wristband

$mkdir dataSet
$sudo python3 recording.py

You should be getting the ble devices around you, put on the wristband and double tap with your fingers to wake the device up

If your wristband is being updated with the latest software from thalmic labs, you will be able to see its mac address, version and myo name( in case the latter differs from yours open open_myo.py and change the version and myo name to coincide with yours)

If everything is ok, you will be able to input the number of iterations that you want to execute, the number of samples to record and the name of the gesture, e.g:
10, 100, fist

Once this program runs, you should have one txt containing 100 recordings per line (each 8 recordings in a line corresponds to one sample)

On your raspberry pi 4
Bring up the terminal and make a directory for the project 

$cd ~/Desktop
$mkdir emg_project_lite
$cd emg_project_lite

As we did before, we will need to install bluez and copy the files contained in the module zip

$sudo pip3 install bluepy

Make a dir called dataSet and run recording.py, check if you are interfacing correctly with the arm-wristband

Finally download the interpreter to load TensorFlow lite models

$pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_armv7l.whl

Instructions to implement our model

Now that we have finished with the set-up, let’s talk about the tools being used to classify the signals. LTSM network or Long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used in the deep learning field that can process entire sequences of data (such as speech or video), In most papers related to signal processing specially EMG it seems like LTSM network is the most ideal candidate for this job, therefore we will implement it in our project.

Now place your myo-wristband at your forearm with the logo facing up, like in the picture below

We will record three gestures (fist, v, spread fingers) with our recording.py script.

record 150 samples per 100 iterations (it might take some time but it will help to increase the accuracy of the model)
$sudo python3 recording.py
>100
>150
>fist

Open the file train_model.py. To implement our lstm network we will need to reformat our recordings.

The current data contains 150 samples per iteration where each sample contains 8 features (emg channels). Therefore we will convert this data into an array of iterationsXsamplesXfeatures size or in this case to 100x150x8 per gesture where 80% of this data will be used as training and 20% as testing with the features being normalized. At the same time, we will make an array of iterations size containing a number as a label that will map with training and testing data which then will be converted into a category (one-hot encoded) with the help tf.keras.utils.to_categorical

By the end of line 70 we have arranged the recordings into:

data_x_train = a three dimensional array containing 80% of all the gestures
data_y_train = a binary class matrix that will be mapped with the data_x_train
data_x_test = a three dimensional array containing the rest of the gestures
data_y_test = a binary class matrix that will be mapped with the data_x_test

Note: the following implementation of a simple lstm network comes from Jason Brownlee

We can implement our lstm network with the following code:

model_signal_emg = tf.keras.models.Sequential()
model_signal_emg.add(tf.keras.layers.LSTM(128,

                                     input_shape=data_x_train.shape[-2:]))

The first line will create a keras model to which we will add an LSTM layer that will take as input dimensions iterationsXsamplesXfeatures and output arrays of shape (*, 128)

model_signal_emg.add(tf.keras.layers.Dense(128,activation='relu'))
model_signal_emg.add(tf.keras.layers.Dense(gesturesTotal,activation = 'softmax'))

We proceed to add a dense layer of input 128 with an activation of the rectified linear unit.
After that we need to add one more dense layer with the total number of gestures and the activation being softmax since we are classifying more than 2 classes.

Now that we have inserted these layers we can compile, fit and evaluate the model with the following code

model_signal_emg.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model_signal_emg.fit(data_x_train, data_y_train, epochs=EPOCHS, batch_size=BATCH_SIZE, verbose=1)
_, accuracy = model_signal_emg.evaluate(data_x_test, data_y_test, batch_size=BATCH_SIZE, verbose=0)

After that we will need to convert this model into a tflite. To do it, we will need to extract a concrete function from the model

numberOfStepsPerSample = data_x_train[0].shape[0]
run_model = tf.function(lambda x: model_signal_emg(x))
BATCH_SIZE = 1
STEPS = numberOfStepsPerSample
INPUT_SIZE = 8
concrete_func = run_model.get_concrete_function(
tf.TensorSpec([BATCH_SIZE, STEPS, INPUT_SIZE], model_signal_emg.inputs[0].dtype))

Once you have extracted the concrete function you will store the model, reload it, convert it and store back to your current directory with the following code:

model_signal_emg.save("saved_model/my_model2",save_format='tf',signatures=concrete_func)
converter = tf.lite.TFLiteConverter.from_saved_model("saved_model/my_model2")
tflite_model = converter.convert()
with tf.io.gfile.GFile('<your path>/emg_project/model2.tflite','wb') as f:
  f.write(tflite_model)

The last thing that we will need to do is to send our model, label.txt to our raspberry pi and run the classify_emg_pi.py contained in the zip file

This is the end of the tutorial. In case that you own a sphero2.0 you could download kulka and run controlSpherePi.py to control it

https://drive.google.com/file/d/1f_ZG38_oQJIL9OXhN2ANXPpF38oC8ahd/view?usp=sharing

Thank you for reading this blog.