Recording detailed brain wave data through Python-OSC

[banjo]

So I'm struggling a bit to record similar data through Python-OSC as is saved in the CSV-files in Dropbox. Specifically I'm looking for these:

Code: Select all

Delta_TP9,Delta_AF7,Delta_AF8,Delta_TP10,Theta_TP9,Theta_AF7,Theta_AF8,Theta_TP10,Alpha_TP9,Alpha_AF7,Alpha_AF8,Alpha_TP10,Beta_TP9,Beta_AF7,Beta_AF8,Beta_TP10,Gamma_TP9,Gamma_AF7,Gamma_AF8,Gamma_TP10

@James has been very kind to create example code in Python, and I thought it would be easiest for me to start from "OSC Receiver Audio Feedback.py" and adapt it. I've been coding for decades, but am quite new to Python and don't understand the code well enough to know how the streaming works, or what the data structure is.

I've tried to insert the recording code in two places (see code below), first in the "abs_handler" function, this saves 1 data point per wave, and sometimes some data points are getting lost, which seems to lead to the rest getting out of order, at least on same line.
Then I tried with the "plot_update" function, this works a bit better, but the saving frequency seems lower compared to what I got from the CSV-file that get saved into Dropbox. I might however be able to live with this for my purpose, but I still get only 1 data point per wave, instead for Delta_AF8,Delta_TP10,Theta_TP9,Theta_AF7,Theta_AF8 etc.

Why do I need all this detailed wave data per sensor?

• because I found out that it gives me excellent results for machine learning compared to using raw data from each sensor

Why can't I use the CSV-file saved into Dropbox then?

• because I want to streamline the process as much as possible, and recording only the data I need should make things faster. If using the Dropbox saving method, I need to have my Python program looking for new files, opening them and splitting them into smaller chunks that can be injected into the machine learning "black box". This would be too laggy and very far from real-time.

PS I know about the FAQ at https://mind-monitor.com/FAQ.php#csvspec, and if I understand it correctly, it should be possible to get what I'm looking for as I can get One average or Four Float values, this according to

Code: Select all

Delta Absolute	/muse/elements/delta_absolute	f | f f f f	One Average or Four Float values	Bels	10Hz
Theta Absolute	/muse/elements/theta_absolute	f | f f f f	One Average or Four Float values	Bels	10Hz
Alpha Absolute	/muse/elements/alpha_absolute	f | f f f f	One Average or Four Float values	Bels	10Hz
Beta Absolute	/muse/elements/beta_absolute	f | f f f f	One Average or Four Float values	Bels	10Hz
Gamma Absolute	/muse/elements/gamma_absolute	f | f f f f	One Average or Four Float values	Bels	10Hz

In MindMonitor I have enabled all values instead of only average.

Here's the program mentioned about above:

Code: Select all

"""
Mind Monitor - OSC Receiver Audio Feedback
Coded: James Clutterbuck (2021)
Requires: python-osc, math, playsound, matplotlib, threading
"""
from pythonosc import dispatcher
from pythonosc import osc_server
import math
from playsound import playsound
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
import threading

#Network Variables
ip = "0.0.0.0"
port = 5000

#Muse Variables
hsi = [4,4,4,4]
hsi_string = ""
abs_waves = [-1,-1,-1,-1,-1]
rel_waves = [-1,-1,-1,-1,-1]

#Audio Variables
alpha_sound_threshold = 0.6
sound_file = "bell.mp3"

#Plot Array
plot_val_count = 200
plot_data = [[0],[0],[0],[0],[0]]

#Global variables
fname = "EEG/Right.Waves.csv"
linenr = 0


#Muse Data handlers
def hsi_handler(address: str,*args):
    global hsi, hsi_string
    hsi = args
    if ((args[0]+args[1]+args[2]+args[3])==4):
        hsi_string_new = "Muse Fit Good"
    else:
        hsi_string_new = "Muse Fit Bad on: "
        if args[0]!=1:
            hsi_string_new += "Left Ear. "
        if args[1]!=1:
            hsi_string_new += "Left Forehead. "
        if args[2]!=1:
            hsi_string_new += "Right Forehead. "
        if args[3]!=1:
            hsi_string_new += "Right Ear."        
    if hsi_string!=hsi_string_new:
        hsi_string = hsi_string_new
        print(hsi_string)    
          
def abs_handler(address: str,*args):
    global hsi, abs_waves, rel_waves
    global linenr

    wave = args[0][0]
    
    #If we have at least one good sensor
    if (hsi[0]==1 or hsi[1]==1 or hsi[2]==1 or hsi[3]==1):
        if (len(args)==2): #If OSC Stream Brainwaves = Average Onle
            abs_waves[wave] = args[1] #Single value for all sensors, already filtered for good data
        if (len(args)==5): #If OSC Stream Brainwaves = All Values
            sumVals=0
            countVals=0            
            for i in [0,1,2,3]:
                if hsi[i]==1: #Only use good sensors
                    countVals+=1
                    sumVals+=args[i+1]

            abs_waves[wave] = sumVals/countVals


#  WHEN USING THIS DATA IS SOMETIMES MISSING FROM A COLUMN
#  seems to lead to that rest will be out of order, at least for that line, perhaps more?
            # with open(fname, "a") as f:
            #     if wave == 0:
            #         f.write(str(abs_waves[wave]))
            #     if wave > 0 and wave < 4:
            #         f.write("," + str(abs_waves[wave]))
            #     elif wave == 4:
            #         f.write("," + str(abs_waves[wave]))
            #         f.write("\n")

            
        rel_waves[wave] = math.pow(10,abs_waves[wave]) / (math.pow(10,abs_waves[0]) + math.pow(10,abs_waves[1]) + math.pow(10,abs_waves[2]) + math.pow(10,abs_waves[3]) + math.pow(10,abs_waves[4]))
        update_plot_vars(wave)

        # DON'T WANT TO DISTURB MY ALREADY DISTURBED MIND WITH A SUDDEN SOUND :D
        # if (wave==2 and len(plot_data[0])>10): #Wait until we have at least 10 values to start testing
        #     test_alpha_relative()

#Audio test
def test_alpha_relative():
    alpha_relative = rel_waves[2]
    if (alpha_relative>alpha_sound_threshold):
        print ("BEEP! Alpha Relative: "+str(alpha_relative))
        playsound(sound_file)        
    
#Live plot
def update_plot_vars(wave):
    global plot_data, rel_waves, plot_val_count
    plot_data[wave].append(rel_waves[wave])
    plot_data[wave] = plot_data[wave][-plot_val_count:]

def plot_update(i):
    global plot_data
    global alpha_sound_threshold
    global linenr

    if len(plot_data[0])<10:
        return
    plt.cla()
    for wave in [0,1,2,3,4]:
        if (wave==0):
            colorStr = 'red'
            waveLabel = 'Delta'
        if (wave==1):
            colorStr = 'purple'
            waveLabel = 'Theta'
        if (wave==2):
            colorStr = 'blue'
            waveLabel = 'Alpha'
        if (wave==3):
            colorStr = 'green'
            waveLabel = 'Beta'
        if (wave==4):
            colorStr = 'orange'
            waveLabel = 'Gamma'

# DISABLED UPDATING THE PLOT TO PERHAPS SPEED UP THINGS A BIT 
#        plt.plot(range(len(plot_data[wave])), plot_data[wave], color=colorStr, label=waveLabel+" {:.4f}".format(plot_data[wave][len(plot_data[wave])-1]))        


# THIS SAVES DATA BUT "ONLY" ONE DATA POINT FROM EACH FREQUENCY BAND (ALPHA, BETA, etc.), and the save frequency seems low
        with open(fname, "a") as f:
            if wave == 0:
                f.write(str(plot_data[wave][len(plot_data[wave])-1]))
            if wave > 0 and wave < 4:
                f.write("," + str(plot_data[wave][len(plot_data[wave])-1]))
            elif wave == 4:
                f.write("," + str(plot_data[wave][len(plot_data[wave])-1]))
                f.write("\n")
                # f.write(str(linenr) + ",")
                linenr += 1


    plt.plot([0,len(plot_data[0])],[alpha_sound_threshold,alpha_sound_threshold],color='black', label='Alpha Sound Threshold',linestyle='dashed')
    plt.ylim([0,1])
    plt.xticks([])
    plt.title('Mind Monitor - Relative Waves')
    plt.legend(loc='upper left')
    
def init_plot():
    ani = FuncAnimation(plt.gcf(), plot_update, interval=100)
    plt.tight_layout()
    plt.show()
        
#Main
if __name__ == "__main__":

    with open(fname, "w") as f:
        f.write('Delta, Theta, Alpha, Beta, Gamma\n')


    #Tread for plot render - Note this generates a warning, but works fine
    thread = threading.Thread(target=init_plot)
    thread.daemon = True
    thread.start()
    
    #Init Muse Listeners    
    dispatcher = dispatcher.Dispatcher()
    dispatcher.map("/muse/elements/horseshoe", hsi_handler)
    
    dispatcher.map("/muse/elements/delta_absolute", abs_handler,0)
    dispatcher.map("/muse/elements/theta_absolute", abs_handler,1)
    dispatcher.map("/muse/elements/alpha_absolute", abs_handler,2)
    dispatcher.map("/muse/elements/beta_absolute", abs_handler,3)
    dispatcher.map("/muse/elements/gamma_absolute", abs_handler,4)

    server = osc_server.ThreadingOSCUDPServer((ip, port), dispatcher)
    print("Listening on UDP port "+str(port))
    server.serve_forever()

[James]

If you want to create a CSV file the "OSC Receiver.py" example does that already with RAW EEG and no extra stuff.

If you want to change it to reccord absolutes, just change the dispatcher from mapping "/muse/eeg" to "/muse/elements/alpha_absolute", one for each wave. You can make a different handler for each, or add an extra variable, like in the above example, so you know which wave you've passed.

The arguments array is enumerated (for arg in args), so you don't need to change anything to work for a single value rather than multiple.

Then just buffer the received values for each wave in global variables and write everything to CSV when you get Alpha, which with OSC absolute data will happen at 10Hz.

[banjo]

Thx! I have a much better understanding now.

Then just buffer the received values for each wave in global variables and write everything to CSV when you get Alpha, which with OSC absolute data will happen at 10Hz.

Interesting that Alpha would be the tail, when I stream (using the code in my topic above), and print the stream to the terminal, I consistently get them in this order once the streaming starts: 2,3,0,1,4 and if these corresponds to the below:

Code: Select all

    dispatcher.map("/muse/elements/delta_absolute", abs_handler,0)
    dispatcher.map("/muse/elements/theta_absolute", abs_handler,1)
    dispatcher.map("/muse/elements/alpha_absolute", abs_handler,2)
    dispatcher.map("/muse/elements/beta_absolute", abs_handler,3)
    dispatcher.map("/muse/elements/gamma_absolute", abs_handler,4)

the streaming order would be "Alpha, Beta, Delta, Theta, Gamma.
Kindly confirm which order is correct.

[James]

It doesn't matter which you trigger on, I just meant you should trigger the write on only one of the values to get a single line CSV with all values.

You should probably add an if statement to only write once each wave has a value, then if you start the python code after the streaming starts, you won't have an issue with a bunch of null values.

[banjo]

Aah, ok.
I just thought that every tenth of a second (10 Hz), I will receive the five waves which all have been measured at exactly the same time by Muse. But nevertheless, I guess it doesn't matter in the end.

[James]

They're sent from the Muse SDK async in order, but you can start recording mid stream so make sure you check for nulls for your first line.
You're right, you should probably trigger on gamma to keep the timestamps more correct.

[banjo]

They're sent from the Muse SDK async in order, but you can start recording mid stream so make sure you check for nulls for your first line.

Thx for the warning, will incorporate this. I initialized the global list with -1, so it should be easy to check. I though need to replace -1 with -100 or something, as -1 might be a value being streamed.

You're right, you should probably trigger on gamma to keep the timestamps more correct.

Ok, now I understand it even better

I'm now able to record what I need to a CSV-file, and I also increased my Python knowledge with 50 % as part of this excercise!

[James]

Python doesn't have nulls, so probably best to just have a separate boolean for alphaReceived, betaReceived etc and only write once you have them all.

[banjo]

Python doesn't have nulls, so probably best to just have a separate boolean for alphaReceived, betaReceived etc and only write once you have them all.

That'd be one solution, another would perhaps be using Python's sum function for summing up the numbers in the list. If the list numbers are initialized to e.g. -100, then if the sum of all items is less than e.g. -50, not all numbers have been received and nothing will be written.
The list could also be reset to -100 after each line write, then any hickups could be discarded.

[James]

That'd likey work 99% of the time because the values produced by the SDK will likely not exceed -2:+2, but that's bad programming because "likely will not" isn't 100%. You should always code expecting your input to be 99% junk and edge cases and still work.