CSV file format errors for markers

[debra peri]

Hello,

I'm encountering errors when reading Muse Monitor CSV files that contain "blink" and "jaw_clench" messages. The format changes, the number of columns are not consistent with the rest of the file. For example:

As per the header layout, the columns are:

# TimeStamp,
# Delta_TP9,Delta_AF7,Delta_AF8,Delta_TP10,
# Theta_TP9,Theta_AF7,Theta_AF8,Theta_TP10,
# Alpha_TP9,Alpha_AF7,Alpha_AF8,Alpha_TP10,
# Beta_TP9,Beta_AF7,Beta_AF8,Beta_TP10,
# Gamma_TP9,Gamma_AF7,Gamma_AF8,Gamma_TP10,
# RAW_TP9,RAW_AF7,RAW_AF8,RAW_TP10,AUX_RIGHT,
# Mellow,Concentration,
# Accelerometer_X,Accelerometer_Y,Accelerometer_Z,
# Gyro_X,Gyro_Y,Gyro_Z,
# HeadBandOn,
# HSI_TP9,HSI_AF7,HSI_AF8,HSI_TP10,
# Battery,
# Elements

However, a row that contains a "blink" or a "jaw_clench" message breaks this format:

# 2019-11-24 18:56:23.826,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,/muse/elements/blink

# 2019-11-24 18:56:23.826,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , , ,
# , ,
# , , ,
# , , ,
# ,
# , , , /muse/elements/blink

# 2019-11-24 18:27:14.523,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,/muse/elements/jaw_clench
# 2019-11-24 18:27:14.523,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , ,
# , , , , ,
# , ,
# , , ,
# , , ,
# ,
# , , , /muse/elements/jaw_clench


As opposed to a typical EEG message:
# 2019-11-24 18:10: 37.955,-0.056779124,-0.4869447,0.0513708,-0.11849284,-0.22336058,-0.6603433,-0.22185676,-0.3285773,-0.29817685,-0.4057947,0.10669098,0.11137576,-0.07063401,-0.44569674,0.4892052,0.107916296,-0.26754653,-0.82047045,0.3018993,-0.117138065,843.73627,847.3626,822.381,853.4066,786.52014,-0.190185546875,0.1046142578125,0.9686279296875,0.28411865234375,1.487884521484375,3.521575927734375,1,1.0,1.0,1.0,1.0,75.09

# 2019-11-24 18:10:37.955,
# -0.056779124, -0.4869447, 0.0513708, -0.11849284,
# -0.22336058 -0.6603433, -0.22185676, -0.3285773,
# -0.29817685, -0.4057947, 0.10669098, 0.11137576,
# -0.07063401, -0.44569674, 0.4892052, 0.107916296,
# -0.26754653, -0.82047045, 0.3018993, -0.117138065,
# 854.21246, 855.82416, 853.4066, 859.0476, 853.4066,
# -0.190185546875, 0.1046142578125, 0.9686279296875,
# 0.28411865234375, 1.487884521484375, 3.521575927734375
# ,1
# ,1.0,1.0,1.0,1.0,
# 5.09


Can you confirm this is what Muse Monitor is outputting and if so please update the app?


Thanks,
Debra Peri

[James]

You're right. That's a bug.
I need to add two extra commas when the Algorithms Mellow and Concentration are enabled.

[James]

Hi Debra,
I checked my code and I am already putting in the correct number of commas.
The only way I am able to re-create the situation you experienced was to have the algorithms enabled, start recording and then disable the the algorithms while recording.

Is this what you did? If not can you please let me know what version of Muse Monitor you are running and what platform, Android or Apple.

[debra peri]

Hi James,

I'm going to be double checking again today, I'm not entirely sure yet but what you describe may have happened.
I found that odd error in an older CSV file.

Muse Monitor is running on Android. I'll try to get to this tonight, very busy with holiday stuff. Thanks for the reply and advice!


--Debra

[debra peri]

Hi James,

I think that the few files I encountered this error were somehow corrupted by changing options while recording. It's the only thing that makes sense.

In the end it was a good exercise to go through, it helped me cleanup some code and to also consider ways of identifying the file before running analysis on the data. I don't seen an option on your app (running on Android) to select or disable the algorithms, so I have to assume it must of been something I fiddled with during the recording process.

Anyway, I think I have it sorted out now!

Just about ready to release a toolset to generate charts from the data. Yay for pandas, matplotlib and numpy!!!


Thanks again,
Debra

[James]

The algorithms are enabled/hidden by tapping on the app name at the bottom of the settings list 10 times.
They are hidden like this because I don't think they're very good! So, I would recommend leaving them off

[debra peri]

Oh! I must have stumbled upon the option accidentally! Another reason to get more sleep ....

I do encourage you to continue to develop algorithms in your app, there's so much fascinating ways to look at the data.


Take care,
Debra