Muse Data Files

Muse generates a lot of data and it's helpful to store that data on disk for later analysis. All applications written by InteraXon for Muse use a data storage format called Protocol Buffers(PB).

Google Protocol Buffers

Protocol buffers are Google's open-source, language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but much smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages – Java, C++, or Python.

Protocol buffers store data in binary form, which means it is very compact. For example, a 5 minute session of data with 220Hz 4-channel EEG and 50Hz 3-channel accelerometer is about 1.5MB on disk. Protocol buffers make it easy to read this data from the programming language of your choice. We also have tools available (MusePlayer) which can convert the data into different formats, such as CSV and MAT.

Ordering of Data

When using a preset that enables compression, the data inside of raw files is only guaranteed to be ordered by type. In other words, all EEG are in order in relationship with each other, and all accelerometer data is in order in relation to each other, but EEG packets are not necessarily ordered in relation to accelerometer packet. EEG samples can arrive up to 72ms after an accelerometer packet with the same timestamp.

File Format

While Protocol Buffers specify the messages themselves, they do not specify how to separate messages from one another. Muse files separate each message with a size(4 byte integer) and a version(2 byte integer). So each message looks like this:

SIZE(4 bytes) VERSION(2 bytes) MESSAGE(SIZE bytes)

"Version" is version number. Currently there is only one version, version "1".

Integrating with Your Own Programs

Protocol Buffers make it easy to read and write Muse data from your own programs. It's easy because the code to parse and write the files is generated for you. To generate the code, download a copy of Protocol Buffers from here and install it. Then you only need to compile the Muse.proto file (attached) for the language of your choice. This will generate native parser code that you can use in your own applications.

C++

protoc --cpp_out=. Muse_v2.proto

Java

protoc --java_out=. Muse_v2.proto

Python

protoc --python-out=. Muse_v2.proto

Muse_v2.proto

(10k)

Paul Baranowski,

Dec 19, 2014, 4:02 PM

v.4

Comments