Manipulating binary data files

Many of you will use the SD card to store the sound samples and image data needed by your E100 program. The best way to store this data on the SD card is as a binary data file. This web page describes how to create and manipulate such data files.

Binary data files

You will probably want to input sound and image data as 32-bit signed numbers. Files that store numbers directly like this are called binary files. You are probably more accustomed to plain-text files, which store ASCII representations of numbers.

To understand the difference between binary data and ASCII representations of numbers, consider two ways to write the number 12 to a file. You could write 12 to a file as two characters '1' and '2'. Each of these two characters would be written by its ASCII value. '1' is represented in ASCII by the number 49, and '2' is represented in ASCII by the number 50. To read this, your E100 program would have to read two numbers (49 and 50), translate each number from ASCII to a decimal digit, then compute the number that those two characters represented. This is unnecessarily cumbersome.

Instead of writing the number 12 as two characters (each represented by a number), you could simply write the number 12 as binary data to a binary file. Then your E100 program could read the single number 12 from the SD card directly into a variable.

You will not be able to view or edit binary files in a normal text editor. Text editors are meant to view plain-text files, so they interpret each number as an ASCII representation of a character. For example, if you used an editor to view a binary file that contained the number 12, your editor would try to display the character represented by ASCII value 12 (which is the line feed character and is unprintable). Instead, you will need to view the file with a program that is intended to handle binary data, such as hexdump or od. For example, you could view (in decimal) each 4-byte word of data.bin by running:

od -Ad -t d4 data.bin

You will probably need to manipulate binary data files, e.g. to combine sound samples or to add header information such as the number of samples. To manipulate binary data, you will probably need to write your own programs, e.g., in C++ or Matlab.

Writing binary data in C++

In C++, you can read and write binary data through the fstream interface's read and write methods. For example, the following program writes the 32-bit number 12 as binary data to a file.

#include <fstream>
#include <stdint>

using namespace std;

main()
{
    ofstream file;
    int32_t number = 12;

    file.open("datafile", ios::out | ios::binary);
    file.write( (char *) &number, sizeof(number) );
}

Writing binary data in Matlab

In MATLAB, you can use the fread and fwrite functions to read and write binary data (remember that this is 32-bit signed data, so use the 'int32' precision). For example, the following program writes the 32-bit number 12 as binary data to a file.

number = 12;
fid = fopen('datafile', 'w');
fwrite(fid, number, 'int32');
fclose(fid);

Combining files

The Unix utility cat can concatenate multiple files into a single file. E.g., to concatenate three files file1.bin, file2.bin, and file3.bin into a single, combined file all.bin, run the following command:

cat file1.bin file2.bin file3.bin > all.bin