Testing off-target

Another efficient way to speed up the development is limiting the interaction, as much as possible, with the actual target. This is of course not always possible, especially when developing device drivers that require to be tested on actual hardware, but tools and methodologies to partially test the software directly on the development machine exist.

Portions of code that are not CPU-specific can be compiled for the host machine architecture and run directly, as long as their surroundings are properly abstracted to simulate the real environment. The software to test can be as small as a single function, and in this case a unit test can be written specifically for the development architecture. Unit tests are in general small applications that verify the behavior of a single component by feeding them with well-known input, and verifying their output. Several tools are available on a Linux system to assist in writing unit tests. The check library provides an interface for defining unit tests by writing a few preprocessor macros. The result is small self-contained applications that can be run every time the code is changed, directly on the host machine. Those components of the system that the function under test depends on are abstracted using mocks. For example, the following code detects and discards a specific escape sequence, Esc + C, from the input from a serial line interface, reading from the serial line until the \0 character is returned:

int serial_parser(char *buffer, uint32_t max_len)
{
 int pos = 0;
 while (pos < max_len) {
 buffer[pos] = read_from_serial();
 if (buffer[pos] == (char)0)
 break;
 if (buffer[pos] == ESC) {
 buffer[++pos] = read_from_serial();
 if (buffer[pos] == ‘c’)
 pos = pos - 1;
 continue;
 pos++;
 }
 return pos;
}

A set of unit tests to verify this function using a check test suite may look like the following:

START_TEST(test_plain) {
 const char test0[] = "hello world!";
 char buffer[40];
 set_mock_buffer(test0);
 fail_if(serial_parser(buffer, 40) != strlen(test0));
 fail_if(strcmp(test0,buffer) != 0);
}
END_TEST

Each test case can be contained in its START_TEST()/END_TEST block, and provide a different initial configuration:

START_TEST(test_escape) {
 const char test0[] = "hello world!";
 const char test1[] = "hello \033cworld!";
 char buffer[40];
 set_mock_buffer(test1);
 fail_if(serial_parser(buffer, 40) != strlen(test0));
 fail_if(strcmp(test0,buffer) != 0);
}
END_TEST


START_TEST(test_other) {
 const char test2[] = "hello \033dworld!";
 char buffer[40];
 set_mock_buffer(test2);
 fail_if(serial_parser(buffer, 40) != strlen(test2));
 fail_if(strcmp(test2,buffer) != 0);
}
END_TEST

This first test_plain test ensures that a string with no escape characters is parsed correctly. The second test ensures that the escape sequence is skipped, and the third one verifies that a similar escape string is left untouched by the output buffer. Serial communication is simulated using a mock function that replaces the original serial_read functionality, which is provided by the driver when running the code on the target. This is a simple mock that feeds the parser with a constant buffer that can be reinitialized using the set_serial_buffer helper function. The mock code looks like this:

static int serial_pos = 0;
static char serial_buffer[40];


char read_from_serial(void) {
 return serial_buffer[serial_pos++];
}


void set_mock_buffer(const char *buf)
{
 serial_pos = 0;
 strncpy(serial_buffer, buf, 20);
}

Unit tests are very useful to improve the quality of the code, but of course achieving a high code coverage consumes a large amount of time and resources in the economy of the project. Functional tests can also be run directly in the development environment, by grouping functions into self-contained modules and implementing simulators that are slightly more complex than mocks for specific test cases. In the example of the serial parser, it would be possible to test the entire application logic on top of a different serial driver on the host machine, which is also able to simulate an entire conversation over the serial line, and interact with other components in the system, such as virtual terminals and other applications generating input sequences. While covering a larger portion of the code within a single test case, the complexity of the simulated environment increases, and so does the amount of work required to reproduce the surroundings of the embedded system on the host machine. Nevertheless, it is good practice, especially when they could be used as verification tools throughout the whole development cycle, and even integrated in the automated test process. Sometimes, implementing a simulator allows for a much more complete set of tests, or it might be the only viable option. Think, for example, about those embedded systems using a GPS receiver for positioning: testing the application logic with negative latitude values would be impossible while sitting in the northern hemisphere, so writing a simulator that imitates the data coming from such a receiver is the quickest way to verify that our final device will not stop working across the equator.