MakeStuff

For a while now I've been getting worried about the amount of boilerplate source code I'm copying around from one project to another, so yesterday I refactored all the general-interest code out of my current mini-project and wrote a cross-platform build infrastructure which can build and test each mini-library in isolation, and automatically integrate with github for fetching dependencies. Doing such a thing in a language like Java is very easy, but doing it in C is far harder.

Firstly the directory structure. Each project lives in a structure like this:

src
|
+-apps
| |
| +-nanduinoJtag
|
+-libs
| |
| +-argtypes
| +-buffer
| +-dump
| +-hexreader
| +-usbwrap
|
+-3rd
  |
  +-argtable2-12
  +-LUFA091223
  +-UnitTest++

The idea is to maintain a clear separation between 3rd-party ("3rd") code, my own library ("libs") code and my own application ("apps") code, and to automatically fetch any dependencies, downloading my own libs if necessary from github and building them, and downloading 3rd-party libs and tools from wherever and building them.

The build infrastructure itself is based on GNU make, and it supports Linux and Windows builds.

Windows Builds

The Windows platform introduces some unique challenges. Tooling on Windows is characterised by large-scale tools, usually relying heavily on IDE integration, rather than the more expressive Unix environment which is based on a large number of small-scale tools which cooperate with eachother. So to get off the ground on Windows you will need a couple of prerequisites:

• A functional GNU environment. I do a lot of stuff using Atmel AVR microcontrollers, so I have WinAVR installed, which comes with such an environment, but you will probably get away with a more minimal cygwin or mingw installation, so long as it has basic stuff like make, tar, mv, cp, wget, etc.
• An installation of Microsoft Visual C++ 2008 Express Edition which is available from Microsoft as a free download. The 2010 edition will not work because vcbuild.exe has been deprecated.

Once you have the prerequisites you can go ahead and build an application (this one really does need WinAVR since it contains AVR code)...

C:\temp>mkdir foo-src
C:\temp>cd foo-src
C:\temp\foo-src>mkdir apps
C:\temp\foo-src>cd apps
C:\temp\foo-src\apps>wget http://github.com/makestuff/nanduinoJtag/tarball/master
C:\temp\foo-src\apps>tar xvzf makestuff-nanduinoJtag-*.tar.gz
C:\temp\foo-src\apps>cd makestuff-nanduinoJtag-*
C:\temp\foo-src\apps\makestuff-nanduinoJtag-c9d3c67>make -f Makefile.win32
C:\temp\foo-src\apps\makestuff-nanduinoJtag-c9d3c67>cd ..\..
C:\temp\foo-src>find . -maxdepth 2 -type d
.
./3rd
./3rd/argtable2-12
./3rd/libusb-win32-device-bin-0.1.12.2
./3rd/LUFA091223
./3rd/UnitTest++
./3rd/unz600
./apps
./apps/makestuff-nanduinoJtag-c9d3c67
./libs
./libs/argtypes
./libs/buffer
./libs/dump
./libs/hexreader
./libs/usbwrap

After the build completes you will notice that the requisite dependencies for this particular project have been downloaded and built for you. Where possible I have used the vcbuild tool rather than invoking the compiler and linker directly, so you should be able to run the apps in the Visual Studio IDE without problems.

Linux Builds

Achieving the same thing on Linux is considerably simpler. The requisite tools are usually already installed, and if not can easily be installed using your favourite package manager.

> mkdir foo-src
> cd foo-src/
> mkdir apps
> cd apps/
> wget http://github.com/makestuff/nanduinoJtag/tarball/master
> tar xvzf makestuff-nanduinoJtag-*.tar.gz
> cd makestuff-nanduinoJtag-*
> make -f Makefile.linux
> cd ../..
> find . -maxdepth 2 -type d
.
./apps
./apps/makestuff-nanduinoJtag-c9d3c67
./libs
./libs/dump
./libs/usbwrap
./libs/hexreader
./libs/buffer
./libs/argtypes
./3rd
./3rd/UnitTest++
./3rd/argtable2-12
./3rd/LUFA091223

After the build completes you will notice that the requisite dependencies for this particular project have been downloaded and built for you.

"There will never be a really free and enlightened State until the State comes to recognize the individual as a higher and independent power, from which all its own power and authority are derived, and treats him accordingly."

- Henry David Thoreau

Recently I bought a machine from Mega Electronics for inserting tiny rivets into 0.7mm diameter drill-holes in home-made PCBs. The result is a fairly low-profile via, with decent electrical conductivity even without soldering. I'm hoping they will be sufficiently flush with the surface of the board to allow me to place vias under the 0.5mm lead-pitch TQFP packages used for Xilinx FPGAs:

The AVR microcontrollers have flash memory and SRAM. Code goes in flash and data goes in SRAM, right?

Well, not quite.

It's useful to add a call to avr-size to AVR makefiles. I realised today that I was missing it, which is a shame because it supplies some very useful information about your application, and how it uses the different types of memory on the micro. For example, you can run it from the directory where you built the Nanduino 20x4 LCD code:

H:\src\nanduino\lcd>avr-size --mcu=at90usb162 --format=avr .build/firmware.elf
AVR Memory Usage
----------------
Device: at90usb162

Program:    1604 bytes (9.8% Full)
(.text + .data + .bootloader)

Data:         62 bytes (12.1% Full)
(.data + .bss + .noinit)

The section names .text, .data, .bss and .noinit are described in detail here, but here's a quick summary:

• The .text section appears only in flash and represents your code plus the AVR-GCC init code
• The .data section appears in both flash and SRAM and represents data that is copied from flash into SRAM on RESET (e.g global variables with initialisers)
• The .bss section appears only in SRAM and represents SRAM locations which are cleared on RESET (e.g global variables without initialisers)

The actual work of copying to and initialising SRAM locations on RESET is done by the __do_clear_bss() and __do_copy_data() functions in the C runtime (i.e the code which executes after RESET but before main() is called).

Typically the cheaper AVR micros have plenty of flash but are short on SRAM. For example the AT90USB162 micro used in the Nanduino has 12kB of useable flash (16kB if you're not using the USB bootloader), but only 512 bytes of SRAM. It is therefore important to conserve SRAM. Unfortunately, SRAM is used implicitly in ways which are often far from obvious. For example:

#include "supergizmo.h"
#include "lcd.h"

#define STATE_IDLE 0
#define STATE_READING 1
#define STATE_WRITING 2

uint8 state = STATE_IDLE;

int main() {
    superGizmoInit();
    lcdInit();
    lcdPrintString("SuperGizmo v1.0\n");
    for ( ; ; ) {
        switch ( state ) {
        case STATE_IDLE:
            :
            break;
        case STATE_READING:
            :
            break;
        case STATE_WRITING:
            :
            break;
        }
    }
}

You would be forgiven for thinking this code uses only one byte of SRAM, for the state byte. Actually, the "SuperGizmo v1.0\n" string literal is written to the .data section, which means it gets written to flash, but is copied into SRAM after RESET, using 17 bytes (15 characters, a newline and a NUL terminator) of precious SRAM.

To get around this problem you need to explicitly state that literals should reside in flash, and should not be copied over into SRAM. So instead of:

lcdPrintString("SuperGizmo v1.0\n");

...we do:

#include <avr/pgmspace.h>
:
lcdPrintFlashString(PSTR("SuperGizmo v1.0\n"));

...which causes the string literal to be allocated in the .text section along with the application code, and referenced by a program memory pointer. This way, no precious SRAM is wasted on it. Unfortunately this .text allocation introduces two challenges.

• Firstly, since the AVR uses a Harvard architecture, a program memory pointer is quite different from a data memory pointer. The AVR has a special machine instruction LPM (Load Program Memory) for accessing data stored in program memory because the normal machine instructions for reading data memory do not work. As a result, it is necessary to write separate implementations of functions accepting strings, one which accepts data-memory pointers, and another which accepts program-memory pointers:

  #include <avr/pgmspace.h>
  
  void lcdPrintFlashString(const char *str) {
      char ch = pgm_read_byte(str);
      while ( ch ) {
      lcdPrintChar(ch);
          str++;
          ch = pgm_read_byte(str);
      }
  }
  
  void lcdPrintString(const char *str) {
      while ( *str ) {
          lcdPrintChar(*str);
          str++;
      }
  }

• Secondly, there is no optimisation of multiple identical PSTR("...") expressions. Each occurrence will result in the string literal being copied faithfully into its own unique location in flash memory. This can be avoided by declaring the literal once and referring to that declaration multiple times:

  #include <avr/pgmspace.h>
  :
  const char LONG_LINE[] PROGMEM = "This is a really really very long line!\n";
  lcdPrintFlashString(LONG_LINE);
  lcdPrintFlashString(LONG_LINE);

This technique of storing static data in flash memory is not only useful for string literals; it can be used for all sorts of static data. For example, you can write an efficient CRC32 routine that uses a precomputed table in program memory thus:

static const uint32 lookupTable[] PROGMEM = {
    0x00000000,
    0x04C11DB7,
    0x09823B6E,
    0x0D4326D9,
    :
};

uint32 crc32Calc(const uint8 *buf, uint16 len) {
    const uint8 *p;
    uint32 crc = 0xffffffff;
    for ( p = buf; len > 0; ++p, --len ) {
        crc = (crc << 8) ^ pgm_read_dword(&lookupTable[(crc >> 24) ^ *p]);
    }
    return ~crc;
}

Dean Camera (of LUFA fame) has written a very nice tutorial about this stuff here.

Yesterday was my birthday. Another year older. Juliet bought me a nifty little microscope. It will be very useful for spotting those soldering errors!

Those pins are on a 0.63mm pitch!