Amongst the other silly games played in the MakeStuff household this Christmas was a game where you each think of a six-letter word and the other person tries to guess it. For each wrong guess you are told how many letters are the same, and in the same place. For example if your word is "carrot" then my guess "carton" scores 4: one point for each of 'c', 'a', 'r' and 'o', but no points for the 't' because it's in a different position. The first to guess the other's word wins. I was pretty useless at it, so I consoled myself by writing it as a computer game:

SixLetters.tar.bz2

With source, built for Windows and Linux. The dictionary is a bit crap, but it was the best I could find. No prizes for finding missing words, garbage words or buffer overrun vulnerabilities.

After spending several hours trying (and failing) to reverse-engineer the USB protocol used to upload designs to the Digilent Nexys2 FPGA devkit, I gave up and resorted to other means. Andy Ross posted a nice perl script for automating the process of programming a Nexys2 from Linux using only open-source tools. Taking inspiration from that I turned my own FX2FPGA board to a makeshift USB-JTAG dongle for programming my other FPGA devkit, the Digilent S3BOARD.

The board connections are as follows (pin numbers for the 56-pin SSOP part):

On the software side I wrote a script to document the process:

s3boardprog.sh

The script has some prerequisites in addition to the Xilinx ISE WebPACK toolchain:

sudo apt-get install sdcc
sudo apt-get install g++
sudo apt-get install libusb-dev
sudo apt-get install urjtag

The script performs six steps:

• Fetch the ixo-jtag firmware source and build it
• Fetch the fx2loader source and build it
• Fetch the S3BOARD demo
• Use the Xilinx iMPACT tool in batch mode to convert the demo .bit file into an .svf file
• Use fx2loader to load the ixo-jtag firmware into the FX2 chip
• Use UrJTAG to play the .svf file via the FX2 chip to the FPGA on the S3BOARD

The usbtest.ko kernel module is installed with the stock kernels on Ubuntu 10.04. Unfortunately it has the nasty habit of hijacking any unrecognised USB device. After a while you get bored with typing sudo rmmod usbtest.

To disable it permanently just create or edit /etc/modprobe.d/blacklist-custom, adding "blacklist usbtest". Then reload udev with sudo service udev restart. On Ubuntu 10.10, reloading udev appears insufficient: it's necessary to reboot.

Recently I had an idea - reverse-engineer the protocol used by the Digilent Nexys2 FPGA development board. That would mean a portable, open-source libusb-based driver library could be developed which could program the FPGA over JTAG and interact with it using Digilent's Enhanced Parallel Port reference design, all using the built-in firmware. And since the donbusb.iic firmware is available as a free download, it would be possible to use it in open-source hardware projects (i.e on non-Digilent hardware). Of course, this post refers to the Nexys2, but these techniques could be used to reverse-engineer any USB device with closed-source Windows drivers.

So I tried a few Windows-based software USB sniffer tools, and eventually settled on the excellent USBTrace. Unfortunately it's only available on a 15-day eval period and is pretty expensive to buy ($195 for a single-user licence), but it's an awesome tool, allowing you to get detailed XML dumps of all the comms traffic on the bus to and from a specific device.

So I set about writing a parser that could read one of these XML dumps, and convert it into an easily-readable (and parseable!) format, and then started writing a program that would replay the same messages using libusb (actually I got bored before adding the libusb code, but it's 95% there).

USBTrace Dump Parser

So you can do:

$ mkdir -p src/apps
$ cd src/apps
$ wget --no-check-certificate https://github.com/makestuff/usbreplay/tarball/master
$ gunzip -c master | tar xf -
$ cd makestuff-usbreplay-*
$ make -f Makefile.linux 
$ ./parse.py /mnt/space/py/read109D55E5from03.xml > out.txt
$ cat out.txt 
R0: C0 E9 0000 0000 -> 05 00 10 00
R0: C0 E6 0000 0000 -> 03 03
R0: C0 E7 0000 0000 -> 0D 00 00 00 00 00 00 00
R0: C0 E9 0000 0000 -> 05 00 10 00
W1: 07 00 03 00 B9 6A 14 01
R1: 05 00 C1 95 EB FE
R0: C0 E9 0000 0000 -> 05 00 10 00
R0: C0 E6 0000 0000 -> 03 03
R0: C0 E7 0000 0000 -> 0D 00 00 00 00 00 00 00
R0: C0 E7 0000 0000 -> 0D 00 00 00 00 00 00 00
W1: 03 04 00 00
R1: 01 00
W1: 08 04 05 00 03 04 00 00 00
R1: 01 00
R6: 10 9D 55 E5
W1: 03 04 85 00
R1: 05 40 04 00 00 00
W1: 03 04 01 00
R1: 01 00

The read109D55E5from03.xml dump was me doing a four-byte read of register 0x03 using the Digilent Adept tool. So what does the out.txt data mean? Initially it's pretty indecipherable, but luckily you can compare the dumps you get for different operations (e.g compare writing 0xAA to register 0x02 with writing 0x55, 0x77 to register 0x01), and gradually piece together the important bits of the protocol. Let's look at it piece by piece.

First four lines

R0: C0 E9 0000 0000 -> 05 00 10 00
R0: C0 E6 0000 0000 -> 03 03
R0: C0 E7 0000 0000 -> 0D 00 00 00 00 00 00 00
R0: C0 E9 0000 0000 -> 05 00 10 00

This is always the same. The host does four control-endpoint reads, using bRequest values 0xE9, 0xE6, 0xE7 and 0xE9 again. My guess is that it's some sort of protocol negotiation to allow the Digilent Adept tool to support multiple firmware versions, but it's not important because it can just be replayed.

Next two lines

W1: 07 00 03 00 B9 6A 14 01
R1: 05 00 C1 95 EB FE

The host writes eight bytes to the first endpoint, and then reads back six bytes. The bytes written by the Digilent Adept tool are always different, but for a given set of bytes the firmware always replies with the the same response bytes. Since it can be replayed, it's not particularly interesting.

Next six lines

R0: C0 E9 0000 0000 -> 05 00 10 00
R0: C0 E6 0000 0000 -> 03 03
R0: C0 E7 0000 0000 -> 0D 00 00 00 00 00 00 00
R0: C0 E7 0000 0000 -> 0D 00 00 00 00 00 00 00
W1: 03 04 00 00
R1: 01 00

Again, some control-endpoint reads, then some writes and reads on the first endpoint. Always the same. Can be replayed. Let's move on.

Next two lines

W1: 08 04 05 00 03 04 00 00 00
R1: 01 00

Here's where it gets interesting. The write to the first endpoint is a command for manipulating the EPP registers which are hopefully implemented by your FPGA design - "08 04 cc 00 rr nn nn nn nn", where:

• the c is 04 for register writes and 05 for register reads,
• the r is the register you want to operate on, and
• the n is the number of times the operation should be done (least significant byte first).

So this is a read of four bytes from register 0x03. The response "00 01" is always the same.

Next line

R6: 10 9D 55 E5

The host reads four bytes from endpoint 6. If this operation was four writes to a register instead of four reads, the host would be writing to endpoint 2.

Next two lines

W1: 03 04 85 00
R1: 05 40 04 00 00 00

The host writes a message meaning "how many bytes were read?" (if it was "03 04 84 00" it would be "how many bytes were written?"). The firmware responds with "05 40 nn nn nn nn" where n is the number of bytes read (least significant byte first). If it was a write, the response would be "05 80 nn nn nn nn".

Last two lines

W1: 03 04 01 00
R1: 01 00

Always the same. Can be replayed.

Next thing I wrote was a program to parse the out.txt file, the idea being to execute each read or write using libusb, and for reads verify that the data actually read is the same as the expected data (i.e according to the dump file). As it happens I stopped short of the integration with libusb, but I did prepare the parameters and the buffers ready for it. I might return to it at a later date if there's interest. Currently it just prints out the message data in a slightly different format:

$ ./replay out.txt 
C: 0xC0 0xE9 0x0000 0x0000 read 4 bytes: 05 00 10 00
C: 0xC0 0xE6 0x0000 0x0000 read 2 bytes: 03 03
C: 0xC0 0xE7 0x0000 0x0000 read 8 bytes: 0D 00 00 00 00 00 00 00
C: 0xC0 0xE9 0x0000 0x0000 read 4 bytes: 05 00 10 00
EP1 wrote 8 bytes: 07 00 03 00 B9 6A 14 01
EP1 read 6 bytes: 05 00 C1 95 EB FE
C: 0xC0 0xE9 0x0000 0x0000 read 4 bytes: 05 00 10 00
C: 0xC0 0xE6 0x0000 0x0000 read 2 bytes: 03 03
C: 0xC0 0xE7 0x0000 0x0000 read 8 bytes: 0D 00 00 00 00 00 00 00
C: 0xC0 0xE7 0x0000 0x0000 read 8 bytes: 0D 00 00 00 00 00 00 00
EP1 wrote 4 bytes: 03 04 00 00
EP1 read 2 bytes: 01 00
EP1 wrote 9 bytes: 08 04 05 00 03 04 00 00 00
EP1 read 2 bytes: 01 00
EP6 read 4 bytes: 10 9D 55 E5
EP1 wrote 4 bytes: 03 04 85 00
EP1 read 6 bytes: 05 40 04 00 00 00
EP1 wrote 4 bytes: 03 04 01 00
EP1 read 2 bytes: 01 00

So far so good with the EPP operations, but the really interesting thing would be to be able to program the FPGA. Unfortunately I have not been able to decipher the protocol used during the JTAG scan chain operation and the FPGA programming operation.

Recently I updated Fx2tools. Now you can:

• Upload SDCC-generated HEX records directly to an FX2LP chip's RAM and EEPROM
• Backup & restore EEPROM (e.g firmware for Digilent's FX2-based FPGA development products)
• Send arbitrary commands to the chip's control endpoint
• Benchmark bulk writes to the FX2LP chip

I have successfully tested it with

• My home-made FX2 board
• The Digilent Nexys2 FPGA devkit