The obvious first question is "what's your data rate?" I2C is pretty slow: it was originally designed for passing control information between digital components in early TVs with digital chips. OTOH, I2C is very forgiving electrically: I2C components include input filtering so it doesn't matter if your SDA and SCL are ringing. This is nice if you're going between boards.
SPI can run much faster, but you have to ensure good signal integrity on your clock.
If you're running very slow, you might consider UART. Then you can debug each board separately using a terminal emulator on a PC. You need to get a USB to UART dongle with the correct voltage for UART signals for your board. Here's one from Adafruit http://www.adafruit.com/products/954.
You can also get dongles that talk I2C or SPI and talk to your boards individually from a PC, or monitor what's happening on the line. However, then you have to write I2C/SPI code on your PC which could be a steeper learning curve than using your embedded boards. Still, here are some cables from FTDI: http://www.ftdichip.com/Products/Cables/USBMPSSE.htm.
As an aside, while I was writing this column, my inventer friend Brian LaGrave dropped by my office with his two sons (Sam and Daniel) bearing gifts -- two large plastic coffee containers -- one loaded with cow manure and the other loaded with horse manure.