It can be fairly subjective, but much of it can be measured. The IPC 610 standard describes things like fillet height, distance for fillet to clipped lean, part position on pad and such.
You could also judge to differnet levels of expertice so the experience folk wouldn't always runaway with the prises. The same standard has different levels: Class I (the minimum acceptable), Class II (appropriate for consumer goods), Class III (military & hi-reliability).
@Max:"It would be great is we had a tri-colored LED (or maybe a number of individual LEDs) and we could arrange it so that bad solder joints affected the LEDs in some way"
Practical Components: http://www.practicalcomponents.com/ offers some soldering training kits. Some of their kits come with a PC board and a special quad flatpack "dummy" chip that has some pins internally connected to each other. The idea is that you solder it to the board and then check the continuity between two pads on the PCB. The circuit zigzags in and out of the chip package through all of the pins, so if you've got continuity, you've either got all the solder joints right (or at least conducting), or you've got a solder bridge.
All this talk about soldering reminds me of Day 2 at college. The instructor passed around squares of plywood filled with nails along the edges. We had to create a criss-crossed array of bare wires using the nails as anchors and solder them at every junction.
Many folks learned soldering through building of kits, Heathkit included a soldering how-to manual with every kit. Funny how the worst solderers were TV repairmen, followed by ham radio operators. Most first-timers followed the tips and did a decent job.
Max - "....if we were to hold a soldering competition at EE Live! 2014,"
If I make it I'd give it a go. Time us Aussies took something else off you guys besides the Americas Cup :-)
" I recently constructed a 4x4x4 tri-colored 3D LED cube kit that required 400-plus soldered joints... needless to say it worked the first time I powered it up (well, actually, if truth be told, this was the first project I've ever built that worked the first time, but let's not rain on my parade as it were)."
Rats! I was looking forward to you sending it to me for final commissioning..... :-)