I was too young to afford a PC in the paper tape, punch card, or cassette days, but I can remember what a pain in the butt trying to use my grampa's trash-80 cassette.
Of course, real old school PC is the MITS Altair -- got to have all those lights and big switches!
But for crazy, its hard to beat my brother, who designed and built a 8080 system, hand translated assembly code into binary, and programmed it to EPROM using DIP switches. Since he's a pack rate, he probably still has that system somewhere! For some reason, it didn't get much use...
OK, I can beat you all. My first computer, a Sinclair ZX81 (used to be called Timex in the states) had just 1K of RAM. I later got a Sinclair Spectrum with 16K. Programs were stored on audio tape cassettes (using FSK). You used a TV as the display. They used BASIC and it's amazing what you could do with that small memory. You could also write and run Z80 machine code programs. They were lots of fun. Data (and programs) expand to fill the space available they say, that's certainly true today as Janine says.
I still have a ST225 20M hard drive kicking around some where, along with the even bigger 80M version. DiskCon (or what's left of it) typically has a historical exhibition of HDDs, including the original IBM drive, and some even bigger (physically) ones made by competitors.
My 1986 Intel databook has lots of great stuff like bubble memory, the iAPX432, and intelligent text display controllers. And although I haven't seen core memory, I know about from my IBM 1620 manual (the 1620 worked in BCD).
Supposedly some military crypto systems used cone memory, but a quick search doesn't turn up anything on that. Of course, on the analog side, the Navy used to use some great technologies like 20 psi proportional air control systems and mag amps (magnetic amplifiers).
I think one of the things that could be a downside to the abundance of available memory is the incredibly bloated code that we now use. I just downloade an update for my IM software, seriously, 51MB for an update?!! (BTW, my first computer had no memory, just two disk drives, one for booting and one for storing)