Well, being one that's not too terribly versed in the field of software and hardware but loves messing around with as many things as possible... I figured I would create a thread for those like me. Everything listed should be free of charge, if not a great demo version.
Tunafish Website
Tunafish is an extremely simple VSTi sequencer. It's interface is extremely user friendly, point, click, make a beat. What comes with it is great, but download other VSTIs and the possibilities are endless. I recommend getting the Minimoog one... probably one of the best when it comes to experimenting with this.
AP Tuner website
AP Tuner is a straight to the point instrument tuner. Place your guitar next to the computer mic (or mic the amp) and you're good to go. Easy and efficient for when you misplaced your guitar tuner.
Audacity website
Audacity is an extremely easy to use sound recorder, I haven't used it too terribly much yet but I've had a lot of success playing with it so far. Record, mix, edit... a studio for anyone messing around. Open source.
Guitar FX Box website
FXBox is a really kick ass program that essentially turns your computer into a guitar processor. Granted, there may be a bit lag if you're running a not so great computer, but otherwise I've noticed little to none. To plug your electric guitar into the computer you will need a 1/4th to 1/8th jack, but you can find these at any regular Radioshack or guitar store. Loads of things to do, but I forget what the non registered version limits you to.
I'll add to the list when I find things, of course feel free to do so as well. Have fun!