Not sure about the folder question....but to delete an app fully you can go to settings-->applications-->manage applications-->select desired app to uninstall-->tap uninstall. You could also go into your market section and then go to downloads.Open the desired app to uninstall and tap "uninstall".
So I have the LG Ally, loving it. But, how do I delete apps I do not use? Can I create a folder to keep them in instead of deleting them? I'd like to have a folder of apps I do not use all the time just in case I decide to use them.
The method that "TheGreatOne" mentioned is probably the easiest, But I use an application called "aTrackDog(SD)" which will allow you to uninstll apps and store them on your MicroSD. If you ever want to reinstall them you can select them from the list of Apps in "aTrackDog" and "Reinstall from MicroSD" is one of the options... It also tracks applications and there current version so you can use it to update apps as well.
I love atrackdog.Plus I like how that app offers you to see the 100 most popular apps. Thers is also another app called Uninstaller by Rhythm Software. You can remove an app by a single click. Users seem to really really love it.