I'm in a laughing fit over this thread...I really am.
If you're thinking about getting a horse and don't have experience, here's what you do. Take 10 $100 bills and flush them down the toilet, if you like that experience, get a horse. If you enjoyed that, now spend 2 hours every day for at least 6 months trying to teach a 5 year old to dance with perfect steps to 8 different dance steps. If you accomplish that, flush another 10 $100 bills down the toilet. When you're not sure where all your money is going, don't worry about it, the vet will be coming by soon to tell you that you need to get a farrier and the horses teeth need to be floated, goodbye another $500. If you get this far, let me know I'll give you some more into on costs. You know, things like wormer, getting a saddle that actually fits and then you can add in all the stupid stuff you don't need but other morons talk you into. In fact, screw it, call me after you've burned up $10k and ridden probably less that 10 miles, then we'll discuss what a horse actually costs.
Want a horse and never owned one, find a 12 year old or older horse that actually know something, since you don't, and ride that around. Buy all the tack they have also, if you're shrew enough they might throw it in for a couple bucks if you remind them that it's probably just taking up space. There are plenty of them out there.
You're welcome to choose either option, just remember they both have the same actual annual cost, one just has a much smaller learning curve and mental well being cost.