We are facing the same problem as you are. Our land has has cattle on it and hay cut off of it for 30 years that I know of. In the past 10 years there has been a lot more taken from the land then put back. Our hay has drastically decreased and the land has gotten poorer. It is now up to us and our pocketbooks to improve our impoverished land. It is not going to get any better on it's own. Every year it will get worse. We had to realize that if we start by putting out lime on this pasture this year and fertilizer on that pasture this year then next year we can do another pasture so we can begin building up the soil a little at a time. It took a long time for it to get this bad and it will take a long time for us to be able to afford to get it where it need to be. We have begun spraying for weeds also, we rotate which pastures we spray each year, hopefully in years down the road we can fertilize, lime, and spray more pastures and we will get more hay and the cows will get better grass. Here's an example of what I am talking about with poor fertility: Our hayfield is +\- 40 acres. 10 years ago they were cutting @300 bales in 3 cuttings. The 1st year (3 yrs ago) that my husband and I started managing the farm we got 87 bales total from the field! That's it! This past year we are up to 140 bales, a far cry from what it used to be, but we have started taking soil samples and fertilizing per the samples and have put out an 18 wheeler load of lime last year (sorry, I can't remember how much it was per acre though). I am determined to make this place better, no matter how little we can afford to do each year, we will at least do a little each year.
Sorry my post was so long