In order to calculate those generations of EPD's, the association's require that breeders turn in data at least on those calves they're registering (some like an average on the entire peer group, as well). This data includes birth weights, weaning weights, yearling weights, whether or not assistance was required at birth, whether or not creep feed was used, maybe more stuff I'm not aware of in some breeds. Then the associations calculate EPD's using generations of this data. Most breeders who are trying to sell bulls do weigh their calves at the appropriate times and try to turn in accurate data. There will always be a few that just guess at it, or a dishonest one here and there that alter that data to try to artificially improve the results, so you have to keep in mind that the first word in EPD is estimate. For the most part they are a good indicator of birth weight and growth potential (and carcass potential and whatever else), but nutrition, environment and management are huge factors as well.
EPD's shouldn't be your only consideration in selection, but rather, as others said, a tool. IMO, use them to help you choose once you've considered phenotype. Many breeders who've tried using EPD's as their only criteria for selection find themselves in trouble with great numbered cattle that aren't efficient, or have structural issues, or don't have enough balance and look for their buyers.
Edit to add: When comparing cattle of different breeds, don't forget that EPD's are not equivalent from one breed to the next. You'd need a pretty good understanding of the EPD's of each of the breeds you're comparing in order to compare them fairly.