A 12 milk epd should be more than plenty to raise a good calf. An EPD of 12 can swing a few pounds depending on how much % reliability you're dealing with but it should stay above average.
A real basic explination of epd's is that they start out as an average of the two parents and then as new information about progeny comes along the EPD changes to show it. So, for example, if you mate a bull with a milk EPD of 10 to a cow with a milk EPD of 10, you'd get a calf with an EPD of ten and you could expect it to be fairly reliably a 10... However, if you mate a Bull with a milk EPD of 20 with a cow with an EPD of 0, your average is 10 but the reality could be anywhere from 0-20 and we won't know until we see how future generations perform.
In your example, the cows that struggle MAY have good milk EPD's on paper but the reality is that they're not living up to their average.