I agree, at weaning. I am not that concerned about the bull calves, its unlikely that they will be able to breed a heifer until they are 10 months old, it doesn't do them much good chasing after and fighting over a in season heifer, though.
Make sure your bull is out of the breeding pasture by the time your oldest heifers are 6 months old at the latest. I had the cleanup bull breed a 6 1/2 month old heifer the other day. Luckily I saw that.
I have lost a heifer in the past that got bred at 6 1/2 months that nobody saw. When she was supposed to be bred she was already springing and a caesarian was the only option.