With 40 cows, you might consider AI? Maybe not all 40, but try a few with Black Simm and/or black Simbrah? This will give you some heteratosis crossed with your Brangus, angus plus, Brangus baldy, I think you said you had an Angus bull now? You can still use him on the f1 Brafords,( one of the best terminal crosses you can produce) and for cleanup on the AI'ed cows. Angus x F1 Hereford is great terminal cross. If you retain heifers from this, you would be correct to think breeding them back to Angus would dilute hybrid vigor. The Simm or Simbrah AI bulls would be ideal for those too. You mentioned that the Brafords will be out eating grass at 3PM while the black ones are in the ponds or under the trees? So? Maybe the black ones are out eating grass at 3AM!!

Nothing wrong with taking a noon day siesta, then going to work later in the afternoon when it cools off some. They can eat/breed., etc, from sunset, all night long til late morning the next day! Whatever you chose, get the best registered bulls you can afford, and stay way from commercial bulls. and for sure, don't be fooled into using a cross bred bull, even if the parents were registered cattle of the 2 different breeds.
That's another plus for AI: The money you'd spend for a good registered bull, will buy a lot of semen from GREAT bulls, and even pay someone to AI them for you.
I think you have a good plan and are obviously putting a lot of time into researching. This is just some options I threw out for you to consider.