Lucky_P is a good one to give you advise. Listen. A calf starter is usually 14-16% protein and some are up to 20%. They need the protein for the bones, ligaments, tendons and all to properly grow.
Some people lump all sweet feed together. Look at the protein %, and see if it is designed for calves. An "all purpose" sweet feed is not enough in the nutrient department. Fine for older more mature animals as a treat but for a growing animal they need what they need. Get a bag of calf starter, and compare the labels. You don't have to get the highest priced one there is, but get a good decent brand. You can mix in the sweet feed you have a little at a time to use it up, and to get them to get used to the new feed. They just need more nutrition when young and growing. You can go to a calf grower after the starter is gone. Giving them a bottle twice a day is one reason why they are looking good and healthy. They are still getting most of their nutrients from the milk replacer.