Trail horses are cheap right now, or so I hear. I would sell him and get one I could handle. If you don't have the skills to handle him, you would be much better off without him.
With that said, I wouldn't feed him on the trailer either. But, if you are inclined to try that method, then that's the only place he'd get a bite or drink of anything, until he started loading regularly and without much asking. Try tying him out like Bigfoot mentioned. I would take a Saturday for example, catch him and tie him to something stout, with lead rope higher than his withers, with a slip knot. He does lead doesn't he? Does he suck back when he's had his fill of being tied too? I'd leave him tied to teach him some patience first. Sun and patience will take it out of them, eventually. Try him at noon, if not leave him. Try him at dark, if not leave him. Try him at daylight, if not... He needs to come back to you, look to you for direction, then follow instructions. He's not doing that, he is dictating to you what he is gonna do or not do. There are a couple of ways to remedy that, and quitting him when he is doing it is not the way to fix it. So, make up your mind that he is going in to that trailer, and stay hooked until he does, by whatever method you choose to use. I like the pressure/release method over the baiting method myself. It will last much longer, and it's better for you both. But, it takes some experience in most cases for a couple of reasons. One is so you can position your body in relation to his to keep from getting hurt, and Two is to be able to use your position to put pressure on him without being in front of him, eyeballing him, and getting your ears pawed off. If you are scared of him, no shame, just get someone to help. If you are scared of him, it will show, and he knows, and he will try you. You don't have to be mean, most horses are trying to do right, but there are some that need way more "pressure" than others. fwiw