When I was a young lad I had the same problem. I have since outgrown it and with the event of cabs and air conditioning it is no longer a problem.
As a young fellow I used wear a mask and goggles when I cut, raked or baled. We used to put the darned small squares in the barn, so I wore it when stacking as well.
Worked for me then - but you will need the entire combo - what you breathe in will help irritate from the inside of your sinuses.
There is a new type of hard hat out with a full face shield. It is attached to a small filter system that hangs from your belt. The fan in the filter system blows the clean air up a tube and into the top of your hat - creating a cooling effect and blowing filtered air down over your face and out the bottom of the shield - preventing dust and pollen from entering.
It runs a couple of hundred bucks - but for those with pollen and dust allergies it really appears to help when working in poor conditions. Made for grain workers and poultry barns and so on. probably work for you as well.
All in all a light and comfortable wear - tried one on at a farm show last summer. You might hunt one of them down.
Take care,
Bez