Not concrete on this, But I would believe its origin lies in "Old Wive's Tales", that generally revolve around what was observed by being near or in hay and gaining a fever afterwards. Hay Fever is...
Hay fever is an occurance that occurs year round when we become allergic to certain things that aren't always blooming, its natural to get sick around the spring and autum because that's the time hay...
Hay fever, like all allergic reactions, is caused by allergens, foreign "invaders" that enter your body by inhalation, by swallowing, or through your skin.