Although American football had more contact than Australian rules I found Australian rules tougher and harder than American football. The fitness level of most Australian footballers would easily put them in the top 1% of American footballers. In American football you can see your opponent 99% of the time but in Australian football with blind turns, twists and running backwards with your eyes on the ball its down to around 80%.

http://en.wikipedia.org/wiki/Issues_in_American_football#_note-2
Can anyone confirm this?

Also, the frequency of the forward pass means you commonly have players whose attention is not at all focused on the men sprinting toward them to drive them backwards or to the ground, and they also may be fully extending their bodies or even leaping, completely unable to brace themselves for impact. Perhaps this happens with long-range kicks in rugby or with marks in Aussie rules, but it’s not common enough for the powers-that-be to mandate padding and helmets.

Basically, it’s like Tony said: it has nothing to do with the relative toughness of the players, but with the differing dynamics of the games.