I think they were "Skirmishers" back in 5th Edition; it was 6th that introduced "Fast Cavalry" and started grouping them under that category of special units instead. I suspect that it had to do with game balance - Skirmish was a pretty potent rule for some types of troops, and cavalry could have put it to even stronger use.
That said, logically...yeah, it would make sense for such units to have the Skirmish rule.