..... just have the flank attack and outnumber for the CR.

That reminds me: according to PhillyLogic, this would also be against the intent of a challenge. How is it a heroic duel when all the guys in the flank and back ranks can still affect the challenge in a way that makes the lone challenger break and run? I'm not having a go at you, Philly. In fact, your impression of how a challenge is supposed to be reflected in rules is quite an enjoyable one. But if your argument of 'nothing can affect a challenger except the other character' would indeed be the intention of the rule, then a challenge should make the combatants ItP, unbreakable, alltogether immune to outside effects and so on. Nice concept, but it's not in the book. Maybe a bit too hero-hammerish, too.
EDIT: On closer inspection, this is hero-hammerish as hell, and should probably never be part of the rules other than house rules...Your flanked and outnumbered PM