You get a spikey knuckle thing for punching (I have never punched anything ever), double barrel shot gun which I shot two times before getting the combat shotgun which is much better, a special dune buggy (I haven't seen it yet), and the Crimson Elite armor which is awesome. But no, you aren't missing anything important.
Apparently, the issue with Rage performance is that it is heavily dependant on raw processor power. Unlike most PC games which rely upon super graphic cards, the people having the least issues with the game are the high end i7 Quad Core users. Luckily I bought an i7 Quad Core and am running at maximum res and 60+ fps. There is some tearing which is an ATI issue.