the size of the mini is fine - what's probably the biggest killer from my pov is the video chipset. Something with a bit more grunt than the current intel integrated chipset would do wonders.
But yes I know about the limits of the motherboard integrated graphics but AMD/Ati and nVidia have managed to have chipsets that performed better than the Intel crap.