It's probably not an OS X Server thing: rather, it seems to be simply the lack of hardware Quartz Extreme support inside the VM.
I have read some forum discussions about this on the web, and it looks like it's an extremely difficult thing to achieve, also due to the fact that Apple doesn't sufficiently document the whole thing.
So, the best we can hope for is that the VMsvga2 developer (or VMware themselves: even better) will eventually produce an updated version of the necessary drivers, with Quartz Extreme support; in the meantime, sadly, there is essentially nothing one can do about this...
