specvirt + my hypervisor is better than yours

I've seen a number of presentations recently where companies are putting up SPECvirt data and drawing some interesting conclusions.   I am not a SPECvirt expert but it seems to me that they're not comparing / contrasting the data properly.    Here's an example.

Note:  There is absolutely nothing wrong with this image.  The interpretation by some people is what appears to be wrong.  I've heard a number of speakers (who are *not* associated with redhat by the way) implying that the data shows KVM being substantially better than vmware esx.

Note:  the above is just an example of the type of metrics i've seen, this is not the actual chart in question - it was just one I found to be handy  :-)

So what am I saying here?  From what I understand the above chart is simply a chart of specvirt numbers.  It cannot be used as a comparison between esx and kvm as the underlying hardware is different.

If we look at the specvirt results themselves then there are only a couple of comparisons between KVM and ESX on the same hardware.   Look for
Hewlett Packard Company ProLiant DL580 G7

RHEL 6.1 (KVM) 3802@234
VMWARE ESX 4.1 3723@228

These numbers are pretty close and I haven't seen enough data to understand if the difference between the two is in the noise for the benchmark

I think we can safely say that the performance is probably the same, maybe an edge to KVM.   I think we're likely to see bigger differences within platforms when say for example in KVM you use vhost_net drivers, remove non-required h/w from the guests, huge pages, pci pass-through etc etc to optimize the platform performance - that could easily lead to a 10% performance change.

SPECvirt is really good and is a fantastic tool for determining relative performance impacts of tuning actions within a h/w platform.   Please don't abuse the numbers.

If i'm wrong, please tell me and hopefully tell me why  :-)