VTd Invalidations

[Anthony_B_Intel]

Would anybody have a pointer to some documentation about how VTd memory mappings are used in current VMMs? It seems most examples imply that a VTd domain is started as each guest is started and remains basically static for the life of the guest. Thus, I/O devices won't have to deal with frequent VTd invalidations since the domains themselves remain fairly persistent.

However, one can imagine a system that seeks to really limit devices to the absolute minimum access rights. Such a system would frequently be updating permissions and revoking them in response to application behavior. This would lead to frequent VTd invalidations.Of course, any VTd caching I/O devices were trying to do would be fairly useless as they would have to flush all the time.While such a system seems like it would be fairly inefficient, it's not beyond the realm of possibility.

Does anybody know of a VMM that does this or of some published documentation that says VMMs really don't (or shouldn't) do this?

[Hussam_Mousa__Intel_ · Accepted solution]

The architecture specifications document probably has the answer to your question:
VT-D Architecture Specifications

In general a Virtual Machine that has a PCI device directly attached to it (i.e. in passthrough mode, which requires the VT-D chipset support) has to have its memory reserved (this is the ESX terminology, I am sure Xen and other VMMs have similar terminology). I think this is their solution to the complication you identified.

There are some chipset structures that deal with MMIO invalidations and there arenewer structures added to support the VT-D feature (like an IOTLB cache). The document has a description of their functionality which might answer your question.

If this doesn't address your question, and you can't find what you are looking for in the specs document, can you rephrase your question more precisely and I will do what I can to get you an answer. An example would be helpful as well.