AFAIK, the HPU is only for calculating sensor inputs to semantic events, not for rendering out stuff (there is an iGPU for it). I could guess newer architectures could cope with such inputs. Microsoft said something about then having the CPU free for calculations of the actual application.
If not, I guess the HPU could perfectly fit in the device/glass itself and only "submit" the semantic events to the host pc. As one would need additional hardware (glasses, cameras, infrared, etc) the HPU would fit it perfectly well