On Thu, Jan 05, 2017 at 03:19:36PM -0500, Jerome Glisse wrote:
> Always having a VMA changes the discussion - the question is how
> create a VMA that reprensents IO device memory, and how do DMA
> consumers extract the correct information from that VMA to pass to the
> kernel DMA API so it can setup peer-peer DMA.
Well my point is that it can't be. In HMM case inside a single VMA
In the GPUDirect case the idea is that you have a specific device
that you map for peer to peer.
I still don't understand what you driving at - you've said in both
cases a user VMA exists.
From my perspective in RDMA, all I want is a core kernel flow to
convert a '__user *' into a scatter list of DMA addresses, that works no
matter what is backing that VMA, be it HMM, a 'hidden' GPU object, or
struct page memory.
A '__user *' pointer is the only way to setup a RDMA MR, and I see no
reason to have another API at this time.
The details of how to translate to a scatter list are a MM subject,
and the MM folks need to get
I just don't care if that routine works at a page level, or a whole
VMA level, or some combination of both, that is up to the MM team to
figure out :)
a page level. Expectation here is that the GPU userspace expose a
API to allow RDMA to directly happen on GPU object allocated through
GPU specific API (ie it is not regular memory and it is not accessible
So, how do you identify these GPU objects? How do you expect RDMA
convert them to scatter lists? How will ODP work?
> We have MMU notifiers to handle this today in RDMA. Async RDMA
> Invalidate like you see in the above out of tree patches is totally
> crazy and shouldn't be in mainline. Use ODP capable RDMA hardware.
Well there is still a large base of hardware that do not have such
feature and some people would like to be able to keep using those.
Hopefully someone will figure out how to do that without the crazy
async MR invalidation.