One other thing to try is to simply reboot the machine, then reserve the 32GB right away.
DPDK's memory manager works by gathering the hugepages into physically continuous
memory "segments", but it has a limited number of total segments that it will
create (256 by default). I don't know why the number of segments is limited - probably
because the list of segments is just held in an array. Either way, if you reboot and then
immediately reserve the pages (or change your boot options to reserve the pages
automatically), you're much more likely to get nice physically contiguous chunks so
you don't hit the max segments issue.
From: SPDK [mailto:
[email protected]] On Behalf Of Wodkowski, PawelX
Sent: Wednesday, August 2, 2017 5:44 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Just grep DPDK files for CONFIG_RTR_MAX_MEMSEG. You will find several places. Change all
of them.
From: SPDK [mailto:
[email protected]] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Wednesday, August 2, 2017 2:35 PM
To: Storage Performance Development Kit
<
[email protected]<mailto:
[email protected]>>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Thanks Pawel.
I am using an old version of DPDK and SPDK. I will try increasing the
CONFIG_RTR_MAX_MEMSEG.
Should we add it to the config file?
Thanking You,
Param.
On 02-Aug-2017, at 4:32 PM, Wodkowski, PawelX
<
[email protected]<mailto:
[email protected]>> wrote:
Yeah, the same message. CONFIG_RTE_MAX_MEMSEG is not big enough for 16GB. Try rebuilding
DPDK and SPDK with CONFIG_RTE_MAX_MEMSEG=512 or 1024. (Just grep for
CONFIG_RTE_MAX_MEMSEG).
BTW: which version of SPDK are you using? In current master or 17.07 error message from
SPDK should be something like this:
Starting DPDK 17.05.0 initialization...
[ DPDK EAL parameters: vhost -c 2 -m 16384 --master-lcore=1 --file-prefix=spdk_pid33057 ]
EAL: Detected 48 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Can only reserve 6375 pages from 9942 requested
Current CONFIG_RTE_MAX_MEMSEG=256 is not enough
Please either increase it or request less amount of memory.
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
Failed to initialize DPDK
Pawel
From: SPDK [mailto:
[email protected]] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Wednesday, August 2, 2017 11:53 AM
To: Storage Performance Development Kit
<
[email protected]<mailto:
[email protected]>>
Subject: Re: [SPDK] Failed to remap 2 MB pages
Hi Pawel,
This is the error.
Thanking You,
Param.
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: cs_dp -c 3 -n 4 -m 16384 --master-lcore=0 --file-prefix=rte0
--proc-type=auto ]
EAL: Detected 40 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: Failed to remap 2 MB pages
PANIC in rte_eal_init():
Cannot init memory
On 02-Aug-2017, at 1:47 PM, Wodkowski, PawelX
<
[email protected]<mailto:
[email protected]>> wrote:
Can you send full log when failure occur.
Pawel
-----Original Message-----
From: SPDK [mailto:
[email protected]] On Behalf Of
Kumaraparameshwaran Rathnavel
Sent: Tuesday, August 1, 2017 3:02 PM
To: Storage Performance Development Kit
<
[email protected]<mailto:
[email protected]>>
Subject: [SPDK] Failed to remap 2 MB pages
Hi All,
I have a system with 256GB. Initially I was giving 16GB for the SPDK to work.
Now I want to increase it to 32GB. I have given 16384 2MB hugepages. But I
am getting the error Failed to remap 2 MB pages. Before the
map_all_hugepages there is a call find_physaddrs and that passes. So can I
assume that physical memory is available and failure is due to some other
reason. Has anyone else observed it and aware of why this is happening.
Thanking You,
Param.
_______________________________________________
SPDK mailing list
[email protected]<mailto:
[email protected]>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
[email protected]<mailto:
[email protected]>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
[email protected]<mailto:
[email protected]>
https://lists.01.org/mailman/listinfo/spdk