Hi,

 

The significant part of the first error message below is this:

 

> EAL: Can only reserve 259 pages from 1024 requested

> Current CONFIG_RTE_MAX_MEMSEG=256 is not enough

> Please either increase it or request less amount of memory.

> EAL: FATAL: Cannot init memory

 

This is saying that DPDK was unable to allocate hugepages to meet your requested memory size in the maximum number of segments (256) configured at DPDK compile time.  You can try rebooting or dropping caches (echo 3 > /proc/sys/vm/drop_caches) to try to defragment the available memory so DPDK will be able to find larger contiguous memory regions and thus require fewer total memory segments.  You can also try reducing the size of memory requested by your app – by default, DPDK will attempt to allocate all available hugepages.

 

Thanks,

-- Daniel

 

From: SPDK [mailto:[email protected]] On Behalf Of PR PR
Sent: Tuesday, April 24, 2018 4:51 PM
To: [email protected]
Subject: [SPDK] spdk in docker container

 

Hi, I am trying to run one of the spdk example (reactor_perf) in docker container. Our build system uses bazel so after compiling spdk natively, I have created a bazel target for linking with spdk (I have set the alwayslink option to 1 - https://github.com/spdk/spdk/issues/262). The container is run with privileged option on docker cmdline and has /dev bind mounted from host. Before running example app, I do run the scripts/setup.sh which succeeds. The example app (reactor_perf) is copied to new workspace location and compiled as a bazel target. For some reason, it fails to allocate huge pages. Some relevant outputs below. Appreciate any pointers.

 

Thanks,

PR 

 

========

After running scripts/setup.sh

========

 

cat /proc/sys/vm/nr_hugepages

1024

cat /sys/devices/system/node/node*/meminfo | fgrep Huge

Node 0 AnonHugePages:   8429568 kB

Node 0 HugePages_Total:   640

Node 0 HugePages_Free:    512

Node 0 HugePages_Surp:      0

Node 1 AnonHugePages:   3663872 kB

Node 1 HugePages_Total:   384

Node 1 HugePages_Free:    384

Node 1 HugePages_Surp:      0

 

=======

Run example app without mem_size option

=======

/ws/bazel-bin/spdk/spdk_test -t 10

Starting DPDK 17.11.0 initialization...

[ DPDK EAL parameters: reactor_perf -c 0x1 --no-pci --file-prefix=spdk_pid28875 ]

EAL: Detected 72 lcore(s)

EAL: No free hugepages reported in hugepages-1048576kB

EAL: Probing VFIO support...

EAL: Can only reserve 259 pages from 1024 requested

Current CONFIG_RTE_MAX_MEMSEG=256 is not enough

Please either increase it or request less amount of memory.

EAL: FATAL: Cannot init memory

 

EAL: Cannot init memory

 

Failed to initialize DPDK

app.c: 372:spdk_app_start: *ERROR*: Unable to initialize SPDK env

 

=======

with mem_size option

=======

/ws/bazel-bin/spdk/spdk_test -t 10

Starting DPDK 17.11.0 initialization...

[ DPDK EAL parameters: reactor_perf -c 0x1 -m 256 --no-pci --file-prefix=spdk_pid29246 ]

EAL: Detected 72 lcore(s)

EAL: No free hugepages reported in hugepages-1048576kB

EAL: Probing VFIO support...

app.c: 377:spdk_app_start: *NOTICE*: Total cores available: 1

reactor.c: 654:spdk_reactors_init: *NOTICE*: Occupied cpu socket mask is 0x1

RING: Cannot reserve memory

reactor.c: 675:spdk_reactors_init: *NOTICE*: Event_mempool creation failed on preferred socket 0.

RING: Cannot reserve memory

reactor.c: 696:spdk_reactors_init: *ERROR*: spdk_event_mempool creation failed

app.c: 385:spdk_app_start: *ERROR*: Invalid reactor mask.

 

 

ps: I tried disabling huge pages but spdk seems to be crashing. Looks like this is not an option. 

===========

with —no-huge option

===========

Starting DPDK 17.11.0 initialization...

[ DPDK EAL parameters: reactor_perf -c 0x1 --no-pci --no-huge --file-prefix=spdk_pid23800 ]

EAL: Detected 72 lcore(s)

EAL: Probing VFIO support...

EAL: Started without hugepages support, physical addresses not available

Unable to unlink shared memory file: /var/run/.spdk_pid23800_hugepage_info. Error code: 2

invalid spdk_mem_register parameters, vaddr=0x7fbf2603b000 len=67108864

app.c: 377:spdk_app_start: *NOTICE*: Total cores available: 1

reactor.c: 654:spdk_reactors_init: *NOTICE*: Occupied cpu socket mask is 0x1

Bus error (core dumped)