Ben, isn't it also the case that the the spdk_malloc routines (I think all of them )
return mem from pre-allocated huge pages. In cases where that's not required,
there's no reason to use that scarce resource (depending on how you configured it) so
it's better to use malloc/calloc. Right?
From: SPDK [mailto:email@example.com] On Behalf Of Walker, Benjamin
Sent: Thursday, July 12, 2018 10:41 AM
Subject: Re: [SPDK] OS memory allocation
On Thu, 2018-07-12 at 07:10 +0000, Ravich, Leonid wrote:
Looks like SPDK uses OS dynamic memory (malloc, zalloc ...) I am not
understand why , It can use spdk_malloc/ spdk_zmalloc instead and be
more "system friendly" .
If there is some good reason for such behavior I would like to suggest
wrapping all the "raw" linux system calls used by SPDK .
This way SPDK will be more flexible in different systems integrations .
SPDK very intentionally depends on the POSIX API for a large number of operations.
We've chosen to go this direction because SPDK is intended to run as regular a user
space application and we want developers to be able to use all of the regular tools
available in that environment. We're concerned that not allowing the use of POSIX APIs
makes it much more difficult for people to jump in and contribute to the code, and also
that it would be quite difficult to enforce in practice. Note that this is also the choice
that DPDK has made.
We have taken some steps to support more exotic environments though. First, all POSIX
include files are only included via include/spdk/stdinc.h, and we actually have tests to
verify that this is the case. If you have a non-POSIX environment, you can swap out that
header to include other implementations and map them to the POSIX calls we make.
SPDK also requires a number of operations that are outside of the purview of POSIX. These
are all encapsulated by include/spdk/env.h. The implementation of that header is in
lib/env_dpdk, and by default uses DPDK to implement all of those operations. However, this
portion is designed to be swapped out. You can point SPDK at a different implementation of
include/spdk/env.h through the configuration script:
We further have a number of abstractions coming for threading that aren't quite done
yet. SPDK is designed around cooperative multi-tasking and many locations in the code need
to be able to send messages to other threads. However, SPDK doesn't want to force
users into a particular green thread/cooperative multitasking/futures+promises framework.
We're currently designing abstractions for "light weight" threads (without
an associated implementation) that users should be able to map trivially to their
I hope that helps clear things up. It's all about making SPDK as flexible as possible
for as many people as we can, but also making the "simple" case as easy to
program and use as possible.
SPDK mailing list