Does BlobFS Asynchronous API support multi thread writing?
by chen.zhenghua＠zte.com.cn
Hi everyone,
I simply tested the BlobFS Asynchronous API by using SPDK events framework to execute multi tasks, each task writes one file.
But it doesn't work, the spdk_file_write_async() reported an error when resizing the file size.
The call stack looks like this:
spdk_file_write_async() -> __readwrite() -> spdk_file_truncate_async() -> spdk_blob_resize()
The resize operation must be done in the metadata thread which invoked the spdk_fs_load(), so only the task dispatched to the metadata CPU core works.
That's to say only one thread can be used to write files. It's hard to use, and performance issues may arise.
Does anyone knows further more about this?
thanks very much
1 month
RFC: NVMf namespace masking
by Jonas Pfefferle
Hi all,
I would be happy to get some feedback on my NVMf target namespace masking
implementation using attach/detach:
https://review.spdk.io/gerrit/c/spdk/spdk/+/7821
The patch introduces namespace masking for NVMe-over-fabrics
targets by allowing to (dynamically) attach and detach
controllers to/from namespaces, cf. NVMe spec 1.4 - section 6.1.4.
Since SPDK only supports the dynamic controller model a new
controller is allocated on every fabric connect command.
This allows to attach/detach controllers of a specific
host NQN to/from a namespace. A host can only perform
operations to an active namespace. Inactive namespaces can
be listed (not supported by SPDK) but no additional
information can be retrieved:
"Unless otherwise noted, specifying an inactive NSID in a
command that uses the Namespace Identifier (NSID) field shall
cause the controller to abort the command with status
Invalid Field in Command" - NVMe spec 1.4 - section 6.1.5
Note that this patch does not implement the NVMe namespace
attachment command but allows to attach/detach via RPCs only.
To preserve current behavior all controllers are auto attached.
To not not auto attach controllers the nvmf_subsystem_add_ns
shall be called with "--no-auto-attach". We introduce two new
RPC calls:
- nvmf_ns_attach_ctrlr <subsysNQN> <NSID> [--host <hostNQN>]
- nvmf_ns_detach_ctrlr <subsysNQN> <NSID> [--host <hostNQN>]
If no host NQN is specified all controllers
(new and currently connected) will attach/detach to/from the
namespace specified.
The list in spdk_nvmf_ns is used to keep track of hostNQNs
which controllers should be attached on connect.
The active_ns array in spdk_nvmf_ctrlr is used for fast lookup
to check whether a NSID is active/inactive on command execution.
Thanks,
Jonas
7 months
test
by xl_3992＠qq.com
test
10 months, 3 weeks
Re: ABI change
by Harris, James R
The dependencies are specified in mk/spdk.lib_deps.mk.
The log isn't very clear on that, I've just pushed a patch out for review to fix it: https://review.spdk.io/gerrit/c/spdk/spdk/+/8499.
-Jim
On 6/24/21, 7:43 AM, "Rodriguez, Edwin" <Ed.Rodriguez(a)netapp.com> wrote:
I'm stumped by the error here: https://ci.spdk.io/results/autotest-per-patch/builds/50808/archive/autobu...
00:07:23.879 ************************************
00:07:23.879 START TEST confirm_makefile_deps
00:07:23.879 ************************************
00:07:23.880 ---------------------------------------------------------------------
00:07:29.634 there was a dependency mismatch in the library bdev_kv_null
00:07:29.634 The makefile lists: ''
00:07:29.634 readelf outputs : 'bdev jsonrpc json log nvme rpc thread util'
00:07:29.634 ---------------------------------------------------------------------
Where does the Makefile list dependencies? And which Makefile?
________________________________
From: Harris, James R <james.r.harris(a)intel.com>
Sent: Wednesday, June 23, 2021 5:33 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Re: ABI change
NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hi Ed,
Neither. You would need to bump SO_VER and set SO_MINOR back to 0. The spdk_nvme_print_command() change is ABI incompatible, requiring the SO_VER bump. We always set SO_MINOR back to 0 after bumping SO_VER.
I would instead suggest a new spdk_nvme_print_command_csi() API, so that you don't have to modify the spdk_nvme_print_command() API. Then you should no longer need any version changes, since SO_MINOR has already been bumped during this release cycle.
-Jim
On 6/23/21, 2:21 PM, "Rodriguez, Edwin" <Ed.Rodriguez(a)netapp.com> wrote:
I've submitted a change which fails with: https://ci.spdk.io/results/autotest-per-patch/builds/50670/archive/autobu...
00:06:41.549 SONAME changed from 'libspdk_nvme.so.5.0' to 'libspdk_nvme.so.5.1'
00:06:41.549
00:06:41.549 15 Added functions:
00:06:41.549
00:06:41.549 [A] 'function int spdk_kv_cmd_fmt_lower(const spdk_nvme_kv_cmd*, char*, size_t)' {spdk_kv_cmd_fmt_lower}
00:06:41.549 [A] 'function int spdk_kv_key_fmt_lower(char*, size_t, uint32_t, const uint8_t*)' {spdk_kv_key_fmt_lower}
00:06:41.549 [A] 'function bool spdk_nvme_ctrlr_is_fabrics(spdk_nvme_ctrlr*)' {spdk_nvme_ctrlr_is_fabrics}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_delete(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_delete}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_exist(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_exist}
00:06:41.549 [A] 'function void spdk_nvme_kv_cmd_get_key(const spdk_nvme_kv_cmd*, spdk_nvme_kv_key_t*)' {spdk_nvme_kv_cmd_get_key}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_list(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_list}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_retrieve(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*, uint32_t)' {spdk_nvme_kv_cmd_retrieve}
00:06:41.549 [A] 'function void spdk_nvme_kv_cmd_set_key(const spdk_nvme_kv_key_t*, spdk_nvme_kv_cmd*)' {spdk_nvme_kv_cmd_set_key}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_store(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*, uint32_t)' {spdk_nvme_kv_cmd_store}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_key_len(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_key_len}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_num_keys(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_num_keys}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_value_len(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_value_len}
00:06:41.549 [A] 'function const spdk_nvme_kv_ns_data* spdk_nvme_kv_ns_get_data(spdk_nvme_ns*)' {spdk_nvme_kv_ns_get_data}
00:06:41.549 [A] 'function int spdk_nvme_ns_cmd_copy(spdk_nvme_ns*, spdk_nvme_qpair*, const spdk_nvme_scc_source_range*, uint16_t, uint64_t, spdk_nvme_cmd_cb, void*)' {spdk_nvme_ns_cmd_copy}
00:06:41.549
00:06:41.549 1 function with some sub-type change:
00:06:41.549
00:06:41.549 [C] 'function void spdk_nvme_print_command(uint16_t, spdk_nvme_cmd*)' at nvme_qpair.c:337:1 has some sub-type changes:
00:06:41.549 parameter 3 of type 'enum spdk_nvme_csi' was added
Do I need to update both SO_VER and SO_MINOR for libnvme? Or will just updating SO_MINOR suffice?
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
10 months, 3 weeks
Re: ABI change
by Harris, James R
Hi Ed,
Neither. You would need to bump SO_VER and set SO_MINOR back to 0. The spdk_nvme_print_command() change is ABI incompatible, requiring the SO_VER bump. We always set SO_MINOR back to 0 after bumping SO_VER.
I would instead suggest a new spdk_nvme_print_command_csi() API, so that you don't have to modify the spdk_nvme_print_command() API. Then you should no longer need any version changes, since SO_MINOR has already been bumped during this release cycle.
-Jim
On 6/23/21, 2:21 PM, "Rodriguez, Edwin" <Ed.Rodriguez(a)netapp.com> wrote:
I've submitted a change which fails with: https://ci.spdk.io/results/autotest-per-patch/builds/50670/archive/autobu...
00:06:41.549 SONAME changed from 'libspdk_nvme.so.5.0' to 'libspdk_nvme.so.5.1'
00:06:41.549
00:06:41.549 15 Added functions:
00:06:41.549
00:06:41.549 [A] 'function int spdk_kv_cmd_fmt_lower(const spdk_nvme_kv_cmd*, char*, size_t)' {spdk_kv_cmd_fmt_lower}
00:06:41.549 [A] 'function int spdk_kv_key_fmt_lower(char*, size_t, uint32_t, const uint8_t*)' {spdk_kv_key_fmt_lower}
00:06:41.549 [A] 'function bool spdk_nvme_ctrlr_is_fabrics(spdk_nvme_ctrlr*)' {spdk_nvme_ctrlr_is_fabrics}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_delete(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_delete}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_exist(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_exist}
00:06:41.549 [A] 'function void spdk_nvme_kv_cmd_get_key(const spdk_nvme_kv_cmd*, spdk_nvme_kv_key_t*)' {spdk_nvme_kv_cmd_get_key}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_list(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_list}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_retrieve(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*, uint32_t)' {spdk_nvme_kv_cmd_retrieve}
00:06:41.549 [A] 'function void spdk_nvme_kv_cmd_set_key(const spdk_nvme_kv_key_t*, spdk_nvme_kv_cmd*)' {spdk_nvme_kv_cmd_set_key}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_store(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*, uint32_t)' {spdk_nvme_kv_cmd_store}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_key_len(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_key_len}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_num_keys(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_num_keys}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_value_len(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_value_len}
00:06:41.549 [A] 'function const spdk_nvme_kv_ns_data* spdk_nvme_kv_ns_get_data(spdk_nvme_ns*)' {spdk_nvme_kv_ns_get_data}
00:06:41.549 [A] 'function int spdk_nvme_ns_cmd_copy(spdk_nvme_ns*, spdk_nvme_qpair*, const spdk_nvme_scc_source_range*, uint16_t, uint64_t, spdk_nvme_cmd_cb, void*)' {spdk_nvme_ns_cmd_copy}
00:06:41.549
00:06:41.549 1 function with some sub-type change:
00:06:41.549
00:06:41.549 [C] 'function void spdk_nvme_print_command(uint16_t, spdk_nvme_cmd*)' at nvme_qpair.c:337:1 has some sub-type changes:
00:06:41.549 parameter 3 of type 'enum spdk_nvme_csi' was added
Do I need to update both SO_VER and SO_MINOR for libnvme? Or will just updating SO_MINOR suffice?
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
10 months, 3 weeks
ABI change
by Rodriguez, Edwin
I've submitted a change which fails with: https://ci.spdk.io/results/autotest-per-patch/builds/50670/archive/autobu...
00:06:41.549 SONAME changed from 'libspdk_nvme.so.5.0' to 'libspdk_nvme.so.5.1'
00:06:41.549
00:06:41.549 15 Added functions:
00:06:41.549
00:06:41.549 [A] 'function int spdk_kv_cmd_fmt_lower(const spdk_nvme_kv_cmd*, char*, size_t)' {spdk_kv_cmd_fmt_lower}
00:06:41.549 [A] 'function int spdk_kv_key_fmt_lower(char*, size_t, uint32_t, const uint8_t*)' {spdk_kv_key_fmt_lower}
00:06:41.549 [A] 'function bool spdk_nvme_ctrlr_is_fabrics(spdk_nvme_ctrlr*)' {spdk_nvme_ctrlr_is_fabrics}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_delete(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_delete}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_exist(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_exist}
00:06:41.549 [A] 'function void spdk_nvme_kv_cmd_get_key(const spdk_nvme_kv_cmd*, spdk_nvme_kv_key_t*)' {spdk_nvme_kv_cmd_get_key}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_list(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*)' {spdk_nvme_kv_cmd_list}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_retrieve(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*, uint32_t)' {spdk_nvme_kv_cmd_retrieve}
00:06:41.549 [A] 'function void spdk_nvme_kv_cmd_set_key(const spdk_nvme_kv_key_t*, spdk_nvme_kv_cmd*)' {spdk_nvme_kv_cmd_set_key}
00:06:41.549 [A] 'function int spdk_nvme_kv_cmd_store(spdk_nvme_ns*, spdk_nvme_qpair*, spdk_nvme_kv_key_t*, void*, uint32_t, spdk_nvme_cmd_cb, void*, uint32_t)' {spdk_nvme_kv_cmd_store}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_key_len(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_key_len}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_num_keys(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_num_keys}
00:06:41.549 [A] 'function uint32_t spdk_nvme_kv_get_max_value_len(spdk_nvme_ns*)' {spdk_nvme_kv_get_max_value_len}
00:06:41.549 [A] 'function const spdk_nvme_kv_ns_data* spdk_nvme_kv_ns_get_data(spdk_nvme_ns*)' {spdk_nvme_kv_ns_get_data}
00:06:41.549 [A] 'function int spdk_nvme_ns_cmd_copy(spdk_nvme_ns*, spdk_nvme_qpair*, const spdk_nvme_scc_source_range*, uint16_t, uint64_t, spdk_nvme_cmd_cb, void*)' {spdk_nvme_ns_cmd_copy}
00:06:41.549
00:06:41.549 1 function with some sub-type change:
00:06:41.549
00:06:41.549 [C] 'function void spdk_nvme_print_command(uint16_t, spdk_nvme_cmd*)' at nvme_qpair.c:337:1 has some sub-type changes:
00:06:41.549 parameter 3 of type 'enum spdk_nvme_csi' was added
Do I need to update both SO_VER and SO_MINOR for libnvme? Or will just updating SO_MINOR suffice?
10 months, 3 weeks
