I simply tested the BlobFS Asynchronous API by using SPDK events framework to execute multi tasks, each task writes one file.
But it doesn't work, the spdk_file_write_async() reported an error when resizing the file size.
The call stack looks like this:
spdk_file_write_async() -> __readwrite() -> spdk_file_truncate_async() -> spdk_blob_resize()
The resize operation must be done in the metadata thread which invoked the spdk_fs_load(), so only the task dispatched to the metadata CPU core works.
That's to say only one thread can be used to write files. It's hard to use, and performance issues may arise.
Does anyone knows further more about this?
thanks very much
Short but important announcement about SPDK Gerrit.
I'd like to notify that SPDK Gerrit at review.spdk.io will be shut down due to maintenance on Wednesday 16th February 2022.
The downtime will start at 8:30AM GMT. Gerrit should be back online at 11:00AM GMT.
I have read the source code of SPDK and implemented my own bdev module by referencing official bdev modules like null, malloc and aio. But I still have some questions and hope someone can explain.
1.When a bdev is deleted, the 'destruct' function pointer of spdk_bdev_fn_table is called and the block device is closed. I find that ongoing spdk_bdev_io is not explicitly completed by calling spdk_bdev_io_complete(). Are these spdk_bdev_io cleaned automatically by SPDK framework?
2.How to write the json config that corresponds to the rpc call of 'log_set_flag'? I have tried many ways but still can not work and I have to implement my own version of rpc call in my own bdev module to enable json config.
As many know we have a few different areas where we support HW accelerators:
* The accelerator framework (fairly new, accel_fw, previously known as the copy framework)
* Supports generic interface for DMA and other data manipulation functions for IOAT and the upcoming DSA (aka IDXD)
* Supports using optimized SW calls for those API not supported by the selected hardware
* The compression and encryption virtual bdev modules
* Supports DPDK CompressDev and CryptoDev APIs
* Able to plug in different PMDs (polled mode drivers) as supported by SPDK, today that includes:
* Compression: QAT and mlx5_pci (HW) and ISAL (SW)
* Crypto: QAT (HW) and ISAL (SW). Ciphers: AES_CBC or AES_XTS (QAT only)
There’s been more than one discussion over the last several months about consolidating this support under one framework, Jim has asked me to work on this for the next release so I’ll be working on (a) adding all accelerator support to the accel_fw and (b) reworking the above bdev modules to use new accel_fw APIs. This should result in a cleaner/extensible architecture and a simpler more usable set of capabilities (crypto/compression and future stuff won’t be limited to the bdev layer).
There are, of course, some challenges in coming up with the new design. In the next few weeks (I’m out next week) I’ll add an agenda item to the Euro/Americas friendly community meeting and would encourage all those interested to attend.
I haven’t put a tremendous amount of thought into this yet but for starters the accel_engine is built to have one and only one initialized HW module that drops to SW if the selected HW doesn’t support the API. Clearly we want to be able to have the accel_fw be initialized to do either IOAT or DSA and crypto and compression, not just one of these things. There’s a few ways to tackle this, I’ll write up a proposal prior to the community meeting in 2 weeks to discuss. Similarly, with IOAT and DSA when we don’t have HW support we use ISA-L directly however with crypto and compression you make a decision at init time whether you want HW or SW and either way you use the DPDK API just with a different PMD. The advantage here is a common API for SPDK to use regardless of the PMD (however the init sequence is slightly different depending on the PMD) but the downside is that we’re using the DPDK API for ISAL SW support when we could code that ourselves directly; this has bitten us 2-3x so far with bugs on the DPDK side so if we skip using the ISAL PMD we have a lot more control. I don’t have an opinion on which is better yet, just something that comes to mind to think about.
If you have some short thoughts over the next few weeks please share on Slack, or here. Slack is preferred but this is a long message so I went with email ☺