Hi Andrey,

 

Your gut is not fooling you.  I think doing this in the context of an existing polled mode thread is a great idea.  (But I’m glad that the spdk_allocate_thread() approach was able to work for you!)

 

You could just always pick the master thread.  This is the thread that calls your module’s init() function.  You can call spdk_poller_register() from your init function to start pollers – and pick a polling frequency that makes sense for the expected rate of synchronization messages.  You can also save spdk_get_thread() to a global variable so that you can do spdk_thread_send_msg() calls to it from other threads – or maybe you create a multi-producer/single-consumer ring that other threads can enqueue to, and then the poller picks up those messages, executes them, and sends a message back to the calling thread.

 

There are a lot of cases in SPDK currently where we start up pollers on a specific thread at application start – i.e. RPC handling (lib/event/rpc.c) and iSCSI acceptors (lib/iscsi/acceptor.c) immediately come to mind.  The virtio bdev driver (lib/bdev/virtio/bdev_virtio_scsi.c) also uses this poller + ring concept to synchronize reset operations.

 

Would this be sufficient for your use case?

 

-Jim

 

 

 

From: SPDK <spdk-bounces@lists.01.org> on behalf of Andrey Kuzmin <andrey.v.kuzmin@gmail.com>
Reply-To: Storage Performance Development Kit <spdk@lists.01.org>
Date: Tuesday, July 31, 2018 at 2:31 AM
To: Storage Performance Development Kit <spdk@lists.01.org>
Subject: [SPDK] Single-threaded semantics in vbdev

 

Hi guys,

 

Working on a vbdev that maintains some global state and underlying data structures, I decided to explore an SPDKish alternative to the traditional synchronization, that of the single-threaded semantics where global data structures are being maintained by a dedicated thread, with requests being served by message passing to/from that thread. 

 

Coding such a thread against the spdk_allocate_thread() guidelines proved to be quite straightforward, and I've got it up and running. At the same time, I'm still unclear as to why I should be essentially duplicating the features of the pinned SPDK thread, with my gut feeling being that the very same single-threaded semantics could have been easily achieved by pinning my execution context to some randomly or round-robin chose pinned thread of the configured allowed cores mask.

 

If that gut feeling doesn't fool me, I will appreciate a pointer to any docs and/or examples showing how one can pin an arbitrary execution context inside a vbdev to an SPDK native thread (similar to app_start and friends, I believe), and do polling on that thread/ messaging to/from that thread similar to what a POSIX thread equipped with spdk thread envelope could do.


Thanks in advance,
Andrey