hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar-group.org | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | This channel is logged: irclog.cct.lsu.edu
hkaiser has quit [Quit: Bye!]
Yorlik has joined #ste||ar
K-ballo has quit [Ping timeout: 240 seconds]
K-ballo has joined #ste||ar
hkaiser has joined #ste||ar
K-ballo1 has joined #ste||ar
K-ballo has quit [Ping timeout: 248 seconds]
K-ballo1 is now known as K-ballo
<gonidelis[m]> K-ballo hkaiser what is `mem_obj` here? i see it takes no args
<hkaiser> that line has no mem_obj
<gonidelis[m]> `invoke_mem_obj` i mean
<hkaiser> that's a wrapper function object that -- when invoked -- returns the value of the member data item it wraps
<hkaiser> using the first argument as the this for the object to access
<hkaiser> (first and only argument)
<satacker[m]> hkaiser: gonidelis I was trying to implement sender_base, sender, and sender_to `concepts` as `tags`. For the sender requirements is it enough to query the completion signatures of the sender?
<satacker[m]> > <@satacker:matrix.org> hkaiser: gonidelis I was trying to implement sender_base, sender, and sender_to `concepts` as `tags`. For the sender requirements is it enough to query the completion signatures of the sender?
<satacker[m]> Or should I simply call the `is_sender_v` ?
<hkaiser> just use is_sender_v
<hkaiser> it does the right thing
<hkaiser> satacker[m]: ^^
<hkaiser> satacker[m]: why do you thing the concepts should be CPOs?
<hkaiser> aren't those simply boolean compiletime conditions?
<hkaiser> think*
<satacker[m]> I thought of `sender_base` as a CPO, it's the base part that made me think so
<satacker[m]> Also I thought of tag dispatching
<satacker[m]> Question 2:
<satacker[m]> Can `sender_of` implementation be done as follows
<satacker[m]> -->Verify if it is a sender and then return the completion signatures using `get_completion_signatures` ?
<hkaiser> sender_of<S, R> should return true/false depending on whether S is a sender and R is a receiver supporting the completion signatures of S
<hkaiser> our is_sender_to does not check the completion signatures part (yet), though
<satacker[m]> So I need to make sure that their return types matches one of the completion signatures (val, error, stopped )?
<hkaiser> what's missing is the is_receiver_of<R, completion_signatures_of_t<S, env_of_t<R>>> trait
<hkaiser> once we have that, is_sender_of is trivial as it simply checks is_sender<S> && is_receiver_of<R, completion_signatures_of_t<S, env_of_t<R>>>
<hkaiser> well, we do have is_receiver_of, but it doesn't check the completion signatures part: https://github.com/STEllAR-GROUP/hpx/blob/master/libs/core/execution_base/include/hpx/execution_base/receiver.hpp#L168-L172
<hkaiser> it currently checks whether set_value would succeed
<satacker[m]> I didn't get why `is_receiver_of` is called to check `is_sender_of`
<hkaiser> satacker[m]: so I would suggest you look into fixing is_receiver_of first, then change is_sender_of
<hkaiser> is_sender_of returns true if S is a sender and R is a receiver that can handle the completions signatures of S
<hkaiser> i.e. is_sender_of<S, R> == is_sender<S> && is_receiver_of<R, completion_signatures_of_t<S, env_of_t<R>>
<satacker[m]> Thanks, that and the definitions clear it
<john98zakaria[m]> I have a very general design question.... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/2ab3b04b42ce1e3f43092198a8f3589102e2194a)
<john98zakaria[m]> I guess the question boils down to "should you use a future as a mutex?
<john98zakaria[m]> * a mutex/barrier ?
<hkaiser> john98zakaria[m]: first - channels support moving data
<hkaiser> whether to use mutexes, futures, barriers or something else really depends on the use case
<john98zakaria[m]> hkaiser: If I do std::move I loose my buffer object
<hkaiser> use a barrier if you need to synchronize between several threads
<john98zakaria[m]> And then I need to reallocate it
<hkaiser> john98zakaria[m]: true
<hkaiser> if everything is on one locality only you can pass a reference to your data
<john98zakaria[m]> That's why I'll rebuild my own one message zero copy channel
<john98zakaria[m]> hkaiser: It's distributed
<hkaiser> we also have the serialize_buffer, that pass references locally and copies remotely
<john98zakaria[m]> hkaiser: But then the receiver needs to allocate new memory each time
<hkaiser> essentially an array wrapper that manages ownership properly depending on whether it is sent locally or remotely
<hkaiser> it needs to allocate only in the remote case
<hkaiser> but I see what you're saying, you want to put the data into an existing receive buffer
<john98zakaria[m]> hkaiser: Which is exactly what I am doing
<john98zakaria[m]> Exactly
<hkaiser> that will work only if the sender has some kind of handle that identifies the receive buffer
<hkaiser> John Biddiscombe (not here right now was working on integrating rdma with the network layer such that it places the received data directly where it should go
<john98zakaria[m]> My plan was to make 2 zero copy servers, each on a different locality.
<john98zakaria[m]> Each locality can write in its own server.
<john98zakaria[m]> The other uses its serialize buffer to grab the data from the remote locality
<hkaiser> and he wanted to use the channel API to hide all of this from the user
<john98zakaria[m]> That's exactly what I am looking for 😍
<hkaiser> if you really want to achieve zero-copy on the receiving end you'll need to drill a hole into the parcelport API or talk directly to the network
<john98zakaria[m]> I don't think I am that capable in c++ yet 😅
<hkaiser> now that I think about it, we might actually be able to do true zero copy without too many changes, just by using special types that have custom serialization support
<hkaiser> well, maybe not - the receiver still would need to allocate the buffer were the network puts the data
<john98zakaria[m]> If we have a way to "inject" the allocator, then it would work.
<hkaiser> serialize_buffer supports that, you're right
<hkaiser> and yes, the zerocopy_rdma example demostrates that - I had completely forgotten about it
<john98zakaria[m]> hkaiser: I imagine the following
<john98zakaria[m]> Each have to allocate memory on channel registration where they want their data to be put .
<john98zakaria[m]> If locality a sends a message larger than the buffer of b, a resize is triggered, otherwise the reception will happen copy free.
<john98zakaria[m]> That's how I would build my thing.
<john98zakaria[m]> I will unfortunately have to do 2 round trips for that, but it's still better that allocating hundreds of mbs
K-ballo1 has joined #ste||ar
K-ballo has quit [Ping timeout: 248 seconds]
K-ballo1 is now known as K-ballo
<hkaiser> john98zakaria[m]: if you do towo roundtrips, then you can use the first to request an allocator that can be used with the second roundtrip to identify the receive buffer
Yorlik has quit [Ping timeout: 272 seconds]