hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar-group.org | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | This channel is logged: irclog.cct.lsu.edu
diehlpk has joined #ste||ar
diehlpk_work has joined #ste||ar
K-ballo has quit [Quit: K-ballo]
hkaiser has quit [Quit: Bye!]
diehlpk has quit [Quit: Leaving.]
diehlpk_work has quit [Remote host closed the connection]
<hkaiser>
the interaction between our scheduler and our threads is done via a coroutine style interface
<hkaiser>
but that has no implications for you (I hope)
<Yorlik>
So it's not really any sort of official functionality I guess?
<Yorlik>
Unfortunately Lua coroutines cannot trivially be moved across Lua states. So doping this on the C++ side of things might be the solution.
<Yorlik>
s/doping/doing/r
<hkaiser>
nod
<hkaiser>
Yorlik: if you used co_await and friends for your coroutine implementtaion (and you should look around, there are some nice libraries helping with this), you can also integrate HPX threads with that as our futures can be used with co_await as well
<hkaiser>
i.e. T result = co_await hpx::async([]() -> T {...}); will work
<Yorlik>
Nice !
<Yorlik>
Now I need to marry this with Lua scripting :D
<Yorlik>
An actor and message based system like we use it sometimes is not the optimum if you need return values like in conventional functions. And future.get can block. So - with coroutines I could try to check a future and if it 's not ready simply try again next frame. So I wouldn't need a ton of continuations and could just run my coroutine each frame, until it's done.
<hkaiser>
Yorlik: sure future::is_ready() is always there
<Yorlik>
hkaiser: Would coroutines be serializable?
<hkaiser>
Yorlik: hmmm
<hkaiser>
might be difficult as you would need to serialize the whole stackframe
<Yorlik>
And I can't access the underlying object if I use the c++20 syntax sugar..
<Yorlik>
hkaiser: I wonder if I should rather use some sort of custom, serializable function object that has the ability to migrate with the object.
<Yorlik>
The main issue is we cannot really use Lua coroutines for this. I need to deeper explore the possibilities I have here.
<hkaiser>
Yorlik: what do you want to migrate? just the function object?
<hkaiser>
hpx::util::function<> gives you that
<hkaiser>
same semantics as std::function<>, just serializable
<hkaiser>
Yorlik: it however relies on the bound function object being serializable
<Yorlik>
hkaiser: I need a way to conveniently have async calls inside Lua, with the ability to play nicely with our actor model. Normally during an update we grab a lua state, let it process the mailbox of our object and then be released back to the pool. This means after initialization, such a lua state may not ever keep local data. All data is kept on the C++ side and the lua states just do the scripted
<Yorlik>
processing of messages. When inside the update process, when the mailbox is processed the need for an async call comes up I'd like to have a syntactically easy way to yield that specific function until the future is ready. Unfortunately Lu a coroutines are bound to the state where they were created which means I cannot release the lua state at the end of the update cycle. I'm looking for a way around this
<Yorlik>
limitation. keeping the lua state around until the next frame is not really an option with potentially hundreds of thousands of objects.
<Yorlik>
A c++ object on the contrary could be kept around.
<Yorlik>
It's not so much about if it is doable, rather if it is doable in a sufficiently convenient way.
diehlpk has quit [Quit: Leaving.]
diehlpk has joined #ste||ar
diehlpk has quit [Quit: Leaving.]
<diehlpk_work>
I just tried to compile hpx kokkos using Cray clang version 12.0.3