hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar-group.org | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | This channel is logged: irclog.cct.lsu.edu
diehlpk has joined #ste||ar
diehlpk_work has joined #ste||ar
K-ballo has quit [Quit: K-ballo]
hkaiser has quit [Quit: Bye!]
diehlpk has quit [Quit: Leaving.]
diehlpk_work has quit [Remote host closed the connection]
Yorlik has joined #ste||ar
nanmiao has quit [Quit: Client closed]
Yorlik has quit [Ping timeout: 268 seconds]
hkaiser has joined #ste||ar
K-ballo has joined #ste||ar
Yorlik has joined #ste||ar
Yorlik has quit [Read error: Connection reset by peer]
<akheir> hkaiser: yes the file system is in the same path on all compute nodes,
<akheir> hkaiser: I don't know how exactly jenkins does it but you can run slurm job with the same way as if you are running slurm
<hkaiser> akheir: great, thanks
hkaiser has quit [Quit: Bye!]
diehlpk has joined #ste||ar
diehlpk has quit [Quit: Leaving.]
diehlpk has joined #ste||ar
diehlpk has quit [Quit: Leaving.]
akheir has quit [Remote host closed the connection]
hkaiser has joined #ste||ar
diehlpk has joined #ste||ar
diehlpk has quit [Client Quit]
diehlpk has joined #ste||ar
hkaiser_ has joined #ste||ar
hkaiser has quit [Ping timeout: 264 seconds]
diehlpk has quit [Quit: Leaving.]
tufei has joined #ste||ar
diehlpk_work has joined #ste||ar
diehlpk has joined #ste||ar
hkaiser_ has quit [Quit: Bye!]
diehlpk has quit [Client Quit]
diehlpk has joined #ste||ar
diehlpk has quit [Quit: Leaving.]
Yorlik has joined #ste||ar
hkaiser has joined #ste||ar
diehlpk has joined #ste||ar
<Yorlik> hkaiser: YT?
<hkaiser> Yorlik: here
<Yorlik> I am thinking about using coroutines for a certain purpose.
<Yorlik> Now I have the choice: C++20 or HPX
<hkaiser> lol
<Yorlik> Not sure what I should use in our context.
<hkaiser> C++20 doesn't give you coroutines, just the tools to use and build them
<Yorlik> I know - did some painful reading today :D
<hkaiser> in general coroutines and HPX are orthogonal
<hkaiser> you can run your coroutines on HPX threads
<Yorlik> The problem is, not everything can be sensibly wrapped into our general actor structure.
<Yorlik> My thoughts on tagged search for example show me a system that spans and concurrently alters the actors - which usually is not allowed.
<Yorlik> If I put everything into the actors I need locks elsewhere and get contention.
<Yorlik> If I want to use this from within the actors, i need to use some sort of future with a continuatioin, which easily gets messy.
<hkaiser> Yorlik: nobody prevents you from running a sequence of coroutines on an HPX thread
<Yorlik> And Lua coroutines are impossible in our case
<hkaiser> I don't see a either/or scenario here
<Yorlik> Me neither.
<Yorlik> Just wondering which coroutines to use.
<Yorlik> Is the any reason not to use the c++20 ones, apart from the requirement to move to c++20 in the first place?
<Yorlik> Or: Why would/should I use the HPX coroutines?
<hkaiser> none
<hkaiser> hpx has no coroutines
<hkaiser> at least no explicit ones
<hkaiser> Yorlik: that's an implementation detail
<Yorlik> Allright. It's C++20 then. :)
<hkaiser> the interaction between our scheduler and our threads is done via a coroutine style interface
<hkaiser> but that has no implications for you (I hope)
<Yorlik> So it's not really any sort of official functionality I guess?
<Yorlik> Unfortunately Lua coroutines cannot trivially be moved across Lua states. So doping this on the C++ side of things might be the solution.
<Yorlik> s/doping/doing/r
<hkaiser> nod
<hkaiser> Yorlik: if you used co_await and friends for your coroutine implementtaion (and you should look around, there are some nice libraries helping with this), you can also integrate HPX threads with that as our futures can be used with co_await as well
<hkaiser> i.e. T result = co_await hpx::async([]() -> T {...}); will work
<Yorlik> Nice !
<Yorlik> Now I need to marry this with Lua scripting :D
<Yorlik> An actor and message based system like we use it sometimes is not the optimum if you need return values like in conventional functions. And future.get can block. So - with coroutines I could try to check a future and if it 's not ready simply try again next frame. So I wouldn't need a ton of continuations and could just run my coroutine each frame, until it's done.
<hkaiser> Yorlik: sure future::is_ready() is always there
<Yorlik> hkaiser: Would coroutines be serializable?
<hkaiser> Yorlik: hmmm
<hkaiser> might be difficult as you would need to serialize the whole stackframe
<Yorlik> And I can't access the underlying object if I use the c++20 syntax sugar..
<Yorlik> hkaiser: I wonder if I should rather use some sort of custom, serializable function object that has the ability to migrate with the object.
<Yorlik> The main issue is we cannot really use Lua coroutines for this. I need to deeper explore the possibilities I have here.
<hkaiser> Yorlik: what do you want to migrate? just the function object?
<hkaiser> hpx::util::function<> gives you that
<hkaiser> same semantics as std::function<>, just serializable
<hkaiser> Yorlik: it however relies on the bound function object being serializable
<Yorlik> hkaiser: I need a way to conveniently have async calls inside Lua, with the ability to play nicely with our actor model. Normally during an update we grab a lua state, let it process the mailbox of our object and then be released back to the pool. This means after initialization, such a lua state may not ever keep local data. All data is kept on the C++ side and the lua states just do the scripted
<Yorlik> processing of messages. When inside the update process, when the mailbox is processed the need for an async call comes up I'd like to have a syntactically easy way to yield that specific function until the future is ready. Unfortunately Lu a coroutines are bound to the state where they were created which means I cannot release the lua state at the end of the update cycle. I'm looking for a way around this
<Yorlik> limitation. keeping the lua state around until the next frame is not really an option with potentially hundreds of thousands of objects.
<Yorlik> A c++ object on the contrary could be kept around.
<Yorlik> It's not so much about if it is doable, rather if it is doable in a sufficiently convenient way.
diehlpk has quit [Quit: Leaving.]
diehlpk has joined #ste||ar
diehlpk has quit [Quit: Leaving.]
<diehlpk_work> I just tried to compile hpx kokkos using Cray clang version 12.0.3
<diehlpk_work> and get the following error message https://pastebin.com/1EfM14PF
<hkaiser> diehlpk_work: uhh
<diehlpk_work> hkaiser, Same thought here
<ms[m]> diehlpk_work: can you open an issue with all the details please?
<hkaiser> also, could you create a preprocessed file for the failing source, please?
<K-ballo> looks like there may be an unqualified call to get somewhere inside map
<K-ballo> it happened with libc++ once
<hkaiser> so disabling std compatibility with hpx::tuple may help?
<hkaiser> diehlpk_work: try setting HPX_DATASTRUCTURES_WITH_ADAPT_STD_TUPLE=OFF
<K-ballo> difficult to say.. chances are hpx is an associated namespace somehow anyhow
<hkaiser> indeed
<K-ballo> worth a try in any case
<diehlpk_work> hkaiser, recompiling
<K-ballo> would help to know the exact libstdc++ version, to match those line numbers to actual code
<diehlpk_work> K-ballo, gibe me a sec
<K-ballo> also maybe try disabling the index pack builtins, just in case
<K-ballo> what's the state of the cmake cache corresponding to HPX_HAVE_BUILTIN_INTEGER_PACK and HPX_HAVE_BUILTIN_MAKE_INTEGER_SEQ ?
<hkaiser> cuda: HPX_HAVE_BUILTIN_MAKE_INTEGER_SEQ_CUDA
Yorlik has quit [Ping timeout: 268 seconds]
<diehlpk_work> K-ballo,
<diehlpk_work> HPX_WITH_BUILTIN_MAKE_INTEGER_SEQ:INTERNAL=TRUE
<diehlpk_work> HPX_WITH_BUILTIN_MAKE_INTEGER_SEQ_RESULT:INTERNAL=TRUE
<hkaiser> diehlpk_work: what about the _CUDA version?
<diehlpk_work> CUDA_TOOLKIT_ROOT_DIR_INTERNAL:INTERNAL=/global/common/software/nersc/cos1.3/cuda/11.3.0
<hkaiser> I meant HPX_HAVE_BUILTIN_MAKE_INTEGER_SEQ_CUDA
<diehlpk_work> That I can not find in CMakeCache.txt
<hkaiser> ok
<diehlpk_work> libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f1969318000)
<diehlpk_work> hkaiser, Recompiling
<diehlpk_work> K-ballo, hkaiser https://pastebin.com/dkFmnmrT
<diehlpk_work> uncomment these lines did not helped
<hkaiser> it's a different error
<diehlpk_work> Yes, sorry
<hkaiser> did you see this error before (possibly after the index_pack issue?
<K-ballo> so something wrong with the builtins then? and apparently completely pointless feature tests
<hkaiser> K-ballo: it's the cuda side of things - everything is possible
<diehlpk_work> Ok, recompiling
<diehlpk_work> I like to break things by always using the latest compilers
<hkaiser> diehlpk_work: have you configured HPX with CUDA (HPX_WITH_CUDA=ON)?
<diehlpk_work> hkaiser, Yes
<hkaiser> k
<diehlpk_work> I am using the same scriprs as I used on summit oder daint
<diehlpk_work> The issue seems to be clang 12
<diehlpk_work> I can compile with clang 11 and CUDA 11.4 on Summit
<diehlpk_work> Exact same versions of hpx and kokkos
<diehlpk_work> hkaiser, K-ballo Thanks
<diehlpk_work> It is compiling
<diehlpk_work> Uncomment the second part did the trick
<diehlpk_work> I will work on hdf5, silo, and octo-tiger later
<diehlpk_work> hkaiser, Should I open a ticket and report what I did?
<hkaiser> diehlpk_work: yes, please
<hkaiser> did you comment both sections?
<diehlpk_work> Yes, that was the trick
<diehlpk_work> the first was not enough
<hkaiser> diehlpk_work: thanks, K-ballo is right then, our features tests for this are apparently useless
K-ballo has quit [Quit: K-ballo]
K-ballo has joined #ste||ar