mdiers_1 has quit [Read error: Connection reset by peer]
Guest70891 has joined #ste||ar
mdiers_ has quit [Ping timeout: 276 seconds]
Guest70891 has quit [Ping timeout: 265 seconds]
Guest70891 has joined #ste||ar
mdiers_ has joined #ste||ar
Guest70891 has quit [Ping timeout: 268 seconds]
Guest70891 has joined #ste||ar
<jbjnr>
heller: if you have a spare moment, could yuo put your task_data future backing for mpi into a gist please. I'd like to possible replace my use of promise with that if it has any advantages. (Either way, I'm curious to see what you did because I can't remember now).
<jbjnr>
aha. that line "return future_access<hpx::future<void>>::create(std::move(data));" that's what I needed. thank you very much.
Guest70891 has quit [Ping timeout: 276 seconds]
Guest70891 has joined #ste||ar
rori has joined #ste||ar
<heller>
simbergm: is merging PRs blocked now?
<simbergm>
heller: no, why?
<heller>
since the daint builders are messed up
<simbergm>
ah, well, wait until the evening
<simbergm>
we can also launch a few builds on rostam if you want something merged quickly
<heller>
I am in no hurry ;)
<heller>
simbergm: what was the general reaction on my talk in zuerich last friday?
<heller>
and also, what do you think?
<simbergm>
heller: good, I'd rather wait than break master
<simbergm>
some were concerned about your virtual function calls :P but I said that wasn't really the main point of your talk, the implementation can still change
<simbergm>
in general they really like the direction
<simbergm>
I like it too
<simbergm>
I'm a bit concerned about how it'll all come together with our "standard compliance" considering things are so much in flux but that's a risk we can't really avoid
Guest70891 has quit [Ping timeout: 268 seconds]
Guest70891 has joined #ste||ar
<heller>
simbergm: the standard compliance storry won't change
<heller>
simbergm: we are in the realm of standard extensions there. And everything will be in line with P0443
<heller>
simbergm: sure, virtual functions, you won't have those in your hot path, hopefully ;)
<heller>
and if you do, hope that the branch predictor works
<heller>
simbergm: the goal now is to have the first standards compliance implementation of P0443 with the goal of allowing those user defined extensions
<heller>
and eventually get rid of the fragmented landscape that you already seem to have at cscs ;0
<heller>
;)
weilewei has quit [Remote host closed the connection]
<simbergm>
heller: sounds good :) note that my concern mainly comes from not knowing well enough how much concensus there is in the committee on these things, but if it's all in line with what's going in that's great
<simbergm>
heller, jbjnr what do you think about #4142... should I go ahead and rename the old thread_pool_executor to embedded_thread_pool_executor? they are in principle in our public api, but they'd usually be used through the <scheduler_name>_executor typedef
<jbjnr>
simbergm: I do not believe anyone is using the mebedded schedulers and if they are, they must be an advanced user and will know what to change once they discover their code is broken
<heller>
simbergm: I'd say we need to clean up the executors significantly
<jbjnr>
+++++11111
<heller>
and that's just dealing with the symptoms...
<simbergm>
I'd agree
<heller>
so i really don't care that much right now ;)
<simbergm>
most of the executors are using the very old interface with add/add_after etc.
<simbergm>
deprecating them in the next release might be a good idea
<simbergm>
so that we can finally get rid of them
<jbjnr>
feel free to remove it all
<jbjnr>
:)
<jbjnr>
do we have a simple lockfree list in hpx?
<jbjnr>
not a dequeue or a queue or a stack, but just a list?
<heller>
jbjnr: no, we don't have anything like that
<jaafar>
The launch policy argument tells how to run the continuation relative to the last supplied input
<jaafar>
what if I want to be launched differently depending on which input it is? Is there some way to accomplish this?
<jaafar>
e.g. I have two dependencies A and B
<jaafar>
If A appears last I want to be run synchronously, so the continuation executes on the same core and the cache is warm
<jaafar>
If B, async is fine
aserio has joined #ste||ar
<jaafar>
dataflow(launch::sync, f, A, B) runs synchronously regardless of whether A or B arrives last, and similar if it were launch::async
<jaafar>
Would be nice to be able to distinguish
<hkaiser>
jaafar: so you want to use sunc if A arrives first, but async if B arrives first?
<hkaiser>
sync*
<hkaiser>
there is no way to distinguish that, atm
<jaafar>
hkaiser: I would say "last" instead of "first" but yes
<jaafar>
the application here is that I know that the supplier of A will have other data I need warm in the cache
<jaafar>
so very helpful for it to simply start executing my continuation
<jbjnr>
I could almost do it with my guided executor, but it'd need some adjustments I think
<jbjnr>
jaafar: if you had a simple test case that demonstrated a gain from cache re-use - I'd take a look at it.
<jaafar>
OK thanks jbjnr I will see what I can do
<jbjnr>
I was pondering the semantics of how to do this just recently (not quite the same, cos it wasn't a dataflow example)
<jbjnr>
the guided executor delays the scheduling of continuations until the input tasks have completed, so it is ideal for this, but I wanted a policy like launch::cache or something
<jbjnr>
but ideally it would be an 'if you can' rather than 'you must' because if we interfere with scheduling too much then we suffer from losing stealing etc
<jaafar>
yep
<jbjnr>
there should be a cost function ideally
<jaafar>
exclusive scan shows strong signs of being dominated by cache misses
<jaafar>
and it makes two passes
<jaafar>
a given chunk is visited twice
<jaafar>
My hypothesis is we'd see a speedup if the two passes were close in time and on the same core
<jaafar>
for a given chunk
<jbjnr>
ok, that ought to be quite testable
<jbjnr>
(naturally I have my own implementation of scan)
<jbjnr>
weilewei: looks like some cruft introduced by the new modularization - simbergm might know the right fix, but it looks as though a dependency is missing. is there a module with a name close to agas/logging that can be added
<jbjnr>
bbiab
<weilewei>
jbjnr not sure about what module should be added, but ideally, the mpi_concurrency_test is independant from hpx, not sure why hpx is involved
<weilewei>
HPX_WITH_LOGGING:BOOL?
<jbjnr>
weilewei: it wants to link in hpx::util::logging - if it doesn't use HPX (I've forgotten that test, so I need to look more closely) - then why is HPX being included at all - check the includes and the cmake too
<jbjnr>
hkaiser or aserio : my calendar just popped up a reminder about an OB meeting, is it happening or a cancelled one
<jbjnr>
weilewei: if I recall right - the DCA++hpx stuff sets a #define to override the threading so that the tests run on hpx threads without the user doing anything - this is going to trigger the pulling in of everything
mbremer has joined #ste||ar
<jbjnr>
but probably we now need to add a dependency to the logging stuff. HPX_WITH_LOGGING OFF might be worth a try
<weilewei>
you use a macro DCA_LOG to log things in mpi concurrency tests
<jbjnr>
weilewei: no. that's just a custom debugging dump that I added, it's safe
<jbjnr>
it just used std::cout to dump messages
<jbjnr>
^uses
<jbjnr>
doesn't depend on agas::logging
<weilewei>
jbjnr ok, I see, I will try build hpx with HPX_WITH_LOGGING OFF
<jbjnr>
jaafar: the latest work on my scheduler has two thread scheduling options one is round_robin the other is thread_parent
<jbjnr>
when thread_parent is used a new thread is scheduled on the same core the parent thread ran on
<aserio>
jbjnr: We switched these meetings to be bi-weekly
<aserio>
jbjnr: the next meeting is scheduled for October 28th
<jbjnr>
but it is done at the executor level rather than the task level, so a schedulehint needs to be generated in the dataflow continuation
<jbjnr>
aserio: thanks
<jbjnr>
will add to calendar
<jbjnr>
weilewei: when is next dca video meeting
<jbjnr>
?
<weilewei>
jbjnr next Monday 10/28, 11:00 AM Eastern Time
<jbjnr>
hmmm. I'm away on 28th actually
<jbjnr>
^^arrghh. both meetings!
<jbjnr>
weilewei: when you finish today, push what you've done to a new weilewei-hpx branch and tomorrow I'll try building dca with hpx from it and see if I can fix anything you haven't fnished
<weilewei>
jbjnr Thanks! I will let you know once I finish and push all the changes
<jbjnr>
ok. but send me an email, cos once I get home, I can't log into this machine and see mesages any more (must fix that)
<weilewei>
jbjnr sure, I found your cscs email on my email, so I will send you an email about everything
<jbjnr>
ta
<weilewei>
welcome :-)
<zao>
You do have the logs in the channel topic, when really desperate?
<jbjnr>
I know, but I never remember any more to check them. we use slack for everything here and I forget IRC exists most of the time now
Guest70891 has quit [Ping timeout: 265 seconds]
Guest70891 has joined #ste||ar
aserio has quit [Quit: aserio]
<weilewei>
jbjnr after applying HPX_WITH_LOGGING OFF, I did not see the logging error but got a linking error
<weilewei>
[ 55%] Linking CXX executable mpi_concurrency_testCMakeFiles/mpi_concurrency_test.dir/mpi_concurrency_test.cpp.o:(.data+0x0): undefined reference to hpx::hpx_check_version_1u_4u' CMakeFiles/mpi_concurrency_test.dir/mpi_concurrency_test.cpp.o:(.data+0x8): undefined reference to hpx::hpx_check_boost_version_106800'