aserio changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
EverYoun_ has joined #ste||ar
EverYoung has quit [Ping timeout: 240 seconds]
EverYoun_ has quit [Ping timeout: 250 seconds]
EverYoun_ has joined #ste||ar
eschnett_ has quit [Quit: eschnett_]
EverYoun_ has quit [Ping timeout: 250 seconds]
EverYoung has joined #ste||ar
EverYoung has quit [Ping timeout: 250 seconds]
eschnett has joined #ste||ar
parsa has joined #ste||ar
parsa has quit [Quit: Zzzzzzzzzzzz]
parsa has joined #ste||ar
parsa has quit [Client Quit]
parsa has joined #ste||ar
parsa has quit [Quit: Zzzzzzzzzzzz]
EverYoung has joined #ste||ar
parsa has joined #ste||ar
EverYoung has quit [Ping timeout: 250 seconds]
jaafar has quit [Ping timeout: 240 seconds]
jaafar has joined #ste||ar
eschnett has quit [Quit: eschnett]
parsa has quit [Quit: Zzzzzzzzzzzz]
eschnett has joined #ste||ar
parsa has joined #ste||ar
hkaiser has quit [Quit: bye]
parsa has quit [Quit: Zzzzzzzzzzzz]
parsa has joined #ste||ar
jaafar has quit [Ping timeout: 268 seconds]
eschnett has quit [Quit: eschnett]
parsa has quit [Quit: Zzzzzzzzzzzz]
parsa has joined #ste||ar
EverYoung has joined #ste||ar
parsa has quit [Quit: Zzzzzzzzzzzz]
parsa has joined #ste||ar
EverYoung has quit [Remote host closed the connection]
parsa has quit [Quit: Zzzzzzzzzzzz]
parsa has joined #ste||ar
EverYoung has joined #ste||ar
EverYoung has quit [Ping timeout: 240 seconds]
<github>
[hpx] sithhell force-pushed remove_component_factory from b16d786 to def8951: https://git.io/vF2gP
<github>
hpx/remove_component_factory def8951 Thomas Heller: Cleanup up and Fixing component creation and deletion...
<github>
[hpx] hkaiser closed pull request #2993: Making sure --hpx:help does not throw for required (but missing) arguments (master...fixing_2990) https://git.io/vF4Ll
<github>
[hpx] hkaiser opened pull request #3000: Adding hpx::util::optional as a first step to replace experimental::ooptional (master...optional) https://git.io/vFVpP
pree has quit [Read error: Connection reset by peer]
<heller>
looks like there is a problem with the new test here
<hkaiser>
heller: no
<hkaiser>
that's a know issue with clang
<hkaiser>
there is even a ticket for that
<heller>
hm, ok
<hkaiser>
see #2931
<heller>
ah, ok
<heller>
might take a look
<heller>
just checking since I'd like to merge the future_data changes
<hkaiser>
right
<heller>
about the for_loop changes in the clang_tidy branch
<heller>
I don't see a way to avoid the multiple copies as is
<hkaiser>
hmm
<heller>
would require more thinking and maybe changing the partitioner
<hkaiser>
let me have a look
<heller>
not to not block that PR further, I'd either revert to what we have on master or stick with the suboptimal solution for now
<heller>
and open a ticket
<hkaiser>
k
<hkaiser>
heller: why the rush?
<heller>
one PR less ;)
<hkaiser>
;)
<heller>
looks like we'll close in to go under three hours on circle ci soon
<heller>
I think the biggest problem we have here is the partitioned vector
<hkaiser>
could be
<heller>
it takes up an insane amount of memory to compile
<hkaiser>
I thikn we have a race in the new unwrapping code
<hkaiser>
when used with dataflow
<heller>
don't have the exact number now, but some of the tests go up higher than 6GB
<heller>
oh oh
<heller>
do you have a testcase or a wild guess?
<hkaiser>
no test case, but a failing app
<heller>
I couldn't see a problem so far
<hkaiser>
it's wild guess, could be a problem in the app itself - still investigating
<hkaiser>
just blows up inside the unwrapping
<hkaiser>
problem is that I don't understand the async unwrapping code :/
<heller>
I am having a hard time as well
<heller>
hkaiser: a blow up in unwrapping might mean that the underlying frame went out of scope
<heller>
not sure how that could happen though
<heller>
hkaiser: about the partitioned vector issue, could you think of a reason why we could not entirely type erase the whole partitiioned vector component stuff?
<hkaiser>
it's complaining about some iterator not being dereferencable
<hkaiser>
I think the async traversal is not thread-safe
<heller>
ok
<hkaiser>
need to talk to Denis
<heller>
so essentially the iterator pointing into the nirvana?
<heller>
could still be that the frame went out of scope too early, no?
<hkaiser>
doesn't really blow up, all memory seems to look ok
<K-ballo>
not thread-safe in which sense?
<hkaiser>
K-ballo: looks like that more than one future becomes ready at the same time in a dataflow(unwrapping(...), ...) causing the async unwrapping code to attempt to make progress concurrently and independently in more than one thread
<heller>
more or less just iterating over the given futures
<github>
[hpx] sithhell deleted future_data_void at 5c8cdaf: https://git.io/vFwGj
<hkaiser>
begin_ == sentinel_ (which was checked right before an attempt to incrementing begin_)
<hkaiser>
everything runs fine with on ethread
<heller>
hkaiser: could you try replace dataflow with when_each? when_each hasn't been ported to unwrapping yet, AFAICT
<hkaiser>
yah, wil experiment
<hkaiser>
first I'll remove unwrapping()
<hkaiser>
hold on, woul dthat happen for any dataflow?
<heller>
I can't even see where the new traversal code sets the completion handler :/
<hkaiser>
even without unwrapping?
<heller>
if you say it is a race, yeah
<hkaiser>
doh
<heller>
when doing unwrapping as the dataflow completion handler, there is no concurrency anymore
<heller>
all futures are ready
<hkaiser>
right
<heller>
so i'd say it is highly unlikely that there's a race in that scenario
<heller>
unless dataflow itself is racey
<hkaiser>
so it's just the dataflow
<hkaiser>
yah
<heller>
one bug I remember we had with the old dataflow was that there was a race between setting the completion handler to call the next element and actually advancing to the next, so a race between is_ready and set_on_completed
<hkaiser>
did we now?
<heller>
i can't even find the place where the new code is setting the completion handler
<hkaiser>
heller: yah, the async frame is not thread-safe, it's racy all over the place :/
jaafar has quit [Ping timeout: 260 seconds]
parsa has joined #ste||ar
<github>
[hpx] hkaiser opened pull request #3001: Attempting to avoid data races in async_traversal while evaluating dataflow() (master...fixing_dataflow) https://git.io/vFwCz
parsa has quit [Quit: Zzzzzzzzzzzz]
K-ballo has quit [Read error: Connection reset by peer]