aserio changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
quaz0r has joined #ste||ar
bikineev_ has quit [Remote host closed the connection]
Matombo has quit [Remote host closed the connection]
EverYoun_ has quit [Ping timeout: 276 seconds]
hkaiser has quit [Quit: bye]
zbyerly_ has joined #ste||ar
ajaivgeorge has joined #ste||ar
ajaivgeorge is now known as ajaivgeorge_
zbyerly_ has quit [Remote host closed the connection]
zbyerly_ has joined #ste||ar
mars0000 has joined #ste||ar
K-ballo has quit [Quit: K-ballo]
vamatya has joined #ste||ar
zbyerly_ has quit [Ping timeout: 246 seconds]
ajaivgeorge_ has quit [Ping timeout: 260 seconds]
ajaivgeorge_ has joined #ste||ar
mars0000 has quit [Ping timeout: 240 seconds]
vamatya has quit [Ping timeout: 240 seconds]
jbjnr has quit [Quit: ChatZilla 0.9.93 [Firefox 54.0.1/20170628075643]]
Matombo has joined #ste||ar
jbjnr has joined #ste||ar
mcopik has joined #ste||ar
Matombo has quit [Remote host closed the connection]
<jbjnr>
hkaiser: mauro asked me about some std:P:future paper from std presented today/yesterday - mostly it looked reasonable.
<heller>
jbjnr: this is template meta programming unleashed
<jbjnr>
^ignore the :P: above
<hkaiser>
jbjnr: yah, lot'sa movement on the 'future' end
<jbjnr>
I will read the metaclasses paper
<hkaiser>
nothing too different from what we have so far, more convenience features....
<jbjnr>
I like the idea of future<int, int> instead of future<tuple<>>
<heller>
jbjnr: if you are of the kind of people thinking that template metaprogramming is total nuts, you will loose your mind on that
<hkaiser>
jbjnr: yah
<jbjnr>
I like metaprogramming in general
<hkaiser>
you need structured bindings for this to really work
<zao>
I'm not sure I trust the C++ community with a tool of this magnitude.
<zao>
Template Haskell was bad enough.
<hkaiser>
zao: what's another turing complete layer amongst friends?
<heller>
zao: and you are totally right with that verdict.
<jbjnr>
only the way libraries like HPX have template dependencies on may things and it actually makes the code harder to read instead of easier
<jbjnr>
^many
<heller>
well, this will make it worse ;)
<jbjnr>
well if it makes i more powerful, I'll have to cope
<heller>
it makes it natural to use
<heller>
it will also improve error reporting drastically
<hkaiser>
heller: if done right
<heller>
sure
<jbjnr>
hkaiser: I'm testing the new threads stuff at the moment. Hoping to get new matix graphs out later today
<hkaiser>
jbjnr: nice
<jbjnr>
might have finally tracked down all the performance problems
<hkaiser>
very nice
<jbjnr>
f@#$ing ironic that they were caused by hpx itself being retarded!
pree has joined #ste||ar
<hkaiser>
jbjnr: sure, did you expect something else?
<jbjnr>
did get some very interesting results from my modified scheduler, and now I know why you did thingy the way you did with the HP queues
<jbjnr>
might have to document my findins somewhere
<hkaiser>
that might become helpful, yes
<hkaiser>
write a paper
<jbjnr>
"scheduling strategins in task based runtimes"
<jbjnr>
^strategies
<heller>
yes!
<jbjnr>
just a generic this works and this doesn't paper
<heller>
well, reviewers will destroy you
<jbjnr>
why?
<hkaiser>
everybody believes they know better
<hkaiser>
even if they don't know anything
<heller>
since they mostly think about complex scheduling strategies and planning, like they do in starpu
<hkaiser>
that as well
<jbjnr>
I wouldn't want to say this is better, or that is better - just that for this application, this works, and this doesn't, but for this ....
<jbjnr>
and put lots of graphs and explanations down so that some poor loser like me doesn't have to relearn it all
<zao>
CMake needs a separate linking target. I can't link examples with more than -j2, while compiles can go -j8 or -j16.
<zao>
Great fun that a 32G machine swaps on building HPX :P
<heller>
metaclasses will fix this!
<heller>
;)
<jbjnr>
lol
<hkaiser>
heller: I now strongly believe that co_await will transform everything we do
<heller>
hkaiser: absolutely
<hkaiser>
no need for things like .then(), when_all(), dataflow() anymore, at least mostly
<heller>
didn't we know that as well?
<hkaiser>
well, not to that extend, I guess
<heller>
why?
<hkaiser>
shrug, I guess I didn't think about it sufficiently hard before, so the implications were not fully clear
<heller>
well, it makes futurization approachable
<hkaiser>
absolutely
<heller>
i think .then, when_all and dataflow are more or less equivalent to co_await
<heller>
what I am not sure about is how to deal with the use cases supported by when_any
<hkaiser>
right
<hkaiser>
also, we need to think about cancellation
<hkaiser>
that has far-reaching consequences as well
<heller>
yeah
<heller>
btw, I have a name for HPX2: DACT, with a robot dactylus as a logo, and the slogan: "Slaying the problems of tomorrow with ideas from the past using modern C++"
<hkaiser>
jbjnr: I think Mauro saw yesterday that our stuff is not that exotic as he believed before
<heller>
DACT: Distributed Asynchronous C++ Tasking
<hkaiser>
heller: lol - you have too much time at your hands
<hkaiser>
heller: what if you looked at some PRs in that case ? ;)
<heller>
or just get back to writing...
<github>
[hpx] hkaiser pushed 1 new commit to throw_with_info: https://git.io/vQ17r
<github>
hpx/throw_with_info 65cbc13 Hartmut Kaiser: Merge branch 'master' into throw_with_info
<Vir>
hkaiser, wash: you're both at the meeting? Apparently LEWG will look at the datapar paper first (or second) thing in the morning. wg21.link/p0350 might also get a view from LEWG, would you be willing to present this? It's basically what you implemented in HPX, i.e. the datapar execution policy
<hkaiser>
Vir: I have a phone call this morning, so can't go there right away...
* Vir
had to stay at home to build a house :-) but I'll be available on Skype
<Vir>
hkaiser: OK, Jens will present P0214, that'll take a bit of time
<hkaiser>
VeXocide: don't want to promise to be able to present this :/
<zao>
*Vir
<Vir>
:-) sure np
<hkaiser>
zao: yah, thanks
<Vir>
I'll handle it via Skype if Jefferey lets me. It's simple enough.
<hkaiser>
k
<hkaiser>
might be the easiest - I'll try to be in the room, though
<Vir>
hkaiser: good, then I'll mention that you have the implementation experience ;-)
<hkaiser>
k
<heller>
does anyone know if there is a DOI of pats thesis?
<heller>
got it
ajaivgeorge_ has quit [Ping timeout: 260 seconds]
ajaivgeorge_ has joined #ste||ar
bikineev has joined #ste||ar
<jbjnr>
heller: HPX2 : DACT. your image of a dinosaur slaying stuff is perfect. I just came from a chat downstairs with Victor who said. I imagined that using HPX would be like riding a high speed horse off across the sands into the sunset - to which I added, instead you found that it's more like carrying a donkey on your back ....
<jbjnr>
I'm in the middle of some testing. I'll rebase my local branch and push later
<jbjnr>
still got a race at start/stop time which causes segfaults sometimes
<jbjnr>
and not done the perf counters either. got to get matrix stuff going
<jbjnr>
(going properly I mean, not the usual shite we do though)
pree has joined #ste||ar
<hkaiser>
jbjnr: yah, I'd appreciate if this didn't end up like the other 13-15 half-ready branches you have in the repo...
<jbjnr>
oooh!
pree has left #ste||ar [#ste||ar]
<jbjnr>
please note that we didn't push this to the hpx repo, just so we could avoid snarky coments like that one and instead wait until the code was ready!
<jbjnr>
code of conduct!
<hkaiser>
jbjnr: hrm, sorry
<jbjnr>
no need
<jbjnr>
I'm enjoying this tremendously!
<hkaiser>
didn't mean to be offensive
pree has joined #ste||ar
Matombo has joined #ste||ar
<jbjnr>
I'd better delete all those branches and go work on some project where they want my help!
<hkaiser>
go for it ;)
pree has quit [Ping timeout: 240 seconds]
Matombo has quit [Ping timeout: 240 seconds]
<denis_blank>
hkaiser: Did you see my PM?
<hkaiser>
denis_blank: just now
pree has joined #ste||ar
Matombo has joined #ste||ar
<diehlpk_work>
hkaiser, I found the issues with the docker image and sent you an mail
<hkaiser>
diehlpk_work: yah, saw the email, but I don't know what to do about this
<diehlpk_work>
hkaiser, I think we have to generate a new docker image for hpx
<hkaiser>
ok
<heller>
diehlpk_work: you should figure out what happens when you do this apt-get stuff
<hkaiser>
I have no idea how ot do that
<diehlpk_work>
It seems that the boost lib there is too old an my apt-get update sees that boost on ubuntu is 1.6.2 default
<zao>
what kind of hosts and images is this on?
<hkaiser>
so you're suggesting to update our circleci testing to rely on boost 1.62
<heller>
diehlpk_work: why do you do an update then?
<heller>
does it work without?
<zao>
and ooh, what compiler is the metaclasses on? clang?
<heller>
zao: yes
<diehlpk_work>
heller, yes without the update it can not find some packages
<zao>
what's our boost minspec now?
<heller>
well, feel free to update it (through a PR)
<K-ballo>
minispec?
<diehlpk_work>
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
<zao>
sounds more like missing pkgs than a version thing at first glance
<K-ballo>
oh, 1.51 I think
<diehlpk_work>
In the hpx dev image we use 1.55
<heller>
diehlpk_work: so any road block to update it?
<diehlpk_work>
heller, I think we just have to generate the docker image again and push it to docker
<diehlpk_work>
So we will habe boost 1.62 there to
pree has quit [Read error: Connection reset by peer]
<heller>
jbjnr: do we know when we get more time on the daint HPX project?
<jbjnr>
?
<jbjnr>
quota used up?
<jbjnr>
PS. didn't hkaiser tell you - I don't work on hpx any more!
<zao>
I hear that jbjnr has exceeded his stale branch quota :P
<jbjnr>
lol
<heller>
jbjnr: yeah, since a while now
<heller>
jbjnr: oh, didn't notice you left ;)
<jbjnr>
it's new news. not to be confused with #fakenews
<hkaiser>
#jbjnrnews, then
<zao>
DACT makes me think of DECT, but otherwise I cannot mangle it into something perverted :)
<hkaiser>
zao: the target of this is perverted enough without this ;)
zbyerly_ has joined #ste||ar
pree has joined #ste||ar
<diehlpk_work>
heller, Actual clang version is 4.0 when we will update
<heller>
Sounds good to me
mars0000 has joined #ste||ar
denis_blank has joined #ste||ar
bibek_desktop has quit [Ping timeout: 246 seconds]
pree has quit [Read error: Connection reset by peer]
<denis_blank>
Is there a std::disjunction like trait in the codebase already, which checks whether one element in a variadic pack satisfies the condition given by a given trait?
<hkaiser>
denis_blank: any_of?
<hkaiser>
all_of?
<hkaiser>
yah, util::detail::any_of
<K-ballo>
there should be something in util/pack
<denis_blank>
The actual issue for me is that all_of is eager so it performs unnecessary evaluation calls to the trait
<K-ballo>
denis_blank: unnecessary, but incorrect?
<K-ballo>
eager is usually faster/lighter on compilation time than lazy
<K-ballo>
though I have a short-circuiting implementation that's still flat, if we have a use case for it I can update all_of/any_of
<denis_blank>
Mh ok, I'll use the eager one, however, I would still prefer a trait which accepts a predicate.
<K-ballo>
it does accept a predicate, doesn't it?
<denis_blank>
No it doesn't. However it calls ::value for non bool values. But this requires the trait to be instantiated already and then it's too late. So the instantiation of the trait should be delayed until the value is requested.
<K-ballo>
but that's what a predicate it
<K-ballo>
*is
<K-ballo>
do you mean a lazy short-circuiting implementation, like std::disjunction?
<K-ballo>
I can give you such an implementation, without changing the interface at all... the question is, is it worth it?
<K-ballo>
it costs more... the flat version doesn't cost as much as std::disjunction is doomed to cost, but still
<denis_blank>
Mh ok... actually no... it should be more like a static_all_of<std::is_void, void, int, bool> -> std::false_type
<denis_blank>
Otherwise I'm required to do something like this: all_of<std::is_void<void>, std::is_void<int>, std::is_void<bool>>
<K-ballo>
mmh, I'm not sure what you mean... could you produce some code, or be explicit about the motivation?
<K-ballo>
yeah, even ::value
<K-ballo>
the most effective way for that would be all_of<pack_c<bool, is_void<X>::value, is_void<Y>::value, ...>
<K-ballo>
the one with the best compilation times, and then eager vs lazy wouldn't matter anyhow
<denis_blank>
However, the difference is that the first example delays the specialization of std::is_void until the value is really requested whereas the second one does a wasted specilization of the last type...
<denis_blank>
Probably it's really minor anyway...
<K-ballo>
yes, that's true, but it's faster!
<K-ballo>
that's why I keep asking if it's merely unnecessary, or incorrect
<denis_blank>
It should be correct
<denis_blank>
So it's just that it may performs unnecessary specializations
<K-ballo>
would you do some benchmarks please, and measure all_of<pack_c<bool, is_void<X>::value, is_void<Y>::value, ...>> vs std::disjunction<is_void<X>, is_void<Y>, ...> ?
<github>
[hpx] hkaiser created remove_make_index_pack_unrolled (+1 new commit): https://git.io/vQMaD
<github>
hpx/remove_make_index_pack_unrolled c960e41 Hartmut Kaiser: Removing make_index_pack_unrolled as it has not proven to be faster than the default make_index_pack implementation
pree has joined #ste||ar
pree has quit [Read error: Connection reset by peer]
<denis_blank>
K-ballo: probably there won't be much difference between both of those versions, however maybe we could receive a benefit from static_all_of<std::is_void, void, int, bool>... but I don't think this is really worth an improvements for now
<K-ballo>
oh there's a substantial difference
<K-ballo>
std::disjunction has the worst interface ever, mandating a recursive implementation
<K-ballo>
recursive implementations lead to long symbol names, accumulated
EverYoung has joined #ste||ar
<K-ballo>
that's the weak spot in today's compilers
<K-ballo>
we are talking linear vs quadratic wrt name mangling, which nowadays happens everywhere since mangled names is what's used as instantiation keys in modern compilers
<denis_blank>
Mh ok, probably you are right then, thanks for the explanation
pree has joined #ste||ar
<K-ballo>
this is an area of particular interest to me, I've done my homework ;)
zbyerly_ has quit [Remote host closed the connection]
<heller>
Vir: do we have a performing knl version now?
<Vir>
heller: dunno, codegen certainly got better from what I've seen. But GCC still needs some hand-holding regarding inlining
<heller>
Ok
<heller>
So David didn't really finish it off?
<Vir>
heller: I recommend to work with native_datapar instead of fixed_size_datapar, though
<heller>
Ok
<heller>
Thanks
<Vir>
he's apparently still on it, just posted a question to gcc-help that I answered :-)
<heller>
Ahh, great
<Vir>
oh and he found that uninitialized arrays of datapars were zero-initialized. I fixed that
<heller>
I'm more or less useless for the next few weeks... Just want to keep pushing from time to time...
<Vir>
well, I'll be rather useless the next months... building a house
<heller>
Have fun!
<Vir>
I will
<heller>
I've been through that already ;)
<Vir>
:)
<github>
[hpx] hkaiser created simple_base_lco (+1 new commit): https://git.io/vQMxn
<github>
hpx/simple_base_lco ae34321 Hartmut Kaiser: Allowing for LCOs to be simple components...
<heller>
Well, didn't build it, just restored an old one
<Vir>
well, just the "Innenausbau". It's getting built in the next 3 days
<heller>
Nice
<heller>
Still no easy task
<heller>
Doing the wiring was the most fun ;)
<Vir>
no, and it's crazy how many errors those people do. I already noticed so many things that I told them to fix
<github>
[hpx] hkaiser opened pull request #2756: Allowing for LCOs to be simple components (master...simple_base_lco) https://git.io/vQMxM
<Vir>
wiring starts in two days
<heller>
Good luck then!
<heller>
Cat 7?
<heller>
I hope!
<heller>
Anyway, gtg
<Vir>
cu
<Vir>
electricity first
taeguk has joined #ste||ar
<github>
[hpx] hkaiser pushed 1 new commit to simple_base_lco: https://git.io/vQMh4
<github>
hpx/simple_base_lco 4fad8c7 Hartmut Kaiser: Converted lcos::channel to be a simple_component...
bikineev has quit [Ping timeout: 260 seconds]
mars0000 has joined #ste||ar
Matombo has quit [Remote host closed the connection]
taeguk has quit [Quit: Page closed]
mcopik has quit [Ping timeout: 276 seconds]
<hkaiser>
parsa[[w]]: yt?
denis_blank has joined #ste||ar
mars0000 has quit [Quit: mars0000]
ajaivgeorge has quit [Ping timeout: 260 seconds]
pree has quit [Ping timeout: 240 seconds]
<hkaiser>
Vir: could you use a new nicely distinguishable version whenever you rename datar to simd?
<Vir>
hkaiser: I can bump the dev version number, yes
<hkaiser>
tks
<hkaiser>
that will simplify the migration for us
ajaivgeorge has joined #ste||ar
zbyerly_ has quit [Remote host closed the connection]
zbyerly_ has joined #ste||ar
<hkaiser>
Vir: the other thing I always forget was that P0350 should not only propose execution::datapar but also execution::dataseq or something
<Vir>
hkaiser: what does that mean? simd<T, scalar>?
mars0000 has joined #ste||ar
<hkaiser>
no simd<T> but run on one thread, while datapar should additionally parallelize
<Vir>
hkaiser: ah, I meant execution::datapar to mean single-threaded
<hkaiser>
yes
<hkaiser>
ahh
<hkaiser>
ok, then we implemented it differently ;)
<hkaiser>
we have both, btw
<Vir>
I believe there are few cases where you want to give up control over what is vectorized and what is threaded
<hkaiser>
why so?
<Vir>
right, it's certainly not wrong to have it. The main reason is locality. I want fine-grained chunking for SIMD units and large-grain chunking for different cores/nodes
<Vir>
So I'd always have an outer loop that does threading and an inner loop that does SIMD
<Vir>
Of course, the parallel algorithms implementation of a combined threading + simd<T> should do that by itself
<Vir>
but, at least in the codes I wrote, the parallelization I exploited went in different dimensions of the problem
<Vir>
i.e. one dimension exclusively for simd<T>
<hkaiser>
Vir: makes sense - having it doesn't hurt, though
pree has joined #ste||ar
<Vir>
hkaiser: it could hurt by making users think less about partitioning the problem space than they should. Still I think it's right to have this convenience when you just don't care enough (or really don't have to care)
<aserio>
hkaiser: yt?
eschnett has quit [Ping timeout: 240 seconds]
hkaiser has quit [Quit: bye]
aserio1 has joined #ste||ar
aserio has quit [Ping timeout: 255 seconds]
aserio1 is now known as aserio
hkaiser has joined #ste||ar
<hkaiser>
aserio: here
<aserio>
hey do you want to do the Phylanx meeting on Monday Tuesday , Tuesday-wed
<hkaiser>
aserio: I'd like to wait for Chris to respond
<hkaiser>
let's wait for another day or two
<aserio>
ok
<aserio>
But what I would like to know is what your preference is if Chris is ok with either
<aserio>
The other option would be to have a three day meeting... but that might be overkill
<hkaiser>
aserio: I don't care ;)
<hkaiser>
yah, I agree
<hkaiser>
Tue-Wed would be good as this wouldn't involve weekend travel for kevin
<aserio>
That's what is optimal... but that would mean we loose 3hrs on Wed.
<aserio>
perhaps we could work that in by having a breakout session then
<hkaiser>
Wed afternoon, right?
<aserio>
yes
<hkaiser>
nod, second day afternoon people tend to become tired anyways
bikineev has joined #ste||ar
<hkaiser>
we could explicitly make it a 1.5 day meeting
<hkaiser>
people could fly back Wed, then
<aserio>
I will decide later I suppose
<hkaiser>
k
zbyerly_ has quit [Remote host closed the connection]
zbyerly_ has joined #ste||ar
<hkaiser>
aserio: see pm, pls
<github>
[hpx] biddisco closed pull request #2656: Reduce MAX_TERMINATED_THREADS default, improve memory use on manycore… (master...terminated_threads) https://git.io/vHCmV
<github>
[hpx] biddisco deleted terminated_threads at 150c584: https://git.io/vQD3I
<github>
[hpx] biddisco closed pull request #2602: Changes to scheduler to steal from one high-priority queue (master...shared_priority_scheduler) https://git.io/v9nTB
<github>
[hpx] biddisco closed pull request #2629: Move unordered_map out of parcelport into hpx/concurrent (master...concurrent_map) https://git.io/v95EI
<github>
[hpx] biddisco closed pull request #2454: Fix a couple of warnings and compiler errors (master...fix_compilation) https://git.io/vMwkD
<github>
[hpx] biddisco deleted fix_compilation at 89cb53b: https://git.io/vQD3g
<K-ballo>
woa
<hkaiser>
K-ballo: yah, jbjnr decided to stop working with hpx...
<hkaiser>
;)
<hkaiser>
jbjnr: thanks for cleaning up things!
<zao>
Was going to say "spring cleaning" but somehow it's already July.