hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
<heller_>
so far, they are offering us one base server (t1.small.x86) and free access to their spot market (with lowest priority). We could then combine buildbot_travis and pycicle to drive builds on those servers (with the hope of high reliability) and switch to a weekly/monthly schedule for tests on daint and other larger resources with less reliable availabilty
<heller_>
what do you think?
ste||ar-github has joined #ste||ar
<ste||ar-github>
[hpx] PixelOfDeath opened pull request #3543: added Ubuntu dependency list to readme (master...readme_add_ubuntu_dependenc_list) https://github.com/STEllAR-GROUP/hpx/pull/3543
ste||ar-github has left #ste||ar [#ste||ar]
quaz0r has quit [Ping timeout: 245 seconds]
quaz0r has joined #ste||ar
hkaiser has joined #ste||ar
hkaiser has quit [Quit: bye]
nanashi55 has quit [Ping timeout: 252 seconds]
nanashi55 has joined #ste||ar
david_pfander has joined #ste||ar
<simbergm>
heller_: sounds cool, but I have a feeling it won't be enough
<simbergm>
we should already be testing more on each PR (although we could be more efficient about it)
<simbergm>
paying is I guess still an option...
<heller_>
Yes
<simbergm>
Atom C2550 :)
<heller_>
The thing I'm unhappy with picycle is that resources are too unreliable
<heller_>
Yes, look up the other options to which we would have access
<heller_>
Through the spot market...
<simbergm>
ok, so what would that mean? whenever they have free instances we get some time there with lowest priority?
<heller_>
And yes, we need faster turnaround times...
<heller_>
Yes
<heller_>
Lowest priority means, that whenever a paying customer comes around, we get suspended
<heller_>
scheduling could then look like this: 1) If we have tasks in the pending queue, run them (no concurrency here) 2) no tasks in the pending queue, try to get some from our staged queue (will need synchronization), if nothing is in our staged queue, steal from neighbors.
<heller_>
the tasks only get a stack when put from staged->run. The stack is being recycled onces the task terminated
<heller_>
`run -> suspended` and `run -> pending` can only be done on `this_thread`. `suspended -> pending` is done from another thread
<heller_>
there needs to be additional state to also support timed suspension and take into account that we can have two reasons for `suspended -> pending`, one is the timeout, the other is the resume due to a condition.
<heller_>
a task goes into the depleted state, if there is another task that might still has a handle (either the timer or the 'condtional' resumer). But this reference counting is again just a thread local operation without any additional synchronization required
<heller_>
in any case ... the review for allscale is next week on friday, then i'll work almost full time on this
<simbergm>
ok
<simbergm>
good luck for that!
<heller_>
simbergm: what I always envision is to run a similar scheduling loop on GPUs ;)
<simbergm>
we'll have the hpx call on the 27th as well, that'll be a good opportunity to discuss
<heller_>
yes
<simbergm>
some day :)
<heller_>
with the exception that we currently can't suspend/resume tasks on GPUs, but that shouldn't a big deal for the algorithms to run on GPUs anyway
<simbergm>
I don't know if you were informed about this, but I did a visit to sandia
<simbergm>
it's slow, but working on setting up some more apps/benchmarks
<heller_>
ok
<heller_>
kokkos should benefit greatly from those things
<heller_>
as it's primary use case is data parallelism
<simbergm>
yes and no, performance wise they can do much better because it's just fork-join
<simbergm>
but they're interested in looking at what we have and experimenting with something like futures in their api
<heller_>
right
<heller_>
there's nothing prevent us from getting better at fork-join as well ...
<heller_>
and I think this is the right step in that direction
<heller_>
in the fork-join model, having futures is a total overkill, for example
<heller_>
unless you do a asynchronous fork-join
<heller_>
simbergm: the problem there however is, that our tasking already comes with soo much overhead, we'll always loose against this simple data parallel fork-join model
<heller_>
at least on fine granularities
<jbjnr__>
heller_: simbergm I've just seen all this writing but cannot read it because I have to go to the other building. Can you create an issue with these ideas in so we can read it properly, comment and put together a plan between us.
<jbjnr__>
(I mean the scheduler clean up stuff)
<heller_>
jbjnr__: I will very soon (promise)
<simbergm>
probably not a bad idea
<heller_>
yeah
<heller_>
I am planning on doing a series of blog posts on that topic...
<simbergm>
ooh, nice
<simbergm>
which reminds me, I said I would do some... :/
<simbergm>
whew, I posted one as well but it's gone again now :)
<simbergm>
(on reddit that is)
<simbergm>
thanks hkaiser
<hkaiser>
simbergm: ohh sorry
<simbergm>
upvoted
<simbergm>
no problem
<hkaiser>
didn't know you already did this
<simbergm>
I was just being slow, you reminded me
<simbergm>
I registered after I saw your isocpp post
<simbergm>
next time I'll really post the link to reddit, now I already have an account ;)
<hkaiser>
heh
<heller_>
hkaiser: hey! How is the demo going?
<heller_>
(and the rest of course)
parsa has joined #ste||ar
<hkaiser>
heller_: running
<hkaiser>
heller_: it's not really selfexplanatory, though
nikunj has joined #ste||ar
david_pfander has quit [Ping timeout: 245 seconds]
hkaiser has quit [Quit: bye]
<zao>
simbergm: I wonder about the reddit announcement, is it technically possible to have the first blurb and/or the highlights inline in the post, to maybe foster a bit of discussion?
<zao>
Oh, wait, it was hkaiser that posted it :)
<simbergm>
zao: that's a good point, but I think not officially
<simbergm>
they have the option of submitting a "post", "image or video" or a "link"
<zao>
(I of course don't know the capabilities of reddit, nor the rules of the subreddit ^^)
<simbergm>
and with the link option there's no place to put a longer description ;?
<zao>
I see.
<simbergm>
hmm, ;? was not what I was going for but maybe it fits
<zao>
Evolutionary emoticonology.
<simbergm>
I guess we could put a link into a post, but I don't know if that's the right way to do it (I don't really know the etiquette on reddit either)
<zao>
Ditto, just idly musing.
jaafar has quit [Quit: Konversation terminated!]
preejackie has joined #ste||ar
mdiers_ has quit [Remote host closed the connection]
mdiers_ has joined #ste||ar
preejackie has left #ste||ar ["WeeChat 1.9.1"]
parsa has quit [Quit: Zzzzzzzzzzzz]
parsa has joined #ste||ar
preejackie has joined #ste||ar
preejackie has quit [Read error: Connection reset by peer]
parsa has quit [Client Quit]
parsa has joined #ste||ar
parsa has quit [Quit: Zzzzzzzzzzzz]
jaafar has joined #ste||ar
parsa has joined #ste||ar
nanashi55 has quit [Ping timeout: 276 seconds]
nanashi55 has joined #ste||ar
parsa has quit [Quit: Zzzzzzzzzzzz]
<heller_>
simbergm: btw, the deadline for the scheduler things to get into master is january ;)
<heller_>
as I have to give a talk on it mid february ;)
<simbergm>
I like that :P
<simbergm>
what is it for? it's not your defense, is it...? (I thought you might have done that already)