aserio changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
EverYoun_ has joined #ste||ar
EverYoung has quit [Ping timeout: 265 seconds]
EverYoun_ has quit [Ping timeout: 255 seconds]
<zao>
Bah, accidentally blew away two build trees. Sleep it is :)
jbjnr_ has joined #ste||ar
jbjnr has quit [Ping timeout: 240 seconds]
EverYoung has joined #ste||ar
EverYoung has quit [Remote host closed the connection]
EverYoung has joined #ste||ar
diehlpk has joined #ste||ar
EverYoung has quit [Remote host closed the connection]
EverYoung has joined #ste||ar
diehlpk has quit [Ping timeout: 256 seconds]
EverYoung has quit [Remote host closed the connection]
EverYoung has joined #ste||ar
EverYoung has quit [Remote host closed the connection]
EverYoung has joined #ste||ar
<zao>
On 29 runs of the merge commit of the migration change, I get 21 passed, 7 timeout, 1 failed.
<Nikunj>
@hkaiser: About the issue #3124. Are we then considering my pull request where I build examples/hello_world_component without using the hello.sh commands?
hkaiser has joined #ste||ar
eschnett has joined #ste||ar
parsa[[[w]]] has joined #ste||ar
parsa[[w]] has quit [Ping timeout: 240 seconds]
santaman has joined #ste||ar
thomas_d has joined #ste||ar
<thomas_d>
hi
<heller_>
hello
<jbjnr_>
hello thomas_d - got a question?
<thomas_d>
I'm playing with some of the HPX examples, and I have a few questions indeed.. :)
<heller_>
the only way to get answers is to ask those questions ;)
<thomas_d>
in the futurization example from the docs, or the "1d_stencil_8.cpp", a config option "hpx.run_hpx_main!=1" is passed. But this actually means: "*do* run hpx-main on all localities", right?
santaman has quit [Quit: Page closed]
<hkaiser>
yes
<hkaiser>
the != does not mean 'not equal'
<hkaiser>
it means 'force the option'
<thomas_d>
yes, that confused me. Is that a "Boost options" thing?
<hkaiser>
no
<hkaiser>
just an internal thing that got a life of its own ;)
<thomas_d>
is it documented somewhere? couldn't find it immediately
<hkaiser>
thomas_d: probably not
<hkaiser>
thomas_d: the bottom line is that those cfg options can be specified on the command line
<thomas_d>
but there it's just "--hpx:run-hpx-main" right?
<hkaiser>
normally HPX rejects cfg settings that it does not 'know' about (doesn't have an explicit default setting)
<hkaiser>
that is to avoid typos
<hkaiser>
the != circumvents that test and accepts the setting no matter what
<hkaiser>
thomas_d: sure, one is the explicit command line option, but there is also the equivalent cfg database key
<thomas_d>
ah, yes, so in the ini file it's "hpx.run_hpx_main = 1"
<hkaiser>
yes
<hkaiser>
you can use any of the ini settings on the command line as well: --hpx:init=...
<hkaiser>
--hpx::ini=, even
<jbjnr_>
(thomas_d: I've been using hpx for years and I have never once actually used an ini file - don't worry too much about those cfg options)
<jbjnr_>
(command line works fine for me)
<thomas_d>
ok, thanks. I just wanted to make sure that I understood that part correctly, before asking anything more complicated...
<jbjnr_>
k
<Nikunj>
@hkaiser: In case we are working on CMakeLists.txt, I can propose 3 ways to implement the options that hello.sh runs.
<thomas_d>
another thing is the "receive_buffer" used in that example. I believe that's not documented either, right?
<jbjnr_>
correct.
<jbjnr_>
documentation is a weak point for hpx in general.
<Nikunj>
1) Since the hello.sh file is always contained in the examples/hello_world_component I can simply run the hello.sh from cmake by using chmod +x hello.sh and then running hello.sh
<hkaiser>
nikunj: yes
<thomas_d>
I'm thinking of writing a 2D heat transfer thing, as a kind of exercise, and the receive_buffer looks like a useful tool. I'll see if I manage to use it correctly based on example code and source.
<jbjnr_>
hkaiser: I'd write more documantation if we could do it in github markdown format instead of needing to use boost docbook or whatever it is. If we could put docs alongside the code in the same folder ...
<hkaiser>
thomas_d: use channels instead, they are much more useful
<thomas_d>
hkaiser: noted, thanks :)
<hkaiser>
jbjnr_: quickbook is almost the same as markdown
<jbjnr_>
also not documented
<jbjnr_>
^ channels I mean
<Nikunj>
2) Create another build.sh file that contains exactly the same code as hello.sh and then running it.
<jbjnr_>
hkaiser: the format isnb't the problem, its the viewing it -with github markdown, you can browse it in the repo
<Nikunj>
3) Directly hard coding the commands into cmake and linking it to an output file of hello_world_client and then running the hello_world_client when building
<jbjnr_>
just put .md files in there
<Nikunj>
@hkaiser: Which one should I prefer?
<hkaiser>
jbjnr_: feel free to write docs in .md, I'd be happy to integrate them into the overall docs
<hkaiser>
it's a nobrainer
<jbjnr_>
can we put the docs in the same folder as the code too -or must it go in separate docs folders?
<jbjnr_>
If I could do that, I'd put a resource_partitioner.md file alongside the header and drop docs in there, it'd be easy to navigate in my editor and I'd actually do it ...
<hkaiser>
well, having the .md files in the folders will not make them easy to find
<jbjnr_>
having docs in a separat place puts me off and makes me "do it tomorrrow ..." hence it never happens
<hkaiser>
nobody knows what folder has what
<hkaiser>
jbjnr_: come on, stop looking for excuses
<jbjnr_>
it's real though. I mean a real excuse. I don;t like writing docs, and I like it even less when I have to run 3rd party tools on them to see how they look.
<hkaiser>
nobody likes writing docs
<hkaiser>
and writing docs just for the sake of it, such that nobody can actually use/browse etc does not make them too useful
<heller_>
regarding docs: I think sphinx + readthedocs would be awesome ;)
<hkaiser>
I don't know of a good documentation system for c++
<hkaiser>
sadly
<hkaiser>
.md is fine to begin with, but people report that it quickly gets at its limits
<hkaiser>
sphinx is nice, but has no c++/doxygen integration
<heller_>
it does
<hkaiser>
so we're stuck with what we have...
<hkaiser>
does it now?
<heller_>
yeah
<hkaiser>
ok
<hkaiser>
didn't know that
<thomas_d>
And a more general question: MPI has "MPI_Cart_create()" and related functions to organise processes ("ranks") in a Cartesian grid, and inform each rank about the id of its grid neighbours. Do similar convenience functions exist to arrange localities in HPX? (the 1d_stencil examle seems to use register_with_basename/find_from_basename to help localities find eachother)
<hkaiser>
thomas_d: not directly
<hkaiser>
you have to organize things yourself - did you see the tutorial code?
<jbjnr_>
thomas_d: we are missing collective operations in hpx - so it's hard to go from an mpi example to an hpx one in most cases
<hkaiser>
thomas_d: in essence, you use register_with_basename to assign names to your grid patches, that can be used to build your communication channels
<hkaiser>
heller_: cool
<heller_>
if we have time eventually...
<hkaiser>
now just finding a way to convert our stuff ;)
<heller_>
I heard there is this GSOC thing :P
<jbjnr_>
heller_: ask simbergm do do it - he seems to enjoy pain.
<hkaiser>
they don't like having doc work
<hkaiser>
lol
<heller_>
just kidding ... gsoc is not supposed to do doc work
<heller_>
simbergm: hey, I heard you volunteer to convert our docs to sphinx and have it generated on readthedocs?
<jbjnr_>
we still have to write some though
<thomas_d>
to add my 2 cents on documentation, as an outsider: I find the existing manual very helpful, but indeed I have the impression that there is much more that is still undocumented.
<jbjnr_>
I want docs I can just look at without having to "publish" them :(
<jbjnr_>
readthdocs is no use to me when I'm writing them
<heller_>
you can generate them locally
<jbjnr_>
the thing I liked about doing our slides using github markdown was that I could write them and see them without having to "generate" anything using extra tools.
<simbergm>
jbjnr_: so you're calling me a masochist?
<jbjnr_>
yup.
<simbergm>
but honestly I'd be interested in using sphinx
<jbjnr_>
volunteering for the release job is proof
<heller_>
jbjnr_: yeah, that was nice indeed ...
<simbergm>
jbjnr_: is the problem that you don't like reading the quickbook markup?
<simbergm>
personally I would change to sphinx just for nicer formatting (themes)
<jbjnr_>
simbergm: it's not the reading the quickbook markup - it's having to run a tool to see the final result and having the docs far away from the code.
<simbergm>
far away from the code I agree
<simbergm>
but for any documentation that's in the code you'll have to run a tool anyway to extract it
<simbergm>
if you want it rendered nicely
<simbergm>
example code in the documentation that can be compiled would be amazing
<thomas_d>
thanks for your time, and until next time ;-)
thomas_d has left #ste||ar ["ERC (IRC client for Emacs 25.3.1)"]
<simbergm>
erc, should maybe try that out...
<heller_>
the current doc toolchain is really just a pain to setup
<simbergm>
yeep
<heller_>
and with readthedocs, we'd have the chance to actually inspect the docs of a PR
<simbergm>
how does that actually work, does it do essentially what circleci does but for docs?
<jbjnr_>
(simbergm: my plan was not to use a tool to generate the code -just put .md files besides the headers and let the user browse - I know. awful. don't tell me)
<heller_>
simbergm: yeah, you can setup github hooks. that being said, probably doesn't work for PRs
<jbjnr_>
<cough>pycicle?<cough>
pradeepisro49 has joined #ste||ar
<heller_>
jbjnr_: sure, an additional doc builder would work
<zao>
hkaiser: As you were offline, I'm seeing roughly equal amounts of failures on that test before merging in heller's code and after merging it.
<zao>
jbjnr_: fwiw, I'm annoyed whenever I touch Boost as they have all that useless crud interspersed in the source tree.
<jbjnr_>
we can't win whatever we do. I'll just do what everyone else decides - but very slowly.
<zao>
:D
<jbjnr_>
hkaiser: let me know if/when you might be free for a chat.
mcopik has joined #ste||ar
<hkaiser>
jbjnr_: would 10am/5pm work?
<hkaiser>
zao: thanks!
<jbjnr_>
might do. pencil me in, if I have to leve before then, I'll warn you. thanks
<hkaiser>
heller_: ^^
<jbjnr_>
^leave
<hkaiser>
jbjnr_: ok, sorry I have a meeting at 9, could be over earlier, though
<hkaiser>
gtg
<jbjnr_>
it's fine. any time this week would be ok
<jbjnr_>
I'll do 5pm
<jbjnr_>
(probably)
hkaiser has quit [Quit: bye]
<simbergm>
jbjnr_: so you complaining about the documentation was you gently saying that you don't want to write RP documentation...?
<jbjnr_>
not entirely
<jbjnr_>
I am doc-averse, but I don't like having to install other tools unless they're super easy and I never found got around to using boost docstuff
<jbjnr_>
^typo heaven
eschnett has quit [Quit: eschnett]
aserio has joined #ste||ar
<simbergm>
jbjnr_: I volunteer to check that your docs build if you write it
EverYoung has joined #ste||ar
diehlpk has joined #ste||ar
hkaiser has joined #ste||ar
EverYoung has quit [Remote host closed the connection]
<github>
hpx/master 6fbd8e1 Adrian Serio: Add Vcpkg to Release Proceedure
EverYoung has joined #ste||ar
EverYoung has quit [Ping timeout: 240 seconds]
diehlpk has quit [Ping timeout: 260 seconds]
diehlpk has joined #ste||ar
<jbjnr_>
sorry nikunj was on a call and have to leave now. will look at your PR later
<Nikunj>
jbjnr_: Ok :)
Nikunj has quit [Ping timeout: 260 seconds]
Nikunj has joined #ste||ar
pradeepisro49 has quit [Quit: Connection closed for inactivity]
aserio has quit [Ping timeout: 265 seconds]
<Zwei>
Hey, I'm currently reading "C++ concurrency in action", since I've realised that my parallel programming knowledge isn't quite up to scratch... I'm on chapter 4. After this book, is there any other resources which you guys recommend?
<zao>
The scars you get as you try to use it in practice are neat.
<Zwei>
heh...
<zao>
In my case, "I want to have background processing with priority of work that the GUI thread issues".
aserio has joined #ste||ar
<zao>
I think I've reinvented pretty much every mistake possible in my work queues.
<Zwei>
I feel like I should learn more about networking/TCP/IP stuff, it goes well with HPC stuff, or am I mistaken in this respect?
<zao>
For codes running on clusters, you tend to be either doing local thread-based parallelism or MPI for inter-node comms.
<zao>
The silly people doing cloud or distributed systems, those need more networking.
<Zwei>
ah
<Zwei>
thanks zao
<zao>
So while you need communication in HPC, it's more about figuring out on which node to compute stuff and where to send data, and less about the actual protocol.
<zao>
So a bit more algorithmically, if that makes sense.
<zao>
(some people are borderline insane and write their own code targetting InfiniBand like some fine people in here)
<Zwei>
Yeah, it does. Just seen a few job adverts for HPC stuff saying "knowledge of networking/TCP/IP is a plus".
<zao>
HPX is nicely abstracted on parcelports or whatever they call them, runs on top of TCP or MPI.
<Zwei>
I'll just learn it if I get bored, maybe take some cisco exam for fun.
<zao>
Zwei: If they involve cluster _admin_ stuff, then yes, networking competence helps, as nothing ever works.
<aserio>
hkaiser: Meeting time
<diehlpk>
aserio, Connected to the meeting
EverYoung has joined #ste||ar
EverYoun_ has joined #ste||ar
EverYoung has quit [Read error: Connection reset by peer]
EverYoun_ has quit [Remote host closed the connection]
EverYoung has quit [Remote host closed the connection]
EverYoung has joined #ste||ar
EverYoung has quit [Remote host closed the connection]
EverYoung has joined #ste||ar
aserio has quit [Ping timeout: 252 seconds]
EverYoun_ has joined #ste||ar
CaptainRubik_ has joined #ste||ar
EverYoung has quit [Ping timeout: 255 seconds]
jaafar has joined #ste||ar
<CaptainRubik_>
(Regd Concurrent Data Structure project) Hi guys. I read Michael's paper on Hazard pointers then read the implementation in libcds which uses atomics . I think it is fairly integrable (as far as I understood the code). Can someone guide me on writing a proposal for the project?
hkaiser has joined #ste||ar
hkaiser has quit [Client Quit]
hkaiser has joined #ste||ar
CaptainRubik_ has quit [Quit: Page closed]
diehlpk has joined #ste||ar
Shikhar has joined #ste||ar
aserio has joined #ste||ar
mcopik has quit [Remote host closed the connection]
mcopik has joined #ste||ar
mcopik has quit [Read error: Connection reset by peer]
<jbjnr_>
have a look at that and make a irst effort, then I'll read it and give you feedback
<jbjnr_>
for the projct proposal section - objectives, challenges, solutions, test cases to use to drive the development
<diehlpk>
CaptainRubik, Please great a Google Document and share it via the mailing list.
<jbjnr_>
in your case, a good test case would be to try replacing one of the lockfree queues in the scheduler with a new one from the libds and then do some timing to see if it makes any differnence
<diehlpk>
And also milestones and deliverable are important
Nikunj has joined #ste||ar
<diehlpk>
We like to know what we can expect from you before the evaluations
<jbjnr_>
or one of the maps we use in the thread queue - lots of locking going on in there and maybe a better map would help
<jbjnr_>
diehlpk: thanks
<diehlpk>
I like to see what you want to finish before each evaluation and the things you like to do should fit into the time slots
<jbjnr_>
for the libcds work - we want, a fork of libcds that comiles with hpx - so there's a lot of intefgration stuff to be done, on the build system and the testing as well as the code itself. Making the code nice without duplication, making the code as mainatainable as possible so that we can pull libcds updates easily and merge them in.
<jbjnr_>
Then choosing an algorithm that uses the concurrent stuff, integrating the libcds-hpx into it, testing, profiling and optimizing if necessary
<diehlpk>
CaptainRubik, Also let have your friends have a look on your proposal is might helpful
<jbjnr_>
use your imagination to think what would be useful, interesting and challenging.
<diehlpk>
And showing it to us early, so we can provide you with remarks and improve it
<jbjnr_>
yes
<jbjnr_>
it doesn't matter if version #1 is rubbish.
<heller_>
And keep in mind that you'll always overestimate yourself.
<heller_>
Listen when we say it's too much
kanav has joined #ste||ar
<heller_>
Think of how long it'll take for you to finish the task. Then multiply that time by π.
kanav has quit [Client Quit]
aserio has quit [Ping timeout: 240 seconds]
Nikunj has quit [Ping timeout: 260 seconds]
Smasher has joined #ste||ar
aserio has joined #ste||ar
captain_rubik has joined #ste||ar
<captain_rubik>
I appreciate your guidance guys. I will do these first thing in the morning.
<heller_>
Which time zone are you in?
<captain_rubik>
GMT+5:30 Indian Its 1:30 here
<captain_rubik>
am*
<jbjnr_>
up late then
<captain_rubik>
Yeah. This project is interesting for me. I like reading papers and implementing them , although I don't have much experience with open source but looking forward to learn. Thanks again.