K-ballo changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
bita has quit [Ping timeout: 256 seconds]
hkaiser has joined #ste||ar
K-ballo has quit [Quit: K-ballo]
nanmiao has quit [Quit: Connection closed]
hkaiser has quit [Quit: bye]
Coldblackice has joined #ste||ar
jehelset has joined #ste||ar
bita has joined #ste||ar
bita has quit [Ping timeout: 265 seconds]
peltonp1 has quit [Remote host closed the connection]
<gonidelis[m]> ms[m]: yt?
<ms[m]> here
<gonidelis[m]> do you know if i put a couple of doxygen comments here
<gonidelis[m]> where will they be rendered
<gonidelis[m]> and how
<gonidelis[m]> ?
<gonidelis[m]> how they will be rendered*
<gonidelis[m]> ms[m]: ^^
<ms[m]> just like the algorithms documentation
<ms[m]> you'll need to start the the comments with ///
<gonidelis[m]> is there any way to redirect from the doc-part to the corresponding source part?
<ms[m]> what do you mean? link to the source files?
<gonidelis[m]> yeah
<ms[m]> hmm, not really in the current setup, the source files are not rendered anywhere in the docs
<ms[m]> you can always link to github, but that's a bit nasty
<ms[m]> what are you thinking to link to exactly?
<gonidelis[m]> ok this is twofold for me. 1) I just want to see the corresponding source code that rendered that doc-block you just sent, that's just for me. but most important 2) i mayyybe think that a mechanism were docs are linked to source code in github would facilitate future developers (or even users)
<gonidelis[m]> but (2) might be a far-fetched
<gonidelis[m]> you are actually right. what do we link exactly...
<ms[m]> gonidelis[m]: yeah, I agree, it would be pretty cool to have that, it's just noone has looked into the options
<ms[m]> maybe something to add to the gsod projects (possibly in the API documentation project)?
<gonidelis[m]> ms[m]: that's what i thought
diehlpk_work has joined #ste||ar
diehlpk_work_ has quit [Ping timeout: 260 seconds]
<gonidelis[m]> <gnikunj[m] "(The one we were talking about i"> gnikunj[m]: hey! i am so sorry. just saw your question! i thought you were just giving me an update
<gonidelis[m]> gnikunj[m]: of course the answer is "definately!"
<gnikunj[m]> gonidelis[m]: sounds good! Let me update it with our names ;)
<gonidelis[m]> actually i cannot find anything on 2021
<gonidelis[m]> gnikunj[m]: ^^
<gnikunj[m]> (trying to find the link)
<gnikunj[m]> gonidelis[m]: ok, I updated the 2020 application: https://github.com/STEllAR-GROUP/hpx/wiki/GSoD-2020-Project-Ideas
<gnikunj[m]> most likely we're gonna update that to 2021
<gonidelis[m]> but where is 2021 anyway ;p ?
<gnikunj[m]> we should ask diehlpk_work and ms about this :P
<ms[m]> gnikunj: gonidelis[m] yeah, it needs to be updated
<ms[m]> would you like to do it? a copy might be good just to keep the old one around for archiving purposes
<ms[m]> (even though it's already been changed...)
<gnikunj[m]> ms: sure. I can add a 2021 page for it.
<gnikunj[m]> I know the changes I made so I can bring back the 2020 version to its original form and have the udpates reflect on 2021 page
<gonidelis[m]> gnikunj[m]: ms[m] ok lets do it
<gnikunj[m]> sounds good. On it!
<ms[m]> thanks guys!
peltonp1 has joined #ste||ar
<gonidelis[m]> gnikunj[m]: you could change 'a "build instructions" header that take you through the quick install' -> '...that take*s* you through...'
<gnikunj[m]> aah right, thanks!
<peltonp1> could someone assist me in linking to hpx with cmake
<peltonp1> I have a header only library which I create like this
<peltonp1> add_library(communication INTERFACE)
<peltonp1> target_include_directories(communication INTERFACE ${CMAKE_CURRENT_SOURCE_DIR})
<peltonp1> target_link_libraries(communication PUBLIC INTERFACE ${HPX_LIBRARIES})
<peltonp1> find_package(HPX REQUIRED)
<peltonp1> target_include_directories(communication PUBLIC INTERFACE ${HPX_INCLUDE_DIRS})
<gonidelis[m]> peltonp1: do you mind creating a git gist or sth just so we be able to read the code?
<peltonp1> I have it publicly available so I can share a link in a second
<gonidelis[m]> great
<ms[m]> peltonp1: if you're on a recent-ish HPX I recommend you just do target_link_libraries(communication INTERFACE HPX::hpx)
<ms[m]> note that if you have an interface library you also have to add include directories and link with INTERFACE
<peltonp1> there is the whole library, Im trying to set-up the lib/communication and interface it through tests in test/test_communication.cpp
<peltonp1> but I get a linker error from file lib/communication/hpx_md_communicatior.hpp (line 43 where an hpx::lcos::channel is created)
<gonidelis[m]> ms[m]: how could we put our emails on the mentors list by using raw.githubusercontent.com/ ?
<ms[m]> peltonp1: please post the full output in a gist or similar
<ms[m]> gonidelis[m]: which emails?
<gonidelis[m]> mine and nikunj's for gsod
<gonidelis[m]> i could pm them to you
<ms[m]> what happens if you try to drag and drop them into the wiki? tbh I think the images for emails are craziness :P but that's just me (spam filters work relatively well)
<gonidelis[m]> lol
<peltonp1> so if I put target_link_libraries(communication PUBLIC INTERFACE HPX::hpx) and when try to test the communication library I just link to communication I should be able to use all the hpx binaries both in test code and communication lib?
<peltonp1> I use the latest version of HPX
<gnikunj[m]> peltonp1: if communication is a library, yes.
<ms[m]> exactly
<ms[m]> note that HPX_LIBRARIES should contain HPX::hpx (and HPX_INCLUDE_DIRECTORIES) is empty, so if everything is set up correctly your current setup should be equivalent
<ms[m]> if it's not something is likely wrong in hpx or your setup
<ms[m]> but we will remove the variables eventually, HPX::hpx is the recommended way to link
<peltonp1> ok, I will test adding source files to communication and to actually create a library binary for that as well since Im doing all kinds of things in test to create the hpx_main which may not be correct
<gnikunj[m]> peltonp1: if you're using the hpx_main header, make sure to link to hpx::wrap_main as well
<peltonp1> thats not in HPX::hpx?
<gnikunj[m]> no that's a different library. Do target_link_libraries(<your target> PRIVATE HPX::hpx HPX::wrap_main <any of your libraries>)
<gnikunj[m]> ms: gonidelis[m] diehlpk_work I've updated the links to the organization application and project ideas (created new pages so that we can keep the old one's). I've improved the grammar of our application a bit. It looks good to me at this point. Please let me know if it works for you.
<gnikunj[m]> here's the link: https://github.com/STEllAR-GROUP/hpx/wiki
<ms[m]> gnikunj: excellent, thanks for doing that!
<gnikunj[m]> :D
<ms[m]> I'll be going through it properly in two weeks
<ms[m]> feel free to modify things as much as you want until then as well :P
<peltonp1> ok lib/communication alone compiles ok so my problem is linking the tests to communication, thanks for the help!
<gnikunj[m]> sounds good! I'll go through the application once more to have more stuff in there.
<gnikunj[m]> <peltonp1 "ok lib/communication alone compi"> nice!
K-ballo has joined #ste||ar
Coldblackice has quit [Ping timeout: 264 seconds]
<peltonp1> and the reason why I was not able to link the tests to communication was that I did not call the HPX_REGISTER_CHANNEL macros for the type that I was trying to open a channel for
hkaiser has joined #ste||ar
<ms[m]> it might also be time to finally just use spack also for the compilers and dependencies instead of just cmake/ninja
<ms[m]> I might play around with it again soon
<K-ballo> what is this spack?
<ms[m]> :P
<ms[m]> I keep mixing up my channels, I need to wake up
<ms[m]> but spack is (yet another) package manager
<gonidelis[m]> Are we talking hpx here ???
<gnikunj[m]> don't know the current state of spack, but my experiences from 2019 have been daunting to say the least :/
<gnikunj[m]> zao: is spack workable now?
<ms[m]> gonidelis[m]: yes and no, spack wouldn't be a dependency any more than vcpkg is a dependency right now, I was just thinking out loud (on the wrong channel) about building compilers and dependencies for ci with spack
<diehlpk_work> gnikunj[m], Please move the old GSoD projects and application ot the archive
<gnikunj[m]> diehlpk_work: already did. Does it not show there?
<diehlpk_work> and thanks for working on it
<diehlpk_work> Oh, sorry for my blindness
<diehlpk_work> The Euro-Par workshop webpage is available: https://amte2021.stellar-group.org/
<gnikunj[m]> diehlpk_work: do let me know if you want me to do anything else
<diehlpk_work> Spread the word and invite people to submit papers
<gnikunj[m]> <diehlpk_work "The Euro-Par workshop webpage is"> \o/
jehelset has quit [Ping timeout: 240 seconds]
nanmiao has joined #ste||ar
jehelset has joined #ste||ar
jehelset has quit [Ping timeout: 240 seconds]
<peltonp1> is there a reason why the partitioned_vector is not used in any of the stencil examples?
<gnikunj[m]> I believe it was used in stencil example 5 or 6
<hkaiser> peltonp1: the only reason would be that nobody has tried implementing it ;-)
<gnikunj[m]> but it lacks some features so it's replaced with serializable_buffer later
<peltonp1> if I call partitioned_vector[i+1] on a locality n and the data is actually at locality n+1, will I get a copy of the data?
<hkaiser> peltonp1: yes
<hkaiser> the global iterators don't give you references, even if it's local
<hkaiser> the local iterators do, however
<gnikunj[m]> hkaiser: are you free for a quick call?
<hkaiser> ure
<hkaiser> sure
<gnikunj[m]> gimme 2 min
bita has joined #ste||ar
bita has quit [Ping timeout: 264 seconds]
<zao> gnikunj[m]: don’t know, I don’t have any need for the competition.
<gnikunj[m]> :D
<tiagofg[m]> could you guys help me in one tiny question?
<tiagofg[m]> Why a hpx application running with mpirun synchronizes all localities in the end? I mean, this is a rule defined for mpi?
<tiagofg[m]> and if I run an application directly, to insure localities synchronizes in the end, I must put --hpx::localityes
<K-ballo> if you don't have --hpx:localities you only have the one
bita has joined #ste||ar
<tiagofg[m]> but with mpirun the localities are defined with "-np", right?
<K-ballo> with mpirun that's all handled for you
<K-ballo> how does your manual command line look?
<tiagofg[m]> that was what i wanted to know.
<tiagofg[m]> mpirun -np 2 ./program
<tiagofg[m]> I don't have to put hpx::localities because mpi handled that for me, ok
<K-ballo> `./program --nodes 2` I think, maybe more
<tiagofg[m]> yes of course
<K-ballo> by manual command line I meant the non-mpirun one
<tiagofg[m]> manually, in two separated consoles, something like that:
<tiagofg[m]> $ ./program --hpx:hpx=localhost:1337 --hpx:agas=localhost:1337 --hpx:expect-connecting-localities --hpx:localities=2
<tiagofg[m]> $ ./program --hpx:hpx=localhost:1338 --hpx:agas=localhost:1337 --hpx:worker
<tiagofg[m]> * manually, in two separated consoles, something like this:
<tiagofg[m]> $ ./program --hpx:hpx=localhost:1337 --hpx:agas=localhost:1337 --hpx:expect-connecting-localities --hpx:localities=2
<tiagofg[m]> $ ./program --hpx:hpx=localhost:1338 --hpx:agas=localhost:1337 --hpx:worker
<tiagofg[m]> so, I already knew that with mpi I don't need any of this, but are these settings hidden by hpx, when mpi is used?
<tiagofg[m]> * manually, in two separated consoles, something like this:
<tiagofg[m]> $ ./program --hpx:hpx=localhost:1337 --hpx:agas=localhost:1337 --hpx:localities=2
<tiagofg[m]> $ ./program --hpx:hpx=localhost:1338 --hpx:agas=localhost:1337 --hpx:worker
<tiagofg[m]> * so, I already knew that with mpi I don't need any of this, but are these settings hidden by hpx when mpi is used?
<tiagofg[m]> I mean, I don't need these settings because hpx hides it for me?
<tiagofg[m]> when using mpi
<hkaiser> yes, just use mpirun, all should be well
<ms[m]> tiago.fg: the only caveat is that hpx can't always detect that it's in an mpi environment (it looks at environment variables, and they can vary from implementation to implementation), but I think that's nowadays mostly a non-problem
<tiagofg[m]> ok, I wanted to know this to be clearer about how things work, thanks both
<jedi18[m]1> ms[m]: Thanks for adding the explanations! I have yet to read it, I started reading about tag invoke but then down a rabbit hole of articles on related concepts (ADL, CPO, tag dispatch, CRTP(not related I know but still)..), plus was a bit busy but now I have time so I intend to read and understand it properly now.
<tiagofg[m]> I remember other thing, to use mpirun to launch processes, only, I don't have to compile hpx with -DHPX_WITH_PARCELPORT_MPI=ON, right?
<hkaiser> jedi18[m]1: CRTP is related at least to our implementation of tag_invoke
<hkaiser> tiagofg[m]: yes and no
<hkaiser> yes because other wise MPI is not available, no because it will still try to run things, but it may fail depending on the situation
jehelset has joined #ste||ar
<tiagofg[m]> hkaiser: humm, so the right thing is if I want to use mpi then I must compile hpx with PARCELPORT_MPI=ON?
<hkaiser> tiagofg[m]: yes
<tiagofg[m]> hkaiser: okok
<hkaiser> otherwise HPX will not know anything about mpi
jehelset has quit [Remote host closed the connection]
weilewei has joined #ste||ar
diehlpk_work has quit [Quit: Leaving]
diehlpk_work has joined #ste||ar
weilewei has quit [Quit: Connection closed]
bita has quit [Ping timeout: 265 seconds]
<gonidelis[m]> hkaiser: any idea why the deprecated `remove` cannot call hpx::ranges::remove_if?
<gonidelis[m]> it says: ‘remove_if’ is not a member of ‘hpx::ranges’
<gonidelis[m]> actually this is the error
<gonidelis[m]> and it happens exactly here
<gonidelis[m]> i don't get it
<gonidelis[m]> K-ballo: any idea?
<K-ballo> it is deprecated
<gonidelis[m]> K-ballo: i know! but it hits on the lambda!
<K-ballo> the call expression spans several lines, one is the lambda
<gonidelis[m]> K-ballo: that happens even if I use detail::remove_if<FwdIter>().call() instead of remove_if
<gonidelis[m]> which (detail::remove_if....etc...) is not dperecated
<K-ballo> where's ‘typename hpx::parallel::util::detail::algorithm_result<ExPolicy, T>::type hpx::parallel::v1::remove(ExPolicy&&, FwdIter, FwdIter, const T&, Proj&&)’ ?
<gonidelis[m]> so it's like, just becaues i am using a lambda in a deprecated call the deprecation "recurs"?
<gonidelis[m]> exactly there?
<K-ballo> ah!
<K-ballo> the deprecated function complains about using itself then?
<gonidelis[m]> yes!
<gonidelis[m]> wtf
<K-ballo> btw you absolutely want to qualify those internal calls
<gonidelis[m]> 1. how is it using itself?
<gonidelis[m]> 2. why does it use it?
<gonidelis[m]> 3. how the hell could it not use it?
<gonidelis[m]> K-ballo: what do you mean?
<K-ballo> make a qualified call to remove_if
<gonidelis[m]> yy.... I tried using both hpx::remove_if and hpx::ranges::remove_if
<gonidelis[m]> it won't find any of the those two
<K-ballo> mmg which remove_if are you trying to call?
<gonidelis[m]> tjos
<gonidelis[m]> this^^
<gonidelis[m]> (i remove the projection)
<gonidelis[m]> or this
<gonidelis[m]> (i dont remove the proj)
<K-ballo> which one is that call actually targeting?
<gonidelis[m]> for the hpx::remove_if and hpx::ranges::remove_if correspondingly
<gonidelis[m]> the second (due to proj)
<K-ballo> then name it qualified
<K-ballo> if that doesn't compile, something else is wrong
<K-ballo> back to your warning, looks bogus, probably due to lambdas being dependent on the function in which they are declared
<K-ballo> which compiler complains, and which ones are fine with it?
<gonidelis[m]> it does not
<gonidelis[m]> gcc certainly complains
<gonidelis[m]> K-ballo: is it bad for a lambda to use its function's template used types
<gonidelis[m]> ?
<K-ballo> ?
<gonidelis[m]> why is it bogus?
<K-ballo> the warning
<gonidelis[m]> because it uses Type which by itself uses FwdIter>
<gonidelis[m]> ?
<K-ballo> looks like a bad warning, compiler flaw
<gonidelis[m]> did i just hack gcc?
<K-ballo> ?
<gonidelis[m]> you said my lambda is bad
<K-ballo> I didn't
<gonidelis[m]> you said it's bogus
<K-ballo> I said the warning looks bogus
<gonidelis[m]> sorry i misread that
<K-ballo> fwiw, you can improve on your implementation by decoupling the lambda from everything but the value type
<gonidelis[m]> so again: do i refer that to the gcc guys or sth?
<K-ballo> try other compilers, as well as newer versions of gcc, see what happens
<gonidelis[m]> hm... ok
<gonidelis[m]> i ll do
<K-ballo> if it's only gcc (which it should), and it hasn't been fixed yet, then report it "to the gcc guys" sure
<gonidelis[m]> <K-ballo "fwiw, you can improve on your im"> is it possible to get rid of Type in this example?
<K-ballo> Type?
<gonidelis[m]> the capture
<gonidelis[m]> Type const& a
<gonidelis[m]> std::iterator_traits<FwdIter>::value_type Type;
<gonidelis[m]> typedef
<K-ballo> the typedef, the capture, or the parameter? ...
<gonidelis[m]> The parameter
<K-ballo> no, a unary predicate must have a parameter
<gonidelis[m]> How do I decouple then
<K-ballo> decouple what?
<gonidelis[m]> <K-ballo "fwiw, you can improve on your im"> .
<K-ballo> form everything but the value type (aka Type)
<gonidelis[m]> I that this was the only coupling
<K-ballo> cosmetic nits: using instead of typedef, and lowercase name for types
<gonidelis[m]> Thought^^
<K-ballo> no, the lambda depends on the entire function
<K-ballo> you get a different lambda for each combination of function parameters, the lambda's "name" itself is hpx::....::remove<FwdIter, Pred, Etc>
<K-ballo> and I'd be surprised (only mildly) if there isn't already an equal_to predicate in hpx
<gonidelis[m]> So it's no mystery that the lands uses the deprecated remove
<K-ballo> the lambda doesn't call the function, it doesn't even name it
<K-ballo> the compiler must be getting confused
<gonidelis[m]> Thanks a lot for the Intel
<K-ballo> the compiler itself must be injecting the function name and then warning on that self-inflicted warning
<gonidelis[m]> I will consider these.fosry thing in the morning
<gonidelis[m]> I will maybe ping you again
<gonidelis[m]> Thanks ;)