hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoD: https://developers.google.com/season-of-docs/
<Yorlik> :)
hkaiser has quit [Quit: bye]
nikunj97 has joined #ste||ar
nikunj has quit [Ping timeout: 244 seconds]
david_pfander has quit [Quit: david_pfander]
david_pfander has joined #ste||ar
rori has joined #ste||ar
<jbjnr> millions of these errors /project/csstaff/biddisco/src/hpx/hpx/hpx_finalize.hpp:12:10: fatal error: hpx/errors.hpp: No such file or directory
hkaiser has joined #ste||ar
<hkaiser> simbergm: yt?
<hkaiser> simbergm: I'm seeing this exception being thrown for all applications during shutdown: https://github.com/STEllAR-GROUP/hpx/blob/master/src/runtime/threads/thread_pool_suspension_helpers.cpp#L142
<hkaiser> it gets swallowed however, as it's very late in the game
<hkaiser> simbergm: also, we're seeing occasional hangs on circleci now, not sure if that's related to the boost::spinlock changes introduced recently
<jbjnr> wiped all the build dirs, tried everything, but
<jbjnr> project/csstaff/biddisco/src/hpx/hpx/hpx_finalize.hpp:12:10: fatal error: hpx/errors.hpp: No such file or directory
<hkaiser> nikunj97: yt?
<hkaiser> jbjnr: works for me (tm)
<jbjnr> works for me on laptop, but not on daint
<simbergm> jbjnr: do you have libs/errors/include/hpx/errors.hpp in your build dir?
<simbergm> hkaiser: damn, thanks for letting me know
<simbergm> any other context for this? phylanx applications? I assume you're not actually calling suspend/resume explicitly?
<simbergm> windows and linux?
<jbjnr> simbergm: no. it does not get generated.
<simbergm> do you have any of the other generated headers e.g. libs/algorithms/include/hpx/algorithms.hpp?
<jbjnr> I wonder if its because I have networking off or something and so a dependency is missing
<simbergm> are you up to date with master?
<jbjnr> AFAICT all the others are generaed properly, just errors that is missing
<jbjnr> but I'm not sure how many others there should be
<simbergm> pretty much one per module
<simbergm> does libs/errors/CMakeLists.txt contain a GLOBAL_HEADER_GEN ON?
<jbjnr> yes
<hkaiser> simbergm: just run hello_world
<hkaiser> simbergm: and have not tried on linux - however, it looks like it's universal, FWICS
<jbjnr> I see. It's the module dependencies. Not looked at this stuff before, but hpx_thread_support does not depend on hpx_errors. Let me try adding it
<simbergm> hkaiser: excellent, thanks
<simbergm> this is new, right? visual studio just does something different to actually report the exception?
<simbergm> (I can reproduce it btw)
<hkaiser> simbergm: don't know what has introduced this, just running in the debugger breaks when exceptions are being thrown...
<hkaiser> good
<simbergm> probably my pr that changed that
<simbergm> it's just silent on pycicle and rostam
<hkaiser> yes, it's being swallowed
<simbergm> I'll do something about it
<hkaiser> thanks
<simbergm> thanks for letting me know
<hkaiser> simbergm: any ideas what's causing the hangs?
<jbjnr> simbergm: lib hpx_wrap uses errors, but I'm not sure where to specify the depndencies
<simbergm> any particular ones? I haven't noticed an increase but I haven't looked very closely
<hkaiser> simbergm: for instance https://circleci.com/gh/STEllAR-GROUP/hpx/119875
<hkaiser> I've seen others
<simbergm> ok, no clue :/
<simbergm> I think those have been problematic for a longer time
<hkaiser> have they? ok
<hkaiser> will watch the space
<jbjnr> hpx_wrap uses hpx_finalize, these are part of the main hpx library, but it isn't declared as a module, so I'm not sure where to add hpx_errors as a dependency
<hkaiser> jbjnr: the main library depends implicitly on all modules
<hkaiser> is your application failing to compile or is it HPX itself failing?
<jbjnr> hpx itself
<hkaiser> jbjnr: Auriane might know :/
<simbergm> jbjnr: can you try a standard configure (no HPX_WITH_NETWORKING=OFF) etc.?
<jbjnr> still waiting for cmake to complete
<jbjnr> daint is too slow, fixing stuff like this can tak hours :(
<simbergm> use apps, configure takes 30 secs or so
<rori> if still not working you can try configuring it in a clean docker image with the same cmake options you are currently using
<jbjnr> simbergm: I turned on timer_pool, IO_pool and networking but the errors.hpp file is still not generated.
<jbjnr> switching to /apps now in the hope the turnaround time improves
<simbergm> jbjnr: master? I don't see any reason why a clean configure should lead to this :/
<jbjnr> 8 minutes so far. still waiting :)
<jbjnr> apps doesn't seem any better
<jbjnr> 21 minutes, configure has finished. waiting for generate
<jbjnr> \o/
<simbergm> jbjnr: success? do you have anything on scratch? compiler? source files? 21 minutes is amazing...
<jbjnr> no. yay \o/ cos cmake completed generating
<jbjnr> ok, apps wins now. much faster second time.
<jbjnr> make -j8
<jbjnr> ^sorry
<jbjnr> master builds. something wrong with my branch it seems. a bad merge somwehere - PITA
<nikunj97> hkaiser: here
<nikunj97> hkaiser: Jackson wanted inverted log scales on x axis, but the points are already respresentative of inverted log (i.e. exponential)
<nikunj97> I can certainly round off the values for simplicity
<hkaiser> nikunj97: I have moved all of hpxr to a hpx_resiliency module (see https://github.com/STEllAR-GROUP/hpx/pull/4037)
<hkaiser> nikunj97: I have fixed a lot of things, bu there are some issues left mainly caused by us using c++14 features
<hkaiser> nikunj97: would you have time to look into fixing those?
<hkaiser> from what I see those can all be easily fixed (worked around)
<nikunj97> hkaiser: will look into it
<hkaiser> thanks! feel free to push directly to that branch
<nikunj97> alright
<nikunj97> what do you mean by "us using c++14 features"
<nikunj97> aren't we supposed to use them?
<hkaiser> well we used for instance [](auto&&) {} which is not a c++11 feature
<hkaiser> or auto return type deduction without trailing type specification etc.
<nikunj97> ohh so we need to make it c++11 compliant?
<hkaiser> HPX has to compile with only c++11
<hkaiser> yes
<nikunj97> gotcha
<nikunj97> are you free today at 10 your time for a call?
<nikunj97> had some final thoughts
<hkaiser> yes
<hkaiser> that should work
<nikunj97> great let's do a call then
hkaiser has quit [Quit: bye]
akheir has quit [Quit: Leaving]
K-ballo has quit [Quit: K-ballo]
K-ballo has joined #ste||ar
hkaiser has joined #ste||ar
lsl88 has quit [Ping timeout: 245 seconds]
lsl88 has joined #ste||ar
lsl88 has quit [Read error: Connection timed out]
lsl88 has joined #ste||ar
<Yorlik> hkaiser: yt?
<Yorlik> Quick question: Using a free function inside a namespace: Instead of Using HPX_DEFINE_PLAIN_ACTION inside the namespace and HPX_REGISTER_ACTION_DECLARATION outside of the namespace, would it suffice to use HPX_PLAIN_ACTION outside the namespace instead? E.G. HPX_PLAIN_ACTION(app::myfunc, myfunc_action) ?)
<Yorlik> I'm working on some documentation of the various use cases thats why I'm asking.
<hkaiser> Yorlik: you need the register_action in any case
<Yorlik> HPX_PLAIN_ACTION doc says it does the def+reg in one step
<hkaiser> the define_action defines the action type, the register_action defines all the boilerplate fro remote invocation
<hkaiser> ahh
<hkaiser> ok
<hkaiser> trust the docs!
<Yorlik> It looks as is HPX_PLAIN_ACTION is really just usable directly in the main app.
<Yorlik> Or inside a sourcefile
<Yorlik> But the action is essentially unusable from a header
<Yorlik> Because you quickly run into double definition errors when linking
<Yorlik> I just tested it
<hkaiser> you can use actions in a header just fine, all you need is the declaration
<Yorlik> There is a bunch of confusing stuff going on that really needs clarification in the docs.
<hkaiser> the definition should go into a source file for sure
<Yorlik> Yep
<Yorlik> HPX_PLAIN_ACTION can never go into a header.
<hkaiser> right
<Yorlik> How wouzld I do just an action declaration in a header then?
<Yorlik> SO - HPX_PLAIN_ACTION to the source - but what is in the header then?
<hkaiser> use the two macros as discussed previously
<hkaiser> not sure if there is a combined macro avaialble, need to check
<Yorlik> That's why I am documenting it - to shed some light on the confusion.
<Yorlik> I have cited the doxygen comment from the source there.
<Yorlik> for HPX_PLAIN_ACTION
<Yorlik> I remember months ago when I asked here in the channel it was confusing even for people familiar with HPX.
<Yorlik> I want to make this stuff really accessible. It just needs to be done once and right. :)
<hkaiser> Yorlik: I don't think you should document the macros, if you have things to add to the docs those should go directly into the doxygen comments (see https://stellar-group.github.io/hpx/docs/sphinx/branches/master/html/api.html?highlight=hpx_plain_action#c.HPX_DEFINE_PLAIN_ACTION)
<hkaiser> what you might want to focus on is not to answer the question "what happens when I do this", but to answer the opposite questions "What do I need to do to achieve this"
<Yorlik> I think some addition deocumentation is helpful - just to explain the underlying mechanics of: Declaration, Definition and Registration and what has to be done where.
<Yorlik> I agree we need more in the doxygen comments
<Yorlik> Some macros are entirely undocumented there.
<hkaiser> Yorlik: I'd prefer if you added the answers to the first question to the doxygen docs (i.e. source code) and link to those from your Cookbook
<Yorlik> But what I want to achieve is to get away from this "black magic" feeling where you just blindly follow a recipe without knowing what you are actually doing.
<hkaiser> but explaining the underlying mechanics is certainly fine on your end
<Yorlik> So - yes - I guess we need both - I might sort out stuff once I have understood it better myself.
<hkaiser> : please cross link with the existing docs
<Yorlik> For now I'm still trying to get my head around it myself.
<Yorlik> I will do!
<Yorlik> I am not trying to create an alternate documentation - that would just be confusing. I want to supplement the existing one.
<Yorlik> Really - I think we should have a voice session for the underlying concepts and how they are linked to the macros
rori has quit [Quit: bye]
vamatya has joined #ste||ar
<nikunj97> hkaiser: yt?
<hkaiser> here
<nikunj97> I've made some changes to the paper
<nikunj97> converted the graph to an exponential one and split the 1d stencil into 2 graphs for better understanding
<nikunj97> I'm free for a call if you wish
<hkaiser> yes, I'm online
nikunj97 has quit [Remote host closed the connection]
nikunj has joined #ste||ar
bita has joined #ste||ar
hkaiser has quit [Quit: bye]
bita has quit [Quit: Leaving]
<Yorlik> hkaiser: Did you have a chance to check out this problem with the vcpkg version of HPX?
<Yorlik> Added some explanation on the use of the free function / plain action macros. If anyone mind skimming over it really quick for gross mistakes? The code examples all compile and run so far: https://mckillroy.github.io/hpx_snippets/r_free_function_to_plain_action.html
hkaiser has joined #ste||ar
<Yorlik> hkaiser - just finished some plain action docs and example code
vamatya has quit [Remote host closed the connection]
<hkaiser> K-ballo: yt?
<hkaiser> Yorlik: show me!
<Yorlik> hkaiser: ^
<Yorlik> There is a little problem linking to the HPX Docs - I linked the macros to the source, but thats for sure not ideal.
<Yorlik> We really should have the Doxygen Docs online somewhere.
<hkaiser> Yorlik: could you link it to the docs instead of the source code?
<hkaiser> we do
<Yorlik> Which links? I don't think such links exist
<Yorlik> I used exactly these links
<Yorlik> But they will change with releases if anyone touches the file
<Yorlik> OK - I'll check that
<Yorlik> All for use caes have been compiled and run
<hkaiser> nice!
<Yorlik> err: all 4 cases
<Yorlik> Next step I think will be components
<Yorlik> Also the Macro naming is somewhat problematic
<Yorlik> Some are confusing
<hkaiser> let's make them more consistent
<Yorlik> We should do a talk - maybe after i did the components
<hkaiser> k
<Yorlik> My docs also need better structure - some stuff is redundant and messy.