hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoD: https://developers.google.com/season-of-docs/
K-ballo has quit [Quit: K-ballo]
hkaiser has quit [Quit: bye]
<nikunj> zao: I'm still trying to make things work here :3
<nikunj> I did set up a completely new vm with Ubuntu 18.04 so that I can install the same set of libraries with exactly same versions.
<nikunj> I may then be able to get to the root of the problem
Coldblackice_ has joined #ste||ar
Coldblackice has quit [Ping timeout: 268 seconds]
Coldblackice has joined #ste||ar
Coldblackice_ has quit [Ping timeout: 268 seconds]
nikunj97 has joined #ste||ar
nikunj has quit [Read error: Connection reset by peer]
rori has joined #ste||ar
K-ballo has joined #ste||ar
<nikunj97> let's continue where I left off
<zao> The above was from building everything I could explicitly with C++14, most things as master.
<zao> (on Arch)
hkaiser has joined #ste||ar
Yorlik has quit [Read error: Connection reset by peer]
<nikunj97> zao: I've lost hopes on arch
<nikunj97> trying everything on ubuntu now
<nikunj97> arch is bleeding edge. Packages tend to break sometimes. Maybe something is wrong inherently with some package that leads to such errors
<zao> :D
<nikunj97> for example: clang 9.0 installs llvm libs for 8.0 xD
<nikunj97> you will have to install llvm libs 9.0 separately to get things working
<nikunj97> let's hope ubuntu doesn't disappoint. It's day 4 and I'm yet to reproduce the error I'm expected to fix :D
nikunj has joined #ste||ar
nikunj97 has quit [Ping timeout: 240 seconds]
<nikunj> installing hpx now
<zao> :D
maxwellr96 has quit [Read error: Connection reset by peer]
diehlpk_work has quit [Remote host closed the connection]
<zao> If you've got any silly questions, I'm happy to give silly answers.
rtohid_ has joined #ste||ar
<nikunj> zao: hpx never failed me once :D
<nikunj> but I have a feeling that I will surely come up with some silly questions today as well
rtohid has quit [Ping timeout: 250 seconds]
<zao> fwiw for arch, it's healthy to occasionally do system upgrades with `pacman -Syu` to ensure that packages that might need reinstallation are actually reinstalled.
<nikunj> zao: yes! I do it every week or so
<nikunj> I need better machines it seems
<nikunj> I'll switch to rostam if it starts taking longer
<nikunj> hkaiser: is there a phylanx wiki by any chance?
<nikunj> never mind, found it
<zao> For reference (may be incorrect), these are the command lines I used to build all the junk last night - https://gist.github.com/zao/4d1cdd634682862dccbe612cf40720bb
hkaiser has quit [Quit: bye]
<nikunj> finally on installing phylanx again
<nikunj> let's hope things don't end up weirdly this time
<nikunj> everything is on C++14
<nikunj> and verbose output looks decent as of now
<K-ballo> nikunj: what are you working on?
<nikunj> K-ballo: the linker error related to __wrap_main that exposed itself a few days back
<nikunj> we need to get that fixed
<nikunj> K-ballo: talking about phylanx here ^^
<K-ballo> yep, got it
aserio has joined #ste||ar
hkaiser has joined #ste||ar
aserio1 has joined #ste||ar
aserio has quit [Ping timeout: 276 seconds]
aserio1 is now known as aserio
<simbergm> nikunj: I still think you just need to use hpx_setup_target instead of linking to the hpx target in phylanx... that's why I was asking about what you were working on before
<simbergm> you could also use the docker container that they use for phylanx ci
<hkaiser> simbergm: well that's the whole point of him looking into things
<hkaiser> that docker image is broken
<hkaiser> could be ok now, however - after your (and rori's) latest changes
<zao> There's a lot of signedness errors in this codebase, hoo boy.
<zao> (ir::range::size() returns ptrdiff and is compared a lot with size_t)
<simbergm> hkaiser: stellargroup/phylanx_base:prerequisites is not broken is it?
<hkaiser> zao: yah, needs some cleanup - I hope that the recent changes to the hpx buildsystem preventing warning related flags to be propagated will help
<simbergm> nah, the wrapping won't be fixed with her changes
<hkaiser> simbergm: right, that one should be ok
<hkaiser> simbergm: also, could we merge #4160, please?
<simbergm> hkaiser: yes, forgot to do so, go for it!
<hkaiser> thanks, done
rori has quit [Ping timeout: 246 seconds]
diehlpk_work has joined #ste||ar
<nikunj> ok strange
<nikunj> the whole branch just built to a 100%
<nikunj> hkaiser, simbergm: did you make any changes?
<simbergm> nikunj try running a test, I think they fail at runtime now
<nikunj> simbergm: alright let me try that
zbyerly has quit [Remote host closed the connection]
<nikunj> simbergm: yes, all the tests failed
<nikunj> simbergm: everything fails since the files with hpx_main.hpp does not link to hpx_wrap.a
<nikunj> there is no mention of hpx_wrap.a anywhere
<simbergm> nikunj runtime not initialized?
<simbergm> Can you check in the cmake files if the tests get created with add_hpx_executable or if they're set up with hpx_setup_target?
<nikunj> turns out that the tests fail since there is no files available in bin. let me make tests first
<zao> I love that trap :)
<hkaiser> simbergm: it has hpx_init and hpx listed in target_link_libraries
<hkaiser> simbergm: should it have hpx_wrap as well (I think I tried that, however)?
<hkaiser> nikunj: is the -W wrap option is not there?
<hkaiser> now*
<nikunj> hkaiser: I don't see hpx_wrap.a and -wrap=main in the build output
<nikunj> this is definitely not the expected behavior. I'm not looking at the changes you made.
<hkaiser> nikunj: ok, shouldn't this fail linking, then?
<nikunj> so the thing is, either it should link with hpx_wrap and also pass the -wrap flag or do nothing
<nikunj> to build
<simbergm> It'll link just fine if both are missing
<nikunj> if the latter happens, then it will throw runtime errors
<nikunj> as simbergm identified
<simbergm> Please use hpx_setup_target, it'll do the right thing for executables and libraries
<nikunj> this is because the hpx_wrap simply shadows the variable defined within the hpx_main file
<simbergm> I know we broke HPX_LIBRARIES and I'm sorry about that but we want to get rid of those asap anyway
<nikunj> simbergm: yes, iirc I did end up making everything work with hpx_setup_target back in gsoc time
<nikunj> so simply using hpx_setup_target should also work
<nikunj> I did also add to HPX_LINK_LIBRARIES the required flags for pkgconfig
<nikunj> so using HPX_LINK_LIBRARIES should also have worked here, unless something happened to the HPX_LINK_LIBRARIES and HPX_LIBRARIES
<nikunj> hkaiser, simbergm ^^
<simbergm> Yep, so use that :)
<hkaiser> simbergm: what does setup_target do in this case?
aserio has quit [Ping timeout: 245 seconds]
<simbergm> Yep, that's the part we broke
<simbergm> hkaiser it'll link to hpx in all cases and add hpx_wrap/init and the correct flags for executables
<hkaiser> simbergm: I tried doing that, but couldn't make it work
<nikunj> also strangely, why is HPX_LINKER_FLAGS not returning anything?
<nikunj> did you remove that from hpx?
<nikunj> that contains the -wrap=main flag
<hkaiser> nikunj: should come to Phylanx through the HPX target
<hkaiser> or the hpx_wrap target, that is
<hkaiser> simbergm: the best would be to make hpx_init depend on hpx_wrap
<simbergm> No, I don't think they're set up for the hpx_wrap target
<hkaiser> ahh! that's the problem, then
<simbergm> They're set in hpx_setup_target
<nikunj> here you use HPX_LINKER_FLAGS. Ideally hpx should've passed down the value as -wrap=main
<hkaiser> simbergm: didn't you say we want to get rid of setup_target?
<nikunj> I remember adding that to the set of exported tags
<simbergm> I though we should but I think they'll be good to leave around for a bit longer
<nikunj> hkaiser: making hpx_init depend on hpx_wrap will make sure that hpx_wrap is linked as well. We do not want that to happen for libraries.
<hkaiser> simbergm: ok, so we need to fix this in the end anyways?
<hkaiser> nikunj: right
<simbergm> I still don't know a good way to set all the compile definitions based on the target names that the macros do
<simbergm> hkaiser yes, of course we want to make the targets work as well but that is a secondary goal when things don't build
<simbergm> all the macros are stil the preferred way to do things
<nikunj> simbergm: could you please tell what happened with hpx? I feel out of this loop
<nikunj> did we break the macros?
<nikunj> or did we break something in hpx related to the hpx_wrap work?
<simbergm> nikunj: this happened
<simbergm> essentially the internals of our build system changed significantly
<simbergm> in the process the HPX_LIBRARIES/HPX_INCLUDE_DIRS variables were removed (they are on they're way back)
<simbergm> the goal is to be able to do target_link_libraries(mytarget hpx) some day
<simbergm> the macros are the only thing that still works of the three potential ways to link to hpx
<simbergm> and the macros should still be used because they set options that we don't have a good way of setting yet through targets
<simbergm> https://github.com/STEllAR-GROUP/hpx/pull/4174 may get the variables working again
<nikunj> simbergm: I see
<nikunj> without the existing macros, phylanx will not build since it requires to set up the link libraries and linker flags as well
<nikunj> that is why hkaiser and I decided to export the flags during gsoc
<nikunj> simbergm, hkaiser: why did you not bring the HPX_LINKER_FLAGS back with https://github.com/STEllAR-GROUP/hpx/pull/4174
<nikunj> you remove it in 4028 and not bring back in 4174
<simbergm> nikunj: unintentional, they'll need come back as well
<simbergm> nikunj: the macros are there!
<nikunj> HPX_LINK_LIBRARIES needs to be back as well
<simbergm> nikunj: the variables are independent of the macros, they are two different ways of setting up targets
<nikunj> Since both were removed, neither hpx_wrap will link nor -wrap=main will be added. Maybe that's why phylanx built to completion
<simbergm> if you use the macros they will be set up with the new internal targets, with the variables you should be able to set things manually
<simbergm> the macros still work the way they used to even though the variables are gone
<nikunj> In phylanx we are currently making use of these variables
<nikunj> simbergm: correct, that's why hpx build correctly includes the link libraries and the linker flags
<simbergm> I'm aware, and I'm trying to ask you to use the macros instead
<nikunj> ohh ok. I finally get it
<nikunj> do we expose the hpx_setup_target?
<simbergm> yep, we do
<nikunj> simbergm: which file had the hpx_add_executable macro?
<nikunj> never mind, found it
<simbergm> nikunj: note that it should be in scope after find_package(HPX)
<simbergm> there's no hpx_wrap in the linux branch
<simbergm> also hkaiser, side question: I get that hpx_wrap might need to be a separate static library; why can't hpx_init be part of hpx?
aserio has joined #ste||ar
<zao> wheeee, `/root/stellar/inst-hpx-master-clang-c++14-debug/include/hpx/hpx_init_impl.hpp:217: undefined reference to `hpx_main(boost::program_options::variables_map&)'`
<zao> (after using hpx_setup_target in as many places as I could, but probably missed one)
<zao> Also hackfixed a bajillion sign mismatches :)
<hkaiser> simbergm: the source files of hpx_init probably could be part of the hpx nowadays, the includes might not, need to check
<hkaiser> simbergm: and also, why doesn't that Phylanx setup work ?
<simbergm> hkaiser gotcha, just curious
<simbergm> You mean the diff up there?
<hkaiser> yah
<simbergm> there's a phylanx_info where you'd want set(phylanx_libs ... hpx_wrap)
<nikunj> btw hkaiser, I think simbergm has a point. We can forward the parameters of phylanx_setup_target to hpx_setup_target and let that do the hpx related stuff instead
<hkaiser> simbergm: doh!
<simbergm> probably would've worked the first time without that ;)
<nikunj> simbergm: ohh yes! we're forgetting the hpx_wrap there
rori has joined #ste||ar
<nikunj> that is crucial
<hkaiser> simbergm: but not even the linker flags go through, and those are definitely set
<hkaiser> nikunj: try it
<nikunj> hkaiser: you forced pushed it
<simbergm> I think the linker flag went through the first time, that's why you got linker errors related to real_main or something
<hkaiser> could be :/
<simbergm> that said, I still recommend you use hpx_setup_target... :P
<nikunj> yes right, the first time I saw the logs, there was no hpx_wrap there
<hkaiser> don't remember, I tried so many things
<nikunj> hold on, there was hpx_wrap but no -wrap=main
<simbergm> the old commit is still on github... does github let you check out a commit without a name?
<zao> You can checkout anything with a name.
<zao> *you can name
<zao> git checkout commitish should be quite fine.
<simbergm> zao: well, I suppose I meant fetch or pull
<simbergm> a remote commit by hash
bibek has joined #ste||ar
<rori> I just did a PR to reestablished HPX_LIBRARIES and HPX_INCLUDE_DIRS
<rori> so that any other projects could still link to those
<zao> simbergm: you can do stuff like git fetch (--depth 1) origin commitish; git checkout FETCH_HEAD
<zao> (if you want to be a bit shallow, otherwise just skip the depth and fetch the whole history)
<nikunj> ok so, this commit which I require is no more in the tree somehow: f102cc0902c0da13c7f76948420253b627253ec9
<nikunj> zao ^^
<nikunj> trying to fetch the hash tells me that: 'error: server does not allow request from unadvertised object f102cc0902c0da13c7f76948420253b627253ec9'
<zao> How did you determine that it's the commit you want? The only reason for something to disappear is if it's been orphaned (via force-push/etc.) and garbage collection has happened.
<rori> The HPX_* cache variables weren't exported, but they are now in one of my PR feel free to merge it asap
<nikunj> well it was forced push yes
<zao> nikunj: The client seems to be saying that for any full-length hash that doesn't correspond to an object.
<nikunj> rori: thanks! That should be able to make things work for phylanx. Let me try to build from your PR.
<nikunj> rori: you're talking about 4165 right?
<zao> Oh, so it's a bit orphaned from being input to a merge and not rooted anywhere else, huh.
<simbergm> zao: it's been force-pushed over, was just curious if there's a way to get it even though it's not reachable since github clearly still knows about it
<zao> You can curl down a diff against master and apply that, I guess.
<nikunj> zao: did exactly that :D
<rori> sorry for breaking everything, I didn't know about the phylanx project and I should have tested it !
<zao> rori: Easy mistake to make thinking that HPX has no users you need to care about ;)
<nikunj> rori: I did exactly the same mistake about a year and half
<K-ballo> what's an "user"?
<nikunj> rori: in fact I remember breaking hpx master for a brief amount of time :D
<nikunj> hkaiser, simbergm, zao: just added the hpx_wrap to the linux portions of things. Let's see if it makes it work
<rori> Haha reassuring, sort of ..
<rori> yeah the HPX_LINK_FLAGS things were in the commits that are cherry picked from chinz07, I should have better reviewed things sorry
<rori> And so do you also need HPX_PKG_LIBRARIES and all configuration variants ?
<nikunj> not sure about HPX_PKG_LIBRARIES but HPX_LIBRARIES, HPX_LINK_LIBRARIES and HPX_LINKER_FLAGS are 3 major variables used by both other cmake configs and pkgconfig
<nikunj> at least until last year, not sure what changes were made to the cmake system this year
<rori> ok thanks !
rori has quit [Ping timeout: 276 seconds]
<simbergm> rori I think HPX_PKG_LIBRARIES was only used to generate pkgconfig files
<hkaiser> simbergm: we're seeing issues with compilation if DATAPAR_VC is enabled
<simbergm> hkaiser: what are the errors?
<simbergm> also, rori ^
<hkaiser> vc includes are missing on the compiler command line
<hkaiser> some examples fail compiling
<hkaiser> we ought to remove all of that vc stuff at some point, however
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
<simbergm> hkaiser: ah, damn
<simbergm> this should be public
<simbergm> rori's promised to go through the link keywords, there might be others that need to be changed
<simbergm> there are a lot of optional libraries which makes testing them quite slow :/
bibek has quit [Client Quit]
bibek has joined #ste||ar
nikunj has quit [Ping timeout: 240 seconds]
<hkaiser> simbergm: sure, no worries
bibek has quit [Client Quit]
<hkaiser> just giving you a hard time as you merged all of this on a Friday afternoon ;-)
bibek has joined #ste||ar
nikunj has joined #ste||ar
<simbergm> hkaiser: I know, I deserve it
bibek has quit [Client Quit]
<simbergm> my only defense is that we had to do it at some point, there's no way we would've caught all of these things before merging
<hkaiser> simbergm: I'm glad you're working on this, really
bibek has joined #ste||ar
<hkaiser> I'm not complaining
<hkaiser> all of this is long overdue and will have a major impact on everything
<hkaiser> simbergm: the work rori and you are doing will lift HPX into a new realm in terms of stability, maintainability, and certainly usability
<simbergm> hkaiser: no worries, I appreciate you bearing with this as well, otherwise we'd end up with all these issues in the next release
bibek has quit [Quit: Konversation terminated!]
<simbergm> btw, hkaiser, "we ought to remove all of that vc stuff"... replace it with new and shiny experimental standard library simd stuff?
<simbergm> hkaiser: see #4175, not tested or anything, but something like that should work
<nikunj> hkaiser: tried adding hpx_wrap where you forgot the last time. Seems like it still won't build the way it should
<nikunj> hkaiser: I'll continue on this tomorrow. I should have a fix by tomorrow. I'm sorry that it is taking this long to fix the issue :/
<simbergm> nikunj do you have logs?
<nikunj> simbergm: my vm stopped responding in between so I had to rerun the make from the middle. Here is the console output: https://gist.github.com/NK-Nikunj/25d9c155a82d946513c3f9c973c74df8
<nikunj> I still don't see hpx_wrap and -wrap=main for examples
<nikunj> I'm too tired to continue now :/
bibek has joined #ste||ar
<nikunj> I will continue with this tomorrow. I should be able to make things working again
<hkaiser> simbergm: this time they have a simd type in the pipeline that will go through - at least it looks like it
<hkaiser> so yah, we will replace it with whatever the standard decides to go with
<hkaiser> once the std libs have caught up
bibek has quit [Client Quit]
<hkaiser> simbergm: and thanks for the quick fix
bibek has joined #ste||ar
hkaiser has quit [Quit: bye]
<simbergm> hkaiser: nice, does that mean it would still go in C++20? or just be finalized-ish soon?
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
akheir has joined #ste||ar
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
aserio has quit [Ping timeout: 276 seconds]
bibek has quit [Read error: Connection reset by peer]
bibek has joined #ste||ar
hkaiser has joined #ste||ar
aserio has joined #ste||ar
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
bibek has quit [Client Quit]
bibek has joined #ste||ar
Coldblackice_ has joined #ste||ar
Coldblackice|2 has joined #ste||ar
Coldblackice has quit [Ping timeout: 264 seconds]
aserio has quit [Quit: aserio]
Coldblackice_ has quit [Ping timeout: 265 seconds]
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
weilewei has joined #ste||ar
bibek has quit [Quit: Konversation terminated!]
_bibek_ has joined #ste||ar
_bibek_ has quit [Client Quit]
_bibek_ has joined #ste||ar
_bibek_ has quit [Client Quit]
_bibek_ has joined #ste||ar
_bibek_ has quit [Quit: Konversation terminated!]
_bibek_ has joined #ste||ar
_bibek_ has quit [Client Quit]
_bibek_ has joined #ste||ar
_bibek_ has quit [Read error: Connection reset by peer]
bibek has joined #ste||ar
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
weilewei has quit [Remote host closed the connection]
nikunj has quit [Quit: Bye]
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
akheir has quit [Remote host closed the connection]
bibek has quit [Quit: Konversation terminated!]
bibek has joined #ste||ar
bibek has quit [Read error: Connection reset by peer]
_bibek_ has joined #ste||ar