hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
daissgr has joined #ste||ar
daissgr has quit [Ping timeout: 252 seconds]
hkaiser has quit [Quit: bye]
jbjnr has joined #ste||ar
daissgr has joined #ste||ar
david_pfander has joined #ste||ar
david_pfander has quit [Ping timeout: 250 seconds]
daissgr has quit [Ping timeout: 252 seconds]
daissgr has joined #ste||ar
<K-ballo> there's a number of uses of HPX_HAVE_CXX17_VARIABLE_TEMPLATES in code that should be CXX14
<K-ballo> seems the cmake code was adjusted but the cpp code kept using the old macro name
hello has joined #ste||ar
hkaiser has joined #ste||ar
<hkaiser> heller_: I know now what goes wrong with the migration test
<hkaiser> just don't know why yet :/
nikunj has joined #ste||ar
hello has quit [Ping timeout: 250 seconds]
<heller_> hkaiser: ok, what did you find out?
hello has joined #ste||ar
hello has quit [Ping timeout: 245 seconds]
diehlpk_mobile has joined #ste||ar
daissgr has quit [Quit: WeeChat 1.9.1]
bibek has quit [Remote host closed the connection]
time_ has joined #ste||ar
time_ is now known as hello
<hkaiser> heller_: there is an object that ends up in the was_object_migrated table even if the migration is done
<hkaiser> this causes route to reschedule another route operation
mbremer has joined #ste||ar
hello has quit [Ping timeout: 272 seconds]
<heller_> hkaiser: hmmm
<hkaiser> heller_: not clear why the migrated object is not taken off the was_migrated table (or why it gets re-inserted)
<heller_> the hang happens when we concurrently migrate and call actions on the component
<hkaiser> yes
<hkaiser> and only for the lazy tests, I think
<Yorlik> I keep hitting this linker bug (on windows) if I try to build with Ninja. I know it's not "officially" supported - still it should not explode like that ("too many sections in object file" when linking hpxd.dll), and I feel it's exposing a weakness in the build system which is not showing up when building with MSBuild. My impression is this bug or specific build issue shouldn't be there.
<hkaiser> Yorlik: missing /bigobj?
<Yorlik> Nope
<heller_> hkaiser: could be a problem with pinned_ptr
<Yorlik> There is one command where only the linker is called
<Yorlik> All objfile builds have /bigobj
<Yorlik> But this is a linker command
<K-ballo> is /bigobj needed during linking as well? that'd be surprising
<Yorlik> It should not and afaik there isn't even a possibility to do so
<Yorlik> I made a gitlab snippet which shows the problem here: https://gitlab.com/snippets/1808744
<K-ballo> is it possible to inspect the actual invocation issued by ninja? is the flag actually present?
<K-ballo> it's more likely that the build scripts somehow miss it
<Yorlik> I'll dig it up
<hkaiser> Yorlik: release? debug?
<Yorlik> Debug Buiuld
<Yorlik> The linker command actually is in the snippet
<Yorlik> If you view it in raw mode you get it with linebreaks
<Yorlik> Build directory was fully cleared ahead of the build attempt
<Yorlik> The Linker command lines of Ninja and MSBuild look quite different, they are even using different linkers.
<Yorlik> MSBuild is using the linker in ..bin\HostX86\x64\link.exe , while Ninja is uing the linker in ..bin\Hostx64\x64\link.exe
<K-ballo> it's possible to select the x64 host for msbuild too, should not be important, I use it on one of my stations
<Yorlik> Yes - I believe it's just a minor detail.
mbremer has quit [Quit: Leaving.]
<K-ballo> I guess I'll have to try a ninja build... Yorlik are you doing anything special besides -GNinja?
<Yorlik> I am running the build from Visual Studio
<Yorlik> Latest version
<Yorlik> It has cmake 3.12 and Ninja built in
<Yorlik> I am setting some variables before I do my "add_subdirectory" into the HPX build
<K-ballo> I guess it'll have to wait then
<Yorlik> I am setting this before: add_compile_definitions(HPX_WITH_BOOST_ALL_DYNAMIC_LINK BOOST_ALL_DYN_LINK BOOST_AUTO_LINK_SYSTEM )
<Yorlik> SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHsc" CACHE INTERNAL "")
<Yorlik> The /EHsc became necessary to eliminate a ton oif warnings
<Yorlik> I can try to run a plain /GNinja from external
hello has joined #ste||ar
<K-ballo> I'll try a plain ninja build just in case, it may reproduce the issue
hello has quit [Ping timeout: 250 seconds]
jbjnr has quit [Ping timeout: 252 seconds]
<K-ballo> > Architecture detected: x84
<K-ballo> go home cmake, you are drunk
<zao> :D
<K-ballo> a bunch of these: Command line warning D9025 : overriding '/Ob2' with '/Ob1'
<Yorlik> hkaiser mentioned that the /Ob2 -> /Ob1 thing is still necessary because of code generation bugs in cl iirc
<K-ballo> Yorlik: just build target hpx successfully
<K-ballo> which target were you having trouble with?
<Yorlik> hpx
<Yorlik> debug version
<Yorlik> using debug versions of boost and hwloc
<K-ballo> debug version as in -DCMAKE_BUILD_TYPE=Debug?
<Yorlik> Yes
<K-ballo> ok, let's try that
<Yorlik> And the debig version of the dependencies
diehlpk_mobile has quit [Quit: Yaaic - Yet another Android IRC client - http://www.yaaic.org]
<K-ballo> I've confirmed that the issue is reproducible with msbuild if I comment out the flag, even for the release dependencies
<Yorlik> I will try to build without visual studio from outside
<K-ballo> Yorlik: can you tell me what flags you see for the failing TU in build.ninja?
<Yorlik> I'm currently running a different build - I'll do a new one that fails and chack back with you
<K-ballo> debug build finished successfully too
<Yorlik> Sometime I wish I had a threadripper .... :)
<Yorlik> My machine is slammed - lol
<zao> Downside of Threadripper core counts is that you need infinite RAM to build HPX :)
<K-ballo> we should be doing slightly better now
<Yorlik> The HPX build system is a beast. I just got past the basics of moderb CMake, but HPX CMake makes ny head explode.
<K-ballo> I've been actively avoiding it for years, any command starting with hpx_ I just ignore, cmake is already messy enough
<hkaiser> K-ballo: nice catch, I must have been drunk that day myself ;-)
<hkaiser> Yorlik: do attempt to build HPX in 32-bit mode?
<Yorlik> LOL - no
<hkaiser> do *you
<hkaiser> ok, good
<Yorlik> That would be a hilarious source for that bug
<Yorlik> I have no problems building it with the MSBuild backend
<Yorlik> It's just that dreaded Ninja build
<K-ballo> ninja works fine, maybe it's a VS open cmake folder bug
<hkaiser> I have never been able to open the HPX cmake files using VS
<Yorlik> Alone MSBuild chosing a different linker than Ninja shows me, that some variables are obviously not controlled by the HPX Cmake scripts, but brobably be the system defaults or defaults of the resp. build tools.
<Yorlik> I am not directly opening the HPX folder in VS
<Yorlik> I have my superbuild and it tries to build HPX by add_subdirectory(<HPX Git>...)
<Yorlik> It first fetches the git and then includes it
<Yorlik> tagged 1.2.1-rc3
<Yorlik> That usually works nicely
<Yorlik> I mean with other projects
<K-ballo> that usually works terrible, unless the project takes special care to support that extraneous workflow
<K-ballo> it's a very popular workflow these days, but it goes against cmake philosphy
<Yorlik> To build HPX from my master CMake script I had to turn to calling Cmake with the MS backend and then MSBuild as an external procerss to build what CMake constructed for it - that works nicely
<Yorlik> Which workflow would you use in a superbuild?
<K-ballo> I just avoid superbuilds, to do them properly one would have to ExternalProject each dependency as well as the project itself, and forward all the right knobs
<K-ballo> the end result is something that breaks as soon as you change platforms
<Yorlik> I do not use externalproject at all
<K-ballo> I know, you are using the fetch+add workflow
<Yorlik> I first build a list with the subprojects I want to build
<Yorlik> Yes
<Yorlik> Because thatw ay with looping I can first build my source tree and then build in order
<zao> Please never make HPC software ;)
<Yorlik> HPC is nice I think.
<Yorlik> I am just currently fighting the VS debugger
<zao> (HPC as in high performance computing)
<K-ballo> as long as you are just abusing cmake to have a sort of script that builds your dependencies for you in your platform of choice, that's fine.. but don't expect a superbuild to behave as an actual proper cmake project
<Yorlik> It forces me to have all dlls inside the binary directory
<Yorlik> All of our own stuff is using targets and target focused commands only
<Yorlik> Automated finds and stuff
<Yorlik> But I'm pondering to take out all the dependencies and separate them from the master script - too much hassle
<K-ballo> zao here is afraid he might to one day build one of your projects in those systems of his
<zao> Exactly ^^
<Yorlik> We are working very modularized
<K-ballo> so if it's just your own internal thing for your own internal needs, then fine, you don't need a proper cmake project
<Yorlik> So we can always take out a portion and make a small nice CMake script for it
<Yorlik> The superbuild is a just a glue script
<Yorlik> Actually one of our requirements is, that all subprojects must build independently as standalone
<zao> (on clusters, we build libraries separately and pull them in via environment modules, and software that bundles/downloads/builds internal versions of things are a mess)
* zao shakes assorted fists at TensorFlow, OpenMolcas, PyTorch, and more :)
<Yorlik> It is importnat the subprojects work as standalones
<Yorlik> So - te superbuild is really just a convenience thing.
<Yorlik> I personally don't really like the term superbuild - sounds way too super to me
hello has joined #ste||ar
time_ has joined #ste||ar
hello has quit [Remote host closed the connection]
time_ is now known as hello
hello has quit [Ping timeout: 245 seconds]