nan11 has quit [Remote host closed the connection]
hkaiser has quit [Quit: bye]
bita has joined #ste||ar
bita has quit [Quit: Leaving]
nikunj has quit [Ping timeout: 240 seconds]
nikunj has joined #ste||ar
<zao>
zao dagur ~ stellar $ rm -rf ./*
<zao>
This wasn't my build directory...
<tarzeau>
i know this might be off-topic, but i can't find another cluster/hpc place, how do you handle low memory situations (linux oom), cgroups? systemd cgroups? cgroups2? oomd? nohang? earlyoom? something else
<zao>
tarzeau: On compute nodes we assign each allocated core a fixed fraction of the system RAM, leaving some headroom for system processes like CVMFS, Lustre, etc.
<zao>
It's enforced by cgroups.
<zao>
On login nodes we currently do not constrain the resources an user may consume, but it's been discussed for years whether we should bother as the login nodes occasionally are slow or killed by users.
<zao>
For batch jobs we've trained our users reasonably well to request enough cores to give them the amount of memory they need.
<ms[m]>
(it's one of the resiliency tests throwing a resiliency related exception)
<nikunj>
ms[m]: it's not something to worry about.
<nikunj>
The async replicate makes 3 replications and run them with the task at hand. If all 3 tasks fails, it returns a std::exception. I'll correct it such that it catches the error in those cases.
<nikunj>
It's essentially an expected but rare behavior
<ms[m]>
nikunj: 👍️ thanks!
hkaiser has joined #ste||ar
gonidelis has joined #ste||ar
gonidelis has quit [Remote host closed the connection]
<ms[m]>
hkaiser (and others): we'll have a very short kokkos meeting today
<hkaiser>
ms[m]: feel free to use the sam zoom link
<ms[m]>
we'll just do a 5 minute update at 15:55 (in 30 minutes) to those who are interested
<ms[m]>
or do you have anything new hkaiser?
<hkaiser>
ms[m]: I don't have any news, unfortunately
<hkaiser>
didn't have time
<diehlpk_work_>
ms[m], Since Rebecca will stop working on the documentation. Is there anything open?
<diehlpk_work_>
parsa, and I will meet with her tomorrow and discuss open items
akheir has joined #ste||ar
akheir has quit [Client Quit]
akheir has joined #ste||ar
rtohid has joined #ste||ar
<ms[m]>
diehlpk_work_: there are still the couple of prs open, but I can also clean them up (it's just rebasing and squashing)
nan11 has joined #ste||ar
bita has joined #ste||ar
<diehlpk_work_>
ms[m], Probably it will be easier if you do that
<ms[m]>
diehlpk_work_: will do
<diehlpk_work_>
ms[m], We have to work on the GSoD application as well
<diehlpk_work_>
I will try to work on it tomorrow
<ms[m]>
true... when was the deadline? I can look tomorrow as well
<diehlpk_work_>
May 4, 2020 at 20:00 UTC
weilewei has joined #ste||ar
<Yorlik>
I'm trying to build HPX on Linux and am getting a weird error: It's complaining not to find boost_filesystem (named libboost_filesystem_clang8_.... etc pp). It seems I'm running into these horrible naming issues again. What would I have to fix? On Windows I had no such issues.
<Yorlik>
So it finds boost but says it can't find specific libraries
<Yorlik>
(filesystem and iostreams)
<hkaiser>
Yorlik: c++14/c++17 issue
<hkaiser>
there is a mismatch between how you compile HPX and the application
<Yorlik>
Uh - wouzld I have to set the c++ standard to 14 for hpx and boost?
<hkaiser>
depends
<hkaiser>
you can do both, c++14 and c++17, just you'll have to do it consistently
<Yorlik>
I didn't change anything compared to windows where everything we compile is c++17
<Yorlik>
I'll double check.
<K-ballo>
did you set c++17 while building boost? more often than not on windows I forget
<K-ballo>
I just realized I forgot yesterday for my 1.73 build :/
<hkaiser>
Yorlik: the problem is that either hpx or your app decide to use std::filesystem, and one or the other does not find std::filesystem (c++14 mode?) in which case it falls back to Boost filesystem
<Yorlik>
We're using std::filesystem - but that problem comes before even compiling my app.
<Yorlik>
I'm just checking my b2 args
<hkaiser>
check whether you have -DHPX_WITH_CXX17=On
<Yorlik>
it on always
<Yorlik>
ON actually - that should do
<Yorlik>
But I also have -DHPX_FILESYSTEM_WITH_BOOST_FILESYSTEM_COMPATIBILITY=ON
<Yorlik>
Might have to ditch that
<Yorlik>
Why do I get this?? HPX_WITH_CXX17_FILESYSTEM - Failed
<Yorlik>
How is it testing this?
<Yorlik>
I set -DHPX_FILESYSTEM_WITH_BOOST_FILESYSTEM_COMPATIBILITY=OFF
<Yorlik>
I compiled boost with "cxxstd=17"
karame_ has joined #ste||ar
<Yorlik>
(b2 param)
<K-ballo>
it tests it by building a program with std::filesystem::path p = std::filesystem::current_path();
<Yorlik>
I can't understand why iot fails Clang8 should be OK, right?
<Yorlik>
and -DHPX_WITH_CXX17=ON
<K-ballo>
for me it builds but doesn't link, does clang8 require extra linking?
<K-ballo>
*it compiles but doesn't link, build fails
<K-ballo>
clang9 links fine, fwtw
<Yorlik>
Not sure - It's been age I built on Linux. I just remember building boost with clang on Windows always broke.
<K-ballo>
I don't know about libc++, but libstdc++ required an additional library to be linked until the abi was considered stable
<K-ballo>
there's some cmake system libraries that one's supposed to use for that
<Yorlik>
What a mess.
<K-ballo>
*cmake variable
<K-ballo>
CMAKE_<LANG>_STANDARD_LIBRARIES
<Yorlik>
libc++ is gss and libstdc++ llvm?
<Yorlik>
gcc
<K-ballo>
other way around
<Yorlik>
OK
<Yorlik>
Allright - redoing everything with clang 11
<Yorlik>
So that's what hpx has to say about Clang 11: "Use a compiler with support for the C++17"
<Yorlik>
... filesystem library ...
<K-ballo>
"the C++17", really?
<K-ballo>
reminds me of polish
<Yorlik>
I wonder if this c++lib issue plays in here
<Yorlik>
The filesystem fix is merged, isn't it?
<weilewei>
If we want to have a continuation after two futures together, is there any functionality in HPX? For attaching a continuation on a future, one can do future.then();
<weilewei>
is it dataflow?
<Yorlik>
Is it OK to manually set HPX_WITH_CXX17_FILESYSTEM
<K-ballo>
weilewei: I think you are talking when_all(f1, f2).then(...), dataflow is more generic and sometimes can be easier to work with
<weilewei>
K-ballo IC, thanks, let me play my code with when_all
<K-ballo>
if dataflow works just keep it
<Yorlik>
Which CMake Variables concerning Boost do you set when compiling HPX?
<Yorlik>
This whole boost compile and integrate with HPX situation is such a steaming pile of junk.
<Yorlik>
And I'm printing them rightbefore I send them to ExternalProject_Add(hpx
<Yorlik>
Especially the first three are supposed to do the trick
rtohid has left #ste||ar [#ste||ar]
<hkaiser>
K-ballo: I don't think we do that anymore
<hkaiser>
but ms[m] should know
<K-ballo>
BOOST_ROOT is all that's needed for a convential install or even simply build tree of boost
<K-ballo>
if you set Boost_DIR then you'll override all the find logic
<K-ballo>
it doesn't make sense to set both of those at the same time
<Yorlik>
Believe me - I tried all combination of them together and in isolation because I know about that. Also with deletion of the entire build tree in between.
<K-ballo>
try simply setting BOOST_ROOT and enabling Boost_DEBUG, on a clean build dir
<Yorlik>
OK
<K-ballo>
....are you building boost from the same cmake project...?
<K-ballo>
it looks like for HPX at least you are doing ExternalProject
<Yorlik>
I am building Bost with b2 through Cmake
<K-ballo>
on its own build, separate from building hpx
<Yorlik>
Yes
<Yorlik>
One after another in separate directories
<Yorlik>
exec_program(./b2 ${BOOST_SOURCE_DIR}
<Yorlik>
#OUTPUT_VARIABLE OUT
<Yorlik>
ARGS ${B2_ARGS}
<Yorlik>
RETURN_VALUE RESULT
<Yorlik>
)
<Yorlik>
Thats my call to b2
<Yorlik>
After building it
<K-ballo>
I hope that's wrapped on an external project too
<Yorlik>
No - I'm just switching it on in isolation after deleting the build directory.
<Yorlik>
Its building in its own binary directory
<K-ballo>
it's building at configuration time, not at build time?
<Yorlik>
Our build system is not perfect, but usually it works.
<K-ballo>
INTERNAL implies FORCE, and Boost_<...> is messing with FindBoost's namespace
<Yorlik>
I always wanted to make a nicer boost wrapper
<K-ballo>
it takes a massive amount of care and work to guarantee everything is built with the same compiler and combination of flags, so that results are compatible.. you are adding b2 compatibility to that, it must be hellish
<Yorlik>
It seems to be happy with having BOOST_ROOT only now
<Yorlik>
But I remember that breaking in the past
<K-ballo>
sure, that's all it takes..
<K-ballo>
probably dirty cache?
<Yorlik>
Gotta work opn hwloc now
<K-ballo>
if you were setting Boost_DIR to empty, in particular, that could have been bad
akheir1 has joined #ste||ar
<K-ballo>
when propagating variables to superprojects you need to check whether they are actually defined or not, as undefined and empty don't always give you the same behavior.. Boost_DIR is one of those
<Yorlik>
I'm hping for the day someone makes a really nice accepted build system for C++
<K-ballo>
heh, the "accepted" part is up to you... vcpkg works fine, conan too
<Yorlik>
They are packagers, not build systems are tehy?
<K-ballo>
just wrapping a bunch of cmake projects on a python script can often be substantially better than a superbuild
<K-ballo>
yeah, but they integrate with cmake for the build system part, and you don't need to wait for everyone to rewrite script files for this new build system
akheir has quit [Ping timeout: 260 seconds]
alireza has joined #ste||ar
<Yorlik>
Our superbuild is sufficiently stable in the moment - after the current milestone we have agreed to rework and polish it.
<K-ballo>
just don't go blaming FindBoost in the meantime ;)
<Yorlik>
I have thought of using a Python or Lua wrapper actually, but I don't want to introduce new technology where I don't have to
david_pf_1 has joined #ste||ar
david_pf_ has quit [Ping timeout: 252 seconds]
david_pf_1 is now known as david_pf_
<Yorlik>
Trust me it didn't work in the past and I had to add include and lib dirs.
akheir1 has quit [Ping timeout: 246 seconds]
<Yorlik>
I guess you hpx guys are playing your own diabolical games with it
<K-ballo>
BOOST_ROOT is all you need to set
<K-ballo>
the include and lib dirs are for unusual locations, for instance I used to have to set Boost_LIBsomething when I had different lib32 and lib64 stage directories
<Yorlik>
These directories were set correctly and setting them worked on windows.
rtohid has joined #ste||ar
<ms[m]>
What did I do?? I don't remember changing findboost behaviour related to pkgconfig, but my memory is short...
<K-ballo>
there's no _need_ to set them if they are in the usual locations
<K-ballo>
setting them to the right values certainly won't make things worse
<ms[m]>
Fwiw, I've never had to set anything but BOOST_ROOT on linux
<ms[m]>
If that doesn't work it's usually because the boost install is "wrong" in some sense
<ms[m]>
And overriding the HPX_WITH_CXX_FOO_FEATURE is not a good idea
<ms[m]>
I a word
<K-ballo>
we still disable boost config package files, I checked.. the generation bug has been fixed since then (1.71? 1.72?), people likely to have installed the incorrectly generated ones are likely to have upgraded by now
<K-ballo>
and in newer cmake versions I understand there's now some colaboration between FindBoost and BoostConfig, and everything should work as expected except for some QUIET flags not being honored
<ms[m]>
Loud is OK :)
david_pf_ has quit [Ping timeout: 240 seconds]
<ms[m]>
What is the but exactly? Do you recommend we enable boost config files again?
<ms[m]>
*bug
<K-ballo>
not really... why is there a bug?
<ms[m]>
"the generation bug"
<ms[m]>
I don't know what refers to
<ms[m]>
That
<ms[m]>
Why do I keep leaving out words
<ms[m]>
I'm happy to keep things as they are if they work, was just curious since you're much deeper into boost
<K-ballo>
when boost first started installing config files, it would generate files not compatible with the FindBoost module
<K-ballo>
the FindBoost module, for arcane reasons, would first lookt for a config file and delegate to it if one existed
<K-ballo>
and there were a couple of bugs in the target generation of the config files
<weilewei>
.
<K-ballo>
which were installed by default.. so working find_package(Boost) calls started failing all over
<ms[m]>
gotcha, thanks
akheir1 has joined #ste||ar
alireza has quit [Ping timeout: 264 seconds]
david_pf_ has joined #ste||ar
jbjnr has joined #ste||ar
jbjnr has quit [Client Quit]
jbjnr has joined #ste||ar
<jbjnr>
If I am writing documentation for a module in the libs/module/docs/index.rst file - is there an easy way of previewing it to see how it will look without installing sphinx, breathe, doxygen (more?) - just a simple preview of the formatting layout of the individual rst file would help - does anyone know?
<jbjnr>
is this the IRC channel or the hpx matrix version - or both?
<jbjnr>
ok. I found an online rst renderer page that lets you type stuff in and see how it looks. good enough for now
<Yorlik>
jbjnr: I use sphinx-autobuild locally (install with pip) to scan files for changes, run a webserver and update automagically when I write .rst
rtohid has quit [Ping timeout: 240 seconds]
<jbjnr>
thanks. I'm not sure if that will work with the hpx docs, but if so I'll try it