K-ballo changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
<diehlpk_work>
ms[m], HPX 1.7.0 does not compile on Fedora
<diehlpk_work>
HPX tries to download asio which is not allowed while building the package
<diehlpk_work>
Can I disable that?
<hkaiser>
diehlpk_work: no
<diehlpk_work>
So all git clone are not allowed
<hkaiser>
ok, then we need to package that somehow and point the build to that dir
<diehlpk_work>
Ok, until we have done that I cna not build HPX on fedora
<hkaiser>
diehlpk_work: I'll look into this tomorrow
<diehlpk_work>
hkaiser, Is that a new feature?
<hkaiser>
yes
<diehlpk_work>
Ok, good, so I understand whay it stopped building
<diehlpk_work>
hkaiser, We have time, the next Fedora release will be in October
<diehlpk_work>
No rush
<hkaiser>
we can easily make that work
hkaiser has quit [Quit: bye]
diehlpk_work has quit [Remote host closed the connection]
<tarzeau_>
1.7.0? can't see it on github
<tarzeau_>
ah can... pre-release, /me tries
<ms[m]>
tarzeau_: (and diehlpk once he's online) the proper thing to do for 1.7.0 would be to package asio separately, but I don't know how much effort that is
<ms[m]>
rachitt_shah: how's it going? I assume you got things going with sphinx and cmake?
<mdiers[m]>
I also stumbled over the additional git dependencies and personally don't like it that much. I would also be happy about a separation.
<rachitt_shah[m]>
<ms[m] "rachitt_shah: how's it going? I "> Yep. Struggled with cmake, but worked out in the end. Docs were really having a large filesize, so took a while to configure the cmake.
<rachitt_shah[m]>
<ms[m] "rachitt_shah: how's it going? I "> I'm trying to make sense of the templates, will ping here. Will require quite a bit of trial and error
<ms[m]>
yeah, configuring should be quick but building can take quite a while
<ms[m]>
I'll send you a diff in a sec which makes it slightly faster (leaves out some of the doxygen for all but one lib)
<rachitt_shah[m]>
<ms[m] "yeah, configuring should be quic"> It did take a while for me, once I start picking it up perhaps I can run builds for specific modules
<rachitt_shah[m]>
<ms[m] "I'll send you a diff in a sec wh"> That would be really helpful
<hkaiser>
or just put hpx and asio side by side in two directories
<ms[m]>
the next best solution is to fetch asio separately (I suppose you can specify other tarballs to be downloaded than just the hpx tarball...?), and point to it using fetchcontent variables
<diehlpk_work>
ms, No only one tar ball
<diehlpk_work>
That is my concern for the multiple HPX repos.
<hkaiser>
diehlpk_work: you can always explicitly have several directories inside youtarball side by side
<ms[m]>
diehlpk_work: for multiple hpx repos, each repo would be a separate fedora package
<diehlpk_work>
ms[m], Yes, but I am not interested to support that
<ms[m]>
ok
<hkaiser>
diehlpk_work: you should cast your vote on the topic
<ms[m]>
^
<diehlpk_work>
One Fedora package is enough work
<diehlpk_work>
and what I heard from other projects this is really painful to handle all dependencies
<diehlpk_work>
hkaiser, I will vote and explain my concerns for the Fedora package
<diehlpk_work>
My favor would be to have a boost like system
<diehlpk_work>
One repo and one can build different modules
<diehlpk_work>
and so on would be mine opinion the way to go
<diehlpk_work>
Another question would be the releases of the different repos alingend?
<diehlpk_work>
If we split the repos, I would prefer to only have one Fedora HPX package having all local and distributed combined
<ms[m]>
there would most likely be at least one local release for each distributed release, but they could also be independent
<diehlpk_work>
My take is that people use the fedora package for testing and will build hpx once the started to use it
<diehlpk_work>
SO I think here one package would be better
<ms[m]>
I don't think I see how the two are related? also, wouldn't you want people to be able to just use the fedora package and not have to build hpx?
<ms[m]>
in any case, please respond with your concerns to the vote
<diehlpk_work>
ms[m], I would say that if people will run long-term runs, they will have to build hpx since the package might not be available on the cluster
chuanqiu has quit [Quit: Connection closed]
<diehlpk_work>
The HPX packages is compiled without any optimization or architecture information
<diehlpk_work>
The compiler flgas are set to have some generic compilation working on most x86 CPUs
<diehlpk_work>
or any other platform
<hkaiser>
diehlpk_work: for V1.7 I'd suggest that you create two directories, one for asio and one for hpx and put the sources for those next to each other, then use the -DASIO_ROOT=<asio_dir> flag to tell cmake to use the asio provided instead of trying to check it out from github
<ms[m]>
diehlpk_work: I don't see how that's related to separate repositories
<ms[m]>
hkaiser: diehlpk_work you must use the fetchcontent variables, because asio has to be installed together with hpx; if you set ASIO_ROOT we assume that asio is already installed and we won't make any effort to try to copy it to an install directory
<hkaiser>
ahh
<hkaiser>
ok, my bad
<hkaiser>
do we support that?
<ms[m]>
that's why it'd be best just to have an asio package, but I can understand if you don't want to create that
<ms[m]>
support what?
<hkaiser>
working with a checked out version of asio that is not installed
<ms[m]>
cmake takes into account some variables related to fetchcontent that should allow doing exactly that
<ms[m]>
I've never tried it myself though... but from the docs it looks like it's what you'd want to do in this case
<hkaiser>
FETCHCONTENT_FULLY_DISCONNECTED looks like a viable option here
Girlwithandroid[ has quit [Quit: Idle for 30+ days]
<hkaiser>
or FETCHCONTENT_SOURCE_DIR_ASIO
<hkaiser>
either way, looks like we need to add things to our cmake scripts to enable builds relying on existing source trees instead of downloading things
<ms[m]>
hkaiser: what would we need to add?
<hkaiser>
we would need to add FETCHCONTENT_SOURCE_DIR_ASIO to the fetch_declare instead of referring to github
<hkaiser>
if I read the docs correctly, that is
<ms[m]>
I think it's enough to point FETCHCONTENT_SOURCE_DIR_ASIO to the root of asio
<hkaiser>
ok, let me try that
<hkaiser>
nope, still fetches asio from github
<hkaiser>
ok, we need to add SOURCE_DIR to fetchcontent_declare
<ms[m]>
hkaiser: seems to work just fine for me... did you set FETCHCONTENT_FULLY_DISCONNECTED (don't know if it has an effect here, but I have it set)?
<hkaiser>
ok, I didn't use that
<diehlpk_work>
hkaiser, I can try your patch later today
<hkaiser>
ms[m]: yes, you're right, setting both options does the trick, thanks!
<hkaiser>
diehlpk_work: no patch needed, add -DFETCHCONTENT_FULLY_DISCONNECTED=ON and -DFETCHCONTENT_SOURCE_DIR_ASIO=<asio_dir> to cmake
<hkaiser>
ms[m]: the only issue is that FETCHCONTENT_FULLY_DISCONNECTED=ON disables it for all fetchcontent operations, not just asio - should we consider having additional options that allow to specify things for asio and apex separately?
<hkaiser>
(not that this is needed for Fedora)
<ms[m]>
only if it's needed, and I don't see it being needed right now
<diehlpk_work>
I refer to this tar ball in the file and upload this to the fedora server
<diehlpk_work>
The Fedora build system has no access to the internet at any point
<diehlpk_work>
and uses the uploaded tar ball which should be the same as the public one
<diehlpk_work>
The principle is that each package has one tar ball with all source code
<diehlpk_work>
this tar ball is extracted and the code compiles using the tar ball or any other fedora package
<hkaiser>
sure
<diehlpk_work>
So I can generate a tar ball having hpx 1.7 and asio and we will upload it to stellar.cct.lsu.edu
<diehlpk_work>
Do we rely on a specific version of asio?
<hkaiser>
ok, let's do that
<hkaiser>
we refer to the latest release, currently it's the tag 'asio-1-18-2'
<diehlpk_work>
Ok, another option would be to install the fedora package
<diehlpk_work>
1.16.1-3.fc34
<hkaiser>
if you configure hpx normally you can find it under <build-dir>/_deps/asio-src
<hkaiser>
diehlpk_work: I have not tried older versions of asio
<hkaiser>
but if you do, simply set ASIO_ROOT to refer to the installed package
<diehlpk_work>
hkaiser, Let me try that first
<hkaiser>
no other options would be needed in this case
<diehlpk_work>
I would prefer this option and not to have an additional tar ball to maintain
<ms[m]>
diehlpk_work: absolutely use the package if there is one and it works (I thought there wasn't one, but it seems my search wasn't very exhaustive...)
<ms[m]>
hpx would otherwise conflict with the asio package
<ms[m]>
hkaiser: I didn't do anything, but hpx is now an option for me on the opencollective invoice page... is it for you as well?
<hkaiser>
ms[m]: I'll have a look
<hkaiser>
yah, I can chose 'Organization Account', but I'm not sure what to put there - do you?
<hkaiser>
ms[m]: once I log into opencollective the organization account is gone
<hkaiser>
wouldn't we have to pay taxes twice if we receive the money personally?
<ms[m]>
hkaiser: I suppose so... but I don't think it can go directly to rachitt
<ms[m]>
I don't know what details to fill in, probably the cct stellar group address etc.?
<hkaiser>
diehlpk_work: +1! is that using the existing asio package?
<diehlpk_work>
Yes
<ms[m]>
diehlpk_work: I think you want ASIO_ROOT=/usr
<diehlpk_work>
We will se it hpx will compile with that
<diehlpk_work>
ms[m], /ust/include/
<diehlpk_work>
*/usr/
<diehlpk_work>
We are at 45% for the gcc build on x86, we will know soon if the package will work
<ms[m]>
diehlpk_work: both will work, but you need ASIO_ROOT, not FETCHCONTENT_SOURCE_DIR_ASIO in any case
<diehlpk_work>
ms[m], -DFETCHCONTENT_FULLY_DISCONNECTED=ON and -DFETCHCONTENT_SOURCE_DIR_ASIO=<asio_dir>
<diehlpk_work>
I added these to
<diehlpk_work>
*two
<ms[m]>
yes, those are for when hpx should install asio, so you want ASIO_ROOT
<diehlpk_work>
Ok, I can change it. Does it make any difference?
<ms[m]>
if you're lucky it'll work because hpx will have nothing to install (if the asio subdirectory that hpx would otherwise install doesn't exist), but if the paths ever change it'll probably break
<diehlpk_work>
If, asio is in /usr/include/ will that not be in the default search path?
<diehlpk_work>
So there is no need for ASIO_ROOT?
<ms[m]>
in principle yes, but we decide to fetchcontent or use an installed asio based on if ASIO_ROOT is set or not
<diehlpk_work>
Ok, I can moce to ASIO_ROOT in the next build
<ms[m]>
we can add another variable to signal that it should use an installed asio, or just try it first, but right now we don't
<diehlpk_work>
87%
<ms[m]>
it'd probably be cleaner to try to use a system asio first anyway
<diehlpk_work>
Since the examples are build now, I would say it worked with the fedora package
<diehlpk_work>
+1 for try to use the system one as default
<diehlpk_work>
ms[m], We should keep Fedora and Ubuntu in mind for the new HPX solution
<diehlpk_work>
I really would keep HPX in Fedora to attract more users
<diehlpk_work>
However, the installation there is very limited
<diehlpk_work>
As long as I can have one tar ball, it would be easy. I can have different packages there
<diehlpk_work>
hpx-local and hpx-distributed
<diehlpk_work>
Again, I would prefer a boost like solution one repo and all parts can be compiled as needed
<diehlpk_work>
ms[m], hkaiser HPX without mpi built using the Fedora asio package
<diehlpk_work>
We might wat to stick with this solution.
<diehlpk_work>
mpi and openmpi should be done in the next hour on x86
<ms[m]>
diehlpk_work: 👍️
<ms[m]>
no reason to use anything else if they system package works
<diehlpk_work>
hkaiser, ms[m] I voted
<hkaiser>
thanks
<ms[m]>
diehlpk_work: thanks
<ms[m]>
hkaiser: yeah, I can try, but could you send me whatever address you think is hpx's address? ;)
<hkaiser>
lol, do we have one?
<hkaiser>
if you mean LSU's address: STE||AR-GROUP, 2156 Digital Media Center, Baton Rouge, LA 70803
<ms[m]>
hkaiser: I've no clue, I suppose it should be something that looks good on the tax form... We'll see if that works
<hkaiser>
lol
<hkaiser>
can we ask them?
<hkaiser>
I assumed the funds would go to opencollective, where Rachit could send his invoice and get paid directly
<hkaiser>
ms[m]: what would happen if we asked him to invoice it based on that link?
chuanqiu has joined #ste||ar
<ms[m]>
hkaiser: so I could choose "Open Collective (Account Balance)" for payout method with hpx as the receiver, if I have myself I only have the options of adding a bank account or paypal account
<ms[m]>
so far so good
<ms[m]>
but I don't know what would happen if we tried to invoice it directly to him
<hkaiser>
this is just confusing
<ms[m]>
I think they prefer not to do that, since they don't want any details on who's been hired, but in practice I don't know if it matters
<hkaiser>
the instructions are ambiguous
<ms[m]>
not really
<hkaiser>
well, for me
<hkaiser>
I don't get how that works
<ms[m]>
I'm also pretty sure they won't approve the invoice for someone who is not on the gsod organization list...
<hkaiser>
right
<ms[m]>
step 8 of their instructions is: Season of Docs admins will review your expense and may request more information or require changes. Once the expense is approved by Season of Docs admins, Open Source Collective will proceed with the payment.
<hkaiser>
ok, let's try it - what could possibly go wrong
<ms[m]>
the world might end... worst case is we have to pay taxes once, and rachitt once, but I don't think we can avoid that really