13:21 <Chewbakka> Hey, is there a special and elegant construct in HPX, which has an equivalent function like "MPI_Allreduce" e.g. given the scenario one wants to calculate the dot product of two vectors, and every node needs a value from everybody in order to compute the dot product. Currently, we use a latch and register a VectorClient in order to gather all intermediate values, but we have the feeling that is not very elegant and doesn't c

13:21 <Chewbakka> onform to HPX ( https://github.com/JeannedArk/smvm/blob/9d91f27b98b2041667a2efb332291bca9e379db9/src/Vector.cpp#L68 )

14:04 <Chewbakka> Hey, is there a special and elegant construct in HPX, which has an equivalent function like "MPI_Allreduce" e.g. given the scenario one wants to calculate the dot product of two vectors, and every node needs a value from everybody in order to compute the dot product.

14:05 <Chewbakka> Currently, we use a latch and register a VectorClient in order to gather all intermediate values, but we have the feeling that is not very elegant and doesn't conform to HPX ( https://github.com/JeannedArk/smvm/blob/9d91f27b98b2041667a2efb332291bca9e379db9/src/Vector.cpp#L68 )

14:07 <hkaiser> Chewbakka: there is no allreduce in hpx, but we have boradcast and gather - allreduce could be implemented in a similar way

14:08 <hkaiser> we also have reduce, iirc

14:13 <Chewbakka> thank you

14:13 <hkaiser> Chewbakka: contributions are most welcome

14:13 <Chewbakka> you probably mean this: https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/lcos/reduce.hpp

14:13 <hkaiser> we don't have allreduce mainly because nobody has felt the need to implement it ;-)

14:14 <hkaiser> yes, that's th eone

14:18 <Chewbakka> I will consider this and the GSoC sounds pretty interesting, but until such point I need to get more familiar with the framework :)

14:18 <hkaiser> Chewbakka: sure

14:18 <hkaiser> the existing facilities (reduce, gather, etc.) are built solely on top of existing hpx features

14:18 <hkaiser> no surprises there

14:22 <Chewbakka> One more question: we have this sparse-matrix-vector-multiplication application and we have the behavior that the application runs fine with --hpx:thread 1, but doesn't run properly properly without this flag or more number of threads, though we don't use any intra-node parallelism.

14:23 <hkaiser> Chewbakka: difficult to tell, sounds like a race somewhere

14:24 <Chewbakka> how is that possible if we do not use local concurrency?

14:25 <hkaiser> I'd assume you use hpx actions, do you?

14:25 <Chewbakka> right now, we only use a latch to sync the localities

14:26 <hkaiser> a latch uses actions

14:26 <hkaiser> but as said, it's difficult to tell without seeing what's going on

17:47 <heller___> libfabric over tcp works splendid now

17:47 <heller___> Did a full system run on my arm cluster today

17:47 <heller___> (128 cores)

17:48 <heller> Everything worked out of the box. Pretty nice

17:48 <hkaiser> heller: even bootstrapping?

17:52 <heller> https://imgur.com/a/qQhLiO6

17:52 <heller> No, that was done over asio

17:52 <hkaiser> k

17:52 <heller> But that's coming soon

17:52 <heller> I know how to do it...

17:54 <heller> If one of you has interest to do some experiments there... Give me your user ID and a ssh key you use

17:55 <heller> One board has a 8 core arm a53

17:56 <heller> BTW, I've been thinking about that lazy task graph building and phylanx

17:57 <heller> Especially regarding the make_ready future overhead

17:58 <heller> When looking at the ast, don't you know how many futures at maximum you'd need?

17:58 <K-ballo> is that a briefcase?

17:58 <heller> Suitecase, yeah

17:59 <K-ballo> that's convenient

18:08 <hkaiser> heller: it might be possible to find out - what do you have in mind?

18:10 <heller> hkaiser: allocation the shared states you need in a single allocation

18:14 <heller> In general, I find these future graphs more and more appealing

18:16 <heller> Especially for combining different future types... Tons of possibilities

18:23 <hkaiser> indeed

18:36 <nikunj> zao: yt?

18:37 <zao> Kind of.

18:38 <nikunj> Do you know where cmake functions for hpx are?

18:39 <nikunj> I want to add another parameter to cmake of HPX_WITH_MAIN so that cmake add extra CXX flags for it

18:40 <diehlpk_work> nikunj, https://github.com/STEllAR-GROUP/hpx/tree/master/cmake/templates

18:41 <nikunj> diehlpk_work, thanks :)

18:41 <diehlpk_work> or https://github.com/STEllAR-GROUP/hpx/tree/master/cmake

18:41 <diehlpk_work> I think there you will find all cmake related stuff

18:43 <nikunj> diehlpk_work, I was searching there itself, but there were many files, so I thought I'd ask. I'll continue looking :)

18:46 <K-ballo> configuration options are more likely to be found in the root cmake lists file

18:47 <nikunj> I think HPX_AddCompileFlag is the one I'm looking for

18:55 <K-ballo> you'll note that no HPX_WITH option is specified there

18:55 <nikunj> hmm

18:56 <nikunj> K-ballo, a little help. I want to add HPX_WITH_MAIN option and add specific CXX flags with it (at linking time)

18:56 <K-ballo> look for uses of hpx_option(HPX_WITH_...

18:57 <K-ballo> most would be in the root CMakeLists.txt file

18:58 <nikunj> ohk. But these CXX flags are for linking time only, don't you think it should be added for HPX_AddLinkFlag in cmake directory?

19:00 <K-ballo> HPX_AddLinkFlag defines the functions that deals with adding linker flags to hpx targets

19:00 <K-ballo> they do not introduce configuration options

19:02 <nikunj> I see

19:02 <nikunj> I'll delve deeper then

19:02 <K-ballo> look for uses of hpx_option(HPX_WITH_..., particularly the ones in the root CMakeLists.txt file

19:02 <nikunj> ok thanks :)

19:47 <heller> nikunj: also, I guess there's a difference between linking libhpx, any other library using hpx and an application striving to use your new technique

19:48 <heller> nikunj: as such, I propose to not make it a global option but add that functionality to hpx_setup_target

19:49 <nikunj> heller: ok

19:51 <heller> Fun fact: shared libraries are executable. Once you execute them though, they segfault because main does not exist

19:52 <nikunj> heller: I'll be placing compiled code (object file) into the lib folder to link with. My current thought process is to create a wrapper file in src folder which would then be compiled (but not linked). Then through one of the options link the .o file with the file user is working with.

19:52 <nikunj> *wrapper.cpp

19:53 <nikunj> https://github.com/NK-Nikunj/GSoC-experimental-codes/tree/master/normal_code/dl_wrap

19:54 <nikunj> In this example I link test.cpp with wrap_libc.o to produce the executable

19:54 <heller> Ok

19:55 <heller> Make that a static library

19:55 <heller> Which is essentially a archive of object files

19:55 <heller> And make sure it's being installed

19:56 <nikunj> heller: ohk

19:57 <nikunj> But it has to link with any file that has hpx_main.hpp

19:57 <heller> The nice thing about making it an archive: you can use cmake magic so that as soon as you're archive/object is a target dependency, your special flags can propagate

19:57 <heller> Aha

19:57 <heller> Hmmm

19:57 <heller> And that's impossible to know upfront

19:57 <nikunj> aah! So I could add it as a target dependency

19:57 <heller> Yes

19:58 <nikunj> Then I can surely make it happen this way.

19:58 <nikunj> I will only have to add a target dependency everywhere

19:58 <nikunj> But this would mean I will have to change a lot of code in cmakelists

19:59 <nikunj> I would personally first try to test my code with a few examples before integrating it directly

20:04 <heller> Sure

20:04 <heller> That's how you'd do it

20:04 <heller> Start with one

20:05 <heller> Do a pr, then we discuss things

20:05 <nikunj> I can integrate one right now, if you want to take a look

20:05 <nikunj> But my code is very basic

20:06 <nikunj> So I do not know if there's any initialization sequencing fiasco problems

20:06 <nikunj> I'll integrate now, so that you can take a look

20:07 <heller> Yes

20:07 <heller> And make it such that it gets tested

20:07 <heller> Hpx should be fine with static init order

20:34 <nikunj> heller: https://github.com/NK-Nikunj/GSoC-experimental-codes/tree/master/hpx_code/dl_wrap

20:34 <nikunj> just run the makefile to create the executable

20:35 <nikunj> and run to see the result

21:04 <nikunj> heller: I have sent you an email updating you with my implementations. It is brief so if you want details on anything, I will explain it in detail.