hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoC2018: https://wp.me/p4pxJf-k1
<diehlpk> yes
<diehlpk> nikunj, Already published
nikunj97 has joined #ste||ar
nikunj1997 has joined #ste||ar
nikunj97 has quit [Remote host closed the connection]
nikunj has quit [Ping timeout: 240 seconds]
<nikunj1997> diehlpk
<nikunj1997> do you need a blog or brief post of my work?
<diehlpk> nikunj1997, No you are fine
<nikunj1997> ok
<diehlpk> You will need a blog post for the final submission
<diehlpk> You can write it on your blog or on our blog
<nikunj1997> diehlpk, understood
<diehlpk> This was done at our blog
<diehlpk> and the student submitted this link to google
<nikunj1997> ok
<diehlpk> Or like this one here https://naios.github.io/gsoc2017/
<diehlpk> I will send a mail with more details after the second evaluation
<nikunj1997> ok, thanks.
nikunj97 has joined #ste||ar
nikunj1997 has quit [Remote host closed the connection]
jaafar has joined #ste||ar
<nikunj97> hkaiser: yt?
<hkaiser> here
<nikunj97> hkaiser: so I tested thoroughly multiple times my mac implementation
<nikunj97> there is one test that fails everytime, tests.regressions.parallel.static_chunker_2282
<hkaiser> ok
<hkaiser> that should be unrelated
<hkaiser> does it use hpx_main?
<nikunj97> no
<nikunj97> it does not
<nikunj97> also, there are 3 other tests which fails on some and passes on others
<nikunj97> tests.unit.components.launch_process, tests.unit.components.distributed.tcp.migrate_component tends to time out
<nikunj97> sorry only 2 other tests which tends to time out
<nikunj97> not on every run though, they vary on different runs
<hkaiser> yah, those are most probably unrelated as well
<nikunj97> ok, so I should not worry about them right?
<hkaiser> no need to worry
<nikunj97> ok, also I think I have a solution for multiple defintion. I can create another file with a struct containing a static bool.
<nikunj97> header file^^
<nikunj97> Then I'll include them in both hpx_wrap and hpx_main. In hpx_wrap I'll make that static member false, in hpx_main I'll make it true
<nikunj97> then I'll create an object of the struct in the functions and check for that variables value
<nikunj97> I think it should work, I haven't tried it though. Just got the idea this morning
<hkaiser> nikunj: using a weak symbol again?
<nikunj97> hkaiser, yes
<hkaiser> nikunj97: very similar to K-ballo's suggestion yesterday
<nikunj97> yes
<hkaiser> ok, sounds good
<nikunj97> that's where I got the idea for
<hkaiser> should work
<hkaiser> thanks for still trying to solve this!
<hkaiser> nikunj97: may I ask you to add some docs anyways?
<nikunj97> I promised to make things robust :)
<hkaiser> :D
jaafar has quit [Ping timeout: 256 seconds]
<nikunj97> where should I add docs?
<hkaiser> sec
<nikunj97> I can add surely update the docs
<hkaiser> nikunj97: ^^
<nikunj97> ok, I'll update it
<hkaiser> thanks
Vir has quit [Ping timeout: 265 seconds]
Vir- has joined #ste||ar
K-ballo has quit [Quit: K-ballo]
nikunj97 has quit [Quit: Leaving]
nikunj has joined #ste||ar
hkaiser has quit [Quit: bye]
jaafar has joined #ste||ar
jaafar_ has joined #ste||ar
jaafar has quit [Ping timeout: 276 seconds]
jaafar_ is now known as jaafar
anushi has quit [Remote host closed the connection]
anushi has joined #ste||ar
jaafar has quit [Ping timeout: 260 seconds]
<github> [hpx] StellarBot pushed 1 new commit to gh-pages: https://git.io/fNvSb
<github> hpx/gh-pages 015ce64 StellarBot: Updating docs
jakub_golinowski has joined #ste||ar
<github> [hpx] NK-Nikunj opened pull request #3368: Updates doc with recent hpx_wrap implementation (master...update_docs) https://git.io/fNvHf
jakub_golinowski has quit [Ping timeout: 245 seconds]
jakub_golinowski has joined #ste||ar
jakub_golinowski has quit [Ping timeout: 256 seconds]
K-ballo has joined #ste||ar
anushi has quit [Ping timeout: 240 seconds]
anushi has joined #ste||ar
hkaiser has joined #ste||ar
anushi has quit [Read error: Connection reset by peer]
<nikunj> hkaiser: bad news, I tried to implement the way to get around the multiple definition error but it didn't work out
<hkaiser> nikunj: you need a template
<hkaiser> a simple type will not do the trick
anushi has joined #ste||ar
<nikunj> ohk
anushi has quit [Read error: Connection reset by peer]
anushi has joined #ste||ar
anushi has quit [Ping timeout: 264 seconds]
anushi has joined #ste||ar
anushi has quit [Read error: Connection reset by peer]
quaz0r has quit [Ping timeout: 240 seconds]
quaz0r has joined #ste||ar
<hkaiser> nikunj: yt?
<nikunj> here
<hkaiser> nikunj: I think all our tests in a project that depend on HPX that use hpx_main do fail now
<nikunj> oh
<nikunj> So we do need to add to HPXConfig
<hkaiser> they fail at runtime
<hkaiser> all of the tests failing using hpx_main
<hkaiser> use*
<nikunj> yes the error is of uninitialized hpx runtime as well
<nikunj> ok, I'll add an entry in HPXConfig
<hkaiser> nanashi55: could you add a config (cmake option) that enables your new code?
<hkaiser> I'd like to make the migration to the new stuff a bit more smooth
<hkaiser> nikunj: ^^
<nikunj> how would you like me to make things happen?
<hkaiser> HPX_WITH_DYNAMIC_HPX_MAIN or something
<hkaiser> that should allow to inforce he old/new behavior at configuration time
<hkaiser> I can try doing this as well
<hkaiser> nikunj: let me do that, I know exactly what I would like to see
<nikunj> hkaiser: understood
<hkaiser> nikunj: this was on my list anyways - we try to introduce breaking changes in stages
<hkaiser> one release have the new behavior in place but users need to explicitly enable it, second release, the new behavior is enabled by default, and third release we remove the old stuff
<nikunj> ohk, sorry for the wrong information I shared
<hkaiser> nikunj: no problem, I didn't communicate this to you
<hkaiser> we might skip the first stage and have it enabled by default, but we need to have a means of enforcing the old behavor
<nikunj> yes
<nikunj> oh so are we adding a transition from the old behavior to the new?
<hkaiser> nikunj: please feel free to comment on this, I hope I got it right
<nikunj> hkaiser: LGTM
quaz0r has quit [Ping timeout: 260 seconds]
<hkaiser> nikunj: what would happen if things are enabled at HPX built time but somebody disables the feature at application build time
<hkaiser> all should be well because of the weak symbol, right?
<nikunj> by built you mean while building hpx?
<nikunj> thing will work like used to before if it's disabled at application build time
<hkaiser> yes, that's what I meant
<hkaiser> I pushed another commit explicitly allowing this
<nikunj> if it's built using that option then the examples and tests will be built linking with my code
<hkaiser> if an application #define HPX_HAVE_DYNAMIC_HPX_MAIN 0 before including hpx/hpx_main.hpp it will disable things
<nikunj> yes
<nikunj> and the code will move on to #define main method
<hkaiser> right
<hkaiser> could you test this PR with your environment, please
<nikunj> also you have specifically mentioned for HPX_HAVE_DYNAMIC_HPX_MAIN in cmake check as well so that would not allow adding the flags as well
<nikunj> ok, on it
<hkaiser> thanks
<nikunj> hkaiser: should I try building phylanx with hpx to test application built?
galabc has joined #ste||ar
<nikunj> hkaiser: the branch builds and can run examples perfectly. Running tests on it now.
quaz0r has joined #ste||ar
<hkaiser> nikunj: feel free to test phylanx as well, sure
<hkaiser> nikunj: circleci for phylanx uses an older version of clang (hmm, so does circleci for hpx)
<hkaiser> could it be that this causes the problems?
<nikunj> I tested the code with clang 3.9, so I don't think it is the problem
<hkaiser> then there is something else ...
<nikunj> Most likely its with HPXConfig
<hkaiser> could be
<hkaiser> Phylanx relies on it
<nikunj> we can now add HPX_WITH_DYNAMIC_HPX_MAIN to the config
<nikunj> that should clear things
<hkaiser> nikunj: pls note that the HPX_WITH_... is used for the cmake part, while HPX_HAVE_... is used as a pp constant at compile time
<nikunj> ohk
<hkaiser> also, the HPX_HAVE_.. is automatically propagated to dependent projects through the hpx/config.hpp
<nikunj> hkaiser: the linker flags are not propagated through HPXConfig
<hkaiser> right
<nikunj> They are explicitly mentioned in HPX_SetupTarget though
<nikunj> oh that explains it!
<nikunj> phylanx doesn't depend on HPX_SetupTarget (as seen from the cmake source)
<nikunj> so it wouldn't run
<nikunj> linker flags are crucial for my code to run as expected
galabc has quit [Quit: Leaving]
<nikunj> hkaiser: just checked out the cmake code for phylanx. It does not make use of hpx_setup_target (where I have added my flags). Therefore it does not work.
<hkaiser> ok
<nikunj> we need to add it along with the necessary flags required for HPX
<hkaiser> nikunj: yes, we essentially copied things over from hpx (see phylanx_setup_target)
<hkaiser> but we need it in hpxconfig anyways
<nikunj> phylanx_setuptarget will not contain my flags as it is not propagated
<hkaiser> right, we need to add it there, but only if HPX_WITH_DYNAMIC_HPX_MAIN is defined in HPXConfig
<nikunj> yes! And HPX_WITH_DYNAMIC_HPX_MAIN will be propagated through cmake
<nikunj> so we need to simply add the flags there and things should start to run as expected
<nikunj> hkaiser: hpxconfig does not seem to contain any variable for link_flags or cxx_flags
<hkaiser> nikunj: it needs to contain HPX_WITH_DYNAMIC_HPX_MAIN, though
<nikunj> hkaiser: yes it should contain it
<nikunj> also why don't we create flags similar to HPX_CUDA_CLANG_FLAGS for our variable?
<nikunj> we can propagate it to all the projects
<nikunj> we will then need to add the flag to the set of link flags and things should run pretty smoothly
<nikunj> this way the user will not have to remember the linker flag I add in HPX_SetupTarget and can simply add it by the variable name
<hkaiser> fine by me
<hkaiser> nikunj: why is it failing if Phylanx does not force-link with your symbols
<hkaiser> should it work anyways (falling back to the old behavior)?
<nikunj> no, phylanx would still fail here still it is being run on inux
<nikunj> so it will enter the #if defined(__linux).. portion
<nikunj> leading to this error
<hkaiser> : sure, but it should hit the weak symbol which defaults to false
<hkaiser> can't we make this work anyways?
<hkaiser> (I would like to change Phylanx as needed, but this is a possible source of errors which I'd like to prevent, if possible)
<nikunj> yes it will hit false, but without the linker flags in place the actual linking will not work
<hkaiser> is there a solution to this?
<nikunj> if the linker flags are present then weak symbol should work.
<nikunj> Yes there is, the one I used at first
<nikunj> creating a separate static lib and using flag to link
<hkaiser> in the end I'd like for the user not to have to do anything in order to have a functioning build, one way or another
<hkaiser> can we at least detect at runtime that there is a problem?
<hkaiser> to report a more meaningful error or somesuch?
<nikunj> unfortunately not, without the addition of the linker flag my code would simply not work.
circleci-bot has joined #ste||ar
<circleci-bot> Success: hkaiser's build (#22058; push) in STEllAR-GROUP/hpx (optional_hpx_main) -- https://circleci.com/gh/STEllAR-GROUP/hpx/22058?utm_campaign=chatroom-integration&amp;utm_medium=referral&amp;utm_source=irc
circleci-bot has quit [Client Quit]
<nikunj> hkaiser: I understand what you're saying. I can try to make it happen
<nikunj> Actually most of it is already implemented. With the addition of your option, and inclusion of the linker flag things will work as expected
<hkaiser> except in cases like Phylanx...
<nikunj> yes. there we will need to add the linker flag, then only things will work
<hkaiser> k
<hkaiser> well, let's try to make things as seamless as possible
<nikunj> hkaiser: I will think of something
<hkaiser> :D
<nikunj> we only need to propagate the linker flag somehow. things will start working naturally
<nikunj> hkaiser: it passed all the tests in my environment
nikunj has quit [Quit: Leaving]
<github> [hpx] hkaiser pushed 1 new commit to optional_hpx_main: https://git.io/fNfed
<github> hpx/optional_hpx_main da8b014 Hartmut Kaiser: Propagate HPX_WITH_DYNAMIC_HPX_MAIN through HPXConfig.cmake...
nikunj has joined #ste||ar
nikunj has quit [Client Quit]
nikunj has joined #ste||ar
<hkaiser> let's see if this fixes things
<hkaiser> I have also added HPX_WITH_DYNAMIC_HPX_MAIN to HPXConfig (#3369)
<nikunj> hkaiser: that should make things run properly, let's wait for the tests.
<nikunj> hkaiser: where are link flags defined in hpx?
<nikunj> hkaiser: question. Does phylanx inherits the hpx related linker flags? if yes then there is a way to make it seamless
jaafar has joined #ste||ar
<heller> jbjnr: the call is still happening in 30 minutes, I guess?
<hkaiser> K-ballo: yt?
<K-ballo> here
<hkaiser> K-ballo: I'd like to use util::format() with variant values
<K-ballo> boost's?
<hkaiser> K-ballo: so I'd like for the format specifier to attempt extracting a particular type from the variant
<hkaiser> no ours
<K-ballo> ah, sure, the specifier is entirely up to your control for user defined types
<hkaiser> can't I use %d etc. with this?
<hkaiser> if I know the variant has an integer, %d would be convenient to use
<K-ballo> ...you could if you wanted to, but I don't think you'd want to.. I may be misunderstanding
<K-ballo> oh, I see
<hkaiser> variant<int, std::string> v = {}; format("%d", v);
<K-ballo> you'd want to define formatting for variants as dispatching
<hkaiser> I thought util::format could use extractor functions for each type
<hkaiser> if it encounters %d, it could call extract<int>(value)
<hkaiser> hwich would throw if the variant doesn't hold an int or something compatile
<K-ballo> sure, the value of the specifier is defined by the source type
<hkaiser> how would I do that?
<hkaiser> defining my own formatter, that is
<K-ballo> er, no.. that's the fallback one
<K-ballo> sorry, one sec
nikunj has quit [Quit: goodnight]
<K-ballo> template <typename ...Ts>
<K-ballo> void format_value(std::ostream& os, boost::string_ref spec, variant<Ts...> const& value) {
<K-ballo> if (spec.starts_with("%d"))
<K-ballo> return format_value(os, spec, get<int>(value))
<K-ballo> else if ...
<K-ballo> }
<K-ballo> throw std::runtime_error("Not a valid format specifier");
<hkaiser> ahh, ok - makes sense
<hkaiser> I'll try that
<K-ballo> whatever you put in the specifier is what you get as the `spec` parameter, totally user defined
<hkaiser> ok
<hkaiser> how does it behave if {%1%} is used?
<hkaiser> just converts it to a string? of just uses operator<<()?
<hkaiser> K-ballo: ^^
<hkaiser> (or whatever the generic syntax is)
<K-ballo> {1} is equivalent to {1:} which is equivalent to an empty spec
<K-ballo> index and specifier are dealt with separately, custom formater only gets the spec
<K-ballo> the empty spec has a special switch somewhere
<hkaiser> {1} just uses operator<<() then?
<K-ballo> no, it depends on the type
<hkaiser> how is that customized?
<K-ballo> {1} for an argument of type variant at that position would call custom formatter for variant, with an empty spec
<hkaiser> ok
<hkaiser> thanks
<K-ballo> if the argument is a variant then the custom formatter function of variant always gets called
<hkaiser> understood
<K-ballo> if there is no custom formatter, the fallback is op <<, yes
<K-ballo> I did not understand the question, but I hope I've answered nonetheless
<K-ballo> I don't think I've been clear.. fundamental types don't use op<<, only user defined types do
<K-ballo> ah! i figured it out
<K-ballo> hkaiser: you actually want to dispatch to format(...), not format_value(...), since the later is only for user defined types
<hkaiser> k
<hkaiser> K-ballo: I think I have a solution to what I want, thanks
<K-ballo> may I see it?
<hkaiser> sure, once it's ready I'll show it to you
<hkaiser> so I need to dispatch to format_to
<hkaiser> K-ballo: another question
<hkaiser> K-ballo: my argument is actually a vector<variant<>>, but I would like to treat the index in the vector as the positional index of the value to format
<hkaiser> K-ballo: i.e. format("%d, %d", vector<>{1, 2});
<hkaiser> K-ballo: i.e. format("%d, %d", vector<int>{1, 2});
<hkaiser> would that be somehow possible?
<hkaiser> I guess I could call detail::format with my argument vector
<K-ballo> no, that doesn't sound possible
<K-ballo> working directly with the implementation, I suppose it would be possible to transform the vector of arguments into a vector of argument descriptors
<heller> jbjnr: yt?
<hkaiser> K-ballo: yah, that's what I did
<K-ballo> is this about formatted output for scripting?
<hkaiser> kind of, yes
<K-ballo> aha
<K-ballo> if the spec is empty it should visit then format, which I assume is what op<< already does?
<hkaiser> yes
<K-ballo> format_args captures by const ref
<hkaiser> ok
<K-ballo> pretty cool
<K-ballo> I wished it could also piggy back on the type specs for fundamentals
<hkaiser> yah
<K-ballo> though I guess the most it could get is `type_specifier<long long>::value()` vs the current `"lld"`
<hkaiser> the library could extract the actula value through a customization point
<hkaiser> sprintf(..., get_value<...>(value))
<K-ballo> does phylanx have booleans?
<hkaiser> yes, but internally expressed as uint8_t
<hkaiser> K-ballo: there is no type_specifier<> for %s
<K-ballo> nope, only for fundamentals
<K-ballo> bool and string are special cased
<K-ballo> feel free to add them, make a comment that they are used elsewhere
<K-ballo> bool and string and pointers
<hkaiser> k
<hkaiser> K-ballo: is there a type_specifier for bool? I don't think so
<K-ballo> no, bool and string are special cased
<K-ballo> (also pointers)
<github> [hpx] hkaiser created format_string_spec (+1 new commit): https://git.io/fNfLU
<github> hpx/format_string_spec 05c395e Hartmut Kaiser: util::format: add type_specifier<> specializations for %s and %ls
<K-ballo> ah, at the printf level you meant
<github> [hpx] hkaiser opened pull request #3370: util::format: add type_specifier<> specializations for %s and %ls (master...format_string_spec) https://git.io/fNfLL
<hkaiser> right
Vir- has quit [Ping timeout: 265 seconds]
Vir has joined #ste||ar
<hkaiser> jbjnr: all pycicle links give a 404 (for instance http://cdash.cscs.ch/None/index.php?project=HPX&date=2018-07-08&filtercount=1&field1=buildname/string&compare1=63&value1=3369-optional_hpx_main)
<github> [hpx] hkaiser pushed 1 new commit to master: https://git.io/fNfmy
<github> hpx/master 91f76e9 Hartmut Kaiser: Merge pull request #3369 from STEllAR-GROUP/optional_hpx_main...
Viraj has joined #ste||ar
Viraj has quit [Client Quit]
diehlpk has quit [Ping timeout: 260 seconds]
<hkaiser> M-ms: all pycicle links give a 404 (for instance http://cdash.cscs.ch/None/index.php?project=HPX&date=2018-07-08&filtercount=1&field1=buildname/string&compare1=63&value1=3369-optional_hpx_main)