hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/ | GSoC: https://github.com/STEllAR-GROUP/hpx/wiki/Google-Summer-of-Code-%28GSoC%29-2020
<gnikunj[m]> hkaiser: yt?
hkaiser has quit [Ping timeout: 260 seconds]
kale[m] has quit [Ping timeout: 240 seconds]
Yorlik has quit [Ping timeout: 265 seconds]
kale[m] has joined #ste||ar
kale[m] has quit [Ping timeout: 256 seconds]
fred[m] has left #ste||ar ["Kicked by @appservice-irc:matrix.org : Idle for 30+ days"]
Yorlik has joined #ste||ar
hkaiser has joined #ste||ar
<gonidelis[m]> @channel Does anyone who uses Visual Studio Code editor know any addon or some setting where the editor could highlight the angle brackets pair (`< >`) like it does with parenthesis and curly brackets?
kale[m] has joined #ste||ar
<gonidelis[m]> hkaiser: K-ballo When I use `util::in_out_result<InIter, OutIter>{std::move(first), std::move(dest)}` with `{}` is it a member init-list? while when I use it with parenthesis `()` like `util::in_out_result<InIter, OutIter>(std::move(first), std::move(dest))` it's a function call? (whch does not succeed as `in_out_result` F.O. does not accept arguments ???)
<K-ballo> it's a braced-init-list, member init-list is the thing after the : in constructors
<gonidelis[m]> K-ballo: ahhh yeahh... you are right. Is there any case where `in_out_result(args)` would be valid?
<K-ballo> if it had a viable constructor for said arguments, or if args where a single instance of in_out_result, or if C++20, or..
<gonidelis[m]> wait... why "if C++20" ?
<K-ballo> C++20 does aggregate initialization for parens too
<zao> I really wish that VC++ had better errors for when you misspell a designated initializer.
<gonidelis[m]> is it me, or people do actually complain like a lot about all these init options?
<zao> It doesn't recognize that it's the wrong name, just gets you a generic "can't convert from aggregate thingiemabob to T"
<K-ballo> did you search for C++ bonkers initialization?
<K-ballo> > I counted eighteen different forms of initialization
<gonidelis[m]> I think I saw a cppcon talk on utube where a guy was explaining all the many options...
<gonidelis[m]> I can't imagine why did they introduce parenthesis, too
<K-ballo> introduce parens?
<gonidelis[m]> actually it makes c++ more generic, so maybe that's the goal after all
<K-ballo> oh you mean aggregate-init via parens?
<gonidelis[m]> K-ballo: as an aggreg-init option
<gnikunj[m]> freenode_gonidelis[m]: you're not alone when it comes to init options. I get confused most of the time myself and have to rely on compiler error report to correct them ;-)
<gonidelis[m]> yup
<K-ballo> because generic libraries everywhere use parens
<K-ballo> consider vector<in_out_results> vs; vs.emplace_back(in, out);
<K-ballo> that's going to end up doing in_out_result(in, out) internally
<gonidelis[m]> yeah yeah... seems more abstract.
<K-ballo> abstract?
<gonidelis[m]> generic
<K-ballo> the alternative was for generic libraries *everywhere* to do a two-step dance in which if initialization with parens fails, then it tries again with braces
<K-ballo> (I voted against that alternative... even though I never believed we would be getting parens for aggregate-init)
<gonidelis[m]> oh you are a member too? how many guys here are members ;p ?
<gonidelis[m]> K-ballo: do you aggree with parens aggreg-list?
<K-ballo> yes, it was the ideal solution.. I just thought we wouldn't be able to afford it
<K-ballo> braces were meant to be "uniform initialization", adding parens for aggregate-init was admitting complete failure
<gonidelis[m]> that's unfortunate ;/
<K-ballo> well, unicorn initialization was a complete failure, so unfortunate but fair
<gonidelis[m]> but it sounds like a good idea...
<K-ballo> what does? uniform initialization? sure
<gonidelis[m]> yeah
<gonidelis[m]> but then you tell me it failed
<gonidelis[m]> hkaiser: yt??
<hkaiser> here
<K-ballo> well it did
<K-ballo> the idea was future C++ code would use braces and braces only for initialization
<K-ballo> but given the awkward weird odd rules we got, the guide today is to always use parens unless braces are explicitly needed
<gonidelis[m]> hkaiser: ok I have completely adapted the 1st overload result type to `in_out_result`. So `make tests.unit.modules.algorithms.transform` compiles fine... but `make tests.unit.modules.algorithms` won't compile as there are same overloads under /container_algorithms
<hkaiser> gonidelis[m]: so those need adaptation too
<gonidelis[m]> The thing is that we will soon change container_agorithms/transform.hpp in order to have overloads under ranges::transform namespace (instead of hpx::parallel::v1)
<hkaiser> yes, but the old overloads should still stay for compatibility reasons (at least for a while)
<gonidelis[m]> yeah I know... the question is. Should I commit and push the adapatation so far?
<hkaiser> if the tests pass, sure
<hkaiser> you can always push to github to see whether all tests are fine
<gonidelis[m]> but it's just the parallel tests. Not the container ones
<gonidelis[m]> ohh ok... So you mean the CI tests and stuff...
<gonidelis[m]> ok the reason I am asking is because I was told not to push code that does not (fully?) compile
<hkaiser> gonidelis[m]: in the end _all_ tests should pass before your start working on the next steps
<hkaiser> gonidelis[m]: nah, you can always force push (with-lease) if you want to amend a commit
<gonidelis[m]> ok I will change the result type of the container transform before pushing then... just to be sure that I won't get caught up into some spaghetti situation again
<hkaiser> (but force-push only on your branch, please)
<gonidelis[m]> hkaiser: What do you mean only on my branch?
<hkaiser> never force push to anything that is not your own work
<hkaiser> at least not without talking to the original author
<gonidelis[m]> oh yeah... of course
<gonidelis[m]> ok copy that
<gonidelis[m]> I am very annoyed by the fact that when I compile `unit.tests.algorithms` there are errors popping on the container side ~.^
<gonidelis[m]> hkaiser: ^^
<hkaiser> :D
<hkaiser> everything is connected... - like in real life
<gonidelis[m]> Isn't there a different target for container algos?
<gonidelis[m]> (ahh yeah actually there isn't)
<gonidelis[m]> hkaiser: ;p
<hkaiser> not sure
<hkaiser> the range algorithms are in the same module with the non-range versions, I think that's why they are under the same target
<gonidelis[m]> K-ballo: weird odd rules?
<gonidelis[m]> hkaiser: yup exactly ;)
<K-ballo> classes with constructors from std::initializer_list get a unique kind of overload resolution mechanism on a two-step overload resolution
<K-ballo> resulting in silly things like `Q x1{}, x2{x1};` where x2 is not copy-constructed from x1
<K-ballo> imagine that.. you have a function that returns Q, and you want to initialize a Q object from it: `Q r{fun()};`, and you get something different than what `fun()` returned
<gonidelis[m]> what? lol why? ;p
<gonidelis[m]> because fun() is treated as an r-value?
<K-ballo> no, because Q has a constructor with std::initializer_list, and so it gets special treatment
<K-ballo> it gets a special round of overload resolution considering only the std::initializer_list-based constructors
<K-ballo> and it matches, so it never goes look into the "normal" constructors, it never finds the copy constructor
<gonidelis[m]> wow... that's ultra stupid :D
<K-ballo> so stupid that two out of three implementations outright refused to implement it
<K-ballo> are you familiar with nlohmann::json?
<gonidelis[m]> ahh no...
<K-ballo> ok well, that's a well-known example of Q
<gonidelis[m]> K-ballo: what do you mean "two out of three impls" ?
<K-ballo> clang and msvc
<gonidelis[m]> wow!
<K-ballo> only gcc fully implements the weird odd rules
<gonidelis[m]> that's a nice mutiny ;p
<gonidelis[m]> hkaiser: Just pushed #4888 . It's just the result_type adaptation but it should compile completely! https://github.com/STEllAR-GROUP/hpx/pull/4888
<gonidelis[m]> ...in case you wanna check
<gonidelis[m]> So! In case tests pass, should I proceed on creating the cpo for the hpx:: namespace?
<gonidelis[m]> and then for the hpx::ranges:: namespace...
<hkaiser> will have a look, thanks!
<hkaiser> yes, cpos next
<gonidelis[m]> hkaiser: thanks!
<gonidelis[m]> is `jenkins/cscs/clang-oldest ` test supposed to fail?
<gonidelis[m]> oh actually I get a clang error on `transform.hpp : line 295` : `undefined ref ...`
<gonidelis[m]> but why?? ~.^
<gonidelis[m]> it seems like it comes from `partitioned_vector_transform_reduce2_test`
<gonidelis[m]> hkaiser: Could it be that the segmented algos don't match ?
<hkaiser> yah, the segmented algos are tricky, I know
<hkaiser> I'd suggest to change them minimally only, just to conform to the result types
<hkaiser> I'd be happy to help with that
<gonidelis[m]> hkaiser: I am changing the `transform_` functions result type undre segmented_algos/
<gonidelis[m]> but... this one transform_ returns a `segmented_transform()` func which by its turn returns a pair...
<gonidelis[m]> So I should change the result type of the segmented_transform too in order to compile
<gonidelis[m]> shouldn't I?
<gonidelis[m]> hkaiser: what if I change only the result type of the non-segmented transform_'s
<gonidelis[m]> Would that fail to compile?
<gonidelis[m]> hkaiser: I tried it and it didn't compile as expected. The two transform_'s should return the same type because they are invoked from `transform()`
<gnikunj[m]> hkaiser: Things aren't exactly working with mpi backend right now as well. I'm trying to get a minimal code that generates the seg fault I'm seeing. I'll add a ticket then for you to go through.
<gonidelis[m]> So... the problem is this: I could try to minimally change the result types of all the transform_ overloads (both segmented and non-segmented). But the segmented ones invoke the `segmented_transform()` functions so we should change them too....
<gonidelis[m]> hkaiser: What's the plan?
<hkaiser> gnikunj[m]: uhh
<hkaiser> gonidelis[m]: yah, I'd need to have a closer look
<hkaiser> but changing the return types shouldn't be difficult
<gonidelis[m]> hkaiser: actually i can see that you changed the return types of segmented_transfer for you copy adaptation! So I will imitate that ;)
<hkaiser> ok
<hkaiser> the infrastructure for in_out_result to work with the segmented algorithms should be in place, so you shouldn't have any trouble
<gonidelis[m]> cool...
<gonidelis[m]> I ll push asap so you can check ;)
<gonidelis[m]> I have this german keyboard (by accident) on my laptop and it has this special `<>|` key and it's like so useful for template programming ;p
<gonidelis[m]> hkaiser: well I 've done most of the job for the first segmented overload
<gonidelis[m]> let me indicate to you two spots that need fixing
<gonidelis[m]> hkaiser: I stared a review where necessary actually ;)
<gnikunj[m]> hkaiser: looks like the minimal file in #4886 throws a seg fault as well
<gnikunj[m]> again, it is happening only with mpi backend as of now. Not with tcp backend. I'll do some more testing to see if I can reproduce the error on tcp backend
<hkaiser> gnikunj[m]: show me the code, I'll have a look
<hkaiser> gnikunj[m]: that worked for me just fine
<gnikunj[m]> yea, that doesn't show up everytime
<gnikunj[m]> try running it for 5-6 times and you'll see it failing at times
<gnikunj[m]> with mpirun it will come to hault after its completed execution and with srun it will throw seg fault
<hkaiser> how does it fail?
<gnikunj[m]> that's beyond my scope of understanding
<gnikunj[m]> it just completes execution
<gnikunj[m]> like outputs the necessary stuff
<gnikunj[m]> and then seg faults
<gnikunj[m]> or haults
<hkaiser> where does it segfault?
<gnikunj[m]> let me create a gist and show you
<hkaiser> gnikunj[m]: have you tried fixing the slurm warnings?
<gnikunj[m]> not exactly
<gnikunj[m]> they're just "warnings" so I thought they'll be harmless
<hkaiser> you allocated 8 nodes through slurm, but ran on 9 localities
<gnikunj[m]> yea that happens with mpirun inside of a screen session
<gnikunj[m]> let me show you srun output. you won't see those outputs there
<hkaiser> ahh, you're running all of the localities nthe same node?
<gnikunj[m]> no
<gnikunj[m]> I realized I was inside a screen session
<hkaiser> yes
<hkaiser> gnikunj[m]: also, does this happen with debug builds too?
<gnikunj[m]> I haven't tried with debug
<gnikunj[m]> let me talk to ali and fix those warnings first, just to make sure that it has nothing to do with them
<hkaiser> the segfault happens during initialization
<gnikunj[m]> is hpx not initialized properly?
<hkaiser> no idea
<hkaiser> the fact is that you requested 8 nodes from slurm, but launched 9 hpx localities
<hkaiser> ok, what's different now?
<gnikunj[m]> its exactly the same. I was trying to explain you that 8 nodes and 9 localities was not the case
<hkaiser> well, you use mpirun -np 9 ...
<gnikunj[m]> yes, that was on a screen session of 9 nodes
<hkaiser> it wouldn't complain about a number mismatch
<gnikunj[m]> it generally does, I've had this issue on other clusters as well
<gnikunj[m]> and its not limited to HPX btw
<hkaiser> well, the hpx::init warning is HPX specific
<gnikunj[m]> I've seen this warning with charm++ as well
<gnikunj[m]> yes, the hpx::init warning is HPX specific. The MPI warnings are general
<hkaiser> I'm talking about the HPX warnings
<gnikunj[m]> I have no clue why it complains though. I use an allocation of 2 nodes. Why would it say that there are only 1?
<hkaiser> no idea what you're talking about
<gnikunj[m]> btw if I reduce to -np 8, it complains: hpx::init: command line warning: --hpx:localities used when running with SLURM, requesting a different number of localities (8) than have been assigned by SLURM (9), the application might not run properly.
<gnikunj[m]> when I do -np 9, it complains: hpx::init: command line warning: --hpx:localities used when running with SLURM, requesting a different number of localities (9) than have been assigned by SLURM (8), the application might not run properly.
<gnikunj[m]> I did screen srun -p marvin -N 9 --pty /bin/bash
<gnikunj[m]> and I'm running that example inside of there
<hkaiser> ok
<gnikunj[m]> hkaiser: I've mailed ali about this. He should be able to tell if it is a cluster related issue.
kale[m] has quit [Ping timeout: 240 seconds]
kale[m] has joined #ste||ar
<gonidelis[m]> hkaiser: yt?
<hkaiser> here
<gonidelis[m]> The copy CPO forwards to `transform_` here
<gonidelis[m]> should I pass std::false_type for the is_segmented arg ?
<gonidelis[m]> hkaiser: ^^
<hkaiser> the copy cpo forwards to transform?
<gonidelis[m]> sorry ;p The transform CPO **
<hkaiser> ok
<hkaiser> if the iterators are non-segmented, then yes
<gonidelis[m]> Well actually there is the question of whether I should forward to `transform_` or `transform` (here https://github.com/STEllAR-GROUP/hpx/blob/eca2139340701b346677723b01187184a6092abb/libs/algorithms/include/hpx/parallel/algorithms/transform.hpp#L132)
<hkaiser> you should dispatch to transform_
<gonidelis[m]> ok so for the overload that takes the ex_policy I shall also pass an is_segmented trait check
<hkaiser> yes
<gonidelis[m]> and for the overload that does not take execution policy I pass `std::False_type`
<gonidelis[m]> great!
hkaiser has quit [Quit: bye]
<gonidelis[m]> what's the difference between `parallel::util::projection_identity()` and `parallel::util::projection_identity{}`
<gonidelis[m]> ?
<K-ballo> none
<gonidelis[m]> ha!
<gonidelis[m]> thanks!
<gonidelis[m]> didn't expect that
<K-ballo> empty type, no constructors, innocent enough
<gonidelis[m]> ahh yeah that's right!
<gonidelis[m]> When the result type according to standrad is ` constexpr OutputIterator` then could the return type be ` friend typename parallel::util::detail::algorithm_result<ExPolicy, in_out_result<FwdIter1,
<gonidelis[m]> FwdIter2>>::type`
<K-ballo> constexpr applies to entities (the function), not types
<gonidelis[m]> I am confused...
<gonidelis[m]> some overloads return `constexpr OutputIterator` and some return `ForwardIterator2`
<K-ballo> constexpr doesn't apply to types, the return type is not `constexpr OutputIterator`, it's just `OutputIterator`
<K-ballo> parallel algorithms require forward iter or better, they do not support plain input/output iter
<gonidelis[m]> oh okk...
<zao> Keyword-wise, it's similar to `static`, `virtual`, affecting the function itself.
<gonidelis[m]> yeah yeah I get that... I have a problem in understanding what the return types are accroding to standard.
<gonidelis[m]> In any case there is no `in_out_result` type retured
<gonidelis[m]> rephrase: There is no `in_out_result` type returned in any non-ranges case
<gonidelis[m]> But we have changed every `transform_` algo to return an `in_out_result`
<K-ballo> non-ranges don't return in, because for them it's simply `last`
<gonidelis[m]> yeah... exactly
<gonidelis[m]> http://eel.is/c++draft/alg.copy this algo has the same result types... I mean `OutputIterator` for algos with no exec-policies and `ForwardIterator2` for algos with exec-policies
<K-ballo> the type corresponding to `result`, yes
<gonidelis[m]> ahhh... I think that's what I need to return for the non-ranges algo *with* the exec-policy `parallel::util::detail::algorithm_result<ExPolicy,
<gonidelis[m]> FwdIter2>::type`
<gonidelis[m]> So the non-ranges algos do not return a pair any more?
<gonidelis[m]> do not have an `.in .out` member
bita has joined #ste||ar
bita has quit [Ping timeout: 246 seconds]
hkaiser has joined #ste||ar
<gonidelis[m]> hkaiser: you might have a github notific ;)
<hkaiser> :D
<hkaiser> do all the tests pass?
<gonidelis[m]> yes locally, they do
<hkaiser> well, then I approve
<gonidelis[m]> the thing is that I altered the tests a little bit accordingly... please check the `tag_invoke` result types (compared to standard) when you have some spare time
<hkaiser> ok, I'll check the branch out and have a look
<gonidelis[m]> hkaiser: thanks a lot... I shall work on the ranges CPO until the github tests complete
diehlpk_work has quit [Ping timeout: 244 seconds]
kale[m] has quit [Ping timeout: 260 seconds]
kale[m] has joined #ste||ar
<gonidelis[m]> Why are there no exec-policies for ranges::algos?
<gonidelis[m]> K-ballo:
<K-ballo> harder to spec
<K-ballo> we are still even missing a bunch of other non-parallel algorithms in range form
<K-ballo> they will happen eventually, given enough time
<gonidelis[m]> but... we do have ranges overloads with exec-policies
<K-ballo> we are not the standard, we don't spec anything
<gonidelis[m]> what do you mean, "spec" ?
<K-ballo> produce a formal specification, what the standard is
<gonidelis[m]> oh ok... So we do provide praallel ranges algos
<K-ballo> we are not the standard
<K-ballo> we can just make up any interface we wish and put it to code
<gonidelis[m]> hmmm.... so standard C++ coders cannot parallelize ranges ?
<K-ballo> I'm not sure what that question means.. is that coders who can only use the standard library and nothing else? can they at least build their own abstractions?
<gonidelis[m]> I mean if I write a program based on std C++20 I can not run sth like ranges::for_each(policy::par, first, last, etc....)
<K-ballo> they can always adapt their ranges to old-style iterators and call the old-style parallel algorithms, with suboptimal results compared to a true ranged approach
<gonidelis[m]> ok that's smart ;p but I am just asking about the default algorithm overloads provided
<K-ballo> there are no ranged parallel algorithms in the standard yet, no
<K-ballo> execution policies are non-ranges only
<zao> Unless library implementers are willing to introduce surprising extensions to their libraries, they're kind of tied to what the standard specifies for current and future versions.
<hkaiser> hah, 'surprising'
<gonidelis[m]> zao: lol
<gonidelis[m]> hkaiser: How do I create the `indirectly_­writable` requirement? Do we have a trait or sth?
<hkaiser> gonidelis[m]: we don't have that
<gonidelis[m]> http://eel.is/c++draft/alg.transform#itemdecl:1 you can see it's needed here
<hkaiser> but we have is_output_iterator which is roughly similar
<gonidelis[m]> hmm.... ok
<gonidelis[m]> so I put it inside `hpx_concept_requires_` macro
<hkaiser> gonidelis[m]: we usually check for is_iterator on the API and have a static_assert for the iterator category to improve error messages
<gonidelis[m]> hkaiser: yeah that sounds right...
<gonidelis[m]> ok
<gonidelis[m]> hkaiser: Do we need overloads withou ex-policies too? I mean for the ranges::algos
<hkaiser> yes
<gonidelis[m]> hkaiser: So is it like, the standard has 3 overloads and then I provide 6? 3 according to std and 3 similar plus the exec-policy?
<hkaiser> yes
<gonidelis[m]> hkaiser: thanks a lot!
<gonidelis[m]> hkaiser: what about `indirect_result_t`??
nanmiao11 has quit [Remote host closed the connection]
<hkaiser> gonidelis[m]: we might have similar things, look at the other algorithms
<gonidelis[m]> hkaiser: ok
<hkaiser> copy should be similar enough
<gonidelis[m]> ahh... there are some changes... we can look into that on tuesday though
<gonidelis[m]> hkaiser: btw I don't think that the failed tests have anything to do with my PR
bita has joined #ste||ar
<hkaiser> gonidelis[m]: yah, those errors are unrelated
kale[m] has quit [Ping timeout: 244 seconds]