hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
jaafar has quit [Ping timeout: 268 seconds]
K-ballo has quit [Quit: K-ballo]
nikunj has quit [Ping timeout: 240 seconds]
nikunj has joined #ste||ar
parsa is now known as parsa_
nikunj has quit [Ping timeout: 272 seconds]
nikunj has joined #ste||ar
hkaiser has quit [Quit: bye]
nikunj has quit [Ping timeout: 240 seconds]
mbremer has quit [Quit: Leaving.]
jaafar has joined #ste||ar
jaafar has quit [Client Quit]
jaafar has joined #ste||ar
nikunj97 has joined #ste||ar
jbjnr has quit [Ping timeout: 250 seconds]
jbjnr has joined #ste||ar
david_pfander has joined #ste||ar
<heller_>
simbergm: good Job
<heller_>
So the only test failing on the release branch is the executor parameters test
<Yorlik>
Is there any recommended way of learning HPX, like studying certain examples, doing certain reads or videos? I know there's a bunch of stuff out to use already, just wondering if there is some recommendation.
<Yorlik>
OFC I could just atart randomly and digging through the material available ...
<Yorlik>
s/atart/start/g
<heller_>
well, what's the first thing you want to try out ;)?
<Yorlik>
I want to write a multithreaded lua scripting environment that starts multiple lua states which run small event handlers written in lua
<Yorlik>
:)
<Yorlik>
The objects sending and receiving these events live on the C++ side of things
<Yorlik>
Thy will be self contained userdata for the lua side.
<Yorlik>
I want any request to be able to be handled on any lua state, since the lua states will not hold any mutable dfata
<Yorlik>
So I want to right jump in :)
K-ballo has joined #ste||ar
<heller_>
Yorlik: break it up in small pieces and then start from there
<Yorlik>
Yup
<Yorlik>
In the moment I am pondering the different ways to start and initialize GPX
<Yorlik>
HPX
<Yorlik>
The Lua Engine will be a component
<Yorlik>
The Quickstart in the docs is nice
<Yorlik>
I think I'll start the third approach described and use the main thread for a basic REPL UI
<heller_>
well, then start of with writing a component and dispatching actions to it
<heller_>
sure
<Yorlik>
This first simulator is meant as a script development environment
<heller_>
that's be a start
nikunj97 has joined #ste||ar
<heller_>
that'd be a start
<Yorlik>
Its not networked and not going to be demonized
<heller_>
this main thread can then later be used to get your events
<heller_>
from some websocket or whatever
<Yorlik>
There are many questions to ponder: e.g. how will we distribute our work over a cluster and move around data for load balancing.
<heller_>
first things first
<Yorlik>
We have an idea for a tiled world map and tiles beimng movable between machines.
<Yorlik>
Client (player) connections would also migrate
<Yorlik>
Thats all for later
diehlpk_work has quit [Remote host closed the connection]
<heller_>
Yorlik: just as a heads up. We don't have any way to secure the network traffic between HPX apps
<Yorlik>
Ow
<heller_>
so regarding your client/server approach, you need some kind of layer in between
<Yorlik>
What about your remote calls?
<Yorlik>
Like a remote future run
<Yorlik>
We are building a networking library for our own purposes
<Yorlik>
However - seems we'll have to look at that
<Yorlik>
Thanks for the heads-up
<heller_>
remote calls use networking, indeed
<heller_>
but no SSL or whatever
<heller_>
just plain network traffix
<heller_>
c
<Yorlik>
So you're living in this cozy world of unuversity clusters where everyone is well educated and not misbehaving ? ;)
<heller_>
so yeah, I am wondering why the heck it was defined in new
aserio1 has joined #ste||ar
aserio has quit [Ping timeout: 250 seconds]
aserio1 is now known as aserio
khuck has joined #ste||ar
david_pfander has quit [Quit: david_pfander]
<khuck>
hkaiser: I confirmed that merging HPX PR 3661 is what broke the Phylanx build. But I don't know why. Is there some dependency between the slice_node_data object in Phylanx and the HPX lcos::latch object?
<K-ballo>
may be using an old version still? no idea how all that stuff is set up
<khuck>
I am using gcc 7.1
<khuck>
not sure what HPX version they are pulling, I would hope current master
<K-ballo>
-- HPX version: 1.3.0 (6c8d2771e0)
<khuck>
ahhhhhh
<K-ballo>
that's the lcos::latch PR merge commit
<khuck>
that's right
<khuck>
weird
<khuck>
CircleCI is configured to build with Clang, not GCC
<hkaiser>
khuck: I frankly don't see how a change in HPX could trigger for a Phylanx template function instantiation to go missing
<khuck>
hkaiser: I'm just the messenger...
<hkaiser>
sure, I'm not shooting
<hkaiser>
K-ballo: so the latch merge commit makes it break?
<khuck>
hkaiser: it does on x86_64 with gcc 7.1 - I am still running the build for clang 7 on power64le
<hkaiser>
strange
<khuck>
agreed. (cue X-Files music...)
<hkaiser>
khuck: we could try adding an PHYLANX_EXPORT to that function, it is instantiated in one translation unit only, somehow the compiler makes it implicitly 'inline' (i.e. local to that TU)
<hkaiser>
but another TU is complaining about it not being available during link time
<khuck>
still building, but got past that linker stage
<hkaiser>
k, thanks
<khuck>
I can submit a PR, if you like
<hkaiser>
yes, please
<hkaiser>
no idea why this is necessary, but hey - it works
<khuck>
this probably breaks the clang build now ;)
<hkaiser>
lol
<hkaiser>
don't think so
<hkaiser>
it removes an optimization
<khuck>
how so?
<khuck>
I thought you guys loved defining functions in header files
<K-ballo>
ouch
jbjnr has quit [Ping timeout: 240 seconds]
aserio has joined #ste||ar
aserio has quit [Quit: aserio]
aserio has joined #ste||ar
<aserio>
khuck: Are you around for the GB Meeting?
<khuck>
aserio: on another call that is wrapping up
<aserio>
Ok :)
<K-ballo>
why do actions need a result type, a remote result type, and a local result type (and why none of those is just R)?
<heller_>
K-ballo: There is the result from the function the action represents, and sometimes, this local result, is being converted before sending over the wire
<heller_>
K-ballo: I think it mostly for historic reasons
<heller_>
we use for gid_type->id_type conversion, IIRC
<heller_>
having just R would be a good thing, I guess
<heller_>
K-ballo: another thing, does it make sense to have a special function(function_ref&&) conversion?
<K-ballo>
what would it do? special casing is usually problematic
<heller_>
forwarding the vtable and calling the move ctor through it?
<heller_>
saving an indirection, basically
<K-ballo>
you'd need to synthesize a special vtable
<heller_>
just a thought
<heller_>
a use case I had in min was future_data_base::set_completion_handler
<heller_>
which could take a function_ref, if the future is ready, call it, if not, "forward" it to a function to store it
<K-ballo>
keep in mind you'd need to guarantee the function_ref will not dangle when stored
<heller_>
yes
<heller_>
you'd have to copy the stored object, somehow
<heller_>
just came to my mind...
<K-ballo>
guy from texas back on the slack channel having trouble
<K-ballo>
"Is "openmpi" or "hpx-openmpi-devel" required to create hpx programs", etc
<diehlpk_work>
K-ballo, Are you talking about the fedora packages?
<K-ballo>
diehlpk_work: no idea, maybe..
<K-ballo>
he does mention fedora29
<diehlpk_work>
Ok, this could be my fedora packages
<diehlpk_work>
hpx-openmpi-devel has openmpi as its dependenci
<K-ballo>
I think he is using your packages, yes
<diehlpk_work>
*dependency
<diehlpk_work>
Could you point him here to irc?
<diehlpk_work>
I would be happy to help him
<diehlpk_work>
simbergm, 50% of the arches built on fedora 30, 29, 28
<diehlpk_work>
libhwloc.so.5()(64bit) on my machine
<diehlpk_work>
Maybe you can not see hwloc on your machine, because it is already installed
<diehlpk_work>
As far as I know dnf install shows only missing dependencies
<diehlpk_work>
Once you installed any other package, which has hwloc as one dependency, it will not be shown again
<grostig>
@diehlpk_work, yes I see it on my system, even in the listing I posted. I did have a problem and installing it via dnf seemed to be required, but I'm not totally sure since I had many other problems.
<grostig>
dnf did do an install of it after installing hpx, so I don't know why that is.
<diehlpk_work>
Ok, so we can close the ticket?
<diehlpk_work>
In addition hpx on fedora just installs the minimal set of dependencies
<heller_>
has the cmake issue been resolved?
<grostig>
@diehlpk_work, I'm not sure of the pressure to close it quickly, maybe that is how people want to do it on github, but I still can't get anything to work. I can test it on another machine and get back to you. As far as I'm concernted it is not _verified_ as fixed yet.
<diehlpk_work>
Ok, grostig So dnf install hpx does not work for you?
<diehlpk_work>
I tested it on fedora 29 ten minutes ago
<grostig>
cmake is still very broken for me on fedora29, but I see a partial fix may be in the works for 1.3, but I don't have a package for that yet. So I'm stuck.
<heller_>
hmm
<heller_>
we are preparing a 1.2.1 right now
<diehlpk_work>
grostig, It would help to see your dnf error message
<heller_>
so a fix for that should definetly go in there
<grostig>
@heller_, did you see my comments on github about the cmake problem?
<heller_>
on it
<grostig>
@diehlpk_work, let me walk over to a clean machine and see what happens, back in 20 minutes.
<heller_>
I am seeing the message, yes
<heller_>
grostig: alright ...
<heller_>
grostig: the fix got merged three days after we released 1.2 ;)
<diehlpk_work>
simbergm, heller_ We should add it to 1.2.1
<heller_>
this will get interesting
aserio has joined #ste||ar
<heller_>
note to ourselves: we shouldn't do point releases, it's never just "that one simple change"
jbjnr has joined #ste||ar
<heller_>
diehlpk_work: is there some way to check if the fedora package actually works? This is kind of embarassing...
<heller_>
and should have been detected early on
<diehlpk_work>
I checked that the examples run and I check to compile a simple program using gcc
<diehlpk_work>
But not for openmpi and mpich
<heller_>
the cmake issue isn't related to openmpi or mpich
<heller_>
it's related to things not being fully installed, for some reason
<heller_>
so: does the fedora build server allow to have a simple "crash&burn" test?
<heller_>
fully automated
<heller_>
that marks the build failed if something basic goes wrong?
<diehlpk_work>
heller_, any ctest you want to run
<diehlpk_work>
I test the examples currently
<heller_>
it's useless to test against the build tree
<heller_>
you need to test against the actual installation
<diehlpk_work>
heller_, I installed hpx on my vm and laptop
<diehlpk_work>
all the examples are running
<diehlpk_work>
I will test cmake now
<heller_>
there's no doubt that the examples won't run, that's not the problem that was reported ;)
<grostig>
@diehlpk_work, ok so I installed hpx (and only hpx), and it installed "hwloc-libs" and only that, not "hwloc". All I can say at this point is that "hwloc" _may_be_ required also to get a compile to work. I don't know cmake so can't tell what the _true_ dependencies are. All I can tell you is that using Qmake, or just make, I seem to recall that I needed to also install "hwloc". Not sure of what the difference is. Once I get something working at all in
<grostig>
fedora29, I work on this subtle issue of "hwloc".
<diehlpk_work>
grostig, What is the issue?
<grostig>
@diehlpk_work, also please note that I appreciate your efforts. please understand that I have been using unix since 1984, but an most on expert in complex build issues with cmake and various libraries.
<heller_>
IIUC, you'd need the devel package to build against HPX, right?
<diehlpk_work>
Sure, I just want to understand what the issue is for hwloc
<grostig>
@diehlpk_work, the issue is that one has to install hwloc, in addition to hwloc-libs, as far as I know. Does the issue have to be closed today?
<heller_>
no
<grostig>
@heller_, yes, that was my next issue. and what about all the other fedora29 packages, what specifically do they do and when are they needed?
<heller_>
I have no idea
hkaiser has joined #ste||ar
<diehlpk_work>
grostig, I just did a dnf install hpx hpx-examples it installed hwloc for me and I could run the examples
<grostig>
@diehlpk_work, when I say other fedora29 packages, I meant the other "hpx*" packages.
jbjnr_ has joined #ste||ar
<grostig>
@diehlpk_work, that sounds good, but the helloworld documentation does not specify one has to install "hwloc-examples", so that does not resolve my issue.
<diehlpk_work>
grostig, dnf info hpx
<diehlpk_work>
Shows you the description of the package
<diehlpk_work>
hpx, hpx-openmpi, and hpx-mpi contains the libraries dependent on the compiler
jbjnr__ has quit [Ping timeout: 240 seconds]
jbjnr has quit [Ping timeout: 240 seconds]
<diehlpk_work>
hpx-*-examples the examples with * as compiler
<diehlpk_work>
hpx-*-devel the development headers for each compiler
<heller_>
so the problem is a lack of documentation? Where is that documented for other packages? Is there a place for that on the fedora site?
<grostig>
@diehlpk_work, yes I figured that out too, but those descriptions are not explicit enough for me (a native english speaker, but not familiar with all "linux" insider information) to be sure what they mean. For example, on "hpx-devel", does "development headers" mean development of programs that use hpx, or develpment of hpx itself. I presumed that latter (2nd). So I installed it.
<grostig>
@diehlpk_work, correction, I presumed the former/1st, ie. writing programs that us HPX, so I installed that.
<diehlpk_work>
grostig, it follows the package guide lines from fedora
<grostig>
@diehlpk_work, "hpx" says "runtime" doesn't sound like it would include headers, but maybe it does.
<grostig>
@diehlpk_work, yes, as I allready stated, I don't know the "insider linux" or fedora knowledge.
<diehlpk_work>
Yes, I understand, but I can not change the names of the packages, sorry for that
<diehlpk_work>
I think it could be different, but I have to follow the guide lines
<grostig>
@diehlpk_work, I'm just asking for documentation on the HPX site, and maybe a bit more documentation in the desrciption of the package. I would not expect the fedora people to document this.
<diehlpk_work>
Sure, we can add more documentation to our wiki
<grostig>
@diehlpk_work, bottom line is you still have not confirmed to me what I need to install and when to do get helloworld to compile and link.
<diehlpk_work>
grostig, Just to be sure, do you want to run the hello_world or do you want to compile it by yourself?
<grostig>
@diehlpk_work, why the wiki, where is that even? Here is the page I would expect it in the early part of the "getting started with compiled packages" in the main documentation.
<diehlpk_work>
So if you just want to execute the compiled hello_world you install hpx-devel and run hpx_hello_world
<grostig>
@diehlpk_work, if you look back at my comments, you will see that I have asked a number of questions which you didn't answer directly, but so indirectly that I could not be sure what you meant if you were trying to answer.
<grostig>
@diehlpk_work, never once did I say i wanted to compile the examples, or run the examples. I want to compile helloworld, just like it says in "Quick start/helloworld".
<diehlpk_work>
Ok, so you want to compile HPX by yourself?
<grostig>
@diehlpk_work, @heller_ as far as I know, it is not defined if I have to install "hpx-devel" or "hwloc" or not, to do helloworld. Neither the documentation, nor the package info, nor the dependencies (i'm assuming they are wrong for now), give that information.
<grostig>
@diehlpk_work, no I just want to install hpx packages (whichever ones I need) for fedora29 and do helloworld. That is all I have ever spoken about.
<heller_>
where would you expect to find that information?
<diehlpk_work>
grostig, sudo dnf install hpx-examples and after that you run hpx_hello_world
<grostig>
@heller_, Firstly the documentation. Which in my mind is the "project specification". All other issues proceed from there.
<diehlpk_work>
dnf install hpx-examples will install the hpx package and hwloc and all other dependencies for yo
khuck has quit []
<grostig>
@diehlpk_work, sorry, but I don't want to install the examples and not "hpx-hello-world", but just "helloworld" from the getting started. Compile that little program, not the whole hpx source.
<diehlpk_work>
Ok, so you install hpx-devel
<diehlpk_work>
and copy the example's source code from github and compile it
<grostig>
@diehlpk_work, I think you have just _proved_ that the documentation is _incorrect_ and dependencies are _not_correct either. But we need a specification to say for sure.
<heller_>
grostig: the issues I have is the following: I don't want to have too much linux distribution specific stuff in our documentation. From my point of view, that should be documented somewhere on fedoras site. from what I can tell, this "-devel" etc lingo comes from fedora, might be phrased strangely, but yeah.
<diehlpk_work>
Oh, I made a proof what a nice day
<heller_>
come on...
<heller_>
let's take a step back.
<hkaiser>
grostig: why so frostig?
<grostig>
@heller_, I understand your viewpoint, however the hpx documentation as it stands is incorrect for fedora and maybe all the other precompile packages, and the precompiled packages don't have the right dependencies.
<heller_>
that may very well be
<grostig>
@hkaiser, I'm not trying to be unpleasant.
<heller_>
restating that over and over again doesn't help neither of us
<diehlpk_work>
grostig, Yes, the hpx-devel package is currently broken
<grostig>
@heller_, It is a lot of work to support precompiled packages, but even more to support source builds.
<heller_>
let's not get into this ;)
<heller_>
anyways
<diehlpk_work>
Something is wrong for the cmake configuration, I will try to repair this
<heller_>
diehlpk_work: there's #3555 which fixed the issue
<grostig>
@diehlpk_work, Again, thank you for your work! :)
<heller_>
grostig: so it looks like building HPX applications using the fedora package is broken
aserio has quit [Quit: aserio]
<heller_>
can you please list the content of /usr/lib64/cmake/HPX?
akheir has joined #ste||ar
<grostig>
@heller_, well in fact, cmake stuff is a minor issue. I just want to use simple make, but I thought I would start documenting the problems I was running into. that was my contribution.
<heller_>
should be easy enough to create a small workaround^Whack to get you started without needing to compile anyting
<heller_>
simple make, sure
<heller_>
that might even work right out of the box ;)
<grostig>
@heller_, I have doen that on slack, so it appears, I should not use slack for complex issues, is that correct?
<heller_>
mostly it's fine
<heller_>
I just didn't pay attention
<heller_>
and ignored the wall of text
<heller_>
anyways
<grostig>
@heller_, I had to learn IRC today, so I don't know how to drop a file into this chat. Can it be done, or copy paste?
<heller_>
gist.github.com, for example
<heller_>
grostig: ha, unix since 1985 but no exposure to IRC, that's funny ;)
<grostig>
@heller_, my real problem is that I got helloworld to compile and link and dynamically link in Qt Creator, but no output on running. That is documented on slack.
<heller_>
ok
<heller_>
How does QtCreator invoke the build?
<grostig>
IRC has changed since then (when I did try it out and reject it), and I much prefer the "real php forums", twitter and slack are so limited. Now this IRC client does have logging, so at least I don't loose my typing and everyones answers. So I vote for a "real fully threaded" support forum.
<heller_>
grostig: anyway, let's solve the issue that you just want to use make
<grostig>
@heller_, yes Qt did invode the build.
<heller_>
sure it did
<heller_>
but how
<heller_>
the recommended way of building against HPX when not using cmake is by using pkg-config
<heller_>
NB: the Makefile docs are horribly outdated
<heller_>
you don't need to export PKG_CONFIG_PATH since that should have been taken care of by fedora package
<heller_>
that should get you going...
<heller_>
grostig: fully threaded forum: mailing list
<grostig>
@heller_, ok another way, yes mailing list is great too. :) I do know how to use email, and even advanced filtering LOL
<grostig>
@heller_, Qt does give show the exact g++ and link commands and output as they are excecuted, so that should be simplest, equivalent to make.
<grostig>
@heller_, not familiar with pkg-config, but I understand that Qt is not supported. :)
<heller_>
so how did you configure it, how did it build stuff and what input did you give?
<heller_>
it's just invoking the compiler
<heller_>
pkg-config and cmake mostly exist to ease our pain of multi platform development in non standard locations
<grostig>
@heller_, Qt setup wasn't difficult if one knows their secrets, to show you it requires pasting two files. Can IRC do that?
<heller_>
having a system wide package makes everything way easier
<heller_>
grostig: use a pastebin site
<heller_>
grostig: for example gist.github.com
<grostig>
@heller_, So I am familiar with gnu autotools but I guess that doesn't work for windows.
<heller_>
nope
<hkaiser>
*shiver*
<heller_>
grostig: FWIW, autotools comes with built in support for pkg-config
<heller_>
pkg-config is the cmake find_package equivalent
<grostig>
@heller_, I can't do the link right now because of the problem with the location of these libraries in the hpx package install. /usr/lib64/hpx They are not found by the dynamic linker.
<heller_>
alright
<heller_>
grostig: in your libs path, you forgot to add hpx_init and hpx_wrap
<grostig>
@heller_ program runs but gives no output is the complicated issue.
<heller_>
grostig: qmake supports pkg-config
<heller_>
sure
<heller_>
because of the missing libraries
<grostig>
@heller_ a silent error? Let me check on those libraries.
<heller_>
grostig: in your .pro file add: CONFIG += link_pkgconfing PKGCONFIG += hpx_application
<heller_>
yeah, the silent error sounds like a bug
<heller_>
but you didn't follow the documentation ;)
<grostig>
@heller_, thanks for correcting the *.a libraries, I just added both and it compiles and ran without needing to relocate the /usr/lib64/hpx/* libaries, so that is great news.
<diehlpk_work>
grostig, Your CMake issue will be addressed in the hpx-1.2.1-1 release on fedora