hkaiser changed the topic of #ste||ar to: STE||AR: Systems Technology, Emergent Parallelism, and Algorithm Research | stellar.cct.lsu.edu | HPX: A cure for performance impaired parallel applications | github.com/STEllAR-GROUP/hpx | Buildbot: http://rostam.cct.lsu.edu/ | Log: http://irclog.cct.lsu.edu/
<zao>
Not sure how C++14-capable the Clang and libstdc++ is supposed to be.
<K-ballo>
seems like libstdc++ pre 8.1 would be the cause
<zao>
I'm gonna go to bed, just wanted to mention this while I remembered it and had the logs around.
<zao>
Was actually going to see if I could reproduce my spurious failure of the MPI migrate_component, but didn't get quite that far :)
<K-ballo>
i'll have to look into it.. libstdc++ seems to be asking whether it can construct an util::bound from an std::tuple
<K-ballo>
the answer would definitively be no, but it's a hard error sfinae unfriendly one
<K-ballo>
one of those bugs with single element tuple constructors
<K-ballo>
should be able to reproduce with an older libstdc++, and add a workaround for it
<K-ballo>
doesn't look like something my recent changes would have caused
K-ballo has quit [Quit: K-ballo]
eschnett has joined #ste||ar
hkaiser has joined #ste||ar
hkaiser has quit [Quit: bye]
eschnett has quit [Quit: eschnett]
<Yorlik>
How threadsafe is std::vector.push_back()? Could 2 threads accidentally write to the same position?
<Yorlik>
OK - made a test and got an access violation - so its not :(
<heller_>
Yes, the std container are not thread safe
<Yorlik>
Its even more weird
<Yorlik>
I am finding data in the vectors I never wrote to them
<Yorlik>
The access violation went away after i did a sufficiently large reserve beforehand
<Yorlik>
But the output from the vector did not only give me the jumbled numbers of the thread writes, but also an occasional dash(-) - though it is a vector of ints I was writing to
<Yorlik>
output like this after an orderly start: -8421504511-84215045143-842150451-
<Yorlik>
code is this for thread 1-8 , just replace the numbers: std::thread t1 ( [ & ] ( ) { for ( int i = 0; i < n; i++ ) { stuff.push_back ( 1 ); } } );
<Yorlik>
so each thread should write its number
<Yorlik>
I wonder if cout or the console are getting mad at me
<Yorlik>
size of the vector is clearly too small - as expected
<heller_>
It's not just the memory allocation
<heller_>
In your case also the update of the size
<Yorlik>
I just don't understand why it's printing dashes
<heller_>
So multiple threads might update both the size and the last element at the same time
<heller_>
Negative numbers
<Yorlik>
The rest makes sense to me
<Yorlik>
Owww
<heller_>
It's undefined behavior
<Yorlik>
That would mean we have partial writes mixed from threads
<heller_>
Something like that
<Yorlik>
Even a simple int write gets jumbled
<Yorlik>
But I see no irregular numbers
<Yorlik>
just 1-8 and the dash
<heller_>
You push back at the same time. All threads update the size and only after that, they write at the same spot at the same time
<zao>
heller_: I don't trust that MPI migrate_component functionality btw. I found an eventually-failed run in one of my consoles and I'm somewhat sure it's a good OpenMPI build.
<Yorlik>
It gives me a large negative number, but only with digits 1-8
<zao>
Haven't gotten around reproducing it, as master's broken on my setup atm :)
<zao>
That number is very specific in hex.
<heller_>
zao: ok
<heller_>
I'll investigate
<Yorlik>
This is all so crazy .. Bottomline is I need a threadsafe vector and queue ...
<zao>
Yorlik: It's 0xCDCDCDCD.
<Yorlik>
Nice find !
<zao>
Any sufficiently advanced C/C++ troubleshooter learns to recognize memory patterns by sight :)
<Yorlik>
I'm not a sufficiently advanced C++ troubleshooter. Rather a sufficiently advanced C++ troublemaker. :O
<zao>
It _can_ be one of the broken OpenMPI builds I manufactured, so it could be a false flag.
<Yorlik>
Time to learn how to make an efficient threadsafe container. Any hints / pointers? I don't just want to copypaste code - rather a good tutorial if any of you know of one.
<zao>
In general, the less generic you can make your requirements, the higher the chance of a decent data structure existing for you.
<Yorlik>
Makes sense
<zao>
In particular around the number of consumers/producers, whether your data is POD and movable, etc.
<zao>
Storing the data in-line in the buffer vs. storing in buckets or indirectly referenced by atomically sized pointer.
<Yorlik>
The data structure I made is based on a vector which is indexed by a map holding keyed indices. It uses a freelist to save space and reuse slots. The items are linked , so I can use them as a que, which has a target and thetarget is in the map. So it's like a vecor holding a bunch of queues automaigally sorted
<Yorlik>
Butit's totally not thread safe
<Yorlik>
Also using vector indices to link is not overly efficient
<Yorlik>
It's a start though
<Yorlik>
there are multiple producers and consumers, but only one consumer per list