concurrent_vector, shrink_to_fit(), compact()

[Sergei]

Hi All.

1. concurrent_vector.h mentions compact() method:

@par Changes since TBB 2.0
[cut]
    - Added compact() method to defragment first segments

But there's no such method.

2. Doc at software.intel.com/en-us/node/506203 states: " The method shrink_to_fit()merges several smaller arrays into a single contiguous array, which may improve access time."
But I found shrink_to_fit() doesn't create a single continuous array, and I need single contiguous array to use older API that expects pointers.

version info:

// Marketing-driven product version
#define TBB_VERSION_MAJOR 2019
#define TBB_VERSION_MINOR 0

// Engineering-focused interface version
#define TBB_INTERFACE_VERSION 11008
#define TBB_INTERFACE_VERSION_MAJOR TBB_INTERFACE_VERSION/1000

Any insights to alleviate my confusion?

Regards,
Sergei.

[Mark_L_Intel]

Hello:

The compact() still shows up in documentation. I have to check why it is the case. My search  shows: "Method shrink_to_fit was called compact() in Intel® TBB 2.1. It was renamed to match the C++0x std::vector::shrink_to_fit()."

I see also in documentation; "The method shrink_to_fit()merges several smaller arrays into a single contiguous array, which may improve access time."

However, you report that shrink_to_fit() "doesn't create a single continuous array". How did you come to this conclusion?

I picked some samples from the web and constructed a sample to experiment with:

#include <iostream>
#include <stdio.h>
#include <tbb/task_scheduler_init.h>
#include <tbb/blocked_range.h>
#include <tbb/parallel_for.h>
#include <tbb/concurrent_vector.h>
#include <tbb/blocked_range.h>

#define N 24

using namespace std;
using namespace tbb;

template<class I>
bool is_contiguous(I first, I last)
{
    auto test = true;
    auto const n = std::distance(first, last);
    for (auto i = 0; i < n && test; ++i) {
        test &= *(std::next(first, i)) == *(std::next(std::addressof(*first), i));
    }
    return test;
}

class FObject {
    private:
        concurrent_vector<int> &cv;

    public:
        FObject( concurrent_vector<int> &_cv ) : cv( _cv ) {}

        void operator( )( const blocked_range<size_t>& r ) const {

            printf("%08d - %08d\n",r.begin(),r.end());

            for ( size_t i=r.begin(); i!=r.end( ); ++i ) {
                cv.push_back(i);
            }
        }
};

void par_func(int x, int y, concurrent_vector<int> &cv)
{
    FObject ob(cv);

    parallel_for(blocked_range<size_t>(x,y,N/4),FObject(ob));
}

int main()
{
    task_scheduler_init init(2);

    concurrent_vector<int> cv;

    par_func(0,N,cv);

    cout << cv.size() << endl;

    for ( int i=0; i<cv.size(); i++ ) {
        printf("cv[%8d] = %8d\n",0,cv);
    }

   //cv.shrink_to_fit();
    std::cout << std::boolalpha << is_contiguous(cv.begin(), cv.end()) << "\n";

    return 0;
}

when I comment out cv.shrink_to_fit(); I'm getting "false" while with cv.shrink_to_fit(); I got "true". This was done with TBB 2019 

[Konstantin_B_Intel1]

Hello,

As documented, method concurrent_vector::shrink_to_fit merges several internal arrays of elements into the single one. So the vector becomes continuous .

But if you then adds some new elements into the vector (with any growth operation), the new elements can be placed into the new array, so the whole vector is not continuous any more.

E.g:

concurrent_vector<int> cv(10); // continuous vector

cv.grow_by(100);

// cv is not continuous after the grow_by call

cv.shrink_to_fit();

// several arrays were merged, so cv is continuous again

cv.push_back(1);

// cv can be not continuous again, so an other shrink_to_fit call is required

Best regards,
Konstantin

[Sergei]

That's the class I used:

template<typename T>
class buffer
{
public:

    buffer() = default;
    buffer(const buffer&) = delete;
    buffer& operator= (const buffer&) = delete;

    void resize(size_t newSize)
    {
        m_Vec.grow_to_at_least(newSize);
        m_compacted = false;
    }
    operator const T* () const // not multithreaded !!! invalidates all iterators/pointers!!!
    {
        if (!m_compacted)
        {
#ifdef _IT_WORKS_
            tbb::concurrent_vector<T> temp(m_Vec.size());
            std::copy(m_Vec.cbegin(), m_Vec.cend(), temp.begin());
            m_Vec.swap(temp);
#else // it doesn't work
            m_Vec.shrink_to_fit();
#endif
            m_compacted = true;
        };
        return &m_Vec.front();
    }
    const T& operator[](size_t i) const
    {
        return m_Vec;
    }
    T& operator[](size_t i)
    {
        return m_Vec;
    }
    auto cbegin() const
    {
        return m_Vec.cbegin();
    }
    auto cend() const
    {
        return m_Vec.cend();
    }
    auto begin() const
    {
        return m_Vec.begin();
    }
    auto end() const
    {
        return m_Vec.end();
    }
    auto size() const
    {
        return m_Vec.size();
    }
private:
    mutable tbb::concurrent_vector<T> m_Vec;
    mutable bool m_compacted = false;
};
 

   buffer aBuffer;

  // ...added data to aBuffer...

    const int* p = aBuffer;

    int val;
    for (int i = 0; i < aBuffer.size(); ++i)
        val = p ;

works with _IT_WORKS_ defined.

with _IT_WORKS_ undefined I can see *p at position 1024 is wrong (less then 0 while it can't). All values in aBuffer in range 0..1023 are valid,

___I was not able to reproduce this problem in a sandbox.___