Community
cancel
Showing results for 
Search instead for 
Did you mean: 
yu__bowen
Beginner
103 Views

VTunes remote analysis with programs using mmap extremely slow in finalizing

Host Platform: Windows
Version: vtune_amplifier_2019.4.0.597835
Analysis: HPC Performance Characterisation [Remote Linux (SSH)]
Target Platform: Ubuntu 16.04
Kernel: 4.4.0-135-generic

Problem: We have a program that uses mmap to read large dataset files (tens-of-GB). However, it seems that VTunes recognize everything mmapped as shared object files. It tries to copy from the target into the host via pscp everything the program has mmapped. This is undesirable because it is very slow to transfer tens-of-GB via network and will drain the host system storage. It is useless because VTunes will eventually state a warning that this file "... has an unsupported or invalid file format" because it is just a data file, not shared objects. 

To repeat, run the following program:

#include <assert.h>

#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>

#include "boost/filesystem.hpp"

int main(int argc, char* argv[]) {
    const char* path = "/mnt/nvme0n1/very_large_about_1GB_dataset_file";
    size_t file_bytes = boost::filesystem::file_size(path);
    int fd = open(path, O_RDONLY);
    assert(fd >= 0);
    void *data = mmap(NULL, file_bytes, PROT_READ, MAP_PRIVATE, fd, 0);
    assert(data != MAP_FAILED);
    printf("Content %.*s\n", 8, static_cast<char *>(data));

    std::vector<char> vec;
    for (size_t i=0; i<file_bytes; i++) {
        vec.push_back(static_cast<char *>(data));
    }

    int err = munmap(data, file_bytes);
    assert(err == 0);
    err = close(fd);
    assert(err == 0);

    return 0;
}

 

0 Kudos
2 Replies
Kirill_U_Intel
Employee
103 Views

Hi.

Let's us check that.

Kirill

Kirill_U_Intel
Employee
103 Views

Ok, we've reproduced that on our side. Have some ideas how that can be fixed.

Thanks for your reproducer.

 

 

Reply