Analyzers
Talk to fellow users of Intel Analyzer tools (Intel VTune™ Profiler, Intel Advisor)
5133 Discussions

VTunes remote analysis with programs using mmap extremely slow in finalizing

yu__bowen
Beginner
842 Views

Host Platform: Windows
Version: vtune_amplifier_2019.4.0.597835
Analysis: HPC Performance Characterisation [Remote Linux (SSH)]
Target Platform: Ubuntu 16.04
Kernel: 4.4.0-135-generic

Problem: We have a program that uses mmap to read large dataset files (tens-of-GB). However, it seems that VTunes recognize everything mmapped as shared object files. It tries to copy from the target into the host via pscp everything the program has mmapped. This is undesirable because it is very slow to transfer tens-of-GB via network and will drain the host system storage. It is useless because VTunes will eventually state a warning that this file "... has an unsupported or invalid file format" because it is just a data file, not shared objects. 

To repeat, run the following program:

#include <assert.h>

#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>

#include "boost/filesystem.hpp"

int main(int argc, char* argv[]) {
    const char* path = "/mnt/nvme0n1/very_large_about_1GB_dataset_file";
    size_t file_bytes = boost::filesystem::file_size(path);
    int fd = open(path, O_RDONLY);
    assert(fd >= 0);
    void *data = mmap(NULL, file_bytes, PROT_READ, MAP_PRIVATE, fd, 0);
    assert(data != MAP_FAILED);
    printf("Content %.*s\n", 8, static_cast<char *>(data));

    std::vector<char> vec;
    for (size_t i=0; i<file_bytes; i++) {
        vec.push_back(static_cast<char *>(data));
    }

    int err = munmap(data, file_bytes);
    assert(err == 0);
    err = close(fd);
    assert(err == 0);

    return 0;
}

 

0 Kudos
2 Replies
Kirill_U_Intel
Employee
842 Views

Hi.

Let's us check that.

Kirill

0 Kudos
Kirill_U_Intel
Employee
842 Views

Ok, we've reproduced that on our side. Have some ideas how that can be fixed.

Thanks for your reproducer.

 

 

0 Kudos
Reply