Host Platform: Windows
Version: vtune_amplifier_2019.4.0.597835
Analysis: HPC Performance Characterisation [Remote Linux (SSH)]
Target Platform: Ubuntu 16.04
Kernel: 4.4.0-135-generic
Problem: We have a program that uses mmap to read large dataset files (tens-of-GB). However, it seems that VTunes recognize everything mmapped as shared object files. It tries to copy from the target into the host via pscp everything the program has mmapped. This is undesirable because it is very slow to transfer tens-of-GB via network and will drain the host system storage. It is useless because VTunes will eventually state a warning that this file "... has an unsupported or invalid file format" because it is just a data file, not shared objects.
To repeat, run the following program:
#include <assert.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include "boost/filesystem.hpp"
int main(int argc, char* argv[]) {
const char* path = "/mnt/nvme0n1/very_large_about_1GB_dataset_file";
size_t file_bytes = boost::filesystem::file_size(path);
int fd = open(path, O_RDONLY);
assert(fd >= 0);
void *data = mmap(NULL, file_bytes, PROT_READ, MAP_PRIVATE, fd, 0);
assert(data != MAP_FAILED);
printf("Content %.*s\n", 8, static_cast<char *>(data));
std::vector<char> vec;
for (size_t i=0; i<file_bytes; i++) {
vec.push_back(static_cast<char *>(data));
}
int err = munmap(data, file_bytes);
assert(err == 0);
err = close(fd);
assert(err == 0);
return 0;
}
