SCIF data transfer

[Kims_W]

Hi, I want to send data using RMA function of SCIF.

Register is success. But do not write to remote memory.

strerror (errno) output "No such device or address" message is displayed.

Below is a simple code of both side(host, mic)

host side---------------

    int len = 1024;                                                                                             
    void *ptr = NULL;                                                                                       
    uint32_t page_size = sysconf(_SC_PAGESIZE);                                                                       
                                                                                                                      
    posix_memalign(&ptr, page_size, len);                                                                       
    scif_register( ep, ptr, len, 0, SCIF_PROT_READ | SCIF_PROT_WRITE, SCIF_MAP_FIXED)
                                                                                                
    void *remote_ptr = NULL;                                                                        
    scif_recv(ep, &remote_ptr, sizeof(void *), SCIF_RECV_BLOCK);  
                                                                                    
    strcpy(ptr, "Hello, World!");                                                                               
    ret = scif_writeto(ep, (long)ptr, strlen(ptr), (long)remote_ptr, 0);                                                                            
    if(ret) {                                                                                                         
        fprintf(stderr, "SCIF Writeto Failed(%s)\n", strerror(errno));         //"No such device or address"
    }                                                                                                                 
                                                                                                                      
    scif_fence_signal( ep, (long)ptr, strlen(ptr), (long)remote_ptr, strlen(ptr)                                              
        , SCIF_FENCE_INIT_SELF|SCIF_SIGNAL_LOCAL|SCIF_SIGNAL_REMOTE );  

mic side---------------

    int len = 1024; 
    void *ptr;                                                                                                        
    uint32_t page_size = sysconf(_SC_PAGESIZE);                                                                                                                          
    
    posix_memalign(&ptr, page_size, len);                                                               
    scif_register( new_ep, ptr, len, 0, SCIF_PROT_READ | SCIF_PROT_WRITE, 0);
                                                                                                                      
    scif_send(new_ep, &ptr, sizeof(void *), SCIF_SEND_BLOCK);   
    sleep(60);   

Please give me a help.

[Evan_P_Intel]

I've trimmed down and annotated your code to highlight the most significant error I saw:

Kims.W wrote:

host side---------------

    scif_register( ep, ptr, len, /* RO_host = */ 0, SCIF_PROT_READ | SCIF_PROT_WRITE, /* note this! */ SCIF_MAP_FIXED)
    ret = scif_writeto(ep, /* RO_host */ (long)ptr, strlen(ptr), /* RO_mic */ (long)remote_ptr, 0);
    scif_fence_signal(ep, /* RO_host */ (long)ptr, strlen(ptr), /* RO_mic */ (long)remote_ptr, strlen(ptr), SCIF_FENCE_INIT_SELF|SCIF_SIGNAL_LOCAL|SCIF_SIGNAL_REMOTE);

mic side---------------

    /* RO_mic = */ scif_register(new_ep, ptr, len, 0, SCIF_PROT_READ | SCIF_PROT_WRITE, 0);

You're casting the local and remote virtual addresses (pointer values) to long and using them as the local and remote window offsets. While it's possible to set up such a 1:1 correspondence between virtual address space and registration offset space, you haven't actually done that. Unless I've made a mistake in my quick scan of your code, the host registration offset you should be passing is 0 and the card registration offset you should be passing is the ignored return value of the card-side scif_register() call.

If you haven't already, you should read the SCIF User Guide. In particular, you may find the bits of the example in section 3.5.1 which are highlighted in red helpful to review.