topic There is no perf counter or in Intel® Moderncode for Parallel Architectures

RTM: Finding memory address at which transaction was aborted

William_Leiserson — Thu, 12 Oct 2017 00:47:16 GMT

If a page is marked Copy-on-Write, and I try to write to it inside of a transaction, the transaction aborts. If I know the address at which it aborted, there is a trivial fix:

int v = addr[0];
__sync_bool_compare_and_swap(addr, v, v); // Force CoW.

And then retry the transaction. Again, this only works if I know the cacheline on which the transaction was aborted. Is there a way to find this? Is it in a performance counter or something?

Three options that I can

jimdempseyatthecove — Thu, 12 Oct 2017 14:43:45 GMT

Three options that I can think of:

a) prior to issuing each instruction that may cause a transaction abort, write the address that may abort into a memory location that won't abort. Somewhat of the same philosophy of a try/catch. Should the transaction abort __sync_fetch_add(loc,0); // RMW

b) prior to entering the transaction region (and inside your retry code) performs something like

     __sync_fetch_add(locA,0); // RMW
     __sync_fetch_add(locB,0);
     ...
     ... start transaction (if abort, go back/redo __sync_fetch_add's, then retry transaction)

c) start transaction, then if abort, perform the __sync_fetch_add's of b) and retry transaction

You will have to determine if choice a), b) or c) is more efficient.

Jim Dempsey

Hi,

Roman_D_Intel — Thu, 12 Oct 2017 15:54:54 GMT

Hi,

before writing to a page you can printf its address using tsx_printf (it escapes transaction also working for transactions that are aborted). You need a processor with Skylake architecture for that.

Roman

Hi Jim and Roman,

William_Leiserson — Fri, 13 Oct 2017 15:27:47 GMT

Hi Jim and Roman,

(a) may be a workable possibility, but it won't catch all cases. Same with tsx_printf (which is very clever, btw; I just read your blog post about it).

Let me provide some context: I'm developing a compiler that allows the code:

xbegin;
// Do some transactional stuff.
xcommit;

If the transaction aborts, it will simply try again until it succeeds. However, the "do some transactional stuff" may call out to C or C++, which contains code that the compiler didn't generate and can't hook. This is expected to be a common scenario in the language. So it could be walking a graph or some such, and the memory is non-trivial to find outside of the transaction.

There is no perf counter or

Roman_D_Intel — Fri, 13 Oct 2017 16:08:12 GMT

There is no perf counter or register with the memory access address that causes the abort. I think the best you can do is to retry the transaction body under a global lock instead of using TSX. In the TSX abort handler you can check the abort status (in EAX register) if the abort was persistent (RETRY bit = 0).

Roman

Roman,

jimdempseyatthecove — Sun, 15 Oct 2017 10:42:00 GMT

Roman,

tsx_printf is an interesting hack. I would have done something different.

#include <stdio.h>
#include <stdarg.h>
#include <stdlib.h>

#define TSX_PRINTF_BUF_PADD 1024
#define TSX_PRINTF_BUF_LEN (1024*1024)
char tsx_printf_str[TSX_PRINTF_BUF_LEN+TSX_PRINTF_BUF_PADD];
int  tsx_printf_fill = 0;
bool tsx_printf_wrapped =  false;

int tsx_printf(const char* format, ...)
{
    va_list list;
    va_start(list, format);
    int ret = vsnprintf(str+tsx_printf_fill, TSX_PRINTF_BUF_PADD, format, list);
    va_end(list);
    if((tsx_printf_fill += ret) == TSX_PRINTF_BUF_LEN)
    {
        tsx_printf_wrapped = true;
        int j = TSX_PRINTF_BUF_LEN;
        tsx_printf_fill = 0;
        for(int i = 0; i < ret; ++i)
        {
            tsx_printf_str[tsx_printf_fill++] = tsx_printf_str[j++];
        }
    }
    return ret;
}

void tsx_printf_dump()
{
    if(tsx_printf_wrapped)
    {
        tsx_printf_str[TSX_PRINTF_BUF_LEN] = 0;
        printf(&tsx_printf_str[tsx_printf_fill]);
    }
    tsx_printf_str[tsx_printf_fill] = 0;
    printf((&tsx_printf_str);
    tsx_printf_fill = 0;
    tsx_printf_wrapped =  false;
}

Jim Dempsey

Okay, yeah. I was afraid of

William_Leiserson — Mon, 16 Oct 2017 00:10:13 GMT

Okay, yeah. I was afraid of that. Thanks Roman.

Jim,

Roman_D_Intel — Mon, 16 Oct 2017 06:14:53 GMT

Jim,

changes to your tsx_printf memory buffer will be lost in case of an abort (e.g. if it happens after tsx_printf). The Intel processor trace records instruction control flow also in aborted transactions. My tsx_printf is (mis-)using it allowing the output data survive aborts.

Best regards,

Roman

Quote:jimdempseyatthecove

William_Leiserson — Tue, 17 Oct 2017 18:28:40 GMT

jimdempseyatthecove wrote:

a) prior to issuing each instruction that may cause a transaction abort, write the address that may abort into a memory location that won't abort. Somewhat of the same philosophy of a try/catch. Should the transaction abort __sync_fetch_add(loc,0); // RMW

Hi Jim,

I've been thinking more about this. Is there a way to specify the memory I don't want added the transaction, or is there pre-defined memory that I can use? Do you have a link with information on how this would work?

Roman, oops. You are right.

jimdempseyatthecove — Wed, 18 Oct 2017 16:28:49 GMT

Roman, oops. You are right.

William, either all memory access is transactional or not. The instruction trace hack is not backed out. You could potentially use the instruction trace information in your transaction abort handler. This should be documented in a systems programmer manual. Perhaps Roman could provide a link. Converting the address into source code line number would be up to you to figure out.

Jim Dempsey

Hey Jim,

William_Leiserson — Wed, 18 Oct 2017 19:04:58 GMT

Hey Jim,

This was what I thought. I must have misunderstood your first comment.

Thanks!

Setting up processor trace

Roman_D_Intel — Fri, 20 Oct 2017 08:23:13 GMT

Setting up processor trace recording and reading the results directly from hardware is only possible in the kernel (ring 0) and not from user space. I am not sure if it is practical for this use case (compiler).

Roman