MSI interrupt in PCIe-MM-DMA example

[OlegT]

Hello.

I am working with C10GX developement kit. As a starting point I use AN829 - Avalon-MM DMA  ref design.

DMA works well. I want to test MSI in that design.

The design contains linux driver, but msi functionality is commented out.

I've made chanes to driver, but have no luck - ISR is not called, when DMA is finished. But status bit is set in lite_table header.

Here are detail of my situation:

1. My initialization code (altera_pci_probe function):

// enable msi in HW

// set interrupt disable for legacy
pci_read_config_dword(bk_ptr->pci_dev, 0x04, &conf_reg);
conf_reg |= (1 << 10);
pci_write_config_dword(bk_ptr->pci_dev, 0x04, conf_reg);

//Set bit[1] (Memory space) and bit[2]
conf_reg |= 0x06;
pci_write_config_dword(bk_ptr->pci_dev, 0x04, conf_reg);

// set msi Ena
pci_read_config_dword(bk_ptr->pci_dev, 0x50, &conf_reg);
conf_reg |= (1 << 16);
pci_write_config_dword(bk_ptr->pci_dev, 0x50, conf_reg);

rc = pci_enable_msi(dev); -- success here

pci_read_config_byte(dev, PCI_REVISION_ID, &bk_ptr->revision);
pci_read_config_byte(dev, PCI_INTERRUPT_PIN, &bk_ptr->irq_pin);
pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &bk_ptr->irq_line);

if (!pci_set_dma_mask(dev, DMA_BIT_MASK(DMAMASK))) {
pci_set_consistent_dma_mask(dev, DMA_BIT_MASK(DMAMASK));
}
else goto err_dma_mask;

rc = request_irq(bk_ptr->irq_line, dma_isr, IRQF_SHARED, ALTERA_DMA_DRIVER_NAME, (void *)bk_ptr); -- success here

2. dma_isr:

     pr_err("+++ dma_isr ++\n"); // not much so far

    return IRQ_HANDLED;

3. After the driver is loaded, in /proc/interrupts I can find my device:

4. When I look on MSIIntfc_o[81:0] (coming from hip), I see that:

MSIIntfc_o[81]: Master enable = 1

MSIIntfc_o[80]: MSI enable = 1

MSIIntfc_o[79:64]: MSI data = 0x29
MSIIntfc_o[63:0]: MSI address = 0xFEE02004

So, the root complex had set up MSI in FPGA.

5. I've captured by signal tap the transactions on TXS interface, and found, that after DMA is finished, there are two memory writes here. The first one - to bus address of lite_table header (to set a flag). The second one - to address from MSIIntfc_o[63 : 0] with MSIIntfc_o[79 : 64] as data. It is an MSI itself.

So FPGA side seems to work correctly.

As I said earlier, I could see, that flag in lite_table is set, but ISR was not called.

If I check /proc/interrupts, then there are still zeros in counters for my device.

Please suggest, what should I check to solve my problem.

Thank you.

[KhaiChein_Y_Intel]

Hi,

Do you see the MSI when all the DMA data transfers are complete?

Thanks

Best regards,

KhaiY

[OlegT]

I do not see interrupt on host PC, but I see, that wr_dcm_master sends irq_data and irq_address to txs.

irq_data and irq_address are the values from MSIIntfc_o.

[KhaiChein_Y_Intel]

Hi,

It seems like the FPGA sent the correct value of MSI signals to the host but the host cannot detect it. I am afraid that I am not an expert in driver writing. We might need the community users to help you on this thread.

Thanks

Best regards,

KhaiY

[OlegT · Accepted solution]

I've replaced legacy pci_enable_msi() with pci_alloc_irq_vectors(). Now ISR for single interrupts is called.

[KhaiChein_Y_Intel]

Hi,

It is glad that you found the solution to modify the driver. Thanks for sharing the solution as it helps other users who have this problem in their driver.

Thanks

Best regards,

KhaiY