Simple PCI Driver for Arria V SoC Card, or Stratix IV card?

[Altera_Forum]

[sorry if you've read this before - the original thread got messed up somehow] 

 Hi - I alluded to this in a prior post but it's become my principal issue. I want to do simple read/write transactions to a dev. board, but via PCI (for better throughput than the JTAG interface). Is there anything out there that will let me, from a windows machine, push reads and writes down to a dev board? (i.e., a windows driver for a PCI device that supports reads/writes?) Thanks!  Ideally, i want to connect the Arria V SoC board (which has a PCI-E connector similar to what you see on a MOBO) to a laptop via USB. So any thoughts on that would also be helpful. Failing this - I can plug a Stratix IV development card into a PCI slot on a PC.  Thanks again!   /j

[Altera_Forum]

--- Quote Start ---   I want to do simple read/write transactions to a dev. board, but via PCI (for better throughput than the JTAG interface). Is there anything out there that will let me, from a windows machine, push reads and writes down to a dev board? (i.e., a windows driver for a PCI device that supports reads/writes?) Thanks!   --- Quote End ---    I know you wanted Windows, but ... this thread has code to use Linux to access PCI/PCIe peripheral boards (no driver required)  http://www.alteraforum.com/forum/showthread.php?t=35678  I think the Jungo drivers would work under Windows, though I have not used them.    --- Quote Start ---   Ideally, i want to connect the Arria V SoC board (which has a PCI-E connector similar to what you see on a MOBO) to a laptop via USB. So any thoughts on that would also be helpful.   --- Quote End ---    Which board specifically? This one:  http://www.altera.com/products/devkits/altera/kit-arria-v-soc.html  Why wouldn't you just use Linux on the SoC to access the PCIe device?  This would be a root-complex rather than a PCIe peripheral    --- Quote Start ---   Failing this - I can plug a Stratix IV development card into a PCI slot on a PC.   --- Quote End ---    This is a PCIe peripheral (not a root-complex). If the connector on the Arria V SoC kit was large enough, you could plug it into that kit :)  OneStopSystems has a series of PCIe cable products.  http://www.onestopsystems.com/components.php http://www.onestopsystems.com/expansion_backplanes.php  For testing the Stratix IV kit, I use a PCIe ExpressCard in my laptop over to a 3-slot motherboard (see the expansion backplanes link).  I then boot my laptop in Linux mode, and poke around the board, per the link in the thread above.  Cheers, Dave

[Altera_Forum]

Hi Dave - Always good to hear from you. I hope you're doing well. The code you point me to looks like it may be just want I need. I think that if I have a Dev. Card plugged into a LINUX box, with the PCI Interface instantiated on the FPGA (via Qsys as an Avalon Bus Master), I can compile this code on the Linux box and it will support read/writes from the PC to the FPGA (sort of a simple System Console). - Great - Thanks! 

 As to why I don't want to boot LINUX on the SoC card and essentially use -that- as the PC: It's a tail-wagging the dog solution. I'm not much of an embedded guy, and it's a lot of overhead to take on for a fairly simple/mundane purpose. (Although I understand that it's doable and is probably the recommended usage model). (so why do we have the SoC card? We plan to use it in future projects (where the SW load is sufficient to get some embedded help)>   I'm not sure we're on the same page for the bottom part of your post. If I do this on my old Stratix-IV card, I can just plug it into a PCI slot on my LINUX box. If I use the SoC card, the problem is that I need to connect that card to a PC. Ideally there would be some sort of PCI-E Male to USB connector (so I could use a laptop) with a driver supplied. But more likely what I'd do is have some sort of Pci-E Male <-> Pci-E Male cable to tether the card to the LINUX box and use your code (above). I'm not clear on how the links you supplied (all card) help?   Thanks very much! /j

[Altera_Forum]

Hi Jeff, 

  --- Quote Start ---  Hi Dave - Always good to hear from you. I hope you're doing well.   --- Quote End ---   Sure. I always try to make time to help out others :)    --- Quote Start ---   The code you point me to looks like it may be just want I need. I think that if I have a Dev. Card plugged into a LINUX box, with the PCI Interface instantiated on the FPGA (via Qsys as an Avalon Bus Master), I can compile this code on the Linux box and it will support read/writes from the PC to the FPGA (sort of a simple System Console). - Great - Thanks!   --- Quote End ---   Yes. The code uses the already-existing PCI device nodes setup automatically by Linux. The only caveat is that to access those default device nodes you need to run the application as root. Not a big deal.    --- Quote Start ---   As to why I don't want to boot LINUX on the SoC card and essentially use -that- as the PC: It's a tail-wagging the dog solution. I'm not much of an embedded guy, and it's a lot of overhead to take on for a fairly simple/mundane purpose. (Although I understand that it's doable and is probably the recommended usage model). (so why do we have the SoC card? We plan to use it in future projects (where the SW load is sufficient to get some embedded help)>    --- Quote End ---   Yeah, I figured it might be something like that. However, given that Rocketboards.org has working Linux images, you could conceivably be "blind" to how they've implemented Linux, and just use it.    --- Quote Start ---   I'm not sure we're on the same page for the bottom part of your post. If I do this on my old Stratix-IV card, I can just plug it into a PCI slot on my LINUX box.    --- Quote End ---   Right, because its a PCIe peripheral board, you can plug it into a PC PCIe slot, or the stand-alone motherboard like I use (which saves me having to deal motherboard chipset issues).    --- Quote Start ---   If I use the SoC card, the problem is that I need to connect that card to a PC. Ideally there would be some sort of PCI-E Male to USB connector (so I could use a laptop) with a driver supplied. But more likely what I'd do is have some sort of Pci-E Male <-> Pci-E Male cable to tether the card to the LINUX box and use your code (above). I'm not clear on how the links you supplied (all card) help?    --- Quote End ---   Herein lies lots of potential issues; that slot will be designed as a PCIe root-complex slot, i.e., it looks like a motherboard PCIe slot as that is how it is supposed to act. You cannot just plug an adapter in and link it to your PC, since the clock from your PC needs to make it over to your Arria V board as the PCIe clock reference. Its possible the Arria V developers thought of operating the board in this mode, but you'd have to check the schematic. In some cases you have to remove the clock source on the development kit (eg., remove source terminations), and then figure out how to get the clock from the PCIe connector over to the Arria V clock pin. In that case, you'd be better off buying a PCIe cable solution from OneStopSystems, i.e., something that links a PCIe periperhal on your PC (or ExpressCard on your laptop) over to the PCIe slot on your Arria V kit.  Still confused? Draw a block diagram of your PCIe link; each link has a master at one end (root-complex) and a slave (peripheral) at the other. Between those two devices are the transceiver lanes, the clock, the reset, and a few other control lines. Those "extra" signals come from the root-complex. You cannot connect two root-complex devices together directly, or two peripheral boards directly.  Cheers, Dave

[Altera_Forum]

Hi Dave - I appreciate all the info! On the last point, I suspected it might not be that simple (i.e., that you can't just connect two sockets together), and if that's what the card is for I will look closer. But I'm assuming then, that the card is transparent from a SW perspective, otherwise drivers (or lack thereof) becomes an issue again. 

 Thanks! /j

[Altera_Forum]

--- Quote Start ---  On the last point, I suspected it might not be that simple (i.e., that you can't just connect two sockets together)   --- Quote End ---   Right, that is what I wanted you to appreciate.    --- Quote Start ---   and if that's what the card is for I will look closer.    --- Quote End ---   If you look at the Arria V SoC schematic, I'm pretty sure you'll fine the PCIe clock, resetN, etc signals are OUTPUTs, i.e., the PCIe connector assumes you will connect to a peripheral board.    --- Quote Start ---   But I'm assuming then, that the card is transparent from a SW perspective, otherwise drivers (or lack thereof) becomes an issue again.   --- Quote End ---   Nothing is transparent :)  If you were to use the Arria V SoC kit to test a PCIe board, then the software running on the Arria V kit is responsible for initializing the PCIe interface, eg., U-Boot or Linux on the ARM core is responsible for enumerating the PCIe bus. This is analogous to your PC BIOS enumerating the Stratix IV PCIe board when plugged into your desktop machine. If the RocketBoards.org U-Boot+Linux combo implements the PCIe interface, then yes, in that respect, the "software is transparent" to your user-space application code, since someone else has already written the low-level hardware initialization code.  Cheers, Dave

[Altera_Forum]

Hi Dave - Ok, I think I get it. If, hypothetically, I wanted to slave the Arria V SoC Card to a PC, I Would get -two- One-Stop cards, plug one into the PC and one into the socket on the Arria V card, and connect them with a cable. Then I would configure the PCI-E block on the FPGA to present an external slave interface (and possibly configure the cards as to which is master and slave), and it should work. Yes? (BTW - the cards list on Amazon for $175 each :-) ) 

 And yes, comparatively speaking, booting LINUX on the SoC Card isn't looking so bad. 8-} I'll checkout RocketBoards.org next. One thing though - if I do boot LINUX on the SoC Card, what stands in for the 'hard disk' (I.e., O/S and file storage)? The SDcard?  Thanks! /j

[Altera_Forum]

--- Quote Start ---  If, hypothetically, I wanted to slave the Arria V SoC Card to a PC, I Would get -two- One-Stop cards, plug one into the PC and one into the socket on the Arria V card, and connect them with a cable.    --- Quote End ---   Yes.    --- Quote Start ---   Then I would configure the PCI-E block on the FPGA to present an external slave interface (and possibly configure the cards as to which is master and slave), and it should work. Yes? (BTW - the cards list on Amazon for $175 each :-) )   --- Quote End ---   You should not have to configure anything, since the Arria V SoC U-Boot or Linux will configure the OneStopSystems board (setup its PCIe base address). What you will be missing is "How to I use the OneStopSystems card?"  For example, if you want to DMA a block of data from your PC over to the Arria V SoC, how do you do something like that? I assume there is some form of PCIe chipset on the OneStop PCIe board, and that you will need to program the DMA controller on those boards. The ExpressCard I have is a direct connection to a PLX chip on the motherboard I have, and the link is effectively transparent. In your case, you want to hook a root-complex to a root-complex through what effectively needs to be a non-transparent bridge, so that both PCIe root-complex devices see a single PCIe peripheral (the OneStopSystems boards).  Those "words" are all the right things to describe what you want, so you can always contact OneStopSystems and ask them this same question directly.    --- Quote Start ---   And yes, comparatively speaking, booting LINUX on the SoC Card isn't looking so bad. 8-}   --- Quote End ---   Right, at which point you may also realize that using a network cable to talk to the SoC kit is also an easier solution than using the OneStopSystems boards.    --- Quote Start ---   One thing though - if I do boot LINUX on the SoC Card, what stands in for the 'hard disk' (I.e., O/S and file storage)? The SDcard?   --- Quote End ---   Yes. Its likely that the board has a Quad SPI flash for FPGA bit-file storage, and possibly U-Boot bootloader storage. Once the bootloader runs, it'll configure the DDR memory, copy Linux "from somewhere" (most likely the SDcard), and then boot. The RocketBoards.org site should explain it somewhere.  Cheers, Dave

[Altera_Forum]

Hi Dave - we're almost on the same page. If I went the 2-cards+cable route, it's because my master is on the PC, and I'm trying to not use the ARM on the SoC chip at all. The OneStopSystems site -says- you don't need any drivers for their cards, but you're right that the devil is in the details. Still - I don't see this as a first-choice path so enough said.  

 Agreed that if I DID boot Linux on the SoC on the dev. board the connection to the world would be Ethernet. Never thought anything different. :-)  Looking at Rocketboards now. Thanks! /j

[Altera_Forum]

--- Quote Start ---  Hi Dave - we're almost on the same page. If I went the 2-cards+cable route, it's because my master is on the PC, and I'm trying to not use the ARM on the SoC chip at all.   --- Quote End ---   Technically both the PC and the Arria V SoC are configured as PCIe masters (root-complexes). The only way you can link them together is via a non-transparent PCIe bridge, where the master on either side of the bridge sees a small block of base address registers (BARs) that make the bridge look like a peripheral to both masters. The DMA controller in the non-transparent bridge can then be used to DMA between the address spaces of the two masters.    --- Quote Start ---   The OneStopSystems site -says- you don't need any drivers for their cards, but you're right that the devil is in the details. Still - I don't see this as a first-choice path so enough said.    --- Quote End ---   Chances are, that is because the cards you looked at are not designed to connect two masters together, but to connect a master over a cable to a peripheral board.  You can look for the phrase "non-transparent bridge" to see if they have a product that allows two root-complexes to connect to each other.    --- Quote Start ---   Agreed that if I DID boot Linux on the SoC on the dev. board the connection to the world would be Ethernet. Never thought anything different. :-)   --- Quote End ---   :)  Cheers, Dave

[Altera_Forum]

HI Dave - I don't attest to knowing much about PCI, and get what you're saying, and it makes sense, but for what it's worth, my Arrow FAE just wrote me to say  

 " I will take a look at [The Arria V] soc [board], I think it might be a cable PCI-E. it is capable of root or endpoint, so there must be something we can do."  So maybe with the right cards it's possible to do a bridge where PC masters the board as a slave. But again, it's moot, you've got me over to Rocketboard. Bringing up the ARM is more consistent with our long-term strategy. I just didn't want to jump into it for this project if there was something much easier, but looks like there isn't, and $$400 - $500 of hardware that will have it's setup quirks and be throwaway is a non-starter.  Best /j  [no, you're not crazy I did post something different and then delete].

[Altera_Forum]

--- Quote Start ---  I don't attest to knowing much about PCI, and get what you're saying, and it makes sense   --- Quote End ---   Ok, I'm glad the explanation was clear.    --- Quote Start ---   my Arrow FAE just wrote me to say   " I will take a look at [The Arria V] soc [board], I think it might be a cable PCI-E. it is capable of root or endpoint, so there must be something we can do."   --- Quote End ---   Cool. Post his response when you get it. The Arria V SoC board would need to have clock multiplexer logic on its PCIe connector.  I suspect that it may support "being a PCIe device" via an FMC connector rather than through the PCIe connector.    --- Quote Start ---   But again, it's moot, you've got me over to Rocketboard.   --- Quote End ---   Yah!  Let me know how it goes :)  I've avoided the SoC boards so far, as I've wanted to avoid the teething stages of the tools (synthesis and software) ... and it would be too much of a fun distraction :)  Cheers, Dave

[Altera_Forum]

Yes - We -WERE- trying to avoid the SoC boards for another quarter or so as well, but we could budget only one board this year, so the plan was to get the SoC board, but use it as a non-SoC board for the current project. Lot of good that plan did us (sigh) 8-} 

 /j