Sorry, you need to enable JavaScript to visit this website.

NVIDIA GPU on axi-pcie bus

3 posts / 0 new
mattriggs's picture
NVIDIA GPU on axi-pcie bus

Hello again!

I'm hoping to get help getting NVIDIA's nvidia-340 module to insert on Ubuntu 14.04 running on the ZYNQ-7000.

On the last post I created, I was trying to get the PCIe bus recognized in Ubuntu 14.04, and I was successful with this forum's help.

I've now got the mini-ITX board running headless with Ubuntu (Linaro) 14.04, running Xilinx's 3.13 kernel. I can see my card, an NVIDIA GTX 660 Ti, on the PCIe bus, I can see that it initializes on the bus properly, and lspci shows that the card looks healthy.

root@linaro-developer:~/samples# lspci
00:00.0 PCI bridge: Xilinx Corporation Device 0706
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 660 Ti] (rev a1)

lshw shows the PCIe bus, and the card shows up and looks healthy, and more importantly hasn't been claimed by any other modules:
description: PCI bridge
product: Xilinx Corporation
vendor: Xilinx Corporation
physical id: 0
bus info: pci@0000:00:00.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: pci pm msi pciexpress normal_decode bus_master cap_list
resources: ioport:1000(size=4096) memory:60000000-6bffffff
*-display UNCLAIMED
description: VGA compatible controller
product: GK104 [GeForce GTX 660 Ti]
vendor: NVIDIA Corporation
physical id: 0
bus info: pci@0000:01:00.0
version: a1
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress vga_controller cap_list
configuration: latency=0
resources: memory:6a000000-6affffff memory:60000000-67ffffff memory:68000000-69ffffff ioport:1000(size=128) memory:6b000000-6b07ffff

I built the nvidia-340 module against my kernel's natively-compiled (I compiled the kernel on the mini-ITX itself to make the correct source available) code and headers, and it built successfully. When I attempt to modprobe the driver, it fails:

root@linaro-developer:~/samples# modprobe nvidia
modprobe: ERROR: could not insert 'nvidia_340': No such device

Looking at dmesg, it complains that it cannot find an IRQ for the NVIDIA card:

root@linaro-developer:~/samples# dmesg | tail
[<c0082f20>] (SyS_finit_module+0x88/0xa0) from [<c000e5a0>] (ret_fast_syscall+0x0/0x48)
---[ end trace 9c2b35f227d16bf6 ]---
NVRM: Can't find an IRQ for your NVIDIA card!
NVRM: Please check your BIOS settings.
NVRM: [Plug & Play OS] should be set to NO
NVRM: [Assign IRQ to VGA] should be set to YES
nvidia: probe of 0000:01:00.0 failed with error -1
NVRM: The NVIDIA probe routine failed for 1 device(s).
NVRM: None of the NVIDIA graphics adapters were initialized!
NVRM: NVIDIA init module failed!

I've tried everything I can think of, and posts on NVIDIA forums haven't turned up anything useful. For reference, here's my current entry in my device tree, where I've tried to use "advanced interrupt mapping" to clarify IRQ mapping:

axi_pcie_0: axi-pcie@50000000 {
#address-cells = <3>;
#size-cells = <2>;
#interrupt-cells = <1>;
compatible = "xlnx,axi-pcie-2.3", "xlnx,axi-pcie-1.05.a";
interrupt-parent = <&ps7_scugic_0>;
interrupts = <0 29 4>;
interrupt-map-mask = <0 0 0 7>;
interrupt-map = <0 0 0 1 &ps7_scugic_0 1>,
<0 0 0 2 &ps7_scugic_0 2>,
<0 0 0 3 &ps7_scugic_0 3>,
<0 0 0 4 &ps7_scugic_0 4>;
ranges = <0x2000000 0x0 0x60000000 0x60000000 0x0 0xc000000>;
reg = <0x50000000 0xc000000>;
device_type = "pci";
xlnx,axi-aclk-freq-hz = <0x7735940>;
xlnx,axibar-0 = <0x60000000>;
xlnx,axibar-1 = <0xffffffff>;
xlnx,axibar-2 = <0xffffffff>;
xlnx,axibar-3 = <0xffffffff>;
xlnx,axibar-4 = <0xffffffff>;
xlnx,axibar-5 = <0xffffffff>;
xlnx,axibar-as-0 = <0x0>;
xlnx,axibar-as-1 = <0x0>;
xlnx,axibar-as-2 = <0x0>;
xlnx,axibar-as-3 = <0x0>;
xlnx,axibar-as-4 = <0x0>;
xlnx,axibar-as-5 = <0x0>;
xlnx,axibar-highaddr-0 = <0x6fffffff>;
xlnx,axibar-highaddr-1 = <0x0>;
xlnx,axibar-highaddr-2 = <0x0>;
xlnx,axibar-highaddr-3 = <0x0>;
xlnx,axibar-highaddr-4 = <0x0>;
xlnx,axibar-highaddr-5 = <0x0>;
xlnx,axibar-num = <0x1>;
xlnx,axibar2pciebar-0 = <0x60000000>;
xlnx,axibar2pciebar-1 = <0x0>;
xlnx,axibar2pciebar-2 = <0x0>;
xlnx,axibar2pciebar-3 = <0x0>;
xlnx,axibar2pciebar-4 = <0x0>;
xlnx,axibar2pciebar-5 = <0x0>;
xlnx,class-code = <0x60400>;
xlnx,comp-timeout = <0x1>;
xlnx,device-id = <0x706>;
xlnx,ep-link-partner-rcb = <0x0>;
xlnx,family = "zynq";
xlnx,include-baroffset-reg = <0x1>;
xlnx,include-rc = <0x1>;
xlnx,instance = "axi_pcie_0";
xlnx,interrupt-pin = <0x0>;
xlnx,max-link-speed = <0x0>;
xlnx,no-of-lanes = <0x4>;
xlnx,num-msi-req = <0x0>;
xlnx,pcie-blk-locn = <0x0>;
xlnx,pcie-cap-slot-implemented = <0x1>;
xlnx,pcie-use-mode = "3.0";
xlnx,pciebar-as = <0x0>;
xlnx,pciebar-len-0 = <0x1e>;
xlnx,pciebar-len-1 = <0xd>;
xlnx,pciebar-len-2 = <0xd>;
xlnx,pciebar-num = <0x1>;
xlnx,pciebar2axibar-0 = <0x0>;
xlnx,pciebar2axibar-0-sec = <0x1>;
xlnx,pciebar2axibar-1 = <0xffffffff>;
xlnx,pciebar2axibar-1-sec = <0x1>;
xlnx,pciebar2axibar-2 = <0xffffffff>;
xlnx,pciebar2axibar-2-sec = <0x1>;
xlnx,ref-clk-freq = <0x0>;
xlnx,ref-clk-freq-hz = <0x5f5e100>;
xlnx,rev-id = <0x0>;
xlnx,s-axi-ctl-aclk-freq-hz = <0x7735940>;
xlnx,s-axi-ctl-protocol = "AXI4LITE";
xlnx,s-axi-id-width = <0x1>;
xlnx,s-axi-support-threads = <0x1>;
xlnx,s-axi-supports-narrow-burst = <0x1>;
xlnx,s-axi-supports-read = <0x1>;
xlnx,s-axi-supports-write = <0x1>;
xlnx,subsystem-id = <0x6100>;
xlnx,subsystem-vendor-id = <0x0>;
xlnx,vendor-id = <0x10ee>;
} ;

Is there any help anyone can give? I'm especially interested in any insight into more advanced device tree configuration for the axi-pcie, or a way to get more debugging information out of modprobe, but any information is much appreciated.

At this point, I'm at a loss, and I'm quickly coming to the conclusion that the miniITX isn't going to work for my application.


hockeyman1972's picture
Wind River Linux 7

 It seems like you are running into too many roadblocks with the infrastructure, and I would like to recommend a different approach for this particular development.  It's our goal to get you (and all our customers!) past all of the problems wrestling with infrastructure, and onto a platform for software development that allows you to focus on your product instead.  
So to that end I have escalated this to our expert Linux partner team at Wind River.  We believe that using the open source Wind River Linux 7 platform we will be able to quickly get over the issues you are currently seeing, and still allow you to continue unimpeded with your existing development environment.  
I will contact you outside this forum to discuss how this can work and see if you would like to try this alternative approach.

henry10210's picture
Re: NVIDIA GPU on axi-pcie bus

Hi, I am VERY interested in this topic, because I plan to migrate to ITX once the Zedbiard prototype works. There is a ITX PCIe root complex reference design, but is there any example of a working nVidia GPU intergration with ITX somewhere?