Example designs

Example designs for several development boards are available in the git repository, under the boards directory.

Ethernet-based designs:

  • Enclustra AX3 module (using the RGMII PHY on the PM3 baseboard)

  • Xilinx KC705 eval board

  • Xilinx KCU105 eval board (using SFP0)

  • Xilinx VCU118 eval board

  • Xilinx ZCU102 eval board

  • PC053a

PCIe-based designs:

  • HiTech Global K800

  • Xilinx VCU118 eval board

Each of the Ethernet-based designs instantiates an Ethernet core, the ipbus_ctrl entity - which contains the IPbus transactor itself - and a small illustrative set of IPbus slaves (a couple of registers, a standard RAM slave, and a ported RAM slave). Each of the PCIe-based designs instantiates the Xilinx XDMA IP core, the ipbus_transport_axi_if entity, the IPbus transactor, and the same small illustrative set of IPbus slaves as the Ethernet-based designs. The address table file for the slaves in the example designs is: components/ipbus_util/addr_table/ipbus_example.xml

Build instructions

Note

The IP and MAC address for each Ethernet-based example design is set by signals in the top-level VDHL file (e.g. boards/kcu105/basex/synth/firmware/hdl/top_kcu105_basex.vhd for the KCU105 design). For most designs, the IP address is 192.168.200.16+n and the MAC address is 0x020ddba1151n, where the value of n is set by the board’s dip switches. Depending on the network setup in your lab, you may want/need to change these addresses before building the example designs.

The example designs can be built using IPBB (the IPBus firmware Build tool), which requires Python (version 2.7 or later) and can be set up as follows:

  1. Prerequisites: pip and virtualenv. Install if not already present, e.g. via YUM: sudo yum install python-pip python-virtualenv

  2. Install IPBB - there are two options:

    1. System-wide installation

      sudo pip install https://github.com/ipbus/ipbb/archive/v0.5.2.tar.gz

    2. Local (VirtualEnv) installation

      curl -L https://github.com/ipbus/ipbb/archive/v0.5.2.tar.gz | tar xvz
source ipbb-0.5.2/env.sh

      N.B. With this VirtualEnv installation, if you start a new shell before building the firmware, then the 2nd of these commands must be run again in order to add the path for the ipbb command to $PATH.

The top-level IPBB .dep files for the example designs can be found under projects/example/firmware/cfg. After setting up IPBB, each of these designs can be built with the following commands (where <board_name> should be replaced by that particular design’s name, e.g. kc705_basex):

ipbb init myFwArea
cd myFwArea

ipbb add git https://github.com/ipbus/ipbus-firmware -b v1.14

ipbb proj create vivado -t top_<board_name>.dep <board_name> ipbus-firmware:projects/example
cd proj/<board_name>
ipbb vivado make-project
ipbb vivado synth -j4 impl -j 4
ipbb vivado package

The bitfile can then be found under the package directory (package/src/top.bit), along with a tarball that contains both the bitfile and the address table (package/<board_name>_<hostname>_<date>_<time>.tgz).

Using the firmware

After loading the bitfile onto the board, you can test communication with it by following the instructions below.

Note

Each example design contains a couple of registers, a standard RAM slave and a ported RAM slave; the corresponding address table file is components/ipbus_util/addr_table/ipbus_example.xml. The csr.ctrl register is used for resets, so if writing random data to the firmware during tests, you should avoid writing to this register.

Ethernet-based designs

First you should check that the board responds to ICMP, by running the ping command, e.g:

ping -c 10 192.168.200.16

Then, you should try to read and write to the reg and ram slaves from the address table, using uHAL and an ipbusudp-2.0 client URI such as ipbusudp-2.0://192.168.200.16:50001 (you may need to change the IP address in this URI to suit the settings in your build and/or the positions of your board’s dip switches). If have not used uHAL before, you may find it useful to read the uHAL quick tutorial page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download read_write_single_register_without_connection_file.py, and run it as follows:

python read_write_single_register_without_connection_file.py ipbusudp-2.0://192.168.200.16:50001 file://ipbus_example.xml reg

PCIe-based designs

The PCIe example designs currently use the Xilinx XMDA IP core, and so before loading the firwmare onto your board, the corresponding driver should be installed by following the instructions at https://www.xilinx.com/support/answers/65444.html

After loading a new bitfile onto the FPGA, in order to refresh the computer’s PCI tree, you should run the rescan command /sys/bus/pci/rescan. The lspci command in Unix can be used to list and display information about devices connected to the system’s PCI(e) buses. After running /sys/bus/pci/rescan, you should run lspci to check that the Xilinx device has been detected correctly - if it is, the output of lspci will contain a line like:

01:00.0 Serial controller: Xilinx Corporation Device 8031

Note

PC reboot sometimes required after loading firmware. There are scenarios - in particular if no functioning PCIe firmware was installed on the FPGA when the host computer was last booted - in which the rescan command will be unable to establish the PCIe link. In that case, the host computer will need to be rebooted after the firmware has been loaded.

The Xilinx XDMA driver creates device files with the prefix xdma<cardNumber>_ (as well as symlinks to these files under /dev/xdma/). Data can be transferred from the CPU to the FPGA using the device file ending h2c_0, and from the FPGA to the CPU using the file ending c2h_0. You should check that you can see devices files matching this format; for example, if there is only one FPGA attached to the CPU, then you should see /dev/xdma0_h2c_0 and /dev/xdma0_c2h_0, and a few other files whose path start with /dev/xdma0_.

Finally, you should try to read and write to the reg and ram slaves from the address table, using uHAL and an ipbuspcie-2.0 client URI such as ipbuspcie-2.0:///dev/xdma0_h2c_0,/dev/xdma0_c2h_0 (if you have multiple XDMA-based cards attached to the same computer, you may need to change xdma0 in the client URI to xdma1, xdma2 or similar). If have not used uHAL before, you may find it useful to read the uHAL quick tutorial page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download read_write_single_register_without_connection_file.py, and run it as follows:

python read_write_single_register_without_connection_file.py ipbuspcie-2.0:///dev/xdma0_h2c_0,/dev/xdma0_c2h_0 file://ipbus_example.xml reg

ZCU102 AXI chip-to-chip example design

The master branch of the repository now contains an example design for the ZCU102 eval board, in which packets of IPbus transactions are sent and received over the PS-PL interface. Vivado 2019.1.1 should be used for building this design.

In practice, this data might be sent over the PS-PL interface either to communicate with firmware that is running on the Zynq PL, or with firmware that is running on another chip that’s connected to the Zynq PL via an AXI chip-to-chip link. In order to include the firmware blocks that are required both for of these scenarios, the ZCU102 ‘chip-to-chip loopback’ design (prototypes/zcu102/c2c_loopback) contains two sets of IPbus modules (IPbus-AXI transport interface, transactor, and example slaves): One set is connected directly to the PL interface (this set referred to as the ‘local’ transactor), and the other is connected via a chip-to-chip link between channel 2 of quad 229 and channel 3 of quad 128 (referred to as the ‘remote’ transactor).

After loading the bitfile onto the FPGA, the example slaves behind the ‘local’ and ‘remote’ transactors can be accessed through uHAL using the ipbusmmap-2.0 client URI, with memory offsets of 0xa0010000 and 0xa1010000 respectively - e.g. ipbusmmap-2.0:///path/to/device-file?offset=0xa0010000 for the ‘local’ transactor. If have not used uHAL before, you may find it useful to read the uHAL quick tutorial page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download read_write_single_register_without_connection_file.py, and run it as follows (for the ‘local’ transactor):

python read_write_single_register_without_connection_file.py ipbusmmap-2.0:///dev/mem?offset=0xa0010000 file://ipbus_example.xml reg