Example designs¶
Example designs for several development boards are available in the git repository, under the boards
directory.
- Ethernet-based designs:
Enclustra AX3 module (using the RGMII PHY on the PM3 baseboard)
Xilinx KC705 eval board
Xilinx KCU105 eval board (using SFP0)
Xilinx VCU118 eval board
Xilinx ZCU102 eval board
PC053a
- PCIe-based designs:
HiTech Global K800
Xilinx VCU118 eval board
Each of the Ethernet-based designs instantiates an Ethernet core, the ipbus_ctrl
entity - which contains the IPbus transactor itself - and a small illustrative set of IPbus slaves (a couple of registers, a standard RAM slave, and a ported RAM slave). Each of the PCIe-based designs instantiates the Xilinx XDMA IP core, the ipbus_transport_axi_if
entity, the IPbus transactor, and the same small illustrative set of IPbus slaves as the Ethernet-based designs. The address table file for the slaves in the example designs is: components/ipbus_util/addr_table/ipbus_example.xml
Build instructions¶
Note
The IP and MAC address for each Ethernet-based example design is set by signals in the top-level VDHL file (e.g. boards/kcu105/basex/synth/firmware/hdl/top_kcu105_basex.vhd
for the KCU105 design). For most designs, the IP address is 192.168.200.16+n and the MAC address is 0x020ddba1151n
, where the value of n
is set by the board’s dip switches. Depending on the network setup in your lab, you may want/need to change these addresses before building the example designs.
The example designs can be built using IPBB (the IPBus firmware Build tool), which requires Python (version 2.7 or later) and can be set up as follows:
Prerequisites:
pip
andvirtualenv
. Install if not already present, e.g. via YUM:sudo yum install python-pip python-virtualenv
Install IPBB - there are two options:
System-wide installation
sudo pip install https://github.com/ipbus/ipbb/archive/v0.5.2.tar.gz
Local (VirtualEnv) installation
curl -L https://github.com/ipbus/ipbb/archive/v0.5.2.tar.gz | tar xvz source ipbb-0.5.2/env.sh
N.B. With this VirtualEnv installation, if you start a new shell before building the firmware, then the 2nd of these commands must be run again in order to add the path for the
ipbb
command to$PATH
.
The top-level IPBB .dep
files for the example designs can be found under projects/example/firmware/cfg
. After setting up IPBB, each of these designs can be built with the following commands (where <board_name>
should be replaced by that particular design’s name, e.g. kc705_basex
):
ipbb init myFwArea
cd myFwArea
ipbb add git https://github.com/ipbus/ipbus-firmware -b v1.15
ipbb proj create vivado -t top_<board_name>.dep <board_name> ipbus-firmware:projects/example
cd proj/<board_name>
ipbb vivado make-project
ipbb vivado synth -j4 impl -j 4
ipbb vivado package
The bitfile can then be found under the package
directory (package/src/top.bit
), along with a tarball that contains both the bitfile and the address table (package/<board_name>_<hostname>_<date>_<time>.tgz
).
Using the firmware¶
After loading the bitfile onto the board, you can test communication with it by following the instructions below.
Note
Each example design contains a couple of registers, a standard RAM slave and a ported RAM slave; the corresponding address table file is components/ipbus_util/addr_table/ipbus_example.xml. The csr.ctrl
register is used for resets, so if writing random data to the firmware during tests, you should avoid writing to this register.
Ethernet-based designs¶
First you should check that the board responds to ICMP, by running the ping
command, e.g:
ping -c 10 192.168.200.16
Then, you should try to read and write to the reg
and ram
slaves from the address table, using uHAL and an ipbusudp-2.0
client URI such as ipbusudp-2.0://192.168.200.16:50001
(you may need to change the IP address in this URI to suit the settings in your build and/or the positions of your board’s dip switches). If have not used uHAL before, you may find it useful to read the uHAL quick tutorial page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download read_write_single_register_without_connection_file.py
, and run it as follows:
python read_write_single_register_without_connection_file.py ipbusudp-2.0://192.168.200.16:50001 file://ipbus_example.xml reg
PCIe-based designs¶
The PCIe example designs currently use the Xilinx XMDA IP core, and so before loading the firwmare onto your board, the corresponding driver should be installed by following the instructions at https://www.xilinx.com/support/answers/65444.html
After loading a new bitfile onto the FPGA, in order to refresh the computer’s PCI tree, you should run the rescan command /sys/bus/pci/rescan
. The lspci
command in Unix can be used to list and display information about devices connected to the system’s PCI(e) buses. After running /sys/bus/pci/rescan
, you should run lspci
to check that the Xilinx device has been detected correctly - if it is, the output of lspci
will contain a line like:
01:00.0 Serial controller: Xilinx Corporation Device 8031
Note
PC reboot sometimes required after loading firmware. There are scenarios - in particular if no functioning PCIe firmware was installed on the FPGA when the host computer was last booted - in which the rescan
command will be unable to establish the PCIe link. In that case, the host computer will need to be rebooted after the firmware has been loaded.
The Xilinx XDMA driver creates device files with the prefix xdma<cardNumber>_
(as well as symlinks to these files under /dev/xdma/
). Data can be transferred from the CPU to the FPGA using the device file ending h2c_0
, and from the FPGA to the CPU using the file ending c2h_0
. You should check that you can see devices files matching this format; for example, if there is only one FPGA attached to the CPU, then you should see /dev/xdma0_h2c_0
and /dev/xdma0_c2h_0
, and a few other files whose path start with /dev/xdma0_
.
Finally, you should try to read and write to the reg
and ram
slaves from the address table, using uHAL and an ipbuspcie-2.0
client URI such as ipbuspcie-2.0:///dev/xdma0_h2c_0,/dev/xdma0_c2h_0
(if you have multiple XDMA-based cards attached to the same computer, you may need to change xdma0
in the client URI to xdma1
, xdma2
or similar). If have not used uHAL before, you may find it useful to read the uHAL quick tutorial page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download read_write_single_register_without_connection_file.py
, and run it as follows:
python read_write_single_register_without_connection_file.py ipbuspcie-2.0:///dev/xdma0_h2c_0,/dev/xdma0_c2h_0 file://ipbus_example.xml reg
ZCU102 AXI chip-to-chip example design¶
The master
branch of the repository now contains an example design for the ZCU102 eval board, in which packets of IPbus transactions are sent and received over the PS-PL interface. Vivado 2019.1.1 should be used for building this design.
In practice, this data might be sent over the PS-PL interface either to communicate with firmware that is running on the Zynq PL, or with firmware that is running on another chip that’s connected to the Zynq PL via an AXI chip-to-chip link. In order to include the firmware blocks that are required both for of these scenarios, the ZCU102 ‘chip-to-chip loopback’ design (prototypes/zcu102/c2c_loopback
) contains two sets of IPbus modules (IPbus-AXI transport interface, transactor, and example slaves): One set is connected directly to the PL interface (this set referred to as the ‘local’ transactor), and the other is connected via a chip-to-chip link between channel 2 of quad 229 and channel 3 of quad 128 (referred to as the ‘remote’ transactor).
After loading the bitfile onto the FPGA, the example slaves behind the ‘local’ and ‘remote’ transactors can be accessed through uHAL using the ipbusmmap-2.0
client URI, with memory offsets of 0xa0010000 and 0xa1010000 respectively - e.g. ipbusmmap-2.0:///path/to/device-file?offset=0xa0010000
for the ‘local’ transactor. If have not used uHAL before, you may find it useful to read the uHAL quick tutorial page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download read_write_single_register_without_connection_file.py
, and run it as follows (for the ‘local’ transactor):
python read_write_single_register_without_connection_file.py ipbusmmap-2.0:///dev/mem?offset=0xa0010000 file://ipbus_example.xml reg