.. _firmware-example-designs:

Example designs
===============

Example designs for several development boards are available in the git repository, under the ``boards`` directory. 

Ethernet-based designs:
  * Enclustra AX3 module (using the RGMII PHY on the PM3 baseboard)
  * Xilinx KC705 eval board
  * Xilinx KCU105 eval board (using SFP0)
  * Xilinx VCU118 eval board
  * Xilinx ZCU102 eval board
  * PC053a

PCIe-based designs:
  * HiTech Global K800
  * Xilinx VCU118 eval board


Each of the Ethernet-based designs instantiates an Ethernet core, the ``ipbus_ctrl`` entity - which contains the IPbus transactor itself - and a small illustrative set of IPbus slaves (a couple of registers, a standard RAM slave, and a ported RAM slave). Each of the PCIe-based designs instantiates the Xilinx XDMA IP core, the ``ipbus_transport_axi_if`` entity, the IPbus transactor, and the same small illustrative set of IPbus slaves as the Ethernet-based designs. The address table file for the slaves in the example designs is: :github-fw-tag-file:`components/ipbus_util/addr_table/ipbus_example.xml`


Build instructions
------------------

.. note::

  The IP and MAC address for each Ethernet-based example design is set by signals in the top-level VDHL file (e.g. ``boards/kcu105/basex/synth/firmware/hdl/top_kcu105_basex.vhd`` for the KCU105 design). For most designs, the IP address is 192.168.200.16+n and the MAC address is ``0x020ddba1151n``, where the value of ``n`` is set by the board's dip switches. Depending on the network setup in your lab, you may want/need to change these addresses before building the example designs.

The example designs can be built using IPBB (the IPBus firmware Build tool), which requires Python (version **2.7 or later**) and can be set up as follows:

1. Prerequisites: ``pip`` and ``virtualenv``. Install if not already present, e.g. via YUM: ``sudo yum install python-pip python-virtualenv``

2. Install IPBB - there are two options:

   a. System-wide installation

      .. code:: sh

        sudo pip install https://github.com/ipbus/ipbb/archive/v0.5.2.tar.gz

   b. Local (VirtualEnv) installation

      .. code:: sh

        curl -L https://github.com/ipbus/ipbb/archive/v0.5.2.tar.gz | tar xvz
        source ipbb-0.5.2/env.sh

      **N.B.** With this VirtualEnv installation, if you start a new shell before building the firmware, then the 2nd of these commands must be run again in order to add the path for the ``ipbb`` command to ``$PATH``.

The top-level IPBB ``.dep`` files for the example designs can be found under ``projects/example/firmware/cfg``. After setting up IPBB, each of these designs can be built with the following commands (where ``<board_name>`` should be replaced by that particular design's name, e.g. ``kc705_basex``):

.. code:: sh

  ipbb init myFwArea
  cd myFwArea

  ipbb add git https://github.com/ipbus/ipbus-firmware -b v1.14

  ipbb proj create vivado -t top_<board_name>.dep <board_name> ipbus-firmware:projects/example
  cd proj/<board_name>
  ipbb vivado make-project
  ipbb vivado synth -j4 impl -j 4
  ipbb vivado package

The bitfile can then be found under the ``package`` directory (``package/src/top.bit``), along with a tarball that contains both the bitfile and the address table  (``package/<board_name>_<hostname>_<date>_<time>.tgz``).


Using the firmware
------------------

After loading the bitfile onto the board, you can test communication with it by following the instructions below.

.. note::

  Each example design contains a couple of registers, a standard RAM slave and a ported RAM slave; the corresponding address table file is :github-fw-tag-file:`components/ipbus_util/addr_table/ipbus_example.xml`. The ``csr.ctrl`` register is used for resets, so if writing random data to the firmware during tests, you should avoid writing to this register.


Ethernet-based designs
^^^^^^^^^^^^^^^^^^^^^^

First you should check that the board responds to ICMP, by running the ``ping`` command, e.g:

.. code:: sh

  ping -c 10 192.168.200.16

Then, you should try to read and write to the ``reg`` and ``ram`` slaves from the address table, using uHAL and an ``ipbusudp-2.0`` client URI such as ``ipbusudp-2.0://192.168.200.16:50001`` (you may need to change the IP address in this URI to suit the settings in your build and/or the positions of your board's dip switches). If have not used uHAL before, you may find it useful to read the :ref:`uhal-quick-tutorial` page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download :download:`read_write_single_register_without_connection_file.py <../software/uhal-example/scripts/read_write_single_register_without_connection_file.py>`, and run it as follows:

.. code:: sh

  python read_write_single_register_without_connection_file.py ipbusudp-2.0://192.168.200.16:50001 file://ipbus_example.xml reg


PCIe-based designs
^^^^^^^^^^^^^^^^^^

The PCIe example designs currently use the Xilinx XMDA IP core, and so before loading the firwmare onto your board, the corresponding driver should be installed by following the instructions at https://www.xilinx.com/support/answers/65444.html

After loading a new bitfile onto the FPGA, in order to refresh the computer's PCI tree, you should run the rescan command ``/sys/bus/pci/rescan``. The ``lspci`` command in Unix can be used to list and display information about devices connected to the system's PCI(e) buses. After running ``/sys/bus/pci/rescan``, you should run ``lspci`` to check that the Xilinx device has been detected correctly - if it is, the output of ``lspci`` will contain a line like::

  01:00.0 Serial controller: Xilinx Corporation Device 8031

.. note::

  **PC reboot sometimes required after loading firmware.** There are scenarios - in particular if no functioning PCIe firmware was installed on the FPGA when the host computer was last booted - in which the ``rescan`` command will be unable to establish the PCIe link. In that case, the host computer will need to be rebooted after the firmware has been loaded.

The Xilinx XDMA driver creates device files with the prefix ``xdma<cardNumber>_`` (as well as symlinks to these files under ``/dev/xdma/``). Data can be transferred from the CPU to the FPGA using the device file ending ``h2c_0``, and from the FPGA to the CPU using the file ending ``c2h_0``. You should check that you can see devices files matching this format; for example, if there is only one FPGA attached to the CPU, then you should see ``/dev/xdma0_h2c_0`` and ``/dev/xdma0_c2h_0``, and a few other files whose path start with ``/dev/xdma0_``.

Finally, you should try to read and write to the ``reg`` and ``ram`` slaves from the address table, using uHAL and an ``ipbuspcie-2.0`` client URI such as ``ipbuspcie-2.0:///dev/xdma0_h2c_0,/dev/xdma0_c2h_0`` (if you have multiple XDMA-based cards attached to the same computer, you may need to change ``xdma0`` in the client URI to ``xdma1``, ``xdma2`` or similar). If have not used uHAL before, you may find it useful to read the :ref:`uhal-quick-tutorial` page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download :download:`read_write_single_register_without_connection_file.py <../software/uhal-example/scripts/read_write_single_register_without_connection_file.py>`, and run it as follows:

.. code:: sh

  python read_write_single_register_without_connection_file.py ipbuspcie-2.0:///dev/xdma0_h2c_0,/dev/xdma0_c2h_0 file://ipbus_example.xml reg



ZCU102 AXI chip-to-chip example design
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``master`` branch of the repository now contains an example design for the ZCU102 eval board, in which packets of IPbus transactions are sent and received over the PS-PL interface. Vivado 2019.1.1 should be used for building this design.

In practice, this data might be sent over the PS-PL interface either to communicate with firmware that is running on the Zynq PL, or with firmware that is running on another chip that's connected to the Zynq PL via an AXI chip-to-chip link. In order to include the firmware blocks that are required both for of these scenarios, the ZCU102 'chip-to-chip loopback' design (``prototypes/zcu102/c2c_loopback``) contains two sets of IPbus modules (IPbus-AXI transport interface, transactor, and example slaves): One set is connected directly to the PL interface (this set referred to as the 'local' transactor), and the other is connected via a chip-to-chip link between channel 2 of quad 229 and channel 3 of quad 128 (referred to as the 'remote' transactor).

After loading the bitfile onto the FPGA, the example slaves behind the 'local' and 'remote' transactors can be accessed through uHAL using the ``ipbusmmap-2.0`` client URI, with memory offsets of 0xa0010000 and 0xa1010000 respectively - e.g.  ``ipbusmmap-2.0:///path/to/device-file?offset=0xa0010000`` for the 'local' transactor. If have not used uHAL before, you may find it useful to read the :ref:`uhal-quick-tutorial` page which summarises classes and functions that should be used to read and write to slaves. As an example, you can download :download:`read_write_single_register_without_connection_file.py <../software/uhal-example/scripts/read_write_single_register_without_connection_file.py>`, and run it as follows (for the 'local' transactor):

.. code:: sh

  python read_write_single_register_without_connection_file.py ipbusmmap-2.0:///dev/mem?offset=0xa0010000 file://ipbus_example.xml reg