Gigabit UDP/Ethernet interface¶
The ipbus_ctrl entity implements an UDP/IP v4 transport layer over 1Gbit Ethernet for an IPbus endpoint, along with simple arbitration between this network access and other ‘out of band’ (OOB) access to the transactor. It supports ARP and ping for ease of commissioning and RARP for IP address resolution, along with a variety of configuration methods. These are either specified by generics, ports on the entity or using an internal configuration space.
The ipbus_ctrl entity is implemented in components/ipbus_util/firmware/hdl/masters/ipbus_ctrl.vhd
Clocks¶
The transport layer within ipbus_ctrl entity runs in the Ethernet MAC clock domain, mac_clk, namely 125MHz, with synchronous reset signal (active high), rst_macclk. The IPbus logic runs in the IPbus clock domain, ipb_clk, typically around 30MHz, with its synchronous reset (again active high) rst_ipb. There is no fixed frequency or phase relationship between these two clocks, the only requirement being that ipb_clk is slower.
Generics¶
MAC_CFG and IP_CFG determine how the MAC and IP addresses respectively are set when RARP is disabled. The values of the mac_address and ip_addr ports will be used as the MAC and IP addresses if the respective generic (MAC_CFG / IP_CFG) is set to EXTERNAL, which is their default value. Whereas if these generics are set to INTERNAL, then the relevant address fields in the transactor’s internal configuration space will be used for the respective address. For a description of the configuration space see the Configuration space section below.
BUFWIDTH determines the number of simultaneous packets in flight supported by the interface, specifically the number of bits in the dual-port RAM address field to select the packet - i.e. the number of simultaneous packets in flight is 2 to the power of BUFWIDTH. The default value is currently 4, giving 16 packets in flight.
INTERNALWIDTH is similarly the number of bits in the address field for ‘other’ packets (ARP, ping, status etc.). The default value is the minimum, namely 1 (2 packets).
ADDRWIDTH is number of address bits within each packet. The maximum packet size is given by 2 to the power of this number. The default value for ADDRWIDTH is 11, giving a 2kByte packet, i.e. standard Ethernet packets. The maximum is 13 for 8kByte packets, slightly smaller than the default for jumbo frames. If you are planning to use jumbo frames, then please note that all MACs and switches along the network path will also need to be configured to support jumbo frames as well.
Note
The total size in bytes of each of the two dual-port RAMs between the transport layer and the endpoint is given by 2 to the power of (BUFWIDTH+ADDRWIDTH) and that of the internal dual-port RAM by 2 to the power of (BUFWIDTH+INTERNALWIDTH), all these RAMs being instantiated by inference (see the notes section below if trying to reduce this to the minimum).
SECONDARYPORT flags whether this IPbus endpoint responds only to traffic to the above UDP port, so as to coexist with another object sharing the Ethernet MAC. Default is ‘0’, where the endpoint responds to ARP and ping (and in RARP mode, issues RARP requests).
N_OOB is the number of out of band interfaces to this IPbus endpoint. The ipbus_ctrl entity always instantiates a UDP master for the transactor, but the out of band interfaces (oob_in and oob_out ports) allow users to connect additional masters (SPI etc.) to the transactor. For example, those out of band interfaces can be used to allow on-board microcontrollers to also issue IPbus writes and reads (which can be useful to initialise the firmware, e.g. to set IP/MAC addresses). The default value for N_OOB is 0.
DHCP_RARP selects which type of IP address management firmware is instantiated - set this to 1 for DHCP, 0 for RARP.
Ports¶
Ports in mac_clk domain¶
mac_rx_data, mac_rx_valid, mac_rx_last, mac_rx_error and mac_tx_ready on input along with mac_tx_data, mac_tx_last and mac_tx_error (never asserted) on output, handle the traffic from and to the Ethernet MAC, satisfying the behaviour corresponding to the Xilinx AXI4_Stream user interface as described in Chapter 7 of the Xilinx UG777 Tri-Mode Ethernet MAC User Guide, as is implemented in Xilinx Spartan 6 and Virtex 6 on. In principle this will work for all speeds supported by this MAC, although all the example designs only implement 1Gbit Ethernet.
With a suitable shim to manipulate the signals, as in eth_v5_gmii.vhd, the interface will also cope with the previous Xilinx V5 TEMAC interface (Xilinx UG194). Porting to the Altera interfaces should not be insurmountable.
ipbus_port is the UDP port number for this IPbus endpoint, default being x”C351”, 50001.
Finally mac_address and ip_addr are assumed stable and are accessed without buffering in the mac_clk domain.
Ports in ipb_clk domain¶
Input ports enable and ipam_select enable the UDP interface and IP Address Management (IPAM) mechanisms respectively when set to ‘1’ (another means of achieving this is through the configuration space described below). When enable is set to ‘0’ all incoming Ethernet packets are ignored, but out of band access is still enabled.
Input port ipb_in and output port ipb_out are the bus signals from and to the slaves. Their fields are defined in ipbus_package.vhd.
pkt_rx and pkt_tx, and their stretched versions, pkt_rx_led and pkt_tx_led, indicate an IPbus packet starting to be decoded by the endpoint and it being finished, i.e. state transitions at the IPbus level, not any network level.
oob_in and oob_out are the bus signals from and to the out of band masters for the transactor. Their fields are defined in ipbus_trans_decl.vhd.
Finally input port ipb_grant and ipb_req allow the possibility of arbitration between multiple bus masters as an alternative to the out of band interface. In standard use this is not enabled and ipb_grant defaults to ‘1’.
RARP and DHCP¶
Each Ethernet MAC should have a globally unique address associated with it and in general it will be this MAC address that is used. There then remains the need for programmatically assigning the IP address in anything other than the smallest systems.
Within xTCA there is the possibility of assigning the IP address over IPMI, for example through the configuration space, but there remains the need for a network-based method of IP address assignment.
The IPbus firmware now supports two methods for IP address assignment: the Reverse ARP mechanism (RARP, RFC903); and the Dynamic Host Configuration Protocol (DHCP). In the latter case, since a full DHCP engine would be very taxing on an FPGA the IPbus firmware implements a simplified DHCP client which does not support DHCP renewal. As such, the IP addresses for ipbus_ctrl endpoints using DHCP must be static reservations in the DHCP server. Users can select which of these two clients are implemented using the DHCP_RARP generic.
In RARP mode, the IPbus endpoint issues RARP requests until such time as a RARP daemon responds. The initial request is issued within 1s of the UDP interface being enabled (enable going high), actual time determined by the bottom two bits of the MAC address, with subsequent requests in the absence of a response issued at random intervals of up to 8s.
Note that until such time as a valid RARP response is received the endpoint will ignore all other network traffic. Out of band access is still enabled.
Configuration space¶
At present the configuration space is defined as std_logic_vector(127 downto 0).
The MAC address is stored in bits 79 down to 32, IP address in bits 127 down to 96, enable as bit 80, RARP enable as bit 81. The UDP interface is enabled using a logical OR of the relevant configuration space bit and the enable input port. Similarly, RARP mode is enabled using a logical OR of the relevant configuration space bit and the ipam_select input port.
Minimising block RAM usage¶
As stated above, the block RAMs are inferred as dual-port block RAM. The RAM between the Ethernet domain and the IPbus logic have asymmetric port widths, 8 bit on the Ethernet side and 32 bit on the IPbus side. Precision Synthesis correctly infers asymmetric port widths from the HDL, whereas Xilinx XST only infers symmetric port widths.
In general this is not an issue, except for the RAM on the rx side, udp_dualportram_rx.vhd, where XST instantiates this as a minimum of 4 block RAMs, even where the RAM would fit in a single block. Hence for the smallest RAM footprint when using XST for synthesis when this dual-port RAM would fit in less than 4 actual block RAMs this module should be replaced by a suitable IP core.