Getting started with Network Tester

This guide aims to introduce the basic concepts required to start working with Network Tester. Complete detailed documentation can be found in the API documentation.

This guide introduces the key concepts and terminology used by Network Tester and walks through the creation of a simple experiment. In this experiment we measure how the SpiNNaker network handles the load produced by a small random network of packet generators.

Introduction & Terminology

Network Tester is a Python library and native SpiNNaker application which generates artificial network traffic in SpiNNaker while recording network performance metrics. In particular, network tester is designed to recreate traffic loads similar to neural applications running on SpiNNaker. This means that the connectivity of a network loads remains fixed throughout experiments but the rate and pattern of injected packets can be varied.

A Network Tester ‘experiment’ consists of a network description along with a series of experimental ‘groups’ during which different traffic patterns or network parameters are applied in sequence.

A network is described as a set of cores between which multicast flows of packets exist. Each flow is sourced by a traffic generator in one core and sunk by traffic consumers in another set of cores. A single core may source and sink many flows simultaneously and thus may contain multiple traffic generators and traffic consumers.

Each experimental group consists of a period of traffic generation and consumption according to a particular set of parameters. A typical experiment may consist of several groups with a single parameter being changed between groups. Various metrics (including packet counts and router diagnostic counters) can be recorded individually for each group and then collected after the experiment into easily manipulated Numpy arrays.

Once an experiment has been defined, Network Tester will automatically load and configure traffic generation software onto a target SpiNNaker machine and execute each experimental group in sequence before collecting results.


The latest stable version of Network Tester library may be installed from PyPI using:

$ pip install network_tester

The standard installation includes precompiled SpiNNaker binaries and should be ready to use ‘out of the box’.

Defining a network

First we must create a new Experiment object which takes a SpiNNaker IP address or hostname as its argument:

>>> from network_tester import Experiment
>>> e = Experiment("")

The first task when defining an experiment is to define a set of cores and flows of network traffic between them. In this example we’ll create a network with 64 cores with random flows between them. First the cores are created using new_core():

>>> cores = [e.new_core() for _ in range(64)]

Next we create a single flow for each core using new_flow() which connects to eight randomly selected cores:

>>> import random
>>> flows = [e.new_flow(core, random.sample(cores, 8))
...          for core in cores]

By default, the cores and flows we’ve defined will be automatically placed and routed in the SpiNNaker machine before we run the experiment. To manually specify which chip each core is added to, this can be given as arguments to new_core(), for example e.new_core(1, 2) would create a core on chip (1, 2). For greater control over the place and route process, see run().

Controling packet generation

Every flow has its own traffic generator on its source core. These traffic generators can be configured to produce a range of different traffic patterns but in this example we’ll configure the traffic generators to produce a simple Bernoulli traffic pattern. In a Bernoulli distribution, each traffic generator will produce a single packet (or not) with a specific probability at a regular interval (the ‘timestep’). By varying the probability of a packet being generated we can change the load our simple example exerts on the SpiNNaker network.

The timestep and packet generation probability are examples of some of the experimental parameters which can be controlled and varied during an experiment. These parameters can be controlled by setting attributes of the Experiment object or Core and Flow objects returned by new_core() and new_flow() respectively.

In our example we’ll set the timestep to 10 microseconds meaning the packet generators in the experiment may generate a packet every 10 microseconds:

>>> e.timestep = 1e-5  # 10 microseconds (in seconds)

In our example experiment we’ll change the probability of a packet being generated (thus changing the network load) and see how the network behaves. To do this we’ll create a number of experimental groups with different probabilities:

>>> num_steps = 10
>>> for step in range(num_steps):
...     with e.new_group() as group:
...         e.probability = step / float(num_steps - 1)
...         group.add_label("probability", e.probability)

The new_group() method creates a new experimental Group object. When a Group object is used with a with statement it causes any parameters changed inside the with block to apply only to that experimental group. In this example we set the probability parameter to a different value for each group.

The Group.add_label() call is optional but adds a custom extra column to the results collected by Network Tester. In this case we add a “probability” column which we set to the probability used in that group. Though the results are automatically broken up into groups, this extra column makes it much easier to plot data straight out of the tool.


Some parameters such as timestep are ‘global’ (i.e. they’re the same for every flow and core) and thus can only changed experiment-wide. Other parameters, such as probability can be set individually for different cores or flows. As a convenience, setting these parameters on the Experiment object sets the ‘default’ value for all cores or flows. For example:

>>> for flow in flows:
...     flow.probability = 0.5

Is equivilent to:

>>> e.probability = 0.5

One last detail is to specify how long to run the traffic generators for each group using duration:

>>> e.duration = 0.1  # Run each group for 1/10th of a second

In experiments with highly static network loads it is important to ‘warm up’ the network to allow it to reach a stable state before recording results for each group. Such a warmup can be added using warmup:

>>> e.warmup = 0.05  # Warm up without recording results for 1/20th of a second

Finally, Network Tester does not attempt to maintain clock synchronisation in long experiments in large SpiNNaker machines. As a result, some traffic generators may finish before others causing artefacts in the results. To help alleviate this a ‘cool down’ period can be added after each group using the cooldown parameter. During the cool down period the traffic generators continue to run but no further results are recorded.

>>> e.cooldown = 0.01  # Cool down without recording results for 1/100th of a second

A complete list of the available parameters is available in the API documentation.

Recording results

Various metrics may be recorded during an experiment. In our example we’ll simply record the number of packets received by the sinks of each flow. Attributes of the Experiment object whose names start with record_ are used to select what metrics are recorded, in this case we enable record_received:

>>> e.record_received = True

The full set of recordable metrics is enumerated in the API documentation and includes per-flow packet counts, router diagnostic counters and packet reinjection statistics.

By default, the recorded metrics are sampled once at the end of each experimental group’s execution but they can alternatively be sampled at a regular interval (see the record_interval parameter).


Unlike the experimental parameters, the set of recorded metrics is fixed for the whole experiment and cannot be changed within groups. Further, individual flows, cores or router’s metrics cannot be enabled and disabled individually. Note, however, that record_interval is an experimental parameter and thus can be set independently for each group.

Running the experiment and plotting results

Once everything has been defined, the experiment is started using run():

>>> results =

Note that the ignore_deadline_errors option is enabled for this experiment. This is necessary since when the injected load is very high the load on the traffic sinks causes the Network Tester to miss its realtime deadlines. In experiments where the network is not expected to saturate this option should not be used.


Running an experiment can take some time. To see informational messages indicating progress you can enable INFO messages in the Python logging module before calling run():

>>> import logging
>>> logging.basicConfig(level=logging.INFO)

The returned Results object provides a number of methods which present the recorded data in useful ways. In this case we’re just interested in the overall behaviour of the network so we’ll grab the totals():

>>> totals = results.totals()
>>> totals.dtype.names
('probability', 'group', 'time', 'received')
>>> totals
[(0.0, <Group 0>, 0.1, 0.0)
 (0.1111111111111111, <Group 1>, 0.1, 566026.0)
 (0.2222222222222222, <Group 2>, 0.1, 1138960.0)
 (0.3333333333333333, <Group 3>, 0.1, 1707350.0)
 (0.4444444444444444, <Group 4>, 0.1, 2277734.0)
 (0.5555555555555556, <Group 5>, 0.1, 2847388.0)
 (0.6666666666666666, <Group 6>, 0.1, 1401762.0)
 (0.7777777777777778, <Group 7>, 0.1, 1377632.0)
 (0.8888888888888888, <Group 8>, 0.1, 1389261.0)
 (1.0, <Group 9>, 0.1, 1393182.0)]

We can then plot this data using pyplot:

>>> import matplotlib.pyplot as plt
>>> plt.plot(totals["probability"], totals["received"])
>>> plt.xlabel("Packet injection probability")
>>> plt.ylabel("Packets received at sinks")

Alternatively, we can export the data as a CSV suitable for processing or plotting with another tool, for example R, using the included network_tester.to_csv() function:

>>> from network_tester import to_csv
>>> print(to_csv(totals))


Unlike the Numpy built-in numpy.savetxt() function, to_csv() automatically adds headers and correctly formats missing elements.