The Network Tester API

The Experiment Class

class network_tester.Experiment

Defines a network experiment to be run on a SpiNNaker machine.

An experiment consists of a defined set of SpiNNaker application cores (new_core()) between which traffic flows (new_flow()).

An experiment is broken up into ‘groups’ (new_group()), during which the traffic generators produce packets according to a specified traffic pattern. Within each group, metrics, such as packet counts, may be recorded. Though the placement of cores and the routing of traffic flows is fixed throughout an experiment, the rate and pattern of the generated traffic flows can be varied between groups allowing, for example, different traffic patterns to be tested.

When the experiment is run(), cores will be loaded with traffic generators and the routing tables initialised. When the experiment completes, recorded results are read back ready for analysis.

__init__(hostname_or_machine_controller)

Create a new network experiment on a particular SpiNNaker machine.

Typical usage:

>>> import sys
>>> from network_tester import Experiment
>>> e = Experiment(sys.argv[1])  # Takes hostname as a CLI argument

The experimental parameters can be set by setting attributes of the Experiment instance like so:

>>> e = Experiment(...)
>>> # Set the probability of a packet being generated at the source
>>> # of each flow every timestep
>>> e.probability = 1.0

Parameters:

hostname_or_machine_controller : str or rig.machine_control.MachineController

The hostname or MachineController of a SpiNNaker machine to run the experiment on.

allocations

A dictionary {Core: {resource: slice}, ...} or None.

Defines the resources allocated to each core. This will include an allocation of the Cores resource representing the core allocated.

See also rig.place_and_route.allocate().

This attribute was made read-only in v0.2.0.

machine

Deprecated. The Machine object describing the SpiNNaker system under test.

Warning

This method is now deprecated, users should use system_info instead. Also, as of v0.3.0, users should not modify this object: all changes are now ignored.

new_core(chip_x=None, chip_y=None, name=None)

Create a new Core.

A SpiNNaker application core which can source and sink a number of traffic flows.

Example:

>>> # Create three cores, the first two are placed on chip (0, 0)
>>> # and final core will be placed automatically onto some
>>> # available core in the system.
>>> c0 = e.new_core(0, 0)
>>> c1 = e.new_core(0, 0)
>>> c2 = e.new_core()

The experimental parameters for each core can also be specified individually if desired:

>>> # Traffice flows sourced at core c2 will transmit with 50%
>>> # probability each timestep
>>> c2.probability = 0.5

Renamed from new_vertex in v0.2.0.

Parameters:

chip_x : int or None

chip_y : int or None

If both chip_x and chip_y are given, this core will be placed on the chip with the specified coordinates. Note: It is not possible to place more cores on a specific chip than the chip has available.

If both chip_x and chip_y are None (the default), the core will be automatically placed on some available chip (see network_tester.Experiment.run()).

Note that either both must be an integer or both must be None.

name

Optional. A name for the core. If not specified the core will be given a number as its name. This name will be used in results tables.

Returns:

Core

An object representing the core.

new_flow(source, sinks, weight=1.0, name=None)

Create a new flow.

A flow of SpiNNaker packets from one source core to many sink cores.

For example:

>>> # A flow with c0 as a source and c1 as a sink.
>>> f0 = e.new_flow(c0, c1)

>>> # Another flow with c0 as a source and both c1 and c2 as sinks.
>>> f1 = e.new_flow(c0, [c1, c2])

The experimental parameters for each flow can be overridden individually if desired. This will take precedence over any values set for the source core of the flow.

For example:

>>> # Flow f0 will generate a packet in 80% of timesteps
>>> f0.probability = 0.8

Renamed from new_net in v0.2.0.

Parameters:

source : Core

The source Core of the flow. A stream of packets will be generated by this core and sent to all sinks.

Only Core objects created by this Experiment may be used.

sinks : Core or [Core, ...]

The sink Core or list of sink cores for the flow.

Only Core objects created by this Experiment may be used.

weight : float

Optional. A hint for the automatic place and route algorithms indicating the relative amount of traffic that may flow through this Flow. This number is not used by the traffic generator.

name

Optional. A name for the flow. If not specified the flow will be given a number as its name. This name will be used in results tables.

Returns:

Flow

An object representing the flow.

new_group(name=None)

Define a new experimental group.

The experiment can be divided up into groups where the traffic pattern generated (but not the structure of connectivity) varies for each group. Results are recorded separately for each group and the network is drained of packets between groups.

The returned Group object can be used as a context manager within which experimental parameters specific to that group may be set, including per-core and per-flow parameters. Note that parameters set globally for the experiment in particular group do not take precedence over per-core or per-flow parameter settings.

For example:

>>> with e.new_group():
...     # Overrides default probability of sending a packet within
...     # the group.
...     e.probability = 0.5
...     # Overrides the probability for c2 within the group
...     c2.probability = 0.25
...     # Overrides the probability for f0 within the group
...     f0.probability = 0.4

Parameters:

name

Optional. A name for the group. If not specified the group will be given a number as its name. This name will be used in results tables.

Returns:

Group

An object representing the group.

placements

A dictionary {Core: (x, y), ...}, or None.

The placements selected for each core, set after calling run().

See also rig.place_and_route.place().

This attribute was made read-only in v0.2.0.

routes

A dictionary {Flow: rig.place_and_route.routing_tree.RoutingTree, ...} or None.

Defines the route used for each flow.

See also rig.place_and_route.route().

This attribute was made read-only in v0.2.0.

run(app_id=219, create_group_if_none_exist=True, ignore_deadline_errors=False, constraints=None, place=<function place>, place_kwargs={}, allocate=<function allocate>, allocate_kwargs={}, route=<function route>, route_kwargs={}, before_load=None, before_group=None, before_read_results=None)

Run the experiment on SpiNNaker and return the results.

Before the experiment is started, any cores whose location was not specified are placed automatically on suitable chips and routes are generated between them. If fine-grained control is required the place, allocate and route arguments allow user-defined placement, allocation and routing algorithms to be used. The result of these processes can be found in the placements, allocations and routes respectively at any time after the before_load callback has been called.

Following placement, the experimental parameters are loaded onto the machine and each experimental group is executed in turn. Results are recorded by the machine and are read back at the end of the experiment.

Warning

Though a global synchronisation barrier is used between the execution of each group, the timers in each core may drift out of sync during each group’s execution. Further, the barrier synchronisation does not give any guarantees about how closely-synchronised the timers will be at the start of each run.

The place_and_route method was removed in v0.2.0 and its functionality merged into this method.

Parameters:

app_id : int

Optional. The SpiNNaker application ID to use for the experiment.

create_group_if_none_exist : bool

Optional. If True (the default), a single group will be automatically created if none have been defined with new_group(). This is the most sensible behaviour for most applications.

If you really want to run an experiment with no experimental groups (where no traffic will ever be generated and no results recorded), you can set this option to False.

ignore_deadline_errors : bool

If True, any realtime deadline-missed errors will no longer cause this method to raise an exception. Other errors will still cause an exception to be raised.

This option is useful when running experiments which involve over-saturating packet sinks or the network in some experimental groups.

constraints : [constraint, ...]

A list of additional place-and-route constraints to apply. In additon to the constraints supplied via this argument, a rig.place_and_route.constraints.ReserveResourceConstraint will be added to reserve the monitor processor on every chip and a rig.place_and_route.constraints.LocationConstraint is also added for all cores whose chip was explicitly specified (and for other special purpose cores such as packet reinjection cores and router-register recording cores).

place : placer

A Rig-API complaint placement algorithm. This will be used to automatically place any cores whose location was not specified.

place_kwargs : dict

Additional algorithm-specific keyword arguments to supply to the placer.

allocate : allocator

A Rig-API complaint allocation algorithm.

allocate_kwargs : dict

Additional algorithm-specific keyword arguments to supply to the allocator.

route : router

A Rig-API complaint route algorithm. Used to route all flows in the experiment.

route_kwargs : dict

Additional algorithm-specific keyword arguments to supply to the router.

before_load : function or None

If not None, this function is called before the network tester application is loaded onto the machine and after place-and-route has been completed. It is called with the Experiment object as its argument. The function may block to postpone the loading process as required.

before_group : function or None

If not None, this function is called before each experimental group is run on the machine. It is called with the Experiment object and Group as its argument. The function may block to postpone the execution of each group as required.

before_read_results : function or None

If not None, this function is called after all experimental groups have run and before the results are read back from the machine. It is called with the Experiment object as its argument. The function may block to postpone the reading of results as required.

Returns:

Results

If no cores reported errors, the experimental results are returned. See the Results object for details.

Raises:

NetworkTesterError

A NetworkTesterError is raised if any cores reported an error. The most common error is likely to be a ‘deadline missed’ error as a result of the experimental timestep being too short or the load on some cores too high in extreme circumstances. Other types of error indicate far more severe problems and are probably a bug in ‘Network Tester’.

Any results recorded during the run will be included in the results attribute of the exception. See the Results object for details.

system_info

The SystemInfo object describing the SpiNNaker system under test.

This value is fetched from the machine once and cached for all future accesses. This value may be modified to influence the place-and-route processes.

Experimental Parameters

Global Parameters

The following experimental parameters apply globally and cannot be overridden on a core-by-core or flow-by-flow basis. They can be changed between groups.

Experiment.timestep

The timestep (in seconds) with which packets are generated by any packet generators in the experiment.

Default value: 0.001 (i.e. 1 ms)

All timing related parameters (e.g. duration and record_interval) are internally converted into multiples of this timestep and rounded to the nearest number of steps

Warning

Setting this value too small will result in the traffic generator software being unable to meet timing deadlines and thus the experiment failing. In practice, 1.5 us is the smallest this can be set but larger values are required if any core is the source of many flows or if large numbers of metrics are being recorded.

Experiment.duration

The duration (in seconds) to run the traffic generators and record results for each experimental group.

Default value: 1.0

See also: warmup, cooldown and Experiment.flush_time.

Experiment.warmup

The duration (in seconds) to run the traffic generators without recording results before beginning result recording for each experimental group.

Default value: 0.0

Many experiments require the network to be in a stable state for their results to be meaningful. To achieve this, a warmup period can be specified during which time the network is allowed to settle into a stable state before any results are recorded.

See also: duration, cooldown and flush_time.

Experiment.cooldown

The duration (in seconds) to run the traffic generators without recording results after result recording is completed for each experimental group.

Default value: 0.0

Since the clocks in individual SpiNNaker cores are not perfectly in sync (and can drift significantly during long-duration experimental groups), it may be necessary for traffic generators to continue producing traffic for a short time after their local timer indicates the end of the experiment in order to maintain consistent network load for any traffic generators which are still running.

See also: duration, warmup and flush_time.

Experiment.flush_time

The pause (in seconds) which is added between experimental groups to allow any packets which remain in the network to be drained.

Default value: 0.01

This is especially important when consume_packets has been set to False.

See also: duration, warmup and cooldown.

Experiment.record_interval

The interval (in seconds) at which metrics are recorded during an experiment.

Default value: 0.0

Special case: If zero, metrics will be recorded once at the end each group’s execution.

See Metric Recording Selection for the set of metrics which can be recorded.

Experiment.router_timeout

Sets the router timeout (in router clock cycles at (usually) 133MHz).

Default value: None (do not change)

If set to an integer, this sets the number of clock cycles before a packet times out and is dropped. Experiment.run() will throw a ValueError if the value presented is not a valid router timeout. Emergency routing will be disabled.

If set to a tuple (wait1, wait2), wait1 sets the number of clock cycles before emergency routing is tried and wait2 gives the number of additional cycles before the packet is dropped. Experiment.run() will throw a ValueError if the values presented are not valid router timeouts.

If None, the timeout will be left as the default when the system was booted.

If this field is set to anything other than None at any point during the experiment, a single core on every chip will be used to set the router timeout. This may result in new cores being created internally and placed on otherwise unused chips.

Experiment.reinject_packets

Enable dropped packet reinjection.

Default value: False (do not use dropped packet reinjection).

Enabling this feature at any point during the experiment will cause core 1 to be reserved on all chips to perform packet reinjection.

See also: Experiment.record_reinjected, Experiment.record_reinject_overflow and Experiment.record_reinject_missed.

Flow/Traffic Generator Parameters

The following experimental parameters apply to each flow (and thus each traffic generator) in the experiment and control the pattern of packets generated and sent down each flow.

The global default for each value can be set by setting the corresponding Experiment attribute.

The default for flows sourced by a particular core can be set by setting the corresponding Core attribute.

These values may also be overridden on a group-by-group basis.

For example:

>>> e = Experiment(...)
>>> c0 = e.new_core()
>>> c1 = e.new_core()
>>> c2 = e.new_core()
>>> f0 = e.new_flow(c0, c1)
>>> f1 = e.new_flow(c0, c3)
>>> f2 = e.new_flow(c2, c3)

>>> e.probability = 1.0
>>> c0.probability = 0.9
>>> f0.probability = 0.8

>>> # First group: f0, f1, f2 all have probability == 0.0
>>> with e.new_group():
...     e.probability = 0.0
...     c0.probability = 0.0
...     f0.probability = 0.0

>>> # Second group: f2 has probability == 0.1 but f0 and f1 have
>>> # probabilities 0.9 and 0.8 respectively.
>>> with e.new_group():
...     e.probability = 0.1

Flow.probability

Core.probability

Experiment.probability

The probability, between 0.0 and 1.0, of a packet being generated in a given timestep.

Default value: 1.0 (Generate a packet every timestep.)

Packet generation is also subject to the burst parameters burst_period, burst_duty, and burst_phase. By default packets are only generated according to probability since bursting behaviour is not enabled.

Flow.packets_per_timestep

Core.packets_per_timestep

Experiment.packets_per_timestep

The number of packets to generate each timestep.

Default value: 1 (Generate a single packet every timestep.)

The probability of each packet being generated is independent and set to Flow.probability.

Flow.num_retries

Core.num_retries

Experiment.num_retries

Number of times to re-try sending a packet if back-pressure from the network blocks it.

Default value: 0 (Give up immediately when faced with back-pressure)

Flow.burst_period

Flow.burst_duty

Flow.burst_phase

Core.burst_period

Core.burst_duty

Core.burst_phase

Experiment.burst_period

Experiment.burst_duty

Experiment.burst_phase

Set the bursting behaviour of the traffic generated for a flow.

Default values: burst_period = 1.0, burst_duty = 0.0 and burst_phase = 0.0. (Not bursting).

If burst_period is 0.0, packets will be generated each timestep with probability probability.

If burst_period is set to a non-zero number of seconds, packets may only be generated the proportion of the time specified by burst_duty, every burst_period seconds.

#
#                                burst_period seconds
#                          |------------------------------|
#              (burst_duty * burst_period) seconds
#                     |-------|
#       (burst_phase * burst_period) seconds
#                     |----|
#                          .                              .
# maybe send          +----.--+                      +----.--+
#                     |    .  |                      |    .  |
#                     |    .  |                      |    .  |
# never send    ...---+    .  +----------------------+    .  +---...
#                          .                              .
#                          .                              .
#                      time = t                 time = t + burst_period

burst_phase gives the initial phase of the bursting behaviour. Note that the phase is not reset at the start of each group. Values outside of the range 0.0 - 1.0 will be wrapped to within this range.

Special case: If burst_phase is set to None, the phase of the bursting behaviour will be chosen randomly.

Flow.use_payload

Core.use_payload

Experiment.use_payload

Should a payload field be added to each generated packet?

Default value: False (Generate 40-bit short packets)

If True, 72-bit ‘long’ multicast packets are generated. If False, 40-bit ‘short’ packets are generated.

Core Parameters

The following experimental parameters apply to each core in the experiment.

The global default for each value can be set by setting the corresponding Experiment attribute.

These values may also be overridden on a group-by-group basis.

Core.seed

Experiment.seed

The seed for the random number generator in the specified core or None to select a random seed automatically.

Default value: None (Seed randomly automatically).

If set to a non-None value, the seed is (re)set at the start of each group.

Core.consume_packets

Experiment.consume_packets

Should the specified core consume packets from the network?

Default value: True (Consume packets from the network)

If set to False, packets sent via any flow to the core will not be consumed from the network. This will cause the packets to back-up in the network and eventually cause routers to drop packets.

See also Experiment.flush_time.

Metric Recording Selection

The following boolean attributes control what metrics are recorded during an experiment. Note that the set of recorded metrics may not be changed during an experiment (though the recording interval can, see Experiment.record_interval).

By default, no metrics are recorded.

Experiment.record_local_multicast

Experiment.record_external_multicast

Experiment.record_local_p2p

Experiment.record_external_p2p

Experiment.record_local_nearest_neighbour

Experiment.record_external_nearest_neighbour

Experiment.record_local_fixed_route

Experiment.record_external_fixed_route

Experiment.record_dropped_multicast

Experiment.record_dropped_p2p

Experiment.record_dropped_nearest_neighbour

Experiment.record_dropped_fixed_route

Experiment.record_counter12

Experiment.record_counter13

Experiment.record_counter14

Experiment.record_counter15

Record changes in each of the SpiNNaker router counter registers.

If any of these metrics are recorded, a single core on every chip will be configured accordingly ensuring router counter values are recorded for all chips in the machine. This may result in new cores being created internally and placed on core 2 of otherwise unused chips.

Experiment.record_reinjected

Experiment.record_reinject_overflow

Experiment.record_reinject_missed

Records droppped packet reinjection metrics.

Experiment.record_reinjected

The number of dropped packets which have been reinjected.

Experiment.record_reinject_overflow

The number of dropped packets which were not reinjected due to the packet reinjector’s buffer being full.

Experiment.record_reinject_missed

A lower bound on the number of dropped packets which were not reinjected due to the packet reinjector being unable to collect them from the router in time.

If any of these metrics are recorded, a single core on every chip will be configured accordingly ensuring router counter values are recorded for all chips in the machine. This may result in new cores being created internally and placed on core 2 of otherwise unused chips.

Additionally, recording any of these values will cause a further core to be reserved on all chips to perform packet reinjection, even if it is not enabled by Experiment.reinject_packets at any point in the experiment.

See also: Experiment.reinject_packets.

Experiment.record_sent

Record the number of packets successfully sent for each flow.

Experiment.record_blocked

Record the number of packets which could not be sent for each flow due to back-pressure from the network. Note that blocked packets are not resent.

Experiment.record_received

Record the number of packets received at each sink of each flow.

The Results Class

class network_tester.Results

The results of an experiment, returned by Experiment.run().

The experimental results may be accessed via one of the methods of this class. These methods produce Numpy ndarray in the form of a structured array. The exact set of fields of this array depend on the method used, however, a number of standard fields are universally present:

Any fields added using the Group.add_label() method

If a group does not have an associated value for a particular label, the value will be set to None.

‘group’

The Group object that result is associated with.

‘time’

The time that the value was recorded. Given in seconds since the start of the group’s execution (not including any warmup time).

A utility function, to_csv(), is also provided which can produce R-compatible CSV files from the output of methods in this class.

core_totals()

Gives the counter totals for each core giving the summed metrics of all flows sourced/sunk there.

In addition to the standard fields, the output of this method has a ‘core’ field containing the Core object associated with each result along with a field for each recorded core-specific metric.

If the number of sent and received packets is recorded, an additional column, ‘ideal_received’, is added which contains total number of packets which would be received if all sent packets arrived at every sink.

flow_counters()

Gives the complete counter values for every flow in the system, listing the counts for every source/sink pair individually.

In addition to the standard fields, the output of this method has:

‘flow’

The Flow object associated with each result.

‘fan_out’

The fan-out of the associated flow (i.e. number of sinks).

‘source_core’

The source Core object.

‘sink_core’

The sink Core object.

‘num_hops’

The number of chip-to-chip hops in the route from source to sink. Note that this is 0 for a pair of cores on the same chip.

A field for each recorded flow-specific metric.

flow_totals()

Gives the counter totals for each flow, summing source and sink specific metrics.

In addition to the standard fields, the output of this method has:

‘flow’

The Flow object associated with each result.

‘fan_out’

The fan-out of the associated flow (i.e. number of sinks).

A field for each recorded flow-specific metric.

router_counters()

Gives the router and reinjector counter values for every chip in the system.

In addition to the standard fields, the output of this method has:

‘x’

The X-coordinate of the chip.

‘y’

The Y-coordinate of the chip.

A field for each recorded router- or reinjector-specific metric.

totals()

Gives the total counts for all recorded metrics.

The output of this method has a field for each recorded metric in addition to the standard fields.

If the number of sent packets is recorded, an additional column, ‘ideal_received’, is added which contains total number of packets which would be received if all sent packets arrived at every sink.

network_tester.to_csv(data, header=True, col_sep=', ', row_sep='\n', none='NA', objects_as_name=True)

Render a structured array produced Results as a CSV complete with headings.

Parameters:

data : np.ndarray

A structured array produced by Results.

header : bool

If True, column headings are included in the output. If False, they are omitted.

col_sep : str

The separator between columns in the output. (Default: ‘,’)

row_sep : str

The separator between rows in the output. (Default: ‘n’)

none : str

The string to use to represent None. (Default: ‘NA’)

objects_as_name : bool

If True, any Group, Core or Flow object in the table of results will be represented by its name attribute rather than the str() of the object.

The Core, Flow and Group Classes

class network_tester.Core

A core in the experiment, created by Experiment.new_core().

A core represents a single core running a traffic generator/consumer and logging results.

See core parameters and flow parameters for experimental parameters associated with cores.

Renamed from Vertex in v0.2.0.

name

The human-readable name of this core.

chip

The (x, y) coordinate of the chip this core should reside on. Alternatively may be set to None if the core should be placed automatically on a suitable chip when the experiment is run.

class network_tester.Flow

A flow of traffic between cores, created by Experiment.new_flow().

This object inherits its attributes from rig.netlist.Net.

See flow parameters for experimental parameters associated with flows.

Renamed from Net in v0.2.0.

name

The human-readable name of this flow.

source

The source core of the flow.

sinks

A list of sink cores for the flow.

weight

The weight placement & routing hint for the flow.

class network_tester.Group

An experimental group, created by Experiment.new_group().

name

The human-readable name of this group.

add_label(name, value)

Set the value of a label results column for this group.

Label columns can be used to give more meaning to each experimental group. For example:

>>> for probability in [0.0, 0.5, 1.0]:
...     with e.new_group() as g:
...         g.add_label("probability", probability)
...         e.probability = probability

In the example above, all results generated would feature a ‘probability’ field with the corresponding value for each group making it much easier to plat experimental results.

Parameters:

name : str

The name of the field.

value

The value of the field for this group.

num_samples

The number of metric recordings which will be made during the execution of this group.