The Network Tester API¶
The Experiment
Class¶
-
class
network_tester.
Experiment
¶ Defines a network experiment to be run on a SpiNNaker machine.
An experiment consists of a defined set of SpiNNaker application cores (
new_core()
) between which traffic flows (new_flow()
).An experiment is broken up into ‘groups’ (
new_group()
), during which the traffic generators produce packets according to a specified traffic pattern. Within each group, metrics, such as packet counts, may be recorded. Though the placement of cores and the routing of traffic flows is fixed throughout an experiment, the rate and pattern of the generated traffic flows can be varied between groups allowing, for example, different traffic patterns to be tested.When the experiment is
run()
, cores will be loaded with traffic generators and the routing tables initialised. When the experiment completes, recorded results are read back ready for analysis.-
__init__
(hostname_or_machine_controller)¶ Create a new network experiment on a particular SpiNNaker machine.
Typical usage:
>>> import sys >>> from network_tester import Experiment >>> e = Experiment(sys.argv[1]) # Takes hostname as a CLI argument
The experimental parameters can be set by setting attributes of the
Experiment
instance like so:>>> e = Experiment(...) >>> # Set the probability of a packet being generated at the source >>> # of each flow every timestep >>> e.probability = 1.0
Parameters: hostname_or_machine_controller : str or
rig.machine_control.MachineController
The hostname or
MachineController
of a SpiNNaker machine to run the experiment on.
-
allocations
¶ A dictionary {
Core
: {resource: slice}, ...} or None.Defines the resources allocated to each core. This will include an allocation of the
Cores
resource representing the core allocated.See also
rig.place_and_route.allocate()
.
-
machine
¶ Deprecated. The
Machine
object describing the SpiNNaker system under test.Warning
This method is now deprecated, users should use
system_info
instead. Also, as of v0.3.0, users should not modify this object: all changes are now ignored.
-
new_core
(chip_x=None, chip_y=None, name=None)¶ Create a new
Core
.A SpiNNaker application core which can source and sink a number of traffic
flows
.Example:
>>> # Create three cores, the first two are placed on chip (0, 0) >>> # and final core will be placed automatically onto some >>> # available core in the system. >>> c0 = e.new_core(0, 0) >>> c1 = e.new_core(0, 0) >>> c2 = e.new_core()
The experimental parameters for each core can also be specified individually if desired:
>>> # Traffice flows sourced at core c2 will transmit with 50% >>> # probability each timestep >>> c2.probability = 0.5
Renamed from new_vertex in v0.2.0.
Parameters: chip_x : int or None
chip_y : int or None
If both
chip_x
andchip_y
are given, this core will be placed on the chip with the specified coordinates. Note: It is not possible to place more cores on a specific chip than the chip has available.If both
chip_x
andchip_y
are None (the default), the core will be automatically placed on some available chip (seenetwork_tester.Experiment.run()
).Note that either both must be an integer or both must be None.
name
Optional. A name for the core. If not specified the core will be given a number as its name. This name will be used in results tables.
Returns: An object representing the core.
-
new_flow
(source, sinks, weight=1.0, name=None)¶ Create a new flow.
A flow of SpiNNaker packets from one source core to many sink cores.
For example:
>>> # A flow with c0 as a source and c1 as a sink. >>> f0 = e.new_flow(c0, c1) >>> # Another flow with c0 as a source and both c1 and c2 as sinks. >>> f1 = e.new_flow(c0, [c1, c2])
The experimental parameters for each flow can be overridden individually if desired. This will take precedence over any values set for the source core of the flow.
For example:
>>> # Flow f0 will generate a packet in 80% of timesteps >>> f0.probability = 0.8
Renamed from new_net in v0.2.0.
Parameters: source :
Core
The source
Core
of the flow. A stream of packets will be generated by this core and sent to all sinks.Only
Core
objects created by thisExperiment
may be used.The sink
Core
or list of sink cores for the flow.Only
Core
objects created by thisExperiment
may be used.weight : float
Optional. A hint for the automatic place and route algorithms indicating the relative amount of traffic that may flow through this Flow. This number is not used by the traffic generator.
name
Optional. A name for the flow. If not specified the flow will be given a number as its name. This name will be used in results tables.
Returns: An object representing the flow.
-
new_group
(name=None)¶ Define a new experimental group.
The experiment can be divided up into groups where the traffic pattern generated (but not the structure of connectivity) varies for each group. Results are recorded separately for each group and the network is drained of packets between groups.
The returned
Group
object can be used as a context manager within which experimental parameters specific to that group may be set, including per-core and per-flow parameters. Note that parameters set globally for the experiment in particular group do not take precedence over per-core or per-flow parameter settings.For example:
>>> with e.new_group(): ... # Overrides default probability of sending a packet within ... # the group. ... e.probability = 0.5 ... # Overrides the probability for c2 within the group ... c2.probability = 0.25 ... # Overrides the probability for f0 within the group ... f0.probability = 0.4
Parameters: name
Optional. A name for the group. If not specified the group will be given a number as its name. This name will be used in results tables.
Returns: An object representing the group.
-
placements
¶ A dictionary {
Core
: (x, y), ...}, or None.The placements selected for each core, set after calling
run()
.See also
rig.place_and_route.place()
.
-
routes
¶ A dictionary {
Flow
:rig.place_and_route.routing_tree.RoutingTree
, ...} or None.Defines the route used for each flow.
See also
rig.place_and_route.route()
.
-
run
(app_id=219, create_group_if_none_exist=True, ignore_deadline_errors=False, constraints=None, place=<function place>, place_kwargs={}, allocate=<function allocate>, allocate_kwargs={}, route=<function route>, route_kwargs={}, before_load=None, before_group=None, before_read_results=None)¶ Run the experiment on SpiNNaker and return the results.
Before the experiment is started, any cores whose location was not specified are placed automatically on suitable chips and routes are generated between them. If fine-grained control is required the
place
,allocate
androute
arguments allow user-definedplacement
,allocation
androuting
algorithms to be used. The result of these processes can be found in theplacements
,allocations
androutes
respectively at any time after thebefore_load
callback has been called.Following placement, the experimental parameters are loaded onto the machine and each experimental group is executed in turn. Results are recorded by the machine and are read back at the end of the experiment.
Warning
Though a global synchronisation barrier is used between the execution of each group, the timers in each core may drift out of sync during each group’s execution. Further, the barrier synchronisation does not give any guarantees about how closely-synchronised the timers will be at the start of each run.
The place_and_route method was removed in v0.2.0 and its functionality merged into this method.
Parameters: app_id : int
Optional. The SpiNNaker application ID to use for the experiment.
create_group_if_none_exist : bool
Optional. If True (the default), a single group will be automatically created if none have been defined with
new_group()
. This is the most sensible behaviour for most applications.If you really want to run an experiment with no experimental groups (where no traffic will ever be generated and no results recorded), you can set this option to False.
ignore_deadline_errors : bool
If True, any realtime deadline-missed errors will no longer cause this method to raise an exception. Other errors will still cause an exception to be raised.
This option is useful when running experiments which involve over-saturating packet sinks or the network in some experimental groups.
constraints : [constraint, ...]
A list of additional place-and-route constraints to apply. In additon to the constraints supplied via this argument, a
rig.place_and_route.constraints.ReserveResourceConstraint
will be added to reserve the monitor processor on every chip and arig.place_and_route.constraints.LocationConstraint
is also added for all cores whose chip was explicitly specified (and for other special purpose cores such as packet reinjection cores and router-register recording cores).place : placer
A
Rig-API complaint
placement algorithm. This will be used to automatically place any cores whose location was not specified.place_kwargs : dict
Additional algorithm-specific keyword arguments to supply to the placer.
allocate : allocator
A
Rig-API complaint
allocation algorithm.allocate_kwargs : dict
Additional algorithm-specific keyword arguments to supply to the allocator.
route : router
A
Rig-API complaint
route algorithm. Used to route all flows in the experiment.route_kwargs : dict
Additional algorithm-specific keyword arguments to supply to the router.
before_load : function or None
If not None, this function is called before the network tester application is loaded onto the machine and after place-and-route has been completed. It is called with the
Experiment
object as its argument. The function may block to postpone the loading process as required.before_group : function or None
If not None, this function is called before each experimental group is run on the machine. It is called with the
Experiment
object andGroup
as its argument. The function may block to postpone the execution of each group as required.before_read_results : function or None
If not None, this function is called after all experimental groups have run and before the results are read back from the machine. It is called with the
Experiment
object as its argument. The function may block to postpone the reading of results as required.Returns: If no cores reported errors, the experimental results are returned. See the
Results
object for details.Raises: NetworkTesterError
A
NetworkTesterError
is raised if any cores reported an error. The most common error is likely to be a ‘deadline missed’ error as a result of the experimental timestep being too short or the load on some cores too high in extreme circumstances. Other types of error indicate far more severe problems and are probably a bug in ‘Network Tester’.Any results recorded during the run will be included in the
results
attribute of the exception. See theResults
object for details.
-
system_info
¶ The
SystemInfo
object describing the SpiNNaker system under test.This value is fetched from the machine once and cached for all future accesses. This value may be modified to influence the place-and-route processes.
-
Experimental Parameters¶
Global Parameters¶
The following experimental parameters apply globally and cannot be overridden on a core-by-core or flow-by-flow basis. They can be changed between groups.
-
Experiment.
timestep
¶ The timestep (in seconds) with which packets are generated by any packet generators in the experiment.
Default value: 0.001 (i.e. 1 ms)
All timing related parameters (e.g.
duration
andrecord_interval
) are internally converted into multiples of this timestep and rounded to the nearest number of stepsWarning
Setting this value too small will result in the traffic generator software being unable to meet timing deadlines and thus the experiment failing. In practice, 1.5 us is the smallest this can be set but larger values are required if any core is the source of many flows or if large numbers of metrics are being recorded.
-
Experiment.
duration
¶ The duration (in seconds) to run the traffic generators and record results for each experimental group.
Default value: 1.0
See also:
warmup
,cooldown
andExperiment.flush_time
.
-
Experiment.
warmup
¶ The duration (in seconds) to run the traffic generators without recording results before beginning result recording for each experimental group.
Default value: 0.0
Many experiments require the network to be in a stable state for their results to be meaningful. To achieve this, a warmup period can be specified during which time the network is allowed to settle into a stable state before any results are recorded.
See also:
duration
,cooldown
andflush_time
.
-
Experiment.
cooldown
¶ The duration (in seconds) to run the traffic generators without recording results after result recording is completed for each experimental group.
Default value: 0.0
Since the clocks in individual SpiNNaker cores are not perfectly in sync (and can drift significantly during long-duration experimental groups), it may be necessary for traffic generators to continue producing traffic for a short time after their local timer indicates the end of the experiment in order to maintain consistent network load for any traffic generators which are still running.
See also:
duration
,warmup
andflush_time
.
-
Experiment.
flush_time
¶ The pause (in seconds) which is added between experimental groups to allow any packets which remain in the network to be drained.
Default value: 0.01
This is especially important when
consume_packets
has been set to False.
-
Experiment.
record_interval
¶ The interval (in seconds) at which metrics are recorded during an experiment.
Default value: 0.0
Special case: If zero, metrics will be recorded once at the end each group’s execution.
See Metric Recording Selection for the set of metrics which can be recorded.
-
Experiment.
router_timeout
¶ Sets the router timeout (in router clock cycles at (usually) 133MHz).
Default value: None (do not change)
If set to an integer, this sets the number of clock cycles before a packet times out and is dropped.
Experiment.run()
will throw aValueError
if the value presented is not a valid router timeout. Emergency routing will be disabled.If set to a tuple (wait1, wait2), wait1 sets the number of clock cycles before emergency routing is tried and wait2 gives the number of additional cycles before the packet is dropped.
Experiment.run()
will throw aValueError
if the values presented are not valid router timeouts.If None, the timeout will be left as the default when the system was booted.
If this field is set to anything other than None at any point during the experiment, a single core on every chip will be used to set the router timeout. This may result in new cores being created internally and placed on otherwise unused chips.
-
Experiment.
reinject_packets
¶ Enable dropped packet reinjection.
Default value: False (do not use dropped packet reinjection).
Enabling this feature at any point during the experiment will cause core 1 to be reserved on all chips to perform packet reinjection.
See also:
Experiment.record_reinjected
,Experiment.record_reinject_overflow
andExperiment.record_reinject_missed
.
Flow/Traffic Generator Parameters¶
The following experimental parameters apply to each flow (and thus each traffic generator) in the experiment and control the pattern of packets generated and sent down each flow.
The global default for each value can be set by setting the corresponding
Experiment
attribute.
The default for flows sourced by a particular core can be set by setting the
corresponding Core
attribute.
These values may also be overridden on a group-by-group basis.
For example:
>>> e = Experiment(...)
>>> c0 = e.new_core()
>>> c1 = e.new_core()
>>> c2 = e.new_core()
>>> f0 = e.new_flow(c0, c1)
>>> f1 = e.new_flow(c0, c3)
>>> f2 = e.new_flow(c2, c3)
>>> e.probability = 1.0
>>> c0.probability = 0.9
>>> f0.probability = 0.8
>>> # First group: f0, f1, f2 all have probability == 0.0
>>> with e.new_group():
... e.probability = 0.0
... c0.probability = 0.0
... f0.probability = 0.0
>>> # Second group: f2 has probability == 0.1 but f0 and f1 have
>>> # probabilities 0.9 and 0.8 respectively.
>>> with e.new_group():
... e.probability = 0.1
-
Flow.
probability
¶ -
Core.
probability
¶ -
Experiment.
probability
¶ The probability, between 0.0 and 1.0, of a packet being generated in a given timestep.
Default value: 1.0 (Generate a packet every timestep.)
Packet generation is also subject to the burst parameters
burst_period
,burst_duty
, andburst_phase
. By default packets are only generated according toprobability
since bursting behaviour is not enabled.
-
Flow.
packets_per_timestep
¶ -
Core.
packets_per_timestep
¶ -
Experiment.
packets_per_timestep
¶ The number of packets to generate each timestep.
Default value: 1 (Generate a single packet every timestep.)
The probability of each packet being generated is independent and set to
Flow.probability
.
-
Flow.
num_retries
¶ -
Core.
num_retries
¶ -
Experiment.
num_retries
¶ Number of times to re-try sending a packet if back-pressure from the network blocks it.
Default value: 0 (Give up immediately when faced with back-pressure)
-
Flow.
burst_period
¶ -
Flow.
burst_duty
¶ -
Flow.
burst_phase
¶ -
Core.
burst_period
¶ -
Core.
burst_duty
¶ -
Core.
burst_phase
¶ -
Experiment.
burst_period
¶ -
Experiment.
burst_duty
¶ -
Experiment.
burst_phase
¶ Set the bursting behaviour of the traffic generated for a flow.
Default values:
burst_period
= 1.0,burst_duty
= 0.0 andburst_phase
= 0.0. (Not bursting).If
burst_period
is 0.0, packets will be generated each timestep with probabilityprobability
.If
burst_period
is set to a non-zero number of seconds, packets may only be generated the proportion of the time specified byburst_duty
, everyburst_period
seconds.# # burst_period seconds # |------------------------------| # (burst_duty * burst_period) seconds # |-------| # (burst_phase * burst_period) seconds # |----| # . . # maybe send +----.--+ +----.--+ # | . | | . | # | . | | . | # never send ...---+ . +----------------------+ . +---... # . . # . . # time = t time = t + burst_period
burst_phase
gives the initial phase of the bursting behaviour. Note that the phase is not reset at the start of each group. Values outside of the range 0.0 - 1.0 will be wrapped to within this range.Special case: If
burst_phase
is set to None, the phase of the bursting behaviour will be chosen randomly.
Core Parameters¶
The following experimental parameters apply to each core in the experiment.
The global default for each value can be set by setting the corresponding
Experiment
attribute.
These values may also be overridden on a group-by-group basis.
-
Core.
seed
¶ -
Experiment.
seed
¶ The seed for the random number generator in the specified core or None to select a random seed automatically.
Default value: None (Seed randomly automatically).
If set to a non-None value, the seed is (re)set at the start of each group.
-
Core.
consume_packets
¶ -
Experiment.
consume_packets
¶ Should the specified core consume packets from the network?
Default value: True (Consume packets from the network)
If set to False, packets sent via any flow to the core will not be consumed from the network. This will cause the packets to back-up in the network and eventually cause routers to drop packets.
See also
Experiment.flush_time
.
Metric Recording Selection¶
The following boolean attributes control what metrics are recorded during an
experiment. Note that the set of recorded metrics may not be changed during an
experiment (though the recording interval can, see
Experiment.record_interval
).
By default, no metrics are recorded.
-
Experiment.
record_local_multicast
¶ -
Experiment.
record_external_multicast
¶ -
Experiment.
record_local_p2p
¶ -
Experiment.
record_external_p2p
¶ -
Experiment.
record_local_nearest_neighbour
¶ -
Experiment.
record_external_nearest_neighbour
¶ -
Experiment.
record_local_fixed_route
¶ -
Experiment.
record_external_fixed_route
¶ -
Experiment.
record_dropped_multicast
¶ -
Experiment.
record_dropped_p2p
¶ -
Experiment.
record_dropped_nearest_neighbour
¶ -
Experiment.
record_dropped_fixed_route
¶ -
Experiment.
record_counter12
¶ -
Experiment.
record_counter13
¶ -
Experiment.
record_counter14
¶ -
Experiment.
record_counter15
¶ Record changes in each of the SpiNNaker router counter registers.
If any of these metrics are recorded, a single core on every chip will be configured accordingly ensuring router counter values are recorded for all chips in the machine. This may result in new cores being created internally and placed on core 2 of otherwise unused chips.
-
Experiment.
record_reinjected
¶ -
Experiment.
record_reinject_overflow
¶ -
Experiment.
record_reinject_missed
¶ Records droppped packet reinjection metrics.
Experiment.record_reinjected
- The number of dropped packets which have been reinjected.
Experiment.record_reinject_overflow
- The number of dropped packets which were not reinjected due to the packet reinjector’s buffer being full.
Experiment.record_reinject_missed
- A lower bound on the number of dropped packets which were not reinjected due to the packet reinjector being unable to collect them from the router in time.
If any of these metrics are recorded, a single core on every chip will be configured accordingly ensuring router counter values are recorded for all chips in the machine. This may result in new cores being created internally and placed on core 2 of otherwise unused chips.
Additionally, recording any of these values will cause a further core to be reserved on all chips to perform packet reinjection, even if it is not enabled by
Experiment.reinject_packets
at any point in the experiment.See also:
Experiment.reinject_packets
.
-
Experiment.
record_sent
¶ Record the number of packets successfully sent for each flow.
-
Experiment.
record_blocked
¶ Record the number of packets which could not be sent for each flow due to back-pressure from the network. Note that blocked packets are not resent.
-
Experiment.
record_received
¶ Record the number of packets received at each sink of each flow.
The Results
Class¶
-
class
network_tester.
Results
¶ The results of an experiment, returned by
Experiment.run()
.The experimental results may be accessed via one of the methods of this class. These methods produce Numpy
ndarray
in the form of a structured array. The exact set of fields of this array depend on the method used, however, a number of standard fields are universally present:- Any fields added using the
Group.add_label()
method - If a group does not have an associated value for a particular label, the value will be set to None.
- ‘group’
- The
Group
object that result is associated with. - ‘time’
- The time that the value was recorded. Given in seconds since the start of the group’s execution (not including any warmup time).
A utility function,
to_csv()
, is also provided which can produce R-compatible CSV files from the output of methods in this class.-
core_totals
()¶ Gives the counter totals for each core giving the summed metrics of all flows sourced/sunk there.
In addition to the standard fields, the output of this method has a ‘core’ field containing the
Core
object associated with each result along with a field for each recorded core-specific metric.If the number of sent and received packets is recorded, an additional column, ‘ideal_received’, is added which contains total number of packets which would be received if all sent packets arrived at every sink.
-
flow_counters
()¶ Gives the complete counter values for every flow in the system, listing the counts for every source/sink pair individually.
In addition to the standard fields, the output of this method has:
- ‘flow’
- The
Flow
object associated with each result. - ‘fan_out’
- The fan-out of the associated flow (i.e. number of sinks).
- ‘source_core’
- The source
Core
object. - ‘sink_core’
- The sink
Core
object. - ‘num_hops’
- The number of chip-to-chip hops in the route from source to sink. Note that this is 0 for a pair of cores on the same chip.
- A field for each recorded flow-specific metric.
-
flow_totals
()¶ Gives the counter totals for each flow, summing source and sink specific metrics.
In addition to the standard fields, the output of this method has:
- ‘flow’
- The
Flow
object associated with each result. - ‘fan_out’
- The fan-out of the associated flow (i.e. number of sinks).
- A field for each recorded flow-specific metric.
-
router_counters
()¶ Gives the router and reinjector counter values for every chip in the system.
In addition to the standard fields, the output of this method has:
- ‘x’
- The X-coordinate of the chip.
- ‘y’
- The Y-coordinate of the chip.
- A field for each recorded router- or reinjector-specific metric.
-
totals
()¶ Gives the total counts for all recorded metrics.
The output of this method has a field for each recorded metric in addition to the standard fields.
If the number of sent packets is recorded, an additional column, ‘ideal_received’, is added which contains total number of packets which would be received if all sent packets arrived at every sink.
- Any fields added using the
-
network_tester.
to_csv
(data, header=True, col_sep=', ', row_sep='\n', none='NA', objects_as_name=True)¶ Render a structured array produced
Results
as a CSV complete with headings.Parameters: data :
np.ndarray
A structured array produced by
Results
.header : bool
If True, column headings are included in the output. If False, they are omitted.
col_sep : str
The separator between columns in the output. (Default: ‘,’)
row_sep : str
The separator between rows in the output. (Default: ‘n’)
none : str
The string to use to represent
None
. (Default: ‘NA’)objects_as_name : bool
The Core
, Flow
and Group
Classes¶
-
class
network_tester.
Core
¶ A core in the experiment, created by
Experiment.new_core()
.A core represents a single core running a traffic generator/consumer and logging results.
See core parameters and flow parameters for experimental parameters associated with cores.
Renamed from Vertex in v0.2.0.
-
name
¶ The human-readable name of this core.
-
chip
¶ The (x, y) coordinate of the chip this core should reside on. Alternatively may be set to None if the core should be placed automatically on a suitable chip when the experiment is run.
-
-
class
network_tester.
Flow
¶ A flow of traffic between cores, created by
Experiment.new_flow()
.This object inherits its attributes from
rig.netlist.Net
.See flow parameters for experimental parameters associated with flows.
-
name
¶ The human-readable name of this flow.
-
source
¶ The source core of the flow.
-
sinks
¶ A list of sink cores for the flow.
-
weight
¶ The weight placement & routing hint for the flow.
-
-
class
network_tester.
Group
¶ An experimental group, created by
Experiment.new_group()
.-
name
¶ The human-readable name of this group.
-
add_label
(name, value)¶ Set the value of a label results column for this group.
Label columns can be used to give more meaning to each experimental group. For example:
>>> for probability in [0.0, 0.5, 1.0]: ... with e.new_group() as g: ... g.add_label("probability", probability) ... e.probability = probability
In the example above, all results generated would feature a ‘probability’ field with the corresponding value for each group making it much easier to plat experimental results.
Parameters: name : str
The name of the field.
value
The value of the field for this group.
-
num_samples
¶ The number of metric recordings which will be made during the execution of this group.
-