Skip to content

Add support for gated queues, credit-based shaping - #1758

Draft
troglobit wants to merge 18 commits into
mainfrom
qos-tsn
Draft

troglobit wants to merge 18 commits into
mainfrom
qos-tsn

Conversation

@troglobit

Copy link
Copy Markdown
Contributor

Description

TBD

Checklist

Tick relevant boxes, this PR is-a or has-a:

  • Bugfix
    • Regression tests
    • ChangeLog updates (for next release)
  • Feature
    • YANG model change => revision updated?
    • Regression tests added?
    • ChangeLog updates (for next release)
    • Documentation added?
  • Test changes
    • Checked in changed Readme.adoc (make test-spec)
    • Added new test to group Readme.adoc and yaml file
  • Code style update (formatting, renaming)
  • Refactoring (please detail in commit messages)
  • Build related changes
  • Documentation content changes
    • ChangeLog updated (for major changes)
  • Other (please describe):

The IEEE 802.1Qcw scheduled traffic module needs the rational and PTP
time groupings, which the 2022-10-29 revision in the tree does not
have.  The LLDP module, the only importer, uses types that are
unchanged between the two.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Nothing in the model can reserve bandwidth for a traffic class or gate
its transmission in time, the two shapers a TSN port needs.  IEEE
802.1DC-2024 models both on ietf-interfaces: the credit-based shaper
of 802.1Qav through ieee802-dot1dc-cbsa-if, and the time-aware shaper
of 802.1Qbv through ieee802-dot1dc-sched-if, each using the parameter
groupings of its 802.1Q module.

Add the four modules as published, and one Infix module of deviations.
The scheduled traffic capability leaves, supported-list-max and its
two siblings, are plain configuration in the standard, with the note
that they may be read-only, and three must statements compare the
admin list, cycle and intervals against them.  Left as published,
every interface fails validation, configured or not, since the leaves
have no value to compare with.  The deviations make the three leaves
read-only and drop the comparisons; the daemon checks the list against
the port instead.

The modules are installed by the new Feature TSN package only, which
also turns on the cbs and taprio queueing disciplines in the kernel,
so a product without the shaper hardware has no TSN nodes in its
model.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
A traffic class could be set to credit-based-shaper, but the daemon
rejected it, having nothing to shape with.  IEEE 802.1Qav reserves
bandwidth for a class by an idle slope, the admin-idle-slope of the
cbsa table beside qos on the interface.

Read the table, and attach a cbs qdisc on the band of every class
whose algorithm is credit-based-shaper, offloaded on a switch port and
in software elsewhere:

    tc qdisc replace dev e1 parent 1:1 handle 17: cbs idleslope 20000 \
        sendslope -980000 hicredit 30 locredit -1492 offload 1

The operator gives the idle slope in bits per second as 802.1Q has it;
the send slope and the two credits follow Annex L from it, the link
rate and one maximum frame, since the kernel passes all four to the
driver and computes nothing.  The ets scheduler gets an explicit
handle so the children have a parent to name.

The algorithm and the slope live in different modules, so the daemon
checks them against each other: a shaped class needs a slope and a
slope needs a shaped class, the slope stays under the link rate, the
shaped classes sit above the strict ones as 802.1Q-2022 34.5 lays a
port out, and rate-limit excludes per-class shaping.

The credits are computed from the link rate at commit; a later speed
change does not yet rebuild them.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Nothing gated a port's transmission in time.  IEEE 802.1Qbv carves
port time into a repeating cycle and opens each traffic class's gate
only in defined windows, configured as a gate control list in the
gate-parameter-table beside qos on the interface.

With gate-enabled set, replace the ets root with taprio: one class per
transmit queue in standard numbering, the base time and cycle from the
PTP timestamp and the rational seconds of the table, and the list as
given, offloaded on a switch port and run against TAI elsewhere:

    tc qdisc replace dev e1 root handle 1: taprio num_tc 8 \
        map 1 0 2 3 4 5 6 7 queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \
        base-time 200 cycle-time 100000 \
        sched-entry S 80 20000 sched-entry S a0 20000 \
        sched-entry S 5f 60000 flags 0x2

A shaped class under gates earns credit only while its gate is open,
so its cbs child gets the slope scaled by the cycle over the class's
open time, 802.1Q-2022 8.6.8.2, and the credits follow the scaled
slope.

Gates take the root and have no weights, so gate-enabled rejects
rate-limit and weighted classes.  The list must have entries, every
interval above zero and only set-gate-states operations, a shaped
class needs a gate that opens, its scaled slope must fit the link, and
the port needs a transmit queue per traffic class, which taprio
requires.  config-change is not consulted: the list applies at commit.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The credit-based and time-aware shapers had no operational state: the
reservation in force, the gate control list the port runs, and whether
either reached the fabric were invisible.

Read them back from the qdisc tree.  A cbs child gives the idle slope
of its class, scaled back to the reservation it came from when the
root is taprio, since confd scaled it up by the cycle over the class's
open time.  A taprio root gives the operational gate control list,
cycle, base time and whether an admin list is pending.  The two
offload enums tell when the shaper runs in the switch fabric.  Nodes
of a module the product has not installed are left out.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The shapers have no coverage.  The credit-based shaper test reserves
10 Mbit/s for the top class, offers three times that through it and
expects the reservation at the listener, reads the reservation back
as the operational idle slope, and checks that the algorithm and the
slope are rejected one without the other.

The time-aware shaper test puts a three-entry gate control list on a
port and reads it back as the operational list with its cycle and
base time, and checks that a rate limit, a weighted class and an
empty list are rejected under gates.  A port with fewer transmit
queues than classes, the virtual rig among them, must reject the
schedule instead, which the test checks and stops at.

The validation test's credit-based-shaper case changes meaning: the
algorithm is accepted now, so what is rejected is a shaped class
without a reservation.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The credit-based and time-aware shaper tests need the IEEE 802.1DC
modules on the virtual rig, which has no shaper hardware but runs
both qdiscs in software.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The QoS guide stops at the port rate limit; the two shapers have no
operator documentation.  Add a section for each: what to set, where
the two halves of a reservation live, the gate control list with its
units, what each excludes, and what the port reports back.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Every port carries its own copy of the classification and traffic
class settings, so a switch with many ports in the same role repeats
them per port and a change means one edit per port.  On fabrics whose
class table and weights are per chip, ports must agree on them, which
per-port copies cannot express.

Move the pipeline into groupings in a new infix-qos module and add a
list of named profiles under /system/qos that instantiates them once
per name.  An interface either references a profile or holds the
settings inline, a choice with inline as the default case, so the two
are exclusive in the schema and the defaults apply to the inline case
only.  A referenced profile cannot be deleted, by the leafref.

The rate limit stays on the port, since it follows the link rather
than the role, and moves from egress to the qos container, which the
inline case now owns.  The model is unreleased, so the path change
costs nothing.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
A port's classification and traffic classes can now come from a named
profile under /system/qos/profile, so every edge port of a site shares
one definition.  The port's own settings and the profile reference are
a choice in the model; the rate limit stays per port.

The profile is resolved once when the port's script is generated and
handed to validation and parsing.  A change to a profile must
regenerate every port that references it, which the dependency pass
adds to the diff; a profile edit that leaves the class layout alone
does not replace the root qdisc, the same rule as for inline edits.
Validation errors name the port and the profile.  The applied profile
is written to /run/qos/<ifname> so operational data can report it
without reading the running configuration.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The operational qos container showed a port's capabilities, but not
the profile it runs or the rate limit in force.

Read the profile from /run/qos/<ifname>, written by the port's qos
script and watched with inotify, and the rate limit from the tbf root
qdisc.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Two ports share one profile and must run its traffic class table and
report it in use; a change to the profile must reach both; deleting a
referenced profile must be rejected; a port taken off the profile runs
its own settings.  The rate limit moved from egress to the port's qos
container, so the tests that set it follow.

The rejection helper the shaper, schedule and validation tests each
carried now lives in infamy.qos, and the profile a port runs is a
property of infamy.capability.Port.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Describe profiles under system qos: what they hold, how a port
references one, what stays on the port, and how a profile change and
a delete behave.  The rate limit example follows the leaf to the
port's qos container.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Status > QoS lists what each port runs: profile, traffic classes, rate
limit and the stages the fabric offloads.  Configure > QoS edits the
profiles: trust, default priority, PCP and DSCP presets, class table
preset, transmission selection and share per class, remarking.  The
interface page gets a QoS section with the profile select and the
rate limit, and on products with the TSN modules the credit-based
shaper table (idle slope per shaped class, with the slope in effect)
and the gate control list (cycle, base time, entries with a gate per
class and an interval, cycle time extension and max SDU under
Advanced).

Pages appear only when the device has the infix-qos module, the TSN
sections only with infix-tsn, the rate limit input only with the
rate-limit feature.  Options, their labels and help, defaults and
input ranges all come from the schema; the schema layer now carries
the description of each enum value and identity on an option, and
the bounds of a range.

The add-row mechanism in app.js takes a <template> element rendered by
the server, so new rows are written in HTML next to the existing ones.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Values, labels, help text, defaults and ranges copied from YANG into
Go or templates drift from the model, which is the contract shared
with the CLI and NETCONF.  Write the rule down where reviewers and
agents look for it.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Profiles never showed up: the datastore wraps the qos container in
the system container it augments, and the decoder expected it bare.
Putting a port on a profile left its inline settings in place, and
the candidate reported their empty containers, so the copy to running
failed the choice validation; the other case is now deleted first and
the copy reads the candidate with defaults trimmed.  Deleting a
profile that ports reference is refused with the ports named, instead
of failing at apply.  The selects were cut at eight rem.

The Advanced tree listed leaves behind features the device does not
enable, and saving one gave "not a child of".  The schema cache now
keeps the features from yang-library beside the modules and hides
nodes whose if-feature expression does not hold.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The status page explained its columns in prose written for the page.
The leaf descriptions say the same, so the column headers carry them
as field help and the note goes.  The notes on the configure pages
that explained what a port without a profile is, and what to do about
an unshaped class, go the same way.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The WebUI alone holds about 90 MiB for the YANG schema, and a page
that queries operational data spawns yanger processes beside statd.
On a 448 MiB machine the kernel killed the WebUI under that load,
which the browser shows as a device that goes unreachable and comes
back on every page.  The Infamy DUTs were raised the same way.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant