mirror of
https://git.proxmox.com/git/mirror_frr
synced 2025-08-02 20:27:14 +00:00
Merge pull request #14448 from qlyoung/doc-add-northbound-api-docs
doc: add northbound api arch docs
This commit is contained in:
Mark Stapp
GitHub
commit
fe1da43cab
@ -22,3 +22,4 @@ FRRouting Developer's Guide
|
||||
path
|
||||
pceplib
|
||||
link-state
|
||||
northbound/northbound
|
||||
|
||||
15
doc/developer/northbound/_sidebar.rst
Normal file
15
doc/developer/northbound/_sidebar.rst
Normal file
@ -0,0 +1,15 @@
|
||||
Northbound:
|
||||
~~~~~~~~~~~
|
||||
|
||||
- [[Architecture]]
|
||||
- [[Transactional CLI]]
|
||||
- [[Retrofitting Configuration Commands]]
|
||||
- [[Operational data, RPCs and Notifications]]
|
||||
- [[Plugins - ConfD]]
|
||||
- [[Plugins: Sysrepo]]
|
||||
- [[Plugins - Writing Your Own]]
|
||||
- [[YANG module translator]]
|
||||
- [[Advanced topics]]
|
||||
- [[YANG tools]]
|
||||
- [[Demos]]
|
||||
- [[Links]]
|
||||
294
doc/developer/northbound/advanced-topics.rst
Normal file
294
doc/developer/northbound/advanced-topics.rst
Normal file
@ -0,0 +1,294 @@
|
||||
Auto-generated CLI commands
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In order to have less code to maintain, it should be possible to write a
|
||||
tool that auto-generates CLI commands based on the FRR YANG models. As a
|
||||
matter of fact, there are already a number of NETCONF-based CLIs that do
|
||||
exactly that (e.g. `Clixon <https://github.com/clicon/clixon>`__,
|
||||
ConfD’s CLI).
|
||||
|
||||
The problem however is that there isn’t an exact one-to-one mapping
|
||||
between the existing CLI commands and the corresponding YANG nodes from
|
||||
the native models. As an example, ripd’s
|
||||
``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` command
|
||||
changes three YANG leaves at the same time. In order to auto-generate
|
||||
CLI commands and retain their original form, it’s necessary to add
|
||||
annotations in the YANG modules to specify how the commands should look
|
||||
like. Without YANG annotations, the CLI auto-generator will generate a
|
||||
command for each YANG leaf, (leaf-)list and presence-container. The
|
||||
ripd’s ``timers basic`` command, for instance, would become three
|
||||
different commands, which would be undesirable.
|
||||
|
||||
This Tail-f’s®
|
||||
`document <http://info.tail-f.com/hubfs/Whitepapers/Tail-f_ConfD-CLI__Cfg_Mode_App_Note_Rev%20C.pdf>`__
|
||||
shows how to customize ConfD auto-generated CLI commands using YANG
|
||||
annotations.
|
||||
|
||||
The good news is that *libyang* allows users to create plugins to
|
||||
implement their own YANG extensions, which can be used to implement CLI
|
||||
annotations. If done properly, a CLI generator can save FRR developers
|
||||
from writing and maintaining hundreds if not thousands of DEFPYs!
|
||||
|
||||
CLI on a separate program
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The flexible design of the northbound architecture opens the door to
|
||||
move the CLI to a separate program in the long-term future. Some
|
||||
advantages of doing so would be: \* Treat the CLI as just another
|
||||
northbound client, instead of having CLI commands embedded in the
|
||||
binaries of all FRR daemons. \* Improved robustness: bugs in CLI
|
||||
commands (e.g. null-pointer dereferences) or in the CLI code itself
|
||||
wouldn’t affect the FRR daemons. \* Foster innovation by allowing other
|
||||
CLI programs to be implemented, possibly using higher level programming
|
||||
languages.
|
||||
|
||||
The problem, however, is that the northbound retrofitting process will
|
||||
convert only the CLI configuration commands and EXEC commands in a first
|
||||
moment. Retrofitting the “show” commands is a completely different story
|
||||
and shouldn’t happen anytime soon. This should hinder progress towards
|
||||
moving the CLI to a separate program.
|
||||
|
||||
Proposed feature: confirmed commits
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Confirmed commits allow the user to request an automatic rollback to the
|
||||
previous configuration if the commit operation is not confirmed within a
|
||||
number of minutes. This is particularly useful when the user is
|
||||
accessing the CLI through the network (e.g. using SSH) and any
|
||||
configuration change might cause an unexpected loss of connectivity
|
||||
between the user and the router (e.g. misconfiguration of a routing
|
||||
protocol). By using a confirmed commit, the user can rest assured the
|
||||
connectivity will be restored after the given timeout expires, avoiding
|
||||
the need to access the router physically to fix the problem.
|
||||
|
||||
Example of how this feature could be provided in the CLI:
|
||||
``commit confirmed [minutes <1-60>]``. The ability to do confirmed
|
||||
commits should also be exposed in the northbound API so that the
|
||||
northbound plugins can also take advantage of it (in the case of the
|
||||
Sysrepo and ConfD plugins, confirmed commits are implemented externally
|
||||
in the *netopeer2-server* and *confd* daemons, respectively).
|
||||
|
||||
Proposed feature: enable/disable configuration commands/sections
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Since the ``lyd_node`` data structure from *libyang* can hold private
|
||||
data, it should be possible to mark configuration commands or sections
|
||||
as active or inactive. This would allow CLI users to leverage this
|
||||
feature to disable parts of the running configuration without actually
|
||||
removing the associated commands, and then re-enable the disabled
|
||||
configuration commands or sections later when necessary. Example:
|
||||
|
||||
::
|
||||
|
||||
ripd(config)# show configuration running
|
||||
Configuration:
|
||||
[snip]
|
||||
!
|
||||
router rip
|
||||
default-metric 2
|
||||
distance 80
|
||||
network eth0
|
||||
network eth1
|
||||
!
|
||||
end
|
||||
ripd(config)# disable router rip
|
||||
ripd(config)# commit
|
||||
% Configuration committed successfully (Transaction ID #7).
|
||||
|
||||
ripd(config)# show configuration running
|
||||
Configuration:
|
||||
[snip]
|
||||
!
|
||||
!router rip
|
||||
!default-metric 2
|
||||
!distance 80
|
||||
!network eth0
|
||||
!network eth1
|
||||
!
|
||||
end
|
||||
ripd(config)# enable router rip
|
||||
ripd(config)# commit
|
||||
% Configuration committed successfully (Transaction ID #8).
|
||||
|
||||
ripd(config)# show configuration running
|
||||
[snip]
|
||||
frr defaults traditional
|
||||
!
|
||||
router rip
|
||||
default-metric 2
|
||||
distance 80
|
||||
network eth0
|
||||
network eth1
|
||||
!
|
||||
end
|
||||
|
||||
This capability could be useful in a number of occasions, like disabling
|
||||
configuration commands that are no longer necessary (e.g. ACLs) but that
|
||||
might be necessary at a later point in the future. Other example is
|
||||
allowing users to disable a configuration section for testing purposes,
|
||||
and then re-enable it easily without needing to copy and paste any
|
||||
command.
|
||||
|
||||
Configuration reloads
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Given the limitations of the previous northbound architecture, the FRR
|
||||
daemons didn’t have the ability to reload their configuration files by
|
||||
themselves. The SIGHUP handler of most daemons would only re-read the
|
||||
configuration file and merge it into the running configuration. In most
|
||||
cases, however, what is desired is to replace the running configuration
|
||||
by the updated configuration file. The *frr-reload.py* script was
|
||||
written to work around this problem and it does it well to a certain
|
||||
extent. The problem with the *frr-reload.py* script is that it’s full of
|
||||
special cases here and there, which makes it fragile and unreliable.
|
||||
Maintaining the script is also an additional burden for FRR developers,
|
||||
few of whom are familiar with its code or know when it needs to be
|
||||
updated to account for a new feature.
|
||||
|
||||
In the new northbound architecture, reloading the configuration file can
|
||||
be easily implemented using a configuration transaction. Once the FRR
|
||||
northbound retrofitting process is complete, all daemons should have the
|
||||
ability to reload their configuration files upon receiving the SIGHUP
|
||||
signal, or when the ``configuration load [...] replace`` command is
|
||||
used. Once that point is reached, the *frr-reload.py* script will no
|
||||
longer be necessary and should be removed from the FRR repository.
|
||||
|
||||
Configuration changes coming from the kernel
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
This
|
||||
`post <http://discuss.tail-f.com/t/who-should-not-set-configuration-once-a-system-is-up-and-running/111>`__
|
||||
from the Tail-f’s® forum describes the problem of letting systems
|
||||
configure themselves behind the users back. Here are some selected
|
||||
snippets from it: > Traditionally, northbound interface users are the
|
||||
ones in charge of providing configuration data for systems. > > In some
|
||||
systems, we see a deviation from this traditional practice; allowing
|
||||
systems to configure “themselves” behind the scenes (or behind the users
|
||||
back). > > While there might be a business case for such a practice,
|
||||
this kind of configuration remains “dangerous” from northbound users
|
||||
perspective and makes systems hard to predict and even harder to debug.
|
||||
(…) > > With the advent of transactional Network configuration, this
|
||||
practice can not work anymore. The fact that systems are given the right
|
||||
to change configuration is a key here in breaking transactional
|
||||
configuration in a Network.
|
||||
|
||||
FRR is immune to some of the problems described in the aforementioned
|
||||
post. Management clients can configure interfaces that don’t yet exist,
|
||||
and once an interface is deleted from the kernel, its configuration is
|
||||
retained in FRR.
|
||||
|
||||
There are however some cases where information learned from the kernel
|
||||
(e.g. using netlink) can affect the running configuration of all FRR
|
||||
daemons. Examples: interface rename events, VRF rename events, interface
|
||||
being moved to a different VRF, etc. In these cases, since these events
|
||||
can’t be ignored, the best we can do is to send YANG notifications to
|
||||
the management clients to inform about the configuration changes. The
|
||||
management clients should then be prepared to handle such notifications
|
||||
and react accordingly.
|
||||
|
||||
Interfaces and VRFs
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
As of now zebra doesn’t have the ability to create VRFs or virtual
|
||||
interfaces in the kernel. The ``vrf`` and ``interface`` commands only
|
||||
create pre-provisioned VRFs and interfaces that are only activated when
|
||||
the corresponding information is learned from the kernel. When
|
||||
configuring FRR using an external management client, like a NETCONF
|
||||
client, it might be desirable to actually create functional VRFs and
|
||||
virtual interfaces (e.g. VLAN subinterfaces, bridges, etc) that are
|
||||
installed in the kernel using OS-specific APIs (e.g. netlink, routing
|
||||
socket, etc). Work needs to be done in this area to make this possible.
|
||||
|
||||
Shared configuration objects
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
One of the existing problems in FRR is that it’s hard to ensure that all
|
||||
daemons are in sync with respect to the shared configuration objects
|
||||
(e.g. interfaces, VRFs, route-maps, ACLs, etc). When a route-map is
|
||||
configured using *vtysh*, the same command is sent to all relevant
|
||||
daemons (the daemons that implement route-maps), which ensures
|
||||
synchronization among them. The problem is when a daemon starts after
|
||||
the route-maps are created. In this case this daemon wouldn’t be aware
|
||||
of the previously configured route-maps (unlike the other daemons),
|
||||
which can lead to a lot of confusion and unexpected problems.
|
||||
|
||||
With the new northbound architecture, configuration objects can be
|
||||
manipulated using higher level abstractions, which opens more
|
||||
possibilities to solve this decades-long problem. As an example, one
|
||||
solution would be to make the FRR daemons fetch the shared configuration
|
||||
objects from zebra using the ZAPI interface during initialization. The
|
||||
shared configuration objects could be requested using a list of XPaths
|
||||
expressions in the ``ZEBRA_HELLO`` message, which zebra would respond by
|
||||
sending the shared configuration objects encoded in the JSON format.
|
||||
This solution however doesn’t address the case where zebra starts or
|
||||
restarts after the other FRR daemons. Other solution would be to store
|
||||
the shared configuration objects in the northbound SQL database and make
|
||||
all daemons fetch these objects from there. So far no work has been made
|
||||
on this area as more investigation needs to be done.
|
||||
|
||||
vtysh support
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
As explained in the [[Transactional CLI]] page, all commands introduced
|
||||
by the transactional CLI are not yet available in *vtysh*. This needs to
|
||||
be addressed in the short term future. Some challenges for doing that
|
||||
work include: \* How to display configurations (running, candidates and
|
||||
rollbacks) in a more clever way? The implementation of the
|
||||
``show running-config`` command in *vtysh* is not something that should
|
||||
be followed as an example. A better idea would be to fetch the desired
|
||||
configuration from all daemons (encoded in JSON for example), merge them
|
||||
all into a single ``lyd_node`` variable and then display the combined
|
||||
configurations from this variable (the configuration merges would
|
||||
transparently take care of combining the shared configuration objects).
|
||||
In order to be able to manipulate the JSON configurations, *vtysh* will
|
||||
need to load the YANG modules from all daemons at startup (this might
|
||||
have a minimal impact on startup time). The only issue with this
|
||||
approach is that the ``cli_show()`` callbacks from all daemons are
|
||||
embedded in their binaries and thus not accessible externally. It might
|
||||
be necessary to compile these callbacks on a separate shared library so
|
||||
that they are accessible to *vtysh* too. Other than that, displaying the
|
||||
combined configurations in the JSON/XML formats should be
|
||||
straightforward. \* With the current design, transaction IDs are
|
||||
per-daemon and not global across all FRR daemons. This means that the
|
||||
same transaction ID can represent different transactions on different
|
||||
daemons. Given this observation, how to implement the
|
||||
``rollback configuration`` command in *vtysh*? The easy solution would
|
||||
be to add a ``daemon WORD`` argument to specify the context of the
|
||||
rollback, but per-daemon rollbacks would certainly be confusing and
|
||||
convoluted to end users. A better idea would be to attack the root of
|
||||
the problem: change configuration transactions to be global instead of
|
||||
being per-daemon. This involves a bigger change in the northbound
|
||||
architecture, and would have implications on how transactions are stored
|
||||
in the SQL database (daemon-specific and shared configuration objects
|
||||
would need to have their own tables or columns). \* Loading
|
||||
configuration files in the JSON or XML formats will be tricky, as
|
||||
*vtysh* will need to know which sections of the configuration should be
|
||||
sent to which daemons. *vtysh* will either need to fetch the YANG
|
||||
modules implemented by all daemons at runtime or obtain this information
|
||||
at compile-time somehow.
|
||||
|
||||
Detecting type mismatches at compile-time
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
As described in the [[Retrofitting Configuration Commands]] page, the
|
||||
northbound configuration callbacks detect type mismatches at runtime
|
||||
when fetching data from the the ``dnode`` parameter (which represents
|
||||
the configuration node being created, modified, deleted or moved). When
|
||||
a type mismatch is detected, the program aborts and displays a backtrace
|
||||
showing where the problem happened. It would be desirable to detect such
|
||||
type mismatches at compile-time, the earlier the problems are detected
|
||||
the sooner they are fixed.
|
||||
|
||||
One possible solution to this problem would be to auto-generate C
|
||||
structures from the YANG models and provide a function that converts a
|
||||
libyang’s ``lyd_node`` variable to a C structure containing the same
|
||||
information. The northbound callbacks could then fetch configuration
|
||||
data from this C structure, which would naturally lead to type
|
||||
mismatches being detected at compile time. One of the challenges of
|
||||
doing this would be the handling of YANG lists and leaf-lists. It would
|
||||
be necessary to use dynamic data structures like hashes or rb-trees to
|
||||
hold all elements of the lists and leaf-lists, and the process of
|
||||
converting a ``lyd_node`` to an auto-generated C-structure could be
|
||||
expensive. At this point it’s unclear if it’s worth adding more
|
||||
complexity in the northbound architecture to solve this specific
|
||||
problem.
|
||||
283
doc/developer/northbound/architecture.rst
Normal file
283
doc/developer/northbound/architecture.rst
Normal file
@ -0,0 +1,283 @@
|
||||
Introduction
|
||||
------------
|
||||
|
||||
The goal of the new northbound API is to provide a better interface to
|
||||
configure and monitor FRR programatically. The current design based on
|
||||
CLI commands is no longer adequate in a world where computer networks
|
||||
are becoming increasingly bigger, more diverse and more complex. Network
|
||||
scripting using *expect* and screen scraping techniques is too primitive
|
||||
and unreliable to be used in large-scale networks. What is proposed is
|
||||
to modernize FRR to turn it into an API-first routing stack, and
|
||||
reposition the CLI on top of this API. The most important change,
|
||||
however, is not the API that will be provided to external users. In
|
||||
fact, multiple APIs will be supported and users will have the ability to
|
||||
write custom management APIs if necessary. The biggest change is the
|
||||
introduction of a model-driven management architecture based on the
|
||||
`YANG <https://tools.ietf.org/html/rfc7950>`__ modeling language.
|
||||
Instead of writing code tied to any particular user interface
|
||||
(e.g. DEFUNs), YANG allows us to write API-agnostic code (in the form of
|
||||
callbacks) that can be used by any management interface. As an example,
|
||||
it shouldn’t matter if a set of configuration changes is coming from a
|
||||
`NETCONF <https://tools.ietf.org/html/rfc6241>`__ session or from a CLI
|
||||
terminal, the same callbacks should be called to process the
|
||||
configuration changes regardless of where they came from. This
|
||||
model-driven design ensures feature parity across all management
|
||||
interfaces supported by FRR.
|
||||
|
||||
Quoting RFC 7950: > YANG is a language originally designed to model data
|
||||
for the NETCONF protocol. A YANG module defines hierarchies of data that
|
||||
can be used for NETCONF-based operations, including configuration, state
|
||||
data, RPCs, and notifications. This allows a complete description of all
|
||||
data sent between a NETCONF client and server. Although out of scope for
|
||||
this specification, YANG can also be used with protocols other than
|
||||
NETCONF.
|
||||
|
||||
While the YANG and NETCONF specifications are tightly coupled with one
|
||||
another, both are independent to a certain extent and are evolving
|
||||
separately. Examples of other management protocols that use YANG include
|
||||
`RESTCONF <https://tools.ietf.org/html/rfc8040>`__,
|
||||
`gNMI <https://github.com/openconfig/reference/tree/master/rpc/gnmi>`__
|
||||
and
|
||||
`CoAP <https://www.ietf.org/archive/id/draft-vanderstok-core-comi-11.txt>`__.
|
||||
|
||||
In addition to being management-protocol independent, some other
|
||||
advantages of using YANG in FRR are listed below: \* Have a formal
|
||||
contract between FRR and application developers (management clients). A
|
||||
management client that has access to the FRR YANG models knows about all
|
||||
existing configuration options available for use. This information can
|
||||
be used to auto-generate user-friendly interfaces like Web-UIs, custom
|
||||
CLIs and even code bindings for several different programming languages.
|
||||
Using `PyangBind <https://github.com/robshakir/pyangbind>`__, for
|
||||
example, it’s possible to generate Python class hierarchies from YANG
|
||||
models and use these classes to instantiate objects that mirror the
|
||||
structure of the YANG modules and can be serialized/deserialized using
|
||||
different encoding formats. \* Support different encoding formats for
|
||||
instance data. Currently only JSON and XML are supported, but
|
||||
`GPB <https://developers.google.com/protocol-buffers/>`__ and
|
||||
`CBOR <http://cbor.io/>`__ are other viable options in the long term.
|
||||
Additional encoding formats can be implemented in the *libyang* library
|
||||
for optimal performance, or externally by translating data to/from one
|
||||
of the supported formats (with a performance penalty). \* Have a formal
|
||||
mechanism to introduce backward-incompatible changes based on `semantic
|
||||
versioning <http://www.openconfig.net/docs/semver/>`__ (not part of the
|
||||
YANG standard, which allows backward-compatible module updates only). \*
|
||||
Provide seamless support to the industry-standard NETCONF/RESTCONF
|
||||
protocols as alternative management APIs. If FRR configuration/state
|
||||
data is modeled using YANG, supporting YANG-based protocols like NETCONF
|
||||
and RESTCONF is much easier.
|
||||
|
||||
As important as shifting to a model-driven management paradigm, the new
|
||||
northbound architecture also introduces the concept of configuration
|
||||
transactions. Configuration transactions allow management clients to
|
||||
commit multiple configuration changes at the same time and rest assured
|
||||
that either all changes will be applied or none will (all-or-nothing).
|
||||
Configuration transactions are implemented as pseudo-atomic operations
|
||||
and facilitate automation by removing the burden of error recovery from
|
||||
the management side. Another property of configuration transactions is
|
||||
that the configuration changes are always processed in a pre-defined
|
||||
order to ensure consistency. Configuration transactions that encompass
|
||||
multiple network devices are called network-wide transactions and are
|
||||
also supported by the new northbound architecture. When FRR is built
|
||||
using the ``--enable-config-rollbacks`` option, all committed
|
||||
transactions are recorded in the FRR rollback log, which can reside
|
||||
either in memory (volatile) or on persistent storage.
|
||||
|
||||
Network-wide Transactions is the most important leap in network
|
||||
management technology since SNMP. The error recovery and sequencing
|
||||
tasks are removed from the manager side. This is usually more than
|
||||
half the cost in a mature system; more than the entire cost of the
|
||||
managed devices.
|
||||
`[source] <https://www.nanog.org/sites/default/files/tuesday_tutorial_moberg_netconf_35.pdf>`__.
|
||||
|
||||
Figures 1 and 2 below illustrate the old and new northbound architecture
|
||||
of FRR, respectively. As it can be seen, in the old architecture the CLI
|
||||
was the only interface used to configure and monitor FRR (the SNMP
|
||||
plugin was’t taken into account given the small number of implemented
|
||||
MIBs). This means that the only way to automate FRR was by writing
|
||||
scripts that send CLI commands and parse the text output (which usually
|
||||
doesn’t have any structure) using screen scraping and regular
|
||||
expressions.
|
||||
|
||||
+-----------------------------------------+
|
||||
| |space-1.jpg| |
|
||||
+=========================================+
|
||||
| *Figure 1: old northbound architecture* |
|
||||
+-----------------------------------------+
|
||||
|
||||
The new northbound architectures, on the other hand, features a
|
||||
multitude of different management APIs, all of them connected to the
|
||||
northbound layer of the FRR daemons. By default, only the CLI interface
|
||||
is compiled built-in in the FRR daemons. The other management interfaces
|
||||
are provided as optional plugins and need to be loaded during the daemon
|
||||
initialization (e.g. *zebra -M confd*). This design makes it possible to
|
||||
integrate FRR with different NETCONF solutions without introducing
|
||||
vendor lock-in. The [[Plugins - Writing Your Own]] page explains how to
|
||||
write custom northbound plugins that can be tailored to all needs
|
||||
(e.g. support custom transport protocols, different data encoding
|
||||
formats, fine-grained access control, etc).
|
||||
|
||||
+-----------------------------------------+
|
||||
| |space-1.jpg| |
|
||||
+=========================================+
|
||||
| *Figure 2: new northbound architecture* |
|
||||
+-----------------------------------------+
|
||||
|
||||
Figure 3 shows the internal view of the FRR northbound architecture. In
|
||||
this image we can see that northbound layer is an abstract entity
|
||||
positioned between the northbound callbacks and the northbound clients.
|
||||
The northbound layer is responsible to process the requests coming from
|
||||
the northbound clients and call the appropriate callbacks to satisfy
|
||||
these requests. The northbound plugins communicate with the northbound
|
||||
layer through a public API, which allow users to write third-party
|
||||
plugins that can be maintained separately. The northbound plugins, in
|
||||
turn, have their own APIs to communicate with external management
|
||||
clients.
|
||||
|
||||
+---------------------------------------------------------+
|
||||
| |space-1.jpg| |
|
||||
+=========================================================+
|
||||
| *Figure 3: new northbound architecture - internal view* |
|
||||
+---------------------------------------------------------+
|
||||
|
||||
Initially the CLI (and all of its commands) will be maintained inside
|
||||
the FRR daemons. In the long term, however, the goal is to move the CLI
|
||||
to a separate program just like any other management client. The
|
||||
[[Advanced Topics]] page describes the motivations and challenges of
|
||||
doing that. Last but not least, the *libyang* block inside the
|
||||
northbound layer is the engine that makes everything possible. The
|
||||
*libyang* library will be described in more detail in the following
|
||||
sections.
|
||||
|
||||
YANG models
|
||||
-----------
|
||||
|
||||
The main decision to be made when using YANG is which models to
|
||||
implement. There’s a general consensus that using standard models is
|
||||
preferable over using custom (native) models. The reasoning is that
|
||||
applications based on standard models can be reused for all network
|
||||
appliances that support those models, whereas the same doesn’t apply for
|
||||
applications written based on custom models.
|
||||
|
||||
That said, there are multiple standards bodies publishing YANG models
|
||||
and unfortunately not all of them are converging (or at least not yet).
|
||||
In the context of FRR, which is a routing stack, the two sets of YANG
|
||||
models that would make sense to implement are the ones from IETF and
|
||||
from the OpenConfig working group. The question that arises is: which
|
||||
one of them should we commit to? Or should we try to support both
|
||||
somehow, at the cost of extra development efforts?
|
||||
|
||||
Another problem, from an implementation point of view, is that it’s
|
||||
challenging to adapt the existing code base to match standard models. A
|
||||
more reasonable solution, at least in a first moment, would be to use
|
||||
YANG deviations and augmentations to do the opposite: adapt the standard
|
||||
models to the existing code. In practice however this is not as simple
|
||||
as it seems. There are cases where the differences are too substantial
|
||||
to be worked around without restructuring the code by changing its data
|
||||
structures and their relationships. As an example, the *ietf-rip* model
|
||||
places per-interface RIP configuration parameters inside the
|
||||
*control-plane-protocol* list (which is augmented by *ietf-rip*). This
|
||||
means that it’s impossible to configure RIP interface parameters without
|
||||
first configuring a RIP routing instance. The *ripd* daemon on the other
|
||||
hand allows the operator to configure RIP interface parameters even if
|
||||
``router rip`` is not configured. If we were to implement the *ietf-rip*
|
||||
module natively, we’d need to change ripd’s CLI commands (and the
|
||||
associated code) to reflect the new configuration hierarchy.
|
||||
|
||||
Taking into account that FRR has a huge code base and that the
|
||||
northbound retrofitting process per-se will cause a lot of impact, it
|
||||
was decided to take a conservative approach and write custom YANG models
|
||||
for FRR modeled after the existing CLI commands. Having YANG models that
|
||||
closely mirror the CLI commands will allow the FRR developers to
|
||||
retrofit the code base much more easily, without introducing
|
||||
backward-incompatible changes in the CLI and reducing the likelihood of
|
||||
introducing bugs. The [[Retrofitting Configuration Commands]] page
|
||||
explains in detail how to convert configuration commands to the new
|
||||
northbound model.
|
||||
|
||||
Even though having native YANG models is not the ideal solution, it will
|
||||
be already a big step forward for FRR to migrate to a model-driven
|
||||
management architecture, with support for configuration transactions and
|
||||
multiple management interfaces, including NETCONF and RESTCONF (through
|
||||
the northbound plugins).
|
||||
|
||||
The new northbound also features an experimental YANG module translator
|
||||
that will allow users to translate to and from standard YANG models by
|
||||
using translation tables. The [[YANG module translator]] page describes
|
||||
this mechanism in more detail. At this point it’s unclear what can be
|
||||
achieved through module translation and if that can be considered as a
|
||||
definitive solution to support standard models or not.
|
||||
|
||||
Northbound Architecture
|
||||
-----------------------
|
||||
|
||||
+-----------------------------------------------+
|
||||
| |space-1.jpg| |
|
||||
+===============================================+
|
||||
| *Figure 4: libyang’s lys_node data structure* |
|
||||
+-----------------------------------------------+
|
||||
|
||||
+-----------------------------------------------+
|
||||
| |space-1.jpg| |
|
||||
+===============================================+
|
||||
| *Figure 5: libyang’s lyd_node data structure* |
|
||||
+-----------------------------------------------+
|
||||
|
||||
+---------------------------------------------+
|
||||
| |space-1.jpg| |
|
||||
+=============================================+
|
||||
| *Figure 6: libyang’s ly_ctx data structure* |
|
||||
+---------------------------------------------+
|
||||
|
||||
+----------------------------------------+
|
||||
| |space-1.jpg| |
|
||||
+========================================+
|
||||
| *Figure 7: configuration transactions* |
|
||||
+----------------------------------------+
|
||||
|
||||
Testing
|
||||
-------
|
||||
|
||||
The new northbound adds the libyang library as a new mandatory
|
||||
dependency for FRR. To obtain and install this library, follow the steps
|
||||
below:
|
||||
|
||||
::
|
||||
|
||||
$ git clone https://github.com/CESNET/libyang
|
||||
$ cd libyang
|
||||
$ git checkout devel
|
||||
$ mkdir build ; cd build
|
||||
$ cmake -DENABLE_LYD_PRIV=ON ..
|
||||
$ make
|
||||
$ sudo make install
|
||||
|
||||
..
|
||||
|
||||
NOTE: first make sure to install the libyang
|
||||
`requirements <https://github.com/CESNET/libyang#build-requirements>`__.
|
||||
|
||||
FRR needs libyang from version 0.16.7 or newer, which is maintained in
|
||||
the ``devel`` branch. libyang 0.15.x is maintained in the ``master``
|
||||
branch and doesn’t contain one small feature used by FRR (the
|
||||
``LY_CTX_DISABLE_SEARCHDIR_CWD`` flag). FRR also makes use of the
|
||||
libyang’s ``ENABLE_LYD_PRIV`` feature, which is disabled by default and
|
||||
needs to be enabled at compile time.
|
||||
|
||||
It’s advisable (but not required) to install sqlite3 and build FRR with
|
||||
``--enable-config-rollbacks`` in order to have access to the
|
||||
configuration rollback feature.
|
||||
|
||||
To test the northbound, the suggested method is to use the
|
||||
[[Transactional CLI]] with the *ripd* daemon and play with the new
|
||||
commands. The ``debug northbound`` command can be used to see which
|
||||
northbound callbacks are called in response to the ``commit`` command.
|
||||
For reference, the [[Demos]] page shows a small demonstration of the
|
||||
transactional CLI in action and what it’s capable of.
|
||||
|
||||
.. |space-1.jpg| image:: https://s22.postimg.cc/se52j8awh/arch-before.png
|
||||
.. |space-1.jpg| image:: https://s22.postimg.cc/fziaiwboh/arch-after.png
|
||||
.. |space-1.jpg| image:: https://s22.postimg.cc/qmc3ocmep/nb-layer.png
|
||||
.. |space-1.jpg| image:: https://s22.postimg.cc/z4ljsodht/lys_node.png
|
||||
.. |space-1.jpg| image:: https://s22.postimg.cc/6eynw1h7l/lyd_node.png
|
||||
.. |space-1.jpg| image:: https://s22.postimg.cc/5cohdhiyp/ly_ctx.png
|
||||
.. |space-1.jpg| image:: https://s22.postimg.cc/8waf3bgjl/transactions.png
|
||||
25
doc/developer/northbound/demos.rst
Normal file
25
doc/developer/northbound/demos.rst
Normal file
@ -0,0 +1,25 @@
|
||||
Transactional CLI
|
||||
-----------------
|
||||
|
||||
This short demo shows some of the capabilities of the new transactional
|
||||
CLI: |asciicast|
|
||||
|
||||
ConfD + NETCONF + Cisco YDK
|
||||
---------------------------
|
||||
|
||||
This is a very simple demo of *ripd* being configured by a python
|
||||
script. The script uses NETCONF to communicate with *ripd*, which has
|
||||
the ConfD plugin loaded. The most interesting part, however, is the fact
|
||||
that the python script is not using handcrafted XML payloads to
|
||||
configure *ripd*. Instead, the script is using python bindings generated
|
||||
using Cisco’s YANG Development Kit (YDK).
|
||||
|
||||
- Script used in the demo:
|
||||
https://gist.github.com/rwestphal/defa9bd1ccf216ab082d4711ae402f95
|
||||
|
||||
|asciicast|
|
||||
|
||||
.. |asciicast| image:: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1.png
|
||||
:target: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1
|
||||
.. |asciicast| image:: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv.png
|
||||
:target: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv
|
||||
233
doc/developer/northbound/links.rst
Normal file
233
doc/developer/northbound/links.rst
Normal file
@ -0,0 +1,233 @@
|
||||
RFCs
|
||||
~~~~
|
||||
|
||||
- `RFC 7950 - The YANG 1.1 Data Modeling
|
||||
Language <https://tools.ietf.org/html/rfc7950>`__
|
||||
- `RFC 7951 - JSON Encoding of Data Modeled with
|
||||
YANG <https://tools.ietf.org/html/rfc7951>`__
|
||||
- `RFC 8342 - Network Management Datastore Architecture
|
||||
(NMDA) <https://tools.ietf.org/html/rfc8342>`__
|
||||
- `RFC 6087 - Guidelines for Authors and Reviewers of YANG Data Model
|
||||
Documents <https://tools.ietf.org/html/rfc6087>`__
|
||||
- `RFC 8340 - YANG Tree
|
||||
Diagrams <https://tools.ietf.org/html/rfc8340>`__
|
||||
- `RFC 6991 - Common YANG Data
|
||||
Types <https://tools.ietf.org/html/rfc6991>`__
|
||||
- `RFC 6241 - Network Configuration Protocol
|
||||
(NETCONF) <https://tools.ietf.org/html/rfc6241>`__
|
||||
- `RFC 8040 - RESTCONF
|
||||
Protocol <https://tools.ietf.org/html/rfc8040>`__
|
||||
|
||||
YANG models
|
||||
~~~~~~~~~~~
|
||||
|
||||
- Collection of several YANG models, including models from standards
|
||||
organizations such as the IETF and vendor specific models:
|
||||
https://github.com/YangModels/yang
|
||||
- OpenConfig: https://github.com/openconfig/public
|
||||
|
||||
Presentations
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
- FRR Advanced Northbound API (May 2018)
|
||||
|
||||
- Slides:
|
||||
https://www.dropbox.com/s/zhybthruwocbqaw/netdef-frr-northbound.pdf?dl=1
|
||||
|
||||
- Ok, We Got Data Models, Now What?
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=2oqkiZ83vAA
|
||||
- Slides:
|
||||
https://www.nanog.org/sites/default/files/20161017_Alvarez_Ok_We_Got_v1.pdf
|
||||
|
||||
- Data Model-Driven Management: Latest Industry and Tool Developments
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=n_oKGJ_jgYQ
|
||||
- Slides:
|
||||
https://pc.nanog.org/static/published/meetings/NANOG72/1559/20180219_Claise_Data_Modeling-Driven_Management__v1.pdf
|
||||
|
||||
- Network Automation And Programmability: Reality Versus The Vendor
|
||||
Hype When Considering Legacy And NFV Networks
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=N5wbYncUS9o
|
||||
- Slides:
|
||||
https://www.nanog.org/sites/default/files/1_Moore_Network_Automation_And_Programmability.pdf
|
||||
|
||||
- Lightning Talk: The API is the new CLI?
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=ngi0erGNi58
|
||||
- Slides:
|
||||
https://pc.nanog.org/static/published/meetings/NANOG72/1638/20180221_Grundemann_Lightning_Talk_The_v1.pdf
|
||||
|
||||
- Lightning Talk: OpenConfig - progress toward vendor-neutral network
|
||||
management
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=10rSUbeMmT4
|
||||
- Slides:
|
||||
https://pc.nanog.org/static/published/meetings/NANOG71/1535/20171004_Shaikh_Lightning_Talk_Openconfig_v1.pdf
|
||||
|
||||
- Getting started with OpenConfig
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=L7trUNK8NJI
|
||||
- Slides:
|
||||
https://pc.nanog.org/static/published/meetings/NANOG71/1456/20171003_Alvarez_Getting_Started_With_v1.pdf
|
||||
|
||||
- Why NETCONF and YANG
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=mp4h8aSTba8
|
||||
|
||||
- NETCONF and YANG Concepts
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=UwYYvT7DBvg
|
||||
|
||||
- NETCONF Tutorial
|
||||
|
||||
- Video: https://www.youtube.com/watch?v=N4vov1mI14U
|
||||
|
||||
Whitepapers
|
||||
~~~~~~~~~~~
|
||||
|
||||
- Automating Network and Service Configuration Using NETCONF and YANG:
|
||||
http://www.tail-f.com/wordpress/wp-content/uploads/2013/02/Tail-f-Presentation-Netconf-Yang.pdf
|
||||
- Creating the Programmable Network: The Business Case for NETCONF/YANG
|
||||
in Network Devices:
|
||||
http://www.tail-f.com/wordpress/wp-content/uploads/2013/10/HR-Tail-f-NETCONF-WP-10-08-13.pdf
|
||||
- NETCONF/YANG: What’s Holding Back Adoption & How to Accelerate It:
|
||||
https://www.oneaccess-net.com/images/public/wp_heavy_reading.pdf
|
||||
- Achieving Automation with YANG Modeling Technologies:
|
||||
https://www.cisco.com/c/dam/en/us/products/collateral/cloud-systems-management/network-services-orchestrator/idc-achieving-automation-wp.pdf
|
||||
|
||||
Blog posts and podcasts
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
- OpenConfig and IETF YANG Models: Can they converge? -
|
||||
http://rob.sh/post/215/
|
||||
- OpenConfig: Standardized Models For Networking -
|
||||
https://packetpushers.net/openconfig-standardized-models-networking/
|
||||
- (Podcast) OpenConfig: From Basics to Implementations -
|
||||
https://blog.ipspace.net/2017/02/openconfig-from-basics-to.html
|
||||
- (Podcast) How Did NETCONF Start on Software Gone Wild -
|
||||
https://blog.ipspace.net/2017/12/how-did-netconf-start-on-software-gone.html
|
||||
- YANG Data Models in the Industry: Current State of Affairs (March
|
||||
2018) -
|
||||
https://www.claise.be/2018/03/yang-data-models-in-the-industry-current-state-of-affairs-march-2018/
|
||||
- Why Data Model-driven Telemetry is the only useful Telemetry? -
|
||||
https://www.claise.be/2018/02/why-data-model-driven-telemetry-is-the-only-useful-telemetry/
|
||||
- NETCONF versus RESTCONF: Capabilitity Comparisons for Data
|
||||
Model-driven Management -
|
||||
https://www.claise.be/2017/10/netconf-versus-restconf-capabilitity-comparisons-for-data-model-driven-management-2/
|
||||
- An Introduction to NETCONF/YANG -
|
||||
https://www.fir3net.com/Networking/Protocols/an-introduction-to-netconf-yang.html
|
||||
- Network Automation and the Rise of NETCONF -
|
||||
https://medium.com/@k.okasha/network-automation-and-the-rise-of-netconf-e96cc33fe28
|
||||
- YANG and the Road to a Model Driven Network -
|
||||
https://medium.com/@k.okasha/yang-and-road-to-a-model-driven-network-e9e52d47148d
|
||||
|
||||
Software
|
||||
~~~~~~~~
|
||||
|
||||
libyang
|
||||
^^^^^^^
|
||||
|
||||
libyang is a YANG data modelling language parser and toolkit written
|
||||
(and providing API) in C.
|
||||
|
||||
- GitHub page: https://github.com/CESNET/libyang
|
||||
- Documentaion: https://netopeer.liberouter.org/doc/libyang/master/
|
||||
|
||||
pyang
|
||||
^^^^^
|
||||
|
||||
pyang is a YANG validator, transformator and code generator, written
|
||||
in python. It can be used to validate YANG modules for correctness,
|
||||
to transform YANG modules into other formats, and to generate code
|
||||
from the modules.
|
||||
|
||||
- GitHub page: https://github.com/mbj4668/pyang
|
||||
- Documentaion: https://github.com/mbj4668/pyang/wiki/Documentation
|
||||
|
||||
ncclient
|
||||
^^^^^^^^
|
||||
|
||||
ncclient is a Python library that facilitates client-side scripting
|
||||
and application development around the NETCONF protocol.
|
||||
|
||||
- GitHub page: https://github.com/ncclient/ncclient
|
||||
- Documentaion: https://ncclient.readthedocs.io/en/latest/
|
||||
|
||||
YDK
|
||||
^^^
|
||||
|
||||
ydk-gen is a developer tool that can generate API’s that are modeled
|
||||
in YANG. Currently, it generates language binding for Python, Go and
|
||||
C++ with planned support for other language bindings in the future.
|
||||
|
||||
- GitHub pages:
|
||||
|
||||
- Generator: https://github.com/CiscoDevNet/ydk-gen
|
||||
- Python: https://github.com/CiscoDevNet/ydk-py
|
||||
|
||||
- Python samples: https://github.com/CiscoDevNet/ydk-py-samples
|
||||
|
||||
- Go: https://github.com/CiscoDevNet/ydk-go
|
||||
- C++: https://github.com/CiscoDevNet/ydk-cpp
|
||||
|
||||
- Documentation:
|
||||
|
||||
- Python: http://ydk.cisco.com/py/docs/
|
||||
- Go: http://ydk.cisco.com/go/docs/
|
||||
- C++: http://ydk.cisco.com/cpp/docs/
|
||||
|
||||
- (Blog post) Simplifying Network Programmability with Model-Driven
|
||||
APIs:
|
||||
https://blogs.cisco.com/sp/simplifying-network-programmability-with-model-driven-apis
|
||||
- (Video introduction) Infrastructure as a Code Using YANG, OpenConfig
|
||||
and YDK: https://www.youtube.com/watch?v=G1b6vJW1R5w
|
||||
|
||||
pyangbind
|
||||
^^^^^^^^^
|
||||
|
||||
A plugin for pyang that creates Python bindings for a YANG model.
|
||||
|
||||
- GitHub page: https://github.com/robshakir/pyangbind
|
||||
- Documentation: http://pynms.io/pyangbind/
|
||||
|
||||
ConfD
|
||||
^^^^^
|
||||
|
||||
- Official webpage (for ConfD Basic):
|
||||
http://www.tail-f.com/confd-basic/
|
||||
- Training Videos: http://www.tail-f.com/confd-training-videos/
|
||||
- Forum: http://discuss.tail-f.com/
|
||||
|
||||
Sysrepo
|
||||
^^^^^^^
|
||||
|
||||
Sysrepo is an YANG-based configuration and operational state data
|
||||
store for Unix/Linux applications.
|
||||
|
||||
- GitHub page: https://github.com/sysrepo/sysrepo
|
||||
- Official webpage: http://www.sysrepo.org/
|
||||
- Documentation: http://www.sysrepo.org/static/doc/html/
|
||||
|
||||
Netopeer2
|
||||
^^^^^^^^^
|
||||
|
||||
Netopeer2 is a set of tools implementing network configuration tools
|
||||
based on the NETCONF Protocol. This is the second generation of the
|
||||
toolset, originally available as the Netopeer project. Netopeer2 is
|
||||
based on the new generation of the NETCONF and YANG libraries -
|
||||
libyang and libnetconf2. The Netopeer server uses sysrepo as a
|
||||
NETCONF datastore implementation.
|
||||
|
||||
- GitHub page: https://github.com/CESNET/Netopeer2
|
||||
|
||||
Clixon
|
||||
^^^^^^
|
||||
|
||||
Clixon is an automatic configuration manager where you generate
|
||||
interactive CLI, NETCONF, RESTCONF and embedded databases with
|
||||
transaction support from a YANG specification.
|
||||
|
||||
- GitHub page: https://github.com/clicon/clixon
|
||||
- Project page: http://www.clicon.org/
|
||||
21
doc/developer/northbound/northbound.rst
Normal file
21
doc/developer/northbound/northbound.rst
Normal file
@ -0,0 +1,21 @@
|
||||
.. _northbound:
|
||||
|
||||
**************
|
||||
Northbound API
|
||||
**************
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
advanced-topics
|
||||
architecture
|
||||
demos
|
||||
links
|
||||
operational-data-rpcs-and-notifications
|
||||
plugins-sysrepo
|
||||
ppr-basic-test-topology
|
||||
ppr-mpls-basic-test-topology
|
||||
retrofitting-configuration-commands
|
||||
transactional-cli
|
||||
yang-module-translator
|
||||
yang-tools
|
||||
@ -0,0 +1,565 @@
|
||||
Operational data
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
Writing API-agnostic code for YANG-modeled operational data is
|
||||
challenging. ConfD and Sysrepo, for instance, have completely different
|
||||
APIs to fetch operational data. So how can we write API-agnostic
|
||||
callbacks that can be used by both the ConfD and Sysrepo plugins, and
|
||||
any other northbound client that might be written in the future?
|
||||
|
||||
As an additional requirement, the callbacks must be designed in a way
|
||||
that makes in-place XPath filtering possible. As an example, a
|
||||
management client might want to retrieve only a subset of a large YANG
|
||||
list (e.g. a BGP table), and for optimal performance it should be
|
||||
possible to filter out the unwanted elements locally in the managed
|
||||
devices instead of returning all elements and performing the filtering
|
||||
on the management application.
|
||||
|
||||
To meet all these requirements, the four callbacks below were introduced
|
||||
in the northbound architecture:
|
||||
|
||||
.. code:: c
|
||||
|
||||
/*
|
||||
* Operational data callback.
|
||||
*
|
||||
* The callback function should return the value of a specific leaf or
|
||||
* inform if a typeless value (presence containers or leafs of type
|
||||
* empty) exists or not.
|
||||
*
|
||||
* xpath
|
||||
* YANG data path of the data we want to get
|
||||
*
|
||||
* list_entry
|
||||
* pointer to list entry
|
||||
*
|
||||
* Returns:
|
||||
* pointer to newly created yang_data structure, or NULL to indicate
|
||||
* the absence of data
|
||||
*/
|
||||
struct yang_data *(*get_elem)(const char *xpath, void *list_entry);
|
||||
|
||||
/*
|
||||
* Operational data callback for YANG lists.
|
||||
*
|
||||
* The callback function should return the next entry in the list. The
|
||||
* 'list_entry' parameter will be NULL on the first invocation.
|
||||
*
|
||||
* list_entry
|
||||
* pointer to a list entry
|
||||
*
|
||||
* Returns:
|
||||
* pointer to the next entry in the list, or NULL to signal that the
|
||||
* end of the list was reached
|
||||
*/
|
||||
void *(*get_next)(void *list_entry);
|
||||
|
||||
/*
|
||||
* Operational data callback for YANG lists.
|
||||
*
|
||||
* The callback function should fill the 'keys' parameter based on the
|
||||
* given list_entry.
|
||||
*
|
||||
* list_entry
|
||||
* pointer to a list entry
|
||||
*
|
||||
* keys
|
||||
* structure to be filled based on the attributes of the provided
|
||||
* list entry
|
||||
*
|
||||
* Returns:
|
||||
* NB_OK on success, NB_ERR otherwise
|
||||
*/
|
||||
int (*get_keys)(void *list_entry, struct yang_list_keys *keys);
|
||||
|
||||
/*
|
||||
* Operational data callback for YANG lists.
|
||||
*
|
||||
* The callback function should return a list entry based on the list
|
||||
* keys given as a parameter.
|
||||
*
|
||||
* keys
|
||||
* structure containing the keys of the list entry
|
||||
*
|
||||
* Returns:
|
||||
* a pointer to the list entry if found, or NULL if not found
|
||||
*/
|
||||
void *(*lookup_entry)(struct yang_list_keys *keys);
|
||||
|
||||
These callbacks were designed to provide maximum flexibility, and borrow
|
||||
a lot of ideas from the ConfD API. Each callback does one and only one
|
||||
task, they are indivisible primitives that can be combined in several
|
||||
different ways to iterate over operational data. The extra flexibility
|
||||
certainly has a performance cost, but it’s the price to pay if we want
|
||||
to expose FRR operational data using several different management
|
||||
interfaces (e.g. NETCONF via either ConfD or Sysrepo+Netopeer2). In the
|
||||
future it might be possible to introduce optional callbacks that do
|
||||
things like returning multiple objects at once. They would provide
|
||||
enhanced performance when iterating over large lists, but their use
|
||||
would be limited by the northbound plugins that can be integrated with
|
||||
them.
|
||||
|
||||
NOTE: using the northbound callbacks as a base, the ConfD plugin can
|
||||
provide up to 100 objects between each round trip between FRR and the
|
||||
*confd* daemon. Preliminary tests showed FRR taking ~7 seconds
|
||||
(asynchronously, without blocking the main pthread) to return a RIP
|
||||
table containing 100k routes to a NETCONF client connected to *confd*
|
||||
(JSON was used as the encoding format). Work needs to be done to find
|
||||
the bottlenecks and optimize this operation.
|
||||
|
||||
The [[Plugins - Writing Your Own]] page explains how the northbound
|
||||
plugins can fetch operational data using the aforementioned northbound
|
||||
callbacks, and how in-place XPath filtering can be implemented.
|
||||
|
||||
Example
|
||||
^^^^^^^
|
||||
|
||||
Now let’s move to an example to show how these callbacks are implemented
|
||||
in practice. The following YANG container is part of the *ietf-rip*
|
||||
module and contains operational data about RIP neighbors:
|
||||
|
||||
.. code:: yang
|
||||
|
||||
container neighbors {
|
||||
description
|
||||
"Neighbor information.";
|
||||
list neighbor {
|
||||
key "address";
|
||||
description
|
||||
"A RIP neighbor.";
|
||||
leaf address {
|
||||
type inet:ipv4-address;
|
||||
description
|
||||
"IP address that a RIP neighbor is using as its
|
||||
source address.";
|
||||
}
|
||||
leaf last-update {
|
||||
type yang:date-and-time;
|
||||
description
|
||||
"The time when the most recent RIP update was
|
||||
received from this neighbor.";
|
||||
}
|
||||
leaf bad-packets-rcvd {
|
||||
type yang:counter32;
|
||||
description
|
||||
"The number of RIP invalid packets received from
|
||||
this neighbor which were subsequently discarded
|
||||
for any reason (e.g. a version 0 packet, or an
|
||||
unknown command type).";
|
||||
}
|
||||
leaf bad-routes-rcvd {
|
||||
type yang:counter32;
|
||||
description
|
||||
"The number of routes received from this neighbor,
|
||||
in valid RIP packets, which were ignored for any
|
||||
reason (e.g. unknown address family, or invalid
|
||||
metric).";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
We know that this is operational data because the ``neighbors``
|
||||
container is within the ``state`` container, which has the
|
||||
``config false;`` property (which is applied recursively).
|
||||
|
||||
As expected, the ``gen_northbound_callbacks`` tool also generates
|
||||
skeleton callbacks for nodes that represent operational data:
|
||||
|
||||
.. code:: c
|
||||
|
||||
{
|
||||
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor",
|
||||
.cbs.get_next = ripd_state_neighbors_neighbor_get_next,
|
||||
.cbs.get_keys = ripd_state_neighbors_neighbor_get_keys,
|
||||
.cbs.lookup_entry = ripd_state_neighbors_neighbor_lookup_entry,
|
||||
},
|
||||
{
|
||||
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/address",
|
||||
.cbs.get_elem = ripd_state_neighbors_neighbor_address_get_elem,
|
||||
},
|
||||
{
|
||||
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/last-update",
|
||||
.cbs.get_elem = ripd_state_neighbors_neighbor_last_update_get_elem,
|
||||
},
|
||||
{
|
||||
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd",
|
||||
.cbs.get_elem = ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem,
|
||||
},
|
||||
{
|
||||
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd",
|
||||
.cbs.get_elem = ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem,
|
||||
},
|
||||
|
||||
The ``/frr-ripd:ripd/state/neighbors/neighbor`` list within the
|
||||
``neighbors`` container has three different callbacks that need to be
|
||||
implemented. Let’s start with the first one, the ``get_next`` callback:
|
||||
|
||||
.. code:: c
|
||||
|
||||
static void *ripd_state_neighbors_neighbor_get_next(void *list_entry)
|
||||
{
|
||||
struct listnode *node;
|
||||
|
||||
if (list_entry == NULL)
|
||||
node = listhead(peer_list);
|
||||
else
|
||||
node = listnextnode((struct listnode *)list_entry);
|
||||
|
||||
return node;
|
||||
}
|
||||
|
||||
Given a list entry, the job of this callback is to find the next element
|
||||
from the list. When the ``list_entry`` parameter is NULL, then the first
|
||||
element of the list should be returned.
|
||||
|
||||
*ripd* uses the ``rip_peer`` structure to represent RIP neighbors, and
|
||||
the ``peer_list`` global variable (linked list) is used to store all RIP
|
||||
neighbors.
|
||||
|
||||
In order to be able to iterate over the list of RIP neighbors, the
|
||||
callback returns a ``listnode`` variable instead of a ``rip_peer``
|
||||
variable. The ``listnextnode`` macro can then be used to find the next
|
||||
element from the linked list.
|
||||
|
||||
Now the second callback, ``get_keys``:
|
||||
|
||||
.. code:: c
|
||||
|
||||
static int ripd_state_neighbors_neighbor_get_keys(void *list_entry,
|
||||
struct yang_list_keys *keys)
|
||||
{
|
||||
struct listnode *node = list_entry;
|
||||
struct rip_peer *peer = listgetdata(node);
|
||||
|
||||
keys->num = 1;
|
||||
(void)inet_ntop(AF_INET, &peer->addr, keys->key[0].value,
|
||||
sizeof(keys->key[0].value));
|
||||
|
||||
return NB_OK;
|
||||
}
|
||||
|
||||
This one is easy. First, we obtain the RIP neighbor from the
|
||||
``listnode`` structure. Then, we fill the ``keys`` parameter according
|
||||
to the attributes of the RIP neighbor. In this case, the ``neighbor``
|
||||
YANG list has only one key: the neighbor IP address. We then use the
|
||||
``inet_ntop()`` function to transform this binary IP address into a
|
||||
string (the lingua franca of the FRR northbound).
|
||||
|
||||
The last callback for the ``neighbor`` YANG list is the ``lookup_entry``
|
||||
callback:
|
||||
|
||||
.. code:: c
|
||||
|
||||
static void *
|
||||
ripd_state_neighbors_neighbor_lookup_entry(struct yang_list_keys *keys)
|
||||
{
|
||||
struct in_addr address;
|
||||
|
||||
yang_str2ipv4(keys->key[0].value, &address);
|
||||
|
||||
return rip_peer_lookup(&address);
|
||||
}
|
||||
|
||||
This callback is the counterpart of the ``get_keys`` callback: given an
|
||||
array of list keys, the associated list entry should be returned. The
|
||||
``yang_str2ipv4()`` function is used to convert the list key (an IP
|
||||
address) from a string to an ``in_addr`` structure. Then the
|
||||
``rip_peer_lookup()`` function is used to find the list entry.
|
||||
|
||||
Finally, each YANG leaf inside the ``neighbor`` list has its associated
|
||||
``get_elem`` callback:
|
||||
|
||||
.. code:: c
|
||||
|
||||
/*
|
||||
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/address
|
||||
*/
|
||||
static struct yang_data *
|
||||
ripd_state_neighbors_neighbor_address_get_elem(const char *xpath,
|
||||
void *list_entry)
|
||||
{
|
||||
struct rip_peer *peer = list_entry;
|
||||
|
||||
return yang_data_new_ipv4(xpath, &peer->addr);
|
||||
}
|
||||
|
||||
/*
|
||||
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/last-update
|
||||
*/
|
||||
static struct yang_data *
|
||||
ripd_state_neighbors_neighbor_last_update_get_elem(const char *xpath,
|
||||
void *list_entry)
|
||||
{
|
||||
/* TODO: yang:date-and-time is tricky */
|
||||
return NULL;
|
||||
}
|
||||
|
||||
/*
|
||||
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd
|
||||
*/
|
||||
static struct yang_data *
|
||||
ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem(const char *xpath,
|
||||
void *list_entry)
|
||||
{
|
||||
struct rip_peer *peer = list_entry;
|
||||
|
||||
return yang_data_new_uint32(xpath, peer->recv_badpackets);
|
||||
}
|
||||
|
||||
/*
|
||||
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd
|
||||
*/
|
||||
static struct yang_data *
|
||||
ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem(const char *xpath,
|
||||
void *list_entry)
|
||||
{
|
||||
struct rip_peer *peer = list_entry;
|
||||
|
||||
return yang_data_new_uint32(xpath, peer->recv_badroutes);
|
||||
}
|
||||
|
||||
These callbacks receive the list entry as parameter and return the
|
||||
corresponding data using the ``yang_data_new_*()`` wrapper functions.
|
||||
Not much to explain here.
|
||||
|
||||
Iterating over operational data without blocking the main pthread
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
One of the problems we have in FRR is that some “show” commands in the
|
||||
CLI can take too long, potentially long enough to the point of
|
||||
triggering some protocol timeouts and bringing sessions down.
|
||||
|
||||
To avoid this kind of problem, northbound clients are encouraged to do
|
||||
one of the following: \* Create a separate pthread for handling requests
|
||||
to fetch operational data. \* Iterate over YANG lists and leaf-lists
|
||||
asynchronously, returning a maximum number of elements per time instead
|
||||
of returning all elements in one shot.
|
||||
|
||||
In order to handle both cases correctly, the ``get_next`` callbacks need
|
||||
to use locks to prevent the YANG lists from being modified while they
|
||||
are being iterated over. If that is not done, the list entry returned by
|
||||
this callback can become a dangling pointer when used in another
|
||||
callback.
|
||||
|
||||
Currently the ConfD and Sysrepo plugins run only in the main pthread.
|
||||
The plan in the short-term is to introduce a separate pthread only for
|
||||
handling operational data, and use the main pthread only for handling
|
||||
configuration changes, RPCs and notifications.
|
||||
|
||||
RPCs and Actions
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
The FRR northbound supports YANG RPCs and Actions through the ``rpc()``
|
||||
callback, which is documented as follows in the *lib/northbound.h* file:
|
||||
|
||||
.. code:: c
|
||||
|
||||
/*
|
||||
* RPC and action callback.
|
||||
*
|
||||
* Both 'input' and 'output' are lists of 'yang_data' structures. The
|
||||
* callback should fetch all the input parameters from the 'input' list,
|
||||
* and add output parameters to the 'output' list if necessary.
|
||||
*
|
||||
* xpath
|
||||
* xpath of the YANG RPC or action
|
||||
*
|
||||
* input
|
||||
* read-only list of input parameters
|
||||
*
|
||||
* output
|
||||
* list of output parameters to be populated by the callback
|
||||
*
|
||||
* Returns:
|
||||
* NB_OK on success, NB_ERR otherwise
|
||||
*/
|
||||
int (*rpc)(const char *xpath, const struct list *input,
|
||||
struct list *output);
|
||||
|
||||
Note that the same callback is used for both RPCs and actions, which are
|
||||
essentially the same thing. In the case of YANG actions, the ``xpath``
|
||||
parameter can be consulted to find the data node associated to the
|
||||
operation.
|
||||
|
||||
As part of the northbound retrofitting process, it’s suggested to model
|
||||
some EXEC-level commands using YANG so that their functionality is
|
||||
exposed to other management interfaces other than the CLI. As an
|
||||
example, if the ``clear bgp`` command is modeled using a YANG RPC, and a
|
||||
corresponding ``rpc`` callback is written, then it should be possible to
|
||||
clear BGP neighbors using NETCONF and RESTCONF with that RPC (the ConfD
|
||||
and Sysrepo plugins have full support for YANG RPCs and actions).
|
||||
|
||||
Here’s an example of a very simple RPC modeled using YANG:
|
||||
|
||||
.. code:: yang
|
||||
|
||||
rpc clear-rip-route {
|
||||
description
|
||||
"Clears RIP routes from the IP routing table and routes
|
||||
redistributed into the RIP protocol.";
|
||||
}
|
||||
|
||||
This RPC doesn’t have any input or output parameters. Below we can see
|
||||
the implementation of the corresponding ``rpc`` callback, whose skeleton
|
||||
was automatically generated by the ``gen_northbound_callbacks`` tool:
|
||||
|
||||
.. code:: c
|
||||
|
||||
/*
|
||||
* XPath: /frr-ripd:clear-rip-route
|
||||
*/
|
||||
static int clear_rip_route_rpc(const char *xpath, const struct list *input,
|
||||
struct list *output)
|
||||
{
|
||||
struct route_node *rp;
|
||||
struct rip_info *rinfo;
|
||||
struct list *list;
|
||||
struct listnode *listnode;
|
||||
|
||||
/* Clear received RIP routes */
|
||||
for (rp = route_top(rip->table); rp; rp = route_next(rp)) {
|
||||
list = rp->info;
|
||||
if (list == NULL)
|
||||
continue;
|
||||
|
||||
for (ALL_LIST_ELEMENTS_RO(list, listnode, rinfo)) {
|
||||
if (!rip_route_rte(rinfo))
|
||||
continue;
|
||||
|
||||
if (CHECK_FLAG(rinfo->flags, RIP_RTF_FIB))
|
||||
rip_zebra_ipv4_delete(rp);
|
||||
break;
|
||||
}
|
||||
|
||||
if (rinfo) {
|
||||
RIP_TIMER_OFF(rinfo->t_timeout);
|
||||
RIP_TIMER_OFF(rinfo->t_garbage_collect);
|
||||
listnode_delete(list, rinfo);
|
||||
rip_info_free(rinfo);
|
||||
}
|
||||
|
||||
if (list_isempty(list)) {
|
||||
list_delete_and_null(&list);
|
||||
rp->info = NULL;
|
||||
route_unlock_node(rp);
|
||||
}
|
||||
}
|
||||
|
||||
return NB_OK;
|
||||
}
|
||||
|
||||
If the ``clear-rip-route`` RPC had any input parameters, they would be
|
||||
available in the ``input`` list given as a parameter to the callback.
|
||||
Similarly, the ``output`` list can be used to append output parameters
|
||||
generated by the RPC, if any are defined in the YANG model.
|
||||
|
||||
The northbound clients (CLI and northbound plugins) have the
|
||||
responsibility to create and delete the ``input`` and ``output`` lists.
|
||||
However, in the cases where the RPC or action doesn’t have any input or
|
||||
output parameters, the northbound client can pass NULL pointers to the
|
||||
``rpc`` callback to avoid creating linked lists unnecessarily. We can
|
||||
see this happening in the example below:
|
||||
|
||||
.. code:: c
|
||||
|
||||
/*
|
||||
* XPath: /frr-ripd:clear-rip-route
|
||||
*/
|
||||
DEFPY (clear_ip_rip,
|
||||
clear_ip_rip_cmd,
|
||||
"clear ip rip",
|
||||
CLEAR_STR
|
||||
IP_STR
|
||||
"Clear IP RIP database\n")
|
||||
{
|
||||
return nb_cli_rpc("/frr-ripd:clear-rip-route", NULL, NULL);
|
||||
}
|
||||
|
||||
``nb_cli_rpc()`` is a helper function that merely finds the appropriate
|
||||
``rpc`` callback based on the XPath provided in the first argument, and
|
||||
map the northbound error code from the ``rpc`` callback to a vty error
|
||||
code (e.g. ``CMD_SUCCESS``, ``CMD_WARNING``). The second and third
|
||||
arguments provided to the function refer to the ``input`` and ``output``
|
||||
lists. In this case, both arguments are set to NULL since the YANG RPC
|
||||
in question doesn’t have any input/output parameters.
|
||||
|
||||
Notifications
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
YANG notifations are sent using the ``nb_notification_send()`` function,
|
||||
documented in the *lib/northbound.h* file as follows:
|
||||
|
||||
.. code:: c
|
||||
|
||||
/*
|
||||
* Send a YANG notification. This is a no-op unless the 'nb_notification_send'
|
||||
* hook was registered by a northbound plugin.
|
||||
*
|
||||
* xpath
|
||||
* xpath of the YANG notification
|
||||
*
|
||||
* arguments
|
||||
* linked list containing the arguments that should be sent. This list is
|
||||
* deleted after being used.
|
||||
*
|
||||
* Returns:
|
||||
* NB_OK on success, NB_ERR otherwise
|
||||
*/
|
||||
extern int nb_notification_send(const char *xpath, struct list *arguments);
|
||||
|
||||
The northbound doesn’t use callbacks for notifications because
|
||||
notifications are generated locally and sent to the northbound clients.
|
||||
This way, whenever a notification needs to be sent, it’s possible to
|
||||
call the appropriate function directly instead of finding a callback
|
||||
based on the XPath of the YANG notification.
|
||||
|
||||
As an example, the *ietf-rip* module contains the following
|
||||
notification:
|
||||
|
||||
.. code:: yang
|
||||
|
||||
notification authentication-failure {
|
||||
description
|
||||
"This notification is sent when the system
|
||||
receives a PDU with the wrong authentication
|
||||
information.";
|
||||
leaf interface-name {
|
||||
type string;
|
||||
description
|
||||
"Describes the name of the RIP interface.";
|
||||
}
|
||||
}
|
||||
|
||||
The following convenience function was implemented in *ripd* to send
|
||||
*authentication-failure* YANG notifications:
|
||||
|
||||
.. code:: c
|
||||
|
||||
/*
|
||||
* XPath: /frr-ripd:authentication-failure
|
||||
*/
|
||||
void ripd_notif_send_auth_failure(const char *ifname)
|
||||
{
|
||||
const char *xpath = "/frr-ripd:authentication-failure";
|
||||
struct list *arguments;
|
||||
char xpath_arg[XPATH_MAXLEN];
|
||||
struct yang_data *data;
|
||||
|
||||
arguments = yang_data_list_new();
|
||||
|
||||
snprintf(xpath_arg, sizeof(xpath_arg), "%s/interface-name", xpath);
|
||||
data = yang_data_new_string(xpath_arg, ifname);
|
||||
listnode_add(arguments, data);
|
||||
|
||||
nb_notification_send(xpath, arguments);
|
||||
}
|
||||
|
||||
Now sending the *authentication-failure* YANG notification should be as
|
||||
simple as calling the above function and provide the appropriate
|
||||
interface name. The notification will be processed by all northbound
|
||||
plugins that subscribed a callback to the ``nb_notification_send`` hook.
|
||||
The ConfD and Sysrepo plugins, for instance, use this hook to relay the
|
||||
notifications to the *confd*/*sysrepod* daemons, which can generate
|
||||
NETCONF notifications to subscribed clients. When no northbound plugin
|
||||
is loaded, ``nb_notification_send()`` doesn’t do anything and the
|
||||
notifications are ignored.
|
||||
137
doc/developer/northbound/plugins-sysrepo.rst
Normal file
137
doc/developer/northbound/plugins-sysrepo.rst
Normal file
@ -0,0 +1,137 @@
|
||||
Installation
|
||||
------------
|
||||
|
||||
Required dependencies
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
::
|
||||
|
||||
# apt-get install git cmake build-essential bison flex libpcre3-dev libev-dev \
|
||||
libavl-dev libprotobuf-c-dev protobuf-c-compiler libcmocka0 \
|
||||
libcmocka-dev doxygen libssl-dev libssl-dev libssh-dev
|
||||
|
||||
libyang
|
||||
^^^^^^^
|
||||
|
||||
::
|
||||
|
||||
# apt-get install libyang0.16 libyang-dev
|
||||
|
||||
Sysrepo
|
||||
^^^^^^^
|
||||
|
||||
::
|
||||
|
||||
$ git clone https://github.com/sysrepo/sysrepo.git
|
||||
$ cd sysrepo/
|
||||
$ mkdir build; cd build
|
||||
$ cmake -DCMAKE_BUILD_TYPE=Release -DGEN_LANGUAGE_BINDINGS=OFF .. && make
|
||||
# make install
|
||||
|
||||
libnetconf2
|
||||
^^^^^^^^^^^
|
||||
|
||||
::
|
||||
|
||||
$ git clone https://github.com/CESNET/libnetconf2.git
|
||||
$ cd libnetconf2/
|
||||
$ mkdir build; cd build
|
||||
$ cmake .. && make
|
||||
# make install
|
||||
|
||||
netopeer2
|
||||
^^^^^^^^^
|
||||
|
||||
::
|
||||
|
||||
$ git clone https://github.com/CESNET/Netopeer2.git
|
||||
$ cd Netopeer2
|
||||
$ cd server
|
||||
$ mkdir build; cd build
|
||||
$ cmake .. && make
|
||||
# make install
|
||||
|
||||
**Note:** If ``make install`` fails as it can’t find
|
||||
``libsysrepo.so.0.7``, then run ``ldconfig`` and try again as it might
|
||||
not have updated the lib search path
|
||||
|
||||
FRR
|
||||
^^^
|
||||
|
||||
Build and install FRR using the ``--enable-sysrepo`` configure-time
|
||||
option.
|
||||
|
||||
Initialization
|
||||
--------------
|
||||
|
||||
Install the FRR YANG modules in the Sysrepo datastore:
|
||||
|
||||
::
|
||||
|
||||
# sysrepoctl --install /usr/local/share/yang/ietf-interfaces@2018-01-09.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-vrf.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-interface.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-route-types.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-filter.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-route-map.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-isisd.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-ripd.yang
|
||||
# sysrepoctl --install /usr/local/share/yang/frr-ripngd.yang
|
||||
# sysrepoctl -c frr-vrf --owner frr --group frr
|
||||
# sysrepoctl -c frr-interface --owner frr --group frr
|
||||
# sysrepoctl -c frr-route-types --owner frr --group frr
|
||||
# sysrepoctl -c frr-filter --owner frr --group frr
|
||||
# sysrepoctl -c frr-route-map --owner frr --group frr
|
||||
# sysrepoctl -c frr-isisd --owner frr --group frr
|
||||
# sysrepoctl -c frr-ripd --owner frr --group frr
|
||||
# sysrepoctl -c frr-ripngd --owner frr --group frr
|
||||
|
||||
Start netopeer2-server:
|
||||
|
||||
::
|
||||
|
||||
# netopeer2-server -d &
|
||||
|
||||
Start the FRR daemons with the sysrepo module:
|
||||
|
||||
::
|
||||
|
||||
# isisd -M sysrepo --log=stdout
|
||||
|
||||
Managing the configuration
|
||||
--------------------------
|
||||
|
||||
The following NETCONF scripts can be used to show and edit the FRR
|
||||
configuration:
|
||||
https://github.com/rzalamena/ietf-hackathon-brazil-201907/tree/master/netconf-scripts
|
||||
|
||||
Example:
|
||||
|
||||
::
|
||||
|
||||
# ./netconf-edit.py 127.0.0.1
|
||||
# ./netconf-get-config.py 127.0.0.1
|
||||
<?xml version="1.0" encoding="UTF-8"?><data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><isis xmlns="http://frrouting.org/yang/isisd"><instance><area-tag>testnet</area-tag><is-type>level-1</is-type></instance></isis></data>
|
||||
|
||||
..
|
||||
|
||||
NOTE: the ncclient library needs to be installed first:
|
||||
``apt install -y python3-ncclient``
|
||||
|
||||
The *sysrepocfg* tool can also be used to show/edit the FRR
|
||||
configuration. Example:
|
||||
|
||||
::
|
||||
|
||||
# sysrepocfg --format=json --import=frr-isisd.json --datastore=running frr-isisd
|
||||
# sysrepocfg --format=json --export --datastore=running frr-isisd
|
||||
{
|
||||
"frr-isisd:isis": {
|
||||
"instance": [
|
||||
{
|
||||
"area-tag": "testnet",
|
||||
"is-type": "level-1"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
1632
doc/developer/northbound/ppr-basic-test-topology.rst
Normal file
1632
doc/developer/northbound/ppr-basic-test-topology.rst
Normal file
File diff suppressed because it is too large
Load Diff
1991
doc/developer/northbound/ppr-mpls-basic-test-topology.rst
Normal file
1991
doc/developer/northbound/ppr-mpls-basic-test-topology.rst
Normal file
File diff suppressed because it is too large
Load Diff
1916
doc/developer/northbound/retrofitting-configuration-commands.rst
Normal file
1916
doc/developer/northbound/retrofitting-configuration-commands.rst
Normal file
File diff suppressed because it is too large
Load Diff
244
doc/developer/northbound/transactional-cli.rst
Normal file
244
doc/developer/northbound/transactional-cli.rst
Normal file
@ -0,0 +1,244 @@
|
||||
Table of Contents
|
||||
-----------------
|
||||
|
||||
- `Introduction <#introduction>`__
|
||||
- `Configuration modes <#config-modes>`__
|
||||
- `New commands <#retrofitting-process>`__
|
||||
|
||||
- `commit check <#cmd1>`__
|
||||
- `commit <#cmd2>`__
|
||||
- `discard <#cmd3>`__
|
||||
- `configuration database max-transactions <#cmd4>`__
|
||||
- `configuration load <#cmd5>`__
|
||||
- `rollback configuration <#cmd6>`__
|
||||
- `show configuration candidate <#cmd7>`__
|
||||
- `show configuration compare <#cmd8>`__
|
||||
- `show configuration running <#cmd9>`__
|
||||
- `show configuration transaction <#cmd10>`__
|
||||
- `show yang module <#cmd11>`__
|
||||
- `show yang module-translator <#cmd12>`__
|
||||
- `update <#cmd13>`__
|
||||
- `yang module-translator load <#cmd14>`__
|
||||
- `yang module-translator unload <#cmd15>`__
|
||||
|
||||
Introduction
|
||||
~~~~~~~~~~~~
|
||||
|
||||
All FRR daemons have built-in support for the CLI, which can be accessed
|
||||
either through local telnet or via the vty socket (e.g. by using
|
||||
*vtysh*). This will not change with the introduction of the Northbound
|
||||
API. However, a new command-line option will be available for all FRR
|
||||
daemons: ``--tcli``. When given, this option makes the daemon start with
|
||||
a transactional CLI and configuration commands behave a bit different.
|
||||
Instead of editing the running configuration, they will edit the
|
||||
candidate configuration. In other words, the configuration commands
|
||||
won’t be applied immediately, that has to be done on a separate step
|
||||
using the new ``commit`` command.
|
||||
|
||||
The transactional CLI simply leverages the new capabilities provided by
|
||||
the Northbound API and exposes the concept of candidate configurations
|
||||
to CLI users too. When the transactional mode is not used, the
|
||||
configuration commands also edit the candidate configuration, but
|
||||
there’s an implicit ``commit`` after each command.
|
||||
|
||||
In order for the transactional CLI to work, all configuration commands
|
||||
need to be converted to the new northbound model. Commands not converted
|
||||
to the new northbound model will change the running configuration
|
||||
directly since they bypass the FRR northbound layer. For this reason,
|
||||
starting a daemon with the transactional CLI is not advisable unless all
|
||||
of its commands have already been converted. When that’s not the case,
|
||||
we can run into a situation like this:
|
||||
|
||||
::
|
||||
|
||||
ospfd(config)# router ospf
|
||||
ospfd(config-router)# ospf router-id 1.1.1.1
|
||||
[segfault in ospfd]
|
||||
|
||||
The segfault above can happen if ``router ospf`` edits the candidate
|
||||
configuration but ``ospf router-id 1.1.1.1`` edits the running
|
||||
configuration. The second command tries to set
|
||||
``ospf->router_id_static`` but, since the previous ``router ospf``
|
||||
command hasn’t been commited yet, the ``ospf`` global variable is set to
|
||||
NULL, which leads to the crash. Besides this problem, having a set of
|
||||
commands that edit the candidate configuration and others that edit the
|
||||
running configuration is confusing at best. The ``--tcli`` option should
|
||||
be used only by developers until the northbound retrofitting process is
|
||||
complete.
|
||||
|
||||
Configuration modes
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
When using the transactional CLI (``--tcli``), FRR supports three
|
||||
different forms of the ``configure`` command: \* ``configure terminal``:
|
||||
in this mode, a single candidate configuration is shared by all users.
|
||||
This means that one user might delete a configuration object that’s
|
||||
being edited by another user, in which case the CLI will detect and
|
||||
report the problem. If one user issues the ``commit`` command, all
|
||||
changes done by all users are committed. \* ``configure private``: users
|
||||
have a private candidate configuration that is edited separately from
|
||||
the other users. The ``commit`` command commits only the changes done by
|
||||
the user. \* ``configure exclusive``: similar to ``configure private``,
|
||||
but also locks the running configuration to prevent other users from
|
||||
changing it. The configuration lock is released when the user exits the
|
||||
configuration mode.
|
||||
|
||||
When using ``configure terminal`` or ``configure private``, the
|
||||
candidate configuration being edited might become outdated if another
|
||||
user commits a different candidate configuration on another session.
|
||||
TODO: show image to illustrate the problem.
|
||||
|
||||
New commands
|
||||
~~~~~~~~~~~~
|
||||
|
||||
The list below contains the new CLI commands introduced by Northbound
|
||||
API. The commands are available when a daemon is started using the
|
||||
transactional CLI (``--tcli``). Currently ``vtysh`` doesn’t support any
|
||||
of these new commands.
|
||||
|
||||
Please refer to the [[Demos]] page to see a demo of the transactional
|
||||
CLI in action.
|
||||
|
||||
--------------
|
||||
|
||||
``commit check``
|
||||
''''''''''''''''
|
||||
|
||||
Check if the candidate configuration is valid or not.
|
||||
|
||||
``commit [force] [comment LINE...]``
|
||||
''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Commit the changes done in the candidate configuration into the running
|
||||
configuration.
|
||||
|
||||
Options: \* ``force``: commit even if the candidate configuration is
|
||||
outdated. It’s usually a better option to use the ``update`` command
|
||||
instead. \* ``comment LINE...``: assign a comment to the configuration
|
||||
transaction. This comment is displayed when viewing the recorded
|
||||
transactions in the output of the ``show configuration transaction``
|
||||
command.
|
||||
|
||||
``discard``
|
||||
'''''''''''
|
||||
|
||||
Discard the changes done in the candidate configuration.
|
||||
|
||||
``configuration database max-transactions (1-100)``
|
||||
'''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Set the maximum number of transactions to store in the rollback log.
|
||||
|
||||
``configuration load <file [<json|xml> [translate WORD]] FILENAME|transaction (1-4294967296)> [replace]``
|
||||
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Load a new configuration into the candidate configuration. When loading
|
||||
the configuration from a file, it’s assumed that the configuration will
|
||||
be in the form of CLI commands by default. The ``json`` and ``xml``
|
||||
options can be used to load configurations in the JSON and XML formats,
|
||||
respectively. It’s also possible to load a configuration from a previous
|
||||
transaction by specifying the desired transaction ID
|
||||
(``(1-4294967296)``).
|
||||
|
||||
Options: \* ``translate WORD``: translate the JSON/XML configuration
|
||||
file using the YANG module translator. \* ``replace``: replace the
|
||||
candidate by the loaded configuration. The default is to merge the
|
||||
loaded configuration into the candidate configuration.
|
||||
|
||||
``rollback configuration (1-4294967296)``
|
||||
'''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Roll back the running configuration to a previous configuration
|
||||
identified by its transaction ID (``(1-4294967296)``).
|
||||
|
||||
``show configuration candidate [<json|xml> [translate WORD]] [<with-defaults|changes>]``
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Show the candidate configuration.
|
||||
|
||||
Options: \* ``json``: show the configuration in the JSON format. \*
|
||||
``xml``: show the configuration in the XML format. \*
|
||||
``translate WORD``: translate the JSON/XML output using the YANG module
|
||||
translator. \* ``with-defaults``: show default values that are hidden by
|
||||
default. \* ``changes``: show only the changes done in the candidate
|
||||
configuration.
|
||||
|
||||
``show configuration compare <candidate|running|transaction (1-4294967296)> <candidate|running|transaction (1-4294967296)> [<json|xml> [translate WORD]]``
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Show the difference between two different configurations.
|
||||
|
||||
Options: \* ``json``: show the configuration differences in the JSON
|
||||
format. \* ``xml``: show the configuration differences in the XML
|
||||
format. \* ``translate WORD``: translate the JSON/XML output using the
|
||||
YANG module translator.
|
||||
|
||||
``show configuration running [<json|xml> [translate WORD]] [with-defaults]``
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Show the running configuration.
|
||||
|
||||
Options: \* ``json``: show the configuration in the JSON format. \*
|
||||
``xml``: show the configuration in the XML format. \*
|
||||
``translate WORD``: translate the JSON/XML output using the YANG module
|
||||
translator. \* ``with-defaults``: show default values that are hidden by
|
||||
default.
|
||||
|
||||
NOTE: ``show configuration running`` shows only the running
|
||||
configuration as known by the northbound layer. Configuration
|
||||
commands not converted to the new northbound model will not be
|
||||
displayed. To show the full running configuration, the legacy
|
||||
``show running-config`` command must be used.
|
||||
|
||||
``show configuration transaction [(1-4294967296) [<json|xml> [translate WORD]] [changes]]``
|
||||
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
When a transaction ID (``(1-4294967296)``) is given, show the
|
||||
configuration associated to the previously committed transaction.
|
||||
|
||||
When a transaction ID is not given, show all recorded transactions in
|
||||
the rollback log.
|
||||
|
||||
Options: \* ``json``: show the configuration in the JSON format. \*
|
||||
``xml``: show the configuration in the XML format. \*
|
||||
``translate WORD``: translate the JSON/XML output using the YANG module
|
||||
translator. \* ``with-defaults``: show default values that are hidden by
|
||||
default. \* ``changes``: show changes compared to the previous
|
||||
transaction.
|
||||
|
||||
``show yang module [module-translator WORD] [WORD <summary|tree|yang|yin>]``
|
||||
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
When a YANG module is not given, show all loaded YANG modules.
|
||||
Otherwise, show detailed information about the given module.
|
||||
|
||||
Options: \* ``module-translator WORD``: change the context to modules
|
||||
loaded by the specified YANG module translator. \* ``summary``: display
|
||||
summary information about the module. \* ``tree``: display module in the
|
||||
tree (RFC 8340) format. \* ``yang``: display module in the YANG format.
|
||||
\* ``yin``: display module in the YIN format.
|
||||
|
||||
``show yang module-translator``
|
||||
'''''''''''''''''''''''''''''''
|
||||
|
||||
Show all loaded YANG module translators.
|
||||
|
||||
``update``
|
||||
''''''''''
|
||||
|
||||
Rebase the candidate configuration on top of the latest running
|
||||
configuration. Conflicts are resolved automatically by giving preference
|
||||
to the changes done in the candidate configuration.
|
||||
|
||||
The candidate configuration might be outdated if the running
|
||||
configuration was updated after the candidate was created.
|
||||
|
||||
``yang module-translator load FILENAME``
|
||||
''''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Load a YANG module translator from the filesystem.
|
||||
|
||||
``yang module-translator unload WORD``
|
||||
''''''''''''''''''''''''''''''''''''''
|
||||
|
||||
Unload a YANG module translator identified by its name.
|
||||
629
doc/developer/northbound/yang-module-translator.rst
Normal file
629
doc/developer/northbound/yang-module-translator.rst
Normal file
@ -0,0 +1,629 @@
|
||||
Table of Contents
|
||||
-----------------
|
||||
|
||||
- `Introduction <#introduction>`__
|
||||
- `Deviation Modules <#deviation-modules>`__
|
||||
- `Translation Tables <#translation-tables>`__
|
||||
- `CLI Demonstration <#cli-demonstration>`__
|
||||
- `Implementation Details <#implementation-details>`__
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
One key requirement for the FRR northbound architecture is that it
|
||||
should be possible to configure/monitor FRR using different sets of YANG
|
||||
models. This is especially important considering that the industry
|
||||
hasn’t reached a consensus to provide a single source of standard models
|
||||
for network management. At this moment both the IETF and OpenConfig
|
||||
models are widely implemented and are unlikely to converge, at least not
|
||||
in the short term. In the ideal scenario, management applications should
|
||||
be able to use either IETF or OpenConfig models to configure and monitor
|
||||
FRR programatically (or even both at the same time!).
|
||||
|
||||
But how can FRR support multiple sets of YANG models at the same time?
|
||||
There must be only a single source of truth that models the existing
|
||||
implementation accurately (the native models). Writing different code
|
||||
paths or callbacks for different models would be inviable, it would lead
|
||||
to a lot of duplicated code and extra maintenance overhead.
|
||||
|
||||
In order to support different sets of YANG modules without introducing
|
||||
the overhead of writing additional code, the solution is to create a
|
||||
mechanism that dynamically translates YANG instance data between
|
||||
non-native models to native models and vice-versa. Based on this idea,
|
||||
an experimental YANG module translator was implemented within the FRR
|
||||
northbound layer. The translator works by translating XPaths at runtime
|
||||
using translation tables provided by the user. The translator itself is
|
||||
modeled using YANG and users can create translators using simple JSON
|
||||
files.
|
||||
|
||||
A YANG module translator consists of two components: deviation modules
|
||||
and translation tables.
|
||||
|
||||
Deviation Modules
|
||||
-----------------
|
||||
|
||||
The first step when writing a YANG module translator is to create a
|
||||
`deviations <https://tools.ietf.org/html/rfc7950#page-131>`__ module for
|
||||
each module that is going be translated. This is necessary because in
|
||||
most cases it won’t be possible to create a perfect translator that
|
||||
covers the non-native models on their entirety. Some non-native modules
|
||||
might contain nodes that can’t be mapped to a corresponding node in the
|
||||
FRR native models. This is either because the corresponding
|
||||
functionality is not implemented in FRR or because it’s modeled in a
|
||||
different way that is incompatible.
|
||||
|
||||
An an example, *ripd* doesn’t have BFD support yet, so we need to create
|
||||
a YANG deviation to modify the *ietf-rip* module and remove the ``bfd``
|
||||
container from it:
|
||||
|
||||
.. code:: yang
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:interfaces/ietf-rip:interface/ietf-rip:bfd" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
In the example below, while both the *frr-ripd* and *ietf-rip* modules
|
||||
support RIP authentication, they model the authentication data in
|
||||
different ways, making translation not possible given the constraints of
|
||||
the current module translator. A new deviation is necessary to remove
|
||||
the ``authentication`` container from the *ietf-rip* module:
|
||||
|
||||
.. code:: yang
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:interfaces/ietf-rip:interface/ietf-rip:authentication" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
..
|
||||
|
||||
NOTE: it should be possible to translate the
|
||||
``ietf-rip:authentication`` container if the *frr-ripd* module is
|
||||
modified to model the corresponding data in a compatible way. Another
|
||||
option is to improve the module translator to make more complex
|
||||
translations possible, instead of requiring one-to-one XPath
|
||||
mappings.
|
||||
|
||||
Sometimes creating a mapping between nodes from the native and
|
||||
non-native models is possible, but the nodes have different properties
|
||||
that need to be normalized to allow the translation. In the example
|
||||
below, a YANG deviation is used to change the type and the default value
|
||||
from a node from the ``ietf-rip`` module.
|
||||
|
||||
.. code:: yang
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:timers/ietf-rip:flush-interval" {
|
||||
deviate replace {
|
||||
default "120";
|
||||
}
|
||||
deviate replace {
|
||||
type uint32;
|
||||
}
|
||||
}
|
||||
|
||||
The deviation modules allow the management applications to know which
|
||||
parts of the custom modules (e.g. IETF/OC) can be used to configure and
|
||||
monitor FRR.
|
||||
|
||||
In order to facilitate the process of creating YANG deviation modules,
|
||||
the *gen_yang_deviations* tool was created to automate part of the
|
||||
process. This tool creates a “not-supported” deviation for all nodes
|
||||
from the given non-native module. Example:
|
||||
|
||||
::
|
||||
|
||||
$ tools/gen_yang_deviations ietf-rip > yang/ietf/frr-deviations-ietf-rip.yang
|
||||
$ head -n 40 yang/ietf/frr-deviations-ietf-rip.yang
|
||||
deviation "/ietf-rip:clear-rip-route" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-rip:clear-rip-route/ietf-rip:input" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-rip:clear-rip-route/ietf-rip:input/ietf-rip:rip-instance" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route/ietf-rip:enabled" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route/ietf-rip:route-policy" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:default-metric" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:distance" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:triggered-update-threshold" {
|
||||
deviate not-supported;
|
||||
}
|
||||
|
||||
Once all existing nodes are listed in the deviation module, it’s easy to
|
||||
check the deviations that need to be removed or modified. This is more
|
||||
convenient than starting with a blank deviations module and listing
|
||||
manually all nodes that need to be deviated.
|
||||
|
||||
After removing and/or modifying the auto-generated deviations, the next
|
||||
step is to write the module XPath translation table as we’ll see in the
|
||||
next section. Before that, it’s possible to use the *yanglint* tool to
|
||||
check how the non-native module looks like after applying the
|
||||
deviations. Example:
|
||||
|
||||
::
|
||||
|
||||
$ yanglint -f tree yang/ietf/ietf-rip@2018-02-03.yang yang/ietf/frr-deviations-ietf-rip.yang
|
||||
module: ietf-rip
|
||||
|
||||
augment /ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol:
|
||||
+--rw rip
|
||||
+--rw originate-default-route
|
||||
| +--rw enabled? boolean <false>
|
||||
+--rw default-metric? uint8 <1>
|
||||
+--rw distance? uint8 <0>
|
||||
+--rw timers
|
||||
| +--rw update-interval? uint32 <30>
|
||||
| +--rw holddown-interval? uint32 <180>
|
||||
| +--rw flush-interval? uint32 <120>
|
||||
+--rw interfaces
|
||||
| +--rw interface* [interface]
|
||||
| +--rw interface ietf-interfaces:interface-ref
|
||||
| +--rw split-horizon? enumeration <simple>
|
||||
+--ro ipv4
|
||||
+--ro neighbors
|
||||
| +--ro neighbor* [ipv4-address]
|
||||
| +--ro ipv4-address ietf-inet-types:ipv4-address
|
||||
| +--ro last-update? ietf-yang-types:date-and-time
|
||||
| +--ro bad-packets-rcvd? ietf-yang-types:counter32
|
||||
| +--ro bad-routes-rcvd? ietf-yang-types:counter32
|
||||
+--ro routes
|
||||
+--ro route* [ipv4-prefix]
|
||||
+--ro ipv4-prefix ietf-inet-types:ipv4-prefix
|
||||
+--ro next-hop? ietf-inet-types:ipv4-address
|
||||
+--ro interface? ietf-interfaces:interface-ref
|
||||
+--ro metric? uint8
|
||||
|
||||
rpcs:
|
||||
+---x clear-rip-route
|
||||
|
||||
..
|
||||
|
||||
NOTE: the same output can be obtained using the
|
||||
``show yang module module-translator ietf ietf-rip tree`` command in
|
||||
FRR once the *ietf* module translator is loaded.
|
||||
|
||||
In the example above, it can be seen that the vast majority of the
|
||||
*ietf-rip* nodes were removed because of the “not-supported” deviations.
|
||||
When a module translator is loaded, FRR calculates the coverage of the
|
||||
translator by dividing the number of YANG nodes before applying the
|
||||
deviations by the number of YANG nodes after applying the deviations.
|
||||
The calculated coverage is displayed in the output of the
|
||||
``show yang module-translator`` command:
|
||||
|
||||
::
|
||||
|
||||
ripd# show yang module-translator
|
||||
Family Module Deviations Coverage (%)
|
||||
-----------------------------------------------------------------------
|
||||
ietf ietf-interfaces frr-deviations-ietf-interfaces 3.92
|
||||
ietf ietf-routing frr-deviations-ietf-routing 1.56
|
||||
ietf ietf-rip frr-deviations-ietf-rip 13.60
|
||||
|
||||
As it can be seen in the output above, the *ietf* module translator
|
||||
covers only ~13% of the original *ietf-rip* module. This is in part
|
||||
because the *ietf-rip* module models both RIPv2 and RIPng. Also,
|
||||
*ietf-rip.yang* contains several knobs that aren’t implemented in *ripd*
|
||||
yet (e.g. BFD support, per-interface timers, statistics, etc). Work can
|
||||
be done over time to increase the coverage to a more reasonable number.
|
||||
|
||||
Translation Tables
|
||||
------------------
|
||||
|
||||
Below is an example of a translator for the IETF family of models:
|
||||
|
||||
.. code:: json
|
||||
|
||||
{
|
||||
"frr-module-translator:frr-module-translator": {
|
||||
"family": "ietf",
|
||||
"module": [
|
||||
{
|
||||
"name": "ietf-interfaces@2018-01-09",
|
||||
"deviations": "frr-deviations-ietf-interfaces",
|
||||
"mappings": [
|
||||
{
|
||||
"custom": "/ietf-interfaces:interfaces/interface[name='KEY1']",
|
||||
"native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-interfaces:interfaces/interface[name='KEY1']/description",
|
||||
"native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']/description"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ietf-routing@2018-01-25",
|
||||
"deviations": "frr-deviations-ietf-routing",
|
||||
"mappings": [
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']",
|
||||
"native": "/frr-ripd:ripd/instance"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ietf-rip@2018-02-03",
|
||||
"deviations": "frr-deviations-ietf-rip",
|
||||
"mappings": [
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/default-metric",
|
||||
"native": "/frr-ripd:ripd/instance/default-metric"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/distance",
|
||||
"native": "/frr-ripd:ripd/instance/distance/default"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/originate-default-route/enabled",
|
||||
"native": "/frr-ripd:ripd/instance/default-information-originate"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/update-interval",
|
||||
"native": "/frr-ripd:ripd/instance/timers/update-interval"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/holddown-interval",
|
||||
"native": "/frr-ripd:ripd/instance/timers/holddown-interval"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/flush-interval",
|
||||
"native": "/frr-ripd:ripd/instance/timers/flush-interval"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']",
|
||||
"native": "/frr-ripd:ripd/instance/interface[.='KEY1']"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']/split-horizon",
|
||||
"native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']/frr-ripd:rip/split-horizon"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']",
|
||||
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/last-update",
|
||||
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/last-update"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/bad-packets-rcvd",
|
||||
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/bad-packets-rcvd"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/bad-routes-rcvd",
|
||||
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/bad-routes-rcvd"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']",
|
||||
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/next-hop",
|
||||
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/next-hop"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/interface",
|
||||
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/interface"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/metric",
|
||||
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/metric"
|
||||
},
|
||||
{
|
||||
"custom": "/ietf-rip:clear-rip-route",
|
||||
"native": "/frr-ripd:clear-rip-route"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
The main motivation to use YANG itself to model YANG module translators
|
||||
was a practical one: leverage *libyang* to validate the structure of the
|
||||
user input (JSON files) instead of doing that manually in the
|
||||
*lib/yang_translator.c* file (tedious and error-prone work).
|
||||
|
||||
Module translators can be loaded using the following CLI command:
|
||||
|
||||
::
|
||||
|
||||
ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json
|
||||
% Module translator "ietf" loaded successfully.
|
||||
|
||||
Module translators can also be loaded/unloaded programatically using the
|
||||
``yang_translator_load()/yang_translator_unload()`` functions within the
|
||||
northbound plugins. These functions are documented in the
|
||||
*lib/yang_translator.h* file.
|
||||
|
||||
Each module translator must be assigned a “family” identifier
|
||||
(e.g. IETF, OpenConfig), and can contain mappings for multiple
|
||||
interrelated YANG modules. The mappings consist of pairs of
|
||||
custom/native XPath expressions that should be equivalent, despite
|
||||
belonging to different YANG modules.
|
||||
|
||||
Example:
|
||||
|
||||
.. code:: json
|
||||
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/default-metric",
|
||||
"native": "/frr-ripd:ripd/instance/default-metric"
|
||||
},
|
||||
|
||||
The nodes pointed by the custom and native XPaths must have compatible
|
||||
types. In the case of the example above, both nodes point to a YANG leaf
|
||||
of type ``uint8``, so the mapping is valid.
|
||||
|
||||
In the example below, the “custom” XPath points to a YANG list
|
||||
(typeless), and the “native” XPath points to a YANG leaf-list of
|
||||
strings. In this exceptional case, the types are also considered to be
|
||||
compatible.
|
||||
|
||||
.. code:: json
|
||||
|
||||
{
|
||||
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']",
|
||||
"native": "/frr-ripd:ripd/instance/interface[.='KEY1']"
|
||||
},
|
||||
|
||||
The ``KEY1..KEY4`` values have a special meaning and are used to
|
||||
preserve the list keys while performing the XPath translation.
|
||||
|
||||
Once a YANG module translator is loaded and validated at a syntactic
|
||||
level using *libyang*, further validations are performed to check for
|
||||
missing mappings (after loading the deviation modules) and incompatible
|
||||
YANG types. Example:
|
||||
|
||||
::
|
||||
|
||||
ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json
|
||||
% Failed to load "/usr/local/share/yang/ietf/frr-ietf-translator.json"
|
||||
|
||||
Please check the logs for more details.
|
||||
|
||||
::
|
||||
|
||||
2018/09/03 15:18:45 RIP: yang_translator_validate_cb: YANG types are incompatible (xpath: "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/default-metric")
|
||||
2018/09/03 15:18:45 RIP: yang_translator_validate_cb: missing mapping for "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/distance"
|
||||
2018/09/03 15:18:45 RIP: yang_translator_validate: failed to validate "ietf" module translator: 2 error(s)
|
||||
|
||||
Overall, this translation mechanism based on XPath mappings is simple
|
||||
and functional, but only to a certain extent. The native models need to
|
||||
be reasonably similar to the models that are going be translated,
|
||||
otherwise the translation is compromised and a good coverage can’t be
|
||||
achieved. Other translation techniques must be investigated to address
|
||||
this shortcoming and make it possible to create more powerful YANG
|
||||
module translators.
|
||||
|
||||
YANG module translators can be evaluated based on the following metrics:
|
||||
\* Translation potential: is it possible to make complex translations,
|
||||
taking several variables into account? \* Complexity: measure of how
|
||||
easy or hard it is to write a module translator. \* Speed: measure of
|
||||
how fast the translation can be achieved. Translation speed is of
|
||||
fundamental importance, especially for operational data. \* Robustness:
|
||||
can the translator be checked for inconsistencies at load time? A module
|
||||
translator based on scripts wouldn’t fare well on this metric. \*
|
||||
Round-trip conversions: can the translated data be translated back to
|
||||
the original format without information loss?
|
||||
|
||||
CLI Demonstration
|
||||
-----------------
|
||||
|
||||
As of now the only northbound client that supports the YANG module
|
||||
translator is the FRR embedded CLI. The confd and sysrepo plugins need
|
||||
to be extended to support the module translator, which might be used not
|
||||
only for configuration data, but also for operational data, RPCs and
|
||||
notifications.
|
||||
|
||||
In this demonstration, we’ll use the CLI ``configuration load`` command
|
||||
to load the following JSON configuration file specified using the IETF
|
||||
data hierarchy:
|
||||
|
||||
.. code:: json
|
||||
|
||||
{
|
||||
"ietf-interfaces:interfaces": {
|
||||
"interface": [
|
||||
{
|
||||
"description": "Engineering",
|
||||
"name": "eth0"
|
||||
}
|
||||
]
|
||||
},
|
||||
"ietf-routing:routing": {
|
||||
"control-plane-protocols": {
|
||||
"control-plane-protocol": [
|
||||
{
|
||||
"name": "main",
|
||||
"type": "ietf-rip:ripv2",
|
||||
"ietf-rip:rip": {
|
||||
"default-metric": "2",
|
||||
"distance": "80",
|
||||
"interfaces": {
|
||||
"interface": [
|
||||
{
|
||||
"interface": "eth0",
|
||||
"split-horizon": "poison-reverse"
|
||||
}
|
||||
]
|
||||
},
|
||||
"originate-default-route": {
|
||||
"enabled": "true"
|
||||
},
|
||||
"timers": {
|
||||
"flush-interval": "241",
|
||||
"holddown-interval": "181",
|
||||
"update-interval": "31"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
In order to load this configuration file, it’s necessary to load the
|
||||
IETF module translator first. Then, when entering the
|
||||
``configuration load`` command, the ``translate ietf`` parameters must
|
||||
be given to specify that the input needs to be translated using the
|
||||
previously loaded ``ietf`` module translator. Example:
|
||||
|
||||
::
|
||||
|
||||
ripd(config)# configuration load file json /mnt/renato/git/frr/yang/example/ietf-rip.json
|
||||
% Failed to load configuration:
|
||||
|
||||
Unknown element "interfaces".
|
||||
ripd(config)#
|
||||
ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json
|
||||
% Module translator "ietf" loaded successfully.
|
||||
|
||||
ripd(config)#
|
||||
ripd(config)# configuration load file json translate ietf /mnt/renato/git/frr/yang/example/ietf-rip.json
|
||||
|
||||
Now let’s check the candidate configuration to see if the configuration
|
||||
file was loaded successfully:
|
||||
|
||||
::
|
||||
|
||||
ripd(config)# show configuration candidate
|
||||
Configuration:
|
||||
!
|
||||
frr version 5.1-dev
|
||||
frr defaults traditional
|
||||
!
|
||||
interface eth0
|
||||
description Engineering
|
||||
ip rip split-horizon poisoned-reverse
|
||||
!
|
||||
router rip
|
||||
default-metric 2
|
||||
distance 80
|
||||
network eth0
|
||||
default-information originate
|
||||
timers basic 31 181 241
|
||||
!
|
||||
end
|
||||
ripd(config)# show configuration candidate json
|
||||
{
|
||||
"frr-interface:lib": {
|
||||
"interface": [
|
||||
{
|
||||
"name": "eth0",
|
||||
"vrf": "default",
|
||||
"description": "Engineering",
|
||||
"frr-ripd:rip": {
|
||||
"split-horizon": "poison-reverse"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"frr-ripd:ripd": {
|
||||
"instance": {
|
||||
"default-metric": 2,
|
||||
"distance": {
|
||||
"default": 80
|
||||
},
|
||||
"interface": [
|
||||
"eth0"
|
||||
],
|
||||
"default-information-originate": true,
|
||||
"timers": {
|
||||
"flush-interval": 241,
|
||||
"holddown-interval": 181,
|
||||
"update-interval": 31
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
As it can be seen, the candidate configuration is identical to the one
|
||||
defined in the *ietf-rip.json* file, only the structure is different.
|
||||
This means that the *ietf-rip.json* file was translated successfully.
|
||||
|
||||
The ``ietf`` module translator can also be used to do the translation in
|
||||
other direction: transform data from the native format to the IETF
|
||||
format. This is shown below by altering the output of the
|
||||
``show configuration candidate json`` command using the
|
||||
``translate ietf`` parameter:
|
||||
|
||||
::
|
||||
|
||||
ripd(config)# show configuration candidate json translate ietf
|
||||
{
|
||||
"ietf-interfaces:interfaces": {
|
||||
"interface": [
|
||||
{
|
||||
"name": "eth0",
|
||||
"description": "Engineering"
|
||||
}
|
||||
]
|
||||
},
|
||||
"ietf-routing:routing": {
|
||||
"control-plane-protocols": {
|
||||
"control-plane-protocol": [
|
||||
{
|
||||
"type": "ietf-rip:ripv2",
|
||||
"name": "main",
|
||||
"ietf-rip:rip": {
|
||||
"interfaces": {
|
||||
"interface": [
|
||||
{
|
||||
"interface": "eth0",
|
||||
"split-horizon": "poison-reverse"
|
||||
}
|
||||
]
|
||||
},
|
||||
"default-metric": 2,
|
||||
"distance": 80,
|
||||
"originate-default-route": {
|
||||
"enabled": true
|
||||
},
|
||||
"timers": {
|
||||
"flush-interval": 241,
|
||||
"holddown-interval": 181,
|
||||
"update-interval": 31
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
As expected, this output is exactly identical to the configuration
|
||||
defined in the *ietf-rip.json* file. The module translator was able to
|
||||
do a round-trip conversion without information loss.
|
||||
|
||||
Implementation Details
|
||||
----------------------
|
||||
|
||||
A different libyang context is allocated for each YANG module
|
||||
translator. This is important to avoid collisions and ensure that
|
||||
non-native data can’t be instantiated in the running and candidate
|
||||
configurations.
|
||||
106
doc/developer/northbound/yang-tools.rst
Normal file
106
doc/developer/northbound/yang-tools.rst
Normal file
@ -0,0 +1,106 @@
|
||||
yanglint cheat sheet
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
libyang project includes a feature-rich tool called yanglint(1) for
|
||||
validation and conversion of the schemas and YANG modeled data. The
|
||||
source codes are located at /tools/lint and can be used to explore
|
||||
how an application is supposed to use the libyang library.
|
||||
yanglint(1) binary as well as its man page are installed together
|
||||
with the library itself.
|
||||
|
||||
Validate a YANG module:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ yanglint -p <yang-search-path> module.yang
|
||||
|
||||
Generate tree representation of a YANG module:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ yanglint -p <yang-search-path> -f tree module.yang
|
||||
|
||||
Validate JSON/XML instance data:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ yanglint -p <yang-search-path> module.yang data.{json,xml}
|
||||
|
||||
Convert JSON/XML instance data to another format:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ yanglint -p <yang-search-path> -f xml module.yang data.json
|
||||
$ yanglint -p <yang-search-path> -f json module.yang data.xml
|
||||
|
||||
*yanglint* also features an interactive mode which is very useful when
|
||||
needing to validate data from multiple modules at the same time. The
|
||||
*yanglint* README provides several examples:
|
||||
https://github.com/CESNET/libyang/blob/master/tools/lint/examples/README.md
|
||||
|
||||
Man page (groff):
|
||||
https://github.com/CESNET/libyang/blob/master/tools/lint/yanglint.1
|
||||
|
||||
pyang cheat sheet
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
pyang is a YANG validator, transformator and code generator, written
|
||||
in python. It can be used to validate YANG modules for correctness,
|
||||
to transform YANG modules into other formats, and to generate code
|
||||
from the modules.
|
||||
|
||||
Obtaining and installing pyang:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ git clone https://github.com/mbj4668/pyang.git
|
||||
$ cd pyang/
|
||||
$ sudo python setup.py install
|
||||
|
||||
Validate a YANG module:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ pyang --ietf -p <yang-search-path> module.yang
|
||||
|
||||
Generate tree representation of a YANG module:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ pyang -f tree -p <yang-search-path> module.yang
|
||||
|
||||
Indent a YANG file:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ pyang -p <yang-search-path> \
|
||||
--keep-comments -f yang --yang-canonical \
|
||||
module.yang -o module.yang
|
||||
|
||||
Generate skeleton instance data: \* XML:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ pyang -p <yang-search-path> \
|
||||
-f sample-xml-skeleton --sample-xml-skeleton-defaults \
|
||||
module.yang [augmented-module1.yang ...] -o module.xml
|
||||
|
||||
- JSON:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ pyang -p <yang-search-path> \
|
||||
-f jsonxsl module.yang -o module.xsl
|
||||
$ xsltproc -o module.json module.xsl module.xml
|
||||
|
||||
Validate XML instance data (works only with YANG 1.0):
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ yang2dsdl -v module.xml module.yang
|
||||
|
||||
vim
|
||||
~~~
|
||||
|
||||
YANG syntax highlighting for vim:
|
||||
https://github.com/nathanalderson/yang.vim
|
||||
@ -67,6 +67,19 @@ dev_RSTFILES = \
|
||||
doc/developer/workflow.rst \
|
||||
doc/developer/xrefs.rst \
|
||||
doc/developer/zebra.rst \
|
||||
doc/developer/northbound/advanced-topics.rst \
|
||||
doc/developer/northbound/architecture.rst \
|
||||
doc/developer/northbound/demos.rst \
|
||||
doc/developer/northbound/links.rst \
|
||||
doc/developer/northbound/northbound.rst \
|
||||
doc/developer/northbound/operational-data-rpcs-and-notifications.rst \
|
||||
doc/developer/northbound/plugins-sysrepo.rst \
|
||||
doc/developer/northbound/ppr-basic-test-topology.rst \
|
||||
doc/developer/northbound/ppr-mpls-basic-test-topology.rst \
|
||||
doc/developer/northbound/retrofitting-configuration-commands.rst \
|
||||
doc/developer/northbound/transactional-cli.rst \
|
||||
doc/developer/northbound/yang-module-translator.rst \
|
||||
doc/developer/northbound/yang-tools.rst \
|
||||
# end
|
||||
|
||||
EXTRA_DIST += \
|
||||
|
||||
Loading…
Reference in New Issue
Block a user