proxmox-mirrors/mirror_frr
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_frr synced 2025-08-08 09:30:30 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #14448 from qlyoung/doc-add-northbound-api-docs

Browse Source 
doc: add northbound api arch docs
This commit is contained in:
Mark Stapp 2023-09-20 07:49:14 -04:00 committed by GitHub
commit fe1da43cab
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
16 changed files with 8105 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/developer/index.rst
View File

@ -22,3 +22,4 @@ FRRouting Developer's Guide
path path
pceplib pceplib
link-state link-state
northbound/northbound

15
doc/developer/northbound/_sidebar.rst Normal file
View File

@ -0,0 +1,15 @@
Northbound:
~~~~~~~~~~~
- [[Architecture]]
- [[Transactional CLI]]
- [[Retrofitting Configuration Commands]]
- [[Operational data, RPCs and Notifications]]
- [[Plugins - ConfD]]
- [[Plugins: Sysrepo]]
- [[Plugins - Writing Your Own]]
- [[YANG module translator]]
- [[Advanced topics]]
- [[YANG tools]]
- [[Demos]]
- [[Links]]

294
doc/developer/northbound/advanced-topics.rst Normal file
View File

@ -0,0 +1,294 @@
Auto-generated CLI commands
~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to have less code to maintain, it should be possible to write a
tool that auto-generates CLI commands based on the FRR YANG models. As a
matter of fact, there are already a number of NETCONF-based CLIs that do
exactly that (e.g. `Clixon <https://github.com/clicon/clixon>`__,
ConfD’s CLI).
The problem however is that there isn’t an exact one-to-one mapping
between the existing CLI commands and the corresponding YANG nodes from
the native models. As an example, ripd’s
``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` command
changes three YANG leaves at the same time. In order to auto-generate
CLI commands and retain their original form, it’s necessary to add
annotations in the YANG modules to specify how the commands should look
like. Without YANG annotations, the CLI auto-generator will generate a
command for each YANG leaf, (leaf-)list and presence-container. The
ripd’s ``timers basic`` command, for instance, would become three
different commands, which would be undesirable.
This Tail-f’s®
`document <http://info.tail-f.com/hubfs/Whitepapers/Tail-f_ConfD-CLI__Cfg_Mode_App_Note_Rev%20C.pdf>`__
shows how to customize ConfD auto-generated CLI commands using YANG
annotations.
The good news is that *libyang* allows users to create plugins to
implement their own YANG extensions, which can be used to implement CLI
annotations. If done properly, a CLI generator can save FRR developers
from writing and maintaining hundreds if not thousands of DEFPYs!
CLI on a separate program
~~~~~~~~~~~~~~~~~~~~~~~~~
The flexible design of the northbound architecture opens the door to
move the CLI to a separate program in the long-term future. Some
advantages of doing so would be: \* Treat the CLI as just another
northbound client, instead of having CLI commands embedded in the
binaries of all FRR daemons. \* Improved robustness: bugs in CLI
commands (e.g. null-pointer dereferences) or in the CLI code itself
wouldn’t affect the FRR daemons. \* Foster innovation by allowing other
CLI programs to be implemented, possibly using higher level programming
languages.
The problem, however, is that the northbound retrofitting process will
convert only the CLI configuration commands and EXEC commands in a first
moment. Retrofitting the “show” commands is a completely different story
and shouldn’t happen anytime soon. This should hinder progress towards
moving the CLI to a separate program.
Proposed feature: confirmed commits
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Confirmed commits allow the user to request an automatic rollback to the
previous configuration if the commit operation is not confirmed within a
number of minutes. This is particularly useful when the user is
accessing the CLI through the network (e.g. using SSH) and any
configuration change might cause an unexpected loss of connectivity
between the user and the router (e.g. misconfiguration of a routing
protocol). By using a confirmed commit, the user can rest assured the
connectivity will be restored after the given timeout expires, avoiding
the need to access the router physically to fix the problem.
Example of how this feature could be provided in the CLI:
``commit confirmed [minutes <1-60>]``. The ability to do confirmed
commits should also be exposed in the northbound API so that the
northbound plugins can also take advantage of it (in the case of the
Sysrepo and ConfD plugins, confirmed commits are implemented externally
in the *netopeer2-server* and *confd* daemons, respectively).
Proposed feature: enable/disable configuration commands/sections
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Since the ``lyd_node`` data structure from *libyang* can hold private
data, it should be possible to mark configuration commands or sections
as active or inactive. This would allow CLI users to leverage this
feature to disable parts of the running configuration without actually
removing the associated commands, and then re-enable the disabled
configuration commands or sections later when necessary. Example:
::
ripd(config)# show configuration running
Configuration:
[snip]
!
router rip
default-metric 2
distance 80
network eth0
network eth1
!
end
ripd(config)# disable router rip
ripd(config)# commit
% Configuration committed successfully (Transaction ID #7).
ripd(config)# show configuration running
Configuration:
[snip]
!
!router rip
!default-metric 2
!distance 80
!network eth0
!network eth1
!
end
ripd(config)# enable router rip
ripd(config)# commit
% Configuration committed successfully (Transaction ID #8).
ripd(config)# show configuration running
[snip]
frr defaults traditional
!
router rip
default-metric 2
distance 80
network eth0
network eth1
!
end
This capability could be useful in a number of occasions, like disabling
configuration commands that are no longer necessary (e.g. ACLs) but that
might be necessary at a later point in the future. Other example is
allowing users to disable a configuration section for testing purposes,
and then re-enable it easily without needing to copy and paste any
command.
Configuration reloads
~~~~~~~~~~~~~~~~~~~~~
Given the limitations of the previous northbound architecture, the FRR
daemons didn’t have the ability to reload their configuration files by
themselves. The SIGHUP handler of most daemons would only re-read the
configuration file and merge it into the running configuration. In most
cases, however, what is desired is to replace the running configuration
by the updated configuration file. The *frr-reload.py* script was
written to work around this problem and it does it well to a certain
extent. The problem with the *frr-reload.py* script is that it’s full of
special cases here and there, which makes it fragile and unreliable.
Maintaining the script is also an additional burden for FRR developers,
few of whom are familiar with its code or know when it needs to be
updated to account for a new feature.
In the new northbound architecture, reloading the configuration file can
be easily implemented using a configuration transaction. Once the FRR
northbound retrofitting process is complete, all daemons should have the
ability to reload their configuration files upon receiving the SIGHUP
signal, or when the ``configuration load [...] replace`` command is
used. Once that point is reached, the *frr-reload.py* script will no
longer be necessary and should be removed from the FRR repository.
Configuration changes coming from the kernel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This
`post <http://discuss.tail-f.com/t/who-should-not-set-configuration-once-a-system-is-up-and-running/111>`__
from the Tail-f’s® forum describes the problem of letting systems
configure themselves behind the users back. Here are some selected
snippets from it: > Traditionally, northbound interface users are the
ones in charge of providing configuration data for systems. > > In some
systems, we see a deviation from this traditional practice; allowing
systems to configure “themselves” behind the scenes (or behind the users
back). > > While there might be a business case for such a practice,
this kind of configuration remains “dangerous” from northbound users
perspective and makes systems hard to predict and even harder to debug.
(…) > > With the advent of transactional Network configuration, this
practice can not work anymore. The fact that systems are given the right
to change configuration is a key here in breaking transactional
configuration in a Network.
FRR is immune to some of the problems described in the aforementioned
post. Management clients can configure interfaces that don’t yet exist,
and once an interface is deleted from the kernel, its configuration is
retained in FRR.
There are however some cases where information learned from the kernel
(e.g. using netlink) can affect the running configuration of all FRR
daemons. Examples: interface rename events, VRF rename events, interface
being moved to a different VRF, etc. In these cases, since these events
can’t be ignored, the best we can do is to send YANG notifications to
the management clients to inform about the configuration changes. The
management clients should then be prepared to handle such notifications
and react accordingly.
Interfaces and VRFs
~~~~~~~~~~~~~~~~~~~
As of now zebra doesn’t have the ability to create VRFs or virtual
interfaces in the kernel. The ``vrf`` and ``interface`` commands only
create pre-provisioned VRFs and interfaces that are only activated when
the corresponding information is learned from the kernel. When
configuring FRR using an external management client, like a NETCONF
client, it might be desirable to actually create functional VRFs and
virtual interfaces (e.g. VLAN subinterfaces, bridges, etc) that are
installed in the kernel using OS-specific APIs (e.g. netlink, routing
socket, etc). Work needs to be done in this area to make this possible.
Shared configuration objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
One of the existing problems in FRR is that it’s hard to ensure that all
daemons are in sync with respect to the shared configuration objects
(e.g. interfaces, VRFs, route-maps, ACLs, etc). When a route-map is
configured using *vtysh*, the same command is sent to all relevant
daemons (the daemons that implement route-maps), which ensures
synchronization among them. The problem is when a daemon starts after
the route-maps are created. In this case this daemon wouldn’t be aware
of the previously configured route-maps (unlike the other daemons),
which can lead to a lot of confusion and unexpected problems.
With the new northbound architecture, configuration objects can be
manipulated using higher level abstractions, which opens more
possibilities to solve this decades-long problem. As an example, one
solution would be to make the FRR daemons fetch the shared configuration
objects from zebra using the ZAPI interface during initialization. The
shared configuration objects could be requested using a list of XPaths
expressions in the ``ZEBRA_HELLO`` message, which zebra would respond by
sending the shared configuration objects encoded in the JSON format.
This solution however doesn’t address the case where zebra starts or
restarts after the other FRR daemons. Other solution would be to store
the shared configuration objects in the northbound SQL database and make
all daemons fetch these objects from there. So far no work has been made
on this area as more investigation needs to be done.
vtysh support
~~~~~~~~~~~~~
As explained in the [[Transactional CLI]] page, all commands introduced
by the transactional CLI are not yet available in *vtysh*. This needs to
be addressed in the short term future. Some challenges for doing that
work include: \* How to display configurations (running, candidates and
rollbacks) in a more clever way? The implementation of the
``show running-config`` command in *vtysh* is not something that should
be followed as an example. A better idea would be to fetch the desired
configuration from all daemons (encoded in JSON for example), merge them
all into a single ``lyd_node`` variable and then display the combined
configurations from this variable (the configuration merges would
transparently take care of combining the shared configuration objects).
In order to be able to manipulate the JSON configurations, *vtysh* will
need to load the YANG modules from all daemons at startup (this might
have a minimal impact on startup time). The only issue with this
approach is that the ``cli_show()`` callbacks from all daemons are
embedded in their binaries and thus not accessible externally. It might
be necessary to compile these callbacks on a separate shared library so
that they are accessible to *vtysh* too. Other than that, displaying the
combined configurations in the JSON/XML formats should be
straightforward. \* With the current design, transaction IDs are
per-daemon and not global across all FRR daemons. This means that the
same transaction ID can represent different transactions on different
daemons. Given this observation, how to implement the
``rollback configuration`` command in *vtysh*? The easy solution would
be to add a ``daemon WORD`` argument to specify the context of the
rollback, but per-daemon rollbacks would certainly be confusing and
convoluted to end users. A better idea would be to attack the root of
the problem: change configuration transactions to be global instead of
being per-daemon. This involves a bigger change in the northbound
architecture, and would have implications on how transactions are stored
in the SQL database (daemon-specific and shared configuration objects
would need to have their own tables or columns). \* Loading
configuration files in the JSON or XML formats will be tricky, as
*vtysh* will need to know which sections of the configuration should be
sent to which daemons. *vtysh* will either need to fetch the YANG
modules implemented by all daemons at runtime or obtain this information
at compile-time somehow.
Detecting type mismatches at compile-time
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As described in the [[Retrofitting Configuration Commands]] page, the
northbound configuration callbacks detect type mismatches at runtime
when fetching data from the the ``dnode`` parameter (which represents
the configuration node being created, modified, deleted or moved). When
a type mismatch is detected, the program aborts and displays a backtrace
showing where the problem happened. It would be desirable to detect such
type mismatches at compile-time, the earlier the problems are detected
the sooner they are fixed.
One possible solution to this problem would be to auto-generate C
structures from the YANG models and provide a function that converts a
libyang’s ``lyd_node`` variable to a C structure containing the same
information. The northbound callbacks could then fetch configuration
data from this C structure, which would naturally lead to type
mismatches being detected at compile time. One of the challenges of
doing this would be the handling of YANG lists and leaf-lists. It would
be necessary to use dynamic data structures like hashes or rb-trees to
hold all elements of the lists and leaf-lists, and the process of
converting a ``lyd_node`` to an auto-generated C-structure could be
expensive. At this point it’s unclear if it’s worth adding more
complexity in the northbound architecture to solve this specific
problem.

283
doc/developer/northbound/architecture.rst Normal file
View File

@ -0,0 +1,283 @@
Introduction
------------
The goal of the new northbound API is to provide a better interface to
configure and monitor FRR programatically. The current design based on
CLI commands is no longer adequate in a world where computer networks
are becoming increasingly bigger, more diverse and more complex. Network
scripting using *expect* and screen scraping techniques is too primitive
and unreliable to be used in large-scale networks. What is proposed is
to modernize FRR to turn it into an API-first routing stack, and
reposition the CLI on top of this API. The most important change,
however, is not the API that will be provided to external users. In
fact, multiple APIs will be supported and users will have the ability to
write custom management APIs if necessary. The biggest change is the
introduction of a model-driven management architecture based on the
`YANG <https://tools.ietf.org/html/rfc7950>`__ modeling language.
Instead of writing code tied to any particular user interface
(e.g. DEFUNs), YANG allows us to write API-agnostic code (in the form of
callbacks) that can be used by any management interface. As an example,
it shouldn’t matter if a set of configuration changes is coming from a
`NETCONF <https://tools.ietf.org/html/rfc6241>`__ session or from a CLI
terminal, the same callbacks should be called to process the
configuration changes regardless of where they came from. This
model-driven design ensures feature parity across all management
interfaces supported by FRR.
Quoting RFC 7950: > YANG is a language originally designed to model data
for the NETCONF protocol. A YANG module defines hierarchies of data that
can be used for NETCONF-based operations, including configuration, state
data, RPCs, and notifications. This allows a complete description of all
data sent between a NETCONF client and server. Although out of scope for
this specification, YANG can also be used with protocols other than
NETCONF.
While the YANG and NETCONF specifications are tightly coupled with one
another, both are independent to a certain extent and are evolving
separately. Examples of other management protocols that use YANG include
`RESTCONF <https://tools.ietf.org/html/rfc8040>`__,
`gNMI <https://github.com/openconfig/reference/tree/master/rpc/gnmi>`__
and
`CoAP <https://www.ietf.org/archive/id/draft-vanderstok-core-comi-11.txt>`__.
In addition to being management-protocol independent, some other
advantages of using YANG in FRR are listed below: \* Have a formal
contract between FRR and application developers (management clients). A
management client that has access to the FRR YANG models knows about all
existing configuration options available for use. This information can
be used to auto-generate user-friendly interfaces like Web-UIs, custom
CLIs and even code bindings for several different programming languages.
Using `PyangBind <https://github.com/robshakir/pyangbind>`__, for
example, it’s possible to generate Python class hierarchies from YANG
models and use these classes to instantiate objects that mirror the
structure of the YANG modules and can be serialized/deserialized using
different encoding formats. \* Support different encoding formats for
instance data. Currently only JSON and XML are supported, but
`GPB <https://developers.google.com/protocol-buffers/>`__ and
`CBOR <http://cbor.io/>`__ are other viable options in the long term.
Additional encoding formats can be implemented in the *libyang* library
for optimal performance, or externally by translating data to/from one
of the supported formats (with a performance penalty). \* Have a formal
mechanism to introduce backward-incompatible changes based on `semantic
versioning <http://www.openconfig.net/docs/semver/>`__ (not part of the
YANG standard, which allows backward-compatible module updates only). \*
Provide seamless support to the industry-standard NETCONF/RESTCONF
protocols as alternative management APIs. If FRR configuration/state
data is modeled using YANG, supporting YANG-based protocols like NETCONF
and RESTCONF is much easier.
As important as shifting to a model-driven management paradigm, the new
northbound architecture also introduces the concept of configuration
transactions. Configuration transactions allow management clients to
commit multiple configuration changes at the same time and rest assured
that either all changes will be applied or none will (all-or-nothing).
Configuration transactions are implemented as pseudo-atomic operations
and facilitate automation by removing the burden of error recovery from
the management side. Another property of configuration transactions is
that the configuration changes are always processed in a pre-defined
order to ensure consistency. Configuration transactions that encompass
multiple network devices are called network-wide transactions and are
also supported by the new northbound architecture. When FRR is built
using the ``--enable-config-rollbacks`` option, all committed
transactions are recorded in the FRR rollback log, which can reside
either in memory (volatile) or on persistent storage.
Network-wide Transactions is the most important leap in network
management technology since SNMP. The error recovery and sequencing
tasks are removed from the manager side. This is usually more than
half the cost in a mature system; more than the entire cost of the
managed devices.
`[source] <https://www.nanog.org/sites/default/files/tuesday_tutorial_moberg_netconf_35.pdf>`__.
Figures 1 and 2 below illustrate the old and new northbound architecture
of FRR, respectively. As it can be seen, in the old architecture the CLI
was the only interface used to configure and monitor FRR (the SNMP
plugin was’t taken into account given the small number of implemented
MIBs). This means that the only way to automate FRR was by writing
scripts that send CLI commands and parse the text output (which usually
doesn’t have any structure) using screen scraping and regular
expressions.
+-----------------------------------------+
| |space-1.jpg| |
+=========================================+
| *Figure 1: old northbound architecture* |
+-----------------------------------------+
The new northbound architectures, on the other hand, features a
multitude of different management APIs, all of them connected to the
northbound layer of the FRR daemons. By default, only the CLI interface
is compiled built-in in the FRR daemons. The other management interfaces
are provided as optional plugins and need to be loaded during the daemon
initialization (e.g. *zebra -M confd*). This design makes it possible to
integrate FRR with different NETCONF solutions without introducing
vendor lock-in. The [[Plugins - Writing Your Own]] page explains how to
write custom northbound plugins that can be tailored to all needs
(e.g. support custom transport protocols, different data encoding
formats, fine-grained access control, etc).
+-----------------------------------------+
| |space-1.jpg| |
+=========================================+
| *Figure 2: new northbound architecture* |
+-----------------------------------------+
Figure 3 shows the internal view of the FRR northbound architecture. In
this image we can see that northbound layer is an abstract entity
positioned between the northbound callbacks and the northbound clients.
The northbound layer is responsible to process the requests coming from
the northbound clients and call the appropriate callbacks to satisfy
these requests. The northbound plugins communicate with the northbound
layer through a public API, which allow users to write third-party
plugins that can be maintained separately. The northbound plugins, in
turn, have their own APIs to communicate with external management
clients.
+---------------------------------------------------------+
| |space-1.jpg| |
+=========================================================+
| *Figure 3: new northbound architecture - internal view* |
+---------------------------------------------------------+
Initially the CLI (and all of its commands) will be maintained inside
the FRR daemons. In the long term, however, the goal is to move the CLI
to a separate program just like any other management client. The
[[Advanced Topics]] page describes the motivations and challenges of
doing that. Last but not least, the *libyang* block inside the
northbound layer is the engine that makes everything possible. The
*libyang* library will be described in more detail in the following
sections.
YANG models
-----------
The main decision to be made when using YANG is which models to
implement. There’s a general consensus that using standard models is
preferable over using custom (native) models. The reasoning is that
applications based on standard models can be reused for all network
appliances that support those models, whereas the same doesn’t apply for
applications written based on custom models.
That said, there are multiple standards bodies publishing YANG models
and unfortunately not all of them are converging (or at least not yet).
In the context of FRR, which is a routing stack, the two sets of YANG
models that would make sense to implement are the ones from IETF and
from the OpenConfig working group. The question that arises is: which
one of them should we commit to? Or should we try to support both
somehow, at the cost of extra development efforts?
Another problem, from an implementation point of view, is that it’s
challenging to adapt the existing code base to match standard models. A
more reasonable solution, at least in a first moment, would be to use
YANG deviations and augmentations to do the opposite: adapt the standard
models to the existing code. In practice however this is not as simple
as it seems. There are cases where the differences are too substantial
to be worked around without restructuring the code by changing its data
structures and their relationships. As an example, the *ietf-rip* model
places per-interface RIP configuration parameters inside the
*control-plane-protocol* list (which is augmented by *ietf-rip*). This
means that it’s impossible to configure RIP interface parameters without
first configuring a RIP routing instance. The *ripd* daemon on the other
hand allows the operator to configure RIP interface parameters even if
``router rip`` is not configured. If we were to implement the *ietf-rip*
module natively, we’d need to change ripd’s CLI commands (and the
associated code) to reflect the new configuration hierarchy.
Taking into account that FRR has a huge code base and that the
northbound retrofitting process per-se will cause a lot of impact, it
was decided to take a conservative approach and write custom YANG models
for FRR modeled after the existing CLI commands. Having YANG models that
closely mirror the CLI commands will allow the FRR developers to
retrofit the code base much more easily, without introducing
backward-incompatible changes in the CLI and reducing the likelihood of
introducing bugs. The [[Retrofitting Configuration Commands]] page
explains in detail how to convert configuration commands to the new
northbound model.
Even though having native YANG models is not the ideal solution, it will
be already a big step forward for FRR to migrate to a model-driven
management architecture, with support for configuration transactions and
multiple management interfaces, including NETCONF and RESTCONF (through
the northbound plugins).
The new northbound also features an experimental YANG module translator
that will allow users to translate to and from standard YANG models by
using translation tables. The [[YANG module translator]] page describes
this mechanism in more detail. At this point it’s unclear what can be
achieved through module translation and if that can be considered as a
definitive solution to support standard models or not.
Northbound Architecture
-----------------------
+-----------------------------------------------+
| |space-1.jpg| |
+===============================================+
| *Figure 4: libyang’s lys_node data structure* |
+-----------------------------------------------+
+-----------------------------------------------+
| |space-1.jpg| |
+===============================================+
| *Figure 5: libyang’s lyd_node data structure* |
+-----------------------------------------------+
+---------------------------------------------+
| |space-1.jpg| |
+=============================================+
| *Figure 6: libyang’s ly_ctx data structure* |
+---------------------------------------------+
+----------------------------------------+
| |space-1.jpg| |
+========================================+
| *Figure 7: configuration transactions* |
+----------------------------------------+
Testing
-------
The new northbound adds the libyang library as a new mandatory
dependency for FRR. To obtain and install this library, follow the steps
below:
::
$ git clone https://github.com/CESNET/libyang
$ cd libyang
$ git checkout devel
$ mkdir build ; cd build
$ cmake -DENABLE_LYD_PRIV=ON ..
$ make
$ sudo make install
..
NOTE: first make sure to install the libyang
`requirements <https://github.com/CESNET/libyang#build-requirements>`__.
FRR needs libyang from version 0.16.7 or newer, which is maintained in
the ``devel`` branch. libyang 0.15.x is maintained in the ``master``
branch and doesn’t contain one small feature used by FRR (the
``LY_CTX_DISABLE_SEARCHDIR_CWD`` flag). FRR also makes use of the
libyang’s ``ENABLE_LYD_PRIV`` feature, which is disabled by default and
needs to be enabled at compile time.
It’s advisable (but not required) to install sqlite3 and build FRR with
``--enable-config-rollbacks`` in order to have access to the
configuration rollback feature.
To test the northbound, the suggested method is to use the
[[Transactional CLI]] with the *ripd* daemon and play with the new
commands. The ``debug northbound`` command can be used to see which
northbound callbacks are called in response to the ``commit`` command.
For reference, the [[Demos]] page shows a small demonstration of the
transactional CLI in action and what it’s capable of.
.. |space-1.jpg| image:: https://s22.postimg.cc/se52j8awh/arch-before.png
.. |space-1.jpg| image:: https://s22.postimg.cc/fziaiwboh/arch-after.png
.. |space-1.jpg| image:: https://s22.postimg.cc/qmc3ocmep/nb-layer.png
.. |space-1.jpg| image:: https://s22.postimg.cc/z4ljsodht/lys_node.png
.. |space-1.jpg| image:: https://s22.postimg.cc/6eynw1h7l/lyd_node.png
.. |space-1.jpg| image:: https://s22.postimg.cc/5cohdhiyp/ly_ctx.png
.. |space-1.jpg| image:: https://s22.postimg.cc/8waf3bgjl/transactions.png

25
doc/developer/northbound/demos.rst Normal file
View File

@ -0,0 +1,25 @@
Transactional CLI
-----------------
This short demo shows some of the capabilities of the new transactional
CLI: |asciicast|
ConfD + NETCONF + Cisco YDK
---------------------------
This is a very simple demo of *ripd* being configured by a python
script. The script uses NETCONF to communicate with *ripd*, which has
the ConfD plugin loaded. The most interesting part, however, is the fact
that the python script is not using handcrafted XML payloads to
configure *ripd*. Instead, the script is using python bindings generated
using Cisco’s YANG Development Kit (YDK).
- Script used in the demo:
https://gist.github.com/rwestphal/defa9bd1ccf216ab082d4711ae402f95
|asciicast|
.. |asciicast| image:: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1.png
:target: https://asciinema.org/a/jL0BS5HfP2kS6N1HfgsZvfZk1
.. |asciicast| image:: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv.png
:target: https://asciinema.org/a/VfMElNxsjLcdvV7484E6ChxWv

233
doc/developer/northbound/links.rst Normal file
View File

@ -0,0 +1,233 @@
RFCs
~~~~
- `RFC 7950 - The YANG 1.1 Data Modeling
Language <https://tools.ietf.org/html/rfc7950>`__
- `RFC 7951 - JSON Encoding of Data Modeled with
YANG <https://tools.ietf.org/html/rfc7951>`__
- `RFC 8342 - Network Management Datastore Architecture
(NMDA) <https://tools.ietf.org/html/rfc8342>`__
- `RFC 6087 - Guidelines for Authors and Reviewers of YANG Data Model
Documents <https://tools.ietf.org/html/rfc6087>`__
- `RFC 8340 - YANG Tree
Diagrams <https://tools.ietf.org/html/rfc8340>`__
- `RFC 6991 - Common YANG Data
Types <https://tools.ietf.org/html/rfc6991>`__
- `RFC 6241 - Network Configuration Protocol
(NETCONF) <https://tools.ietf.org/html/rfc6241>`__
- `RFC 8040 - RESTCONF
Protocol <https://tools.ietf.org/html/rfc8040>`__
YANG models
~~~~~~~~~~~
- Collection of several YANG models, including models from standards
organizations such as the IETF and vendor specific models:
https://github.com/YangModels/yang
- OpenConfig: https://github.com/openconfig/public
Presentations
~~~~~~~~~~~~~
- FRR Advanced Northbound API (May 2018)
- Slides:
https://www.dropbox.com/s/zhybthruwocbqaw/netdef-frr-northbound.pdf?dl=1
- Ok, We Got Data Models, Now What?
- Video: https://www.youtube.com/watch?v=2oqkiZ83vAA
- Slides:
https://www.nanog.org/sites/default/files/20161017_Alvarez_Ok_We_Got_v1.pdf
- Data Model-Driven Management: Latest Industry and Tool Developments
- Video: https://www.youtube.com/watch?v=n_oKGJ_jgYQ
- Slides:
https://pc.nanog.org/static/published/meetings/NANOG72/1559/20180219_Claise_Data_Modeling-Driven_Management__v1.pdf
- Network Automation And Programmability: Reality Versus The Vendor
Hype When Considering Legacy And NFV Networks
- Video: https://www.youtube.com/watch?v=N5wbYncUS9o
- Slides:
https://www.nanog.org/sites/default/files/1_Moore_Network_Automation_And_Programmability.pdf
- Lightning Talk: The API is the new CLI?
- Video: https://www.youtube.com/watch?v=ngi0erGNi58
- Slides:
https://pc.nanog.org/static/published/meetings/NANOG72/1638/20180221_Grundemann_Lightning_Talk_The_v1.pdf
- Lightning Talk: OpenConfig - progress toward vendor-neutral network
management
- Video: https://www.youtube.com/watch?v=10rSUbeMmT4
- Slides:
https://pc.nanog.org/static/published/meetings/NANOG71/1535/20171004_Shaikh_Lightning_Talk_Openconfig_v1.pdf
- Getting started with OpenConfig
- Video: https://www.youtube.com/watch?v=L7trUNK8NJI
- Slides:
https://pc.nanog.org/static/published/meetings/NANOG71/1456/20171003_Alvarez_Getting_Started_With_v1.pdf
- Why NETCONF and YANG
- Video: https://www.youtube.com/watch?v=mp4h8aSTba8
- NETCONF and YANG Concepts
- Video: https://www.youtube.com/watch?v=UwYYvT7DBvg
- NETCONF Tutorial
- Video: https://www.youtube.com/watch?v=N4vov1mI14U
Whitepapers
~~~~~~~~~~~
- Automating Network and Service Configuration Using NETCONF and YANG:
http://www.tail-f.com/wordpress/wp-content/uploads/2013/02/Tail-f-Presentation-Netconf-Yang.pdf
- Creating the Programmable Network: The Business Case for NETCONF/YANG
in Network Devices:
http://www.tail-f.com/wordpress/wp-content/uploads/2013/10/HR-Tail-f-NETCONF-WP-10-08-13.pdf
- NETCONF/YANG: What’s Holding Back Adoption & How to Accelerate It:
https://www.oneaccess-net.com/images/public/wp_heavy_reading.pdf
- Achieving Automation with YANG Modeling Technologies:
https://www.cisco.com/c/dam/en/us/products/collateral/cloud-systems-management/network-services-orchestrator/idc-achieving-automation-wp.pdf
Blog posts and podcasts
~~~~~~~~~~~~~~~~~~~~~~~
- OpenConfig and IETF YANG Models: Can they converge? -
http://rob.sh/post/215/
- OpenConfig: Standardized Models For Networking -
https://packetpushers.net/openconfig-standardized-models-networking/
- (Podcast) OpenConfig: From Basics to Implementations -
https://blog.ipspace.net/2017/02/openconfig-from-basics-to.html
- (Podcast) How Did NETCONF Start on Software Gone Wild -
https://blog.ipspace.net/2017/12/how-did-netconf-start-on-software-gone.html
- YANG Data Models in the Industry: Current State of Affairs (March
2018) -
https://www.claise.be/2018/03/yang-data-models-in-the-industry-current-state-of-affairs-march-2018/
- Why Data Model-driven Telemetry is the only useful Telemetry? -
https://www.claise.be/2018/02/why-data-model-driven-telemetry-is-the-only-useful-telemetry/
- NETCONF versus RESTCONF: Capabilitity Comparisons for Data
Model-driven Management -
https://www.claise.be/2017/10/netconf-versus-restconf-capabilitity-comparisons-for-data-model-driven-management-2/
- An Introduction to NETCONF/YANG -
https://www.fir3net.com/Networking/Protocols/an-introduction-to-netconf-yang.html
- Network Automation and the Rise of NETCONF -
https://medium.com/@k.okasha/network-automation-and-the-rise-of-netconf-e96cc33fe28
- YANG and the Road to a Model Driven Network -
https://medium.com/@k.okasha/yang-and-road-to-a-model-driven-network-e9e52d47148d
Software
~~~~~~~~
libyang
^^^^^^^
libyang is a YANG data modelling language parser and toolkit written
(and providing API) in C.
- GitHub page: https://github.com/CESNET/libyang
- Documentaion: https://netopeer.liberouter.org/doc/libyang/master/
pyang
^^^^^
pyang is a YANG validator, transformator and code generator, written
in python. It can be used to validate YANG modules for correctness,
to transform YANG modules into other formats, and to generate code
from the modules.
- GitHub page: https://github.com/mbj4668/pyang
- Documentaion: https://github.com/mbj4668/pyang/wiki/Documentation
ncclient
^^^^^^^^
ncclient is a Python library that facilitates client-side scripting
and application development around the NETCONF protocol.
- GitHub page: https://github.com/ncclient/ncclient
- Documentaion: https://ncclient.readthedocs.io/en/latest/
YDK
^^^
ydk-gen is a developer tool that can generate API’s that are modeled
in YANG. Currently, it generates language binding for Python, Go and
C++ with planned support for other language bindings in the future.
- GitHub pages:
- Generator: https://github.com/CiscoDevNet/ydk-gen
- Python: https://github.com/CiscoDevNet/ydk-py
- Python samples: https://github.com/CiscoDevNet/ydk-py-samples
- Go: https://github.com/CiscoDevNet/ydk-go
- C++: https://github.com/CiscoDevNet/ydk-cpp
- Documentation:
- Python: http://ydk.cisco.com/py/docs/
- Go: http://ydk.cisco.com/go/docs/
- C++: http://ydk.cisco.com/cpp/docs/
- (Blog post) Simplifying Network Programmability with Model-Driven
APIs:
https://blogs.cisco.com/sp/simplifying-network-programmability-with-model-driven-apis
- (Video introduction) Infrastructure as a Code Using YANG, OpenConfig
and YDK: https://www.youtube.com/watch?v=G1b6vJW1R5w
pyangbind
^^^^^^^^^
A plugin for pyang that creates Python bindings for a YANG model.
- GitHub page: https://github.com/robshakir/pyangbind
- Documentation: http://pynms.io/pyangbind/
ConfD
^^^^^
- Official webpage (for ConfD Basic):
http://www.tail-f.com/confd-basic/
- Training Videos: http://www.tail-f.com/confd-training-videos/
- Forum: http://discuss.tail-f.com/
Sysrepo
^^^^^^^
Sysrepo is an YANG-based configuration and operational state data
store for Unix/Linux applications.
- GitHub page: https://github.com/sysrepo/sysrepo
- Official webpage: http://www.sysrepo.org/
- Documentation: http://www.sysrepo.org/static/doc/html/
Netopeer2
^^^^^^^^^
Netopeer2 is a set of tools implementing network configuration tools
based on the NETCONF Protocol. This is the second generation of the
toolset, originally available as the Netopeer project. Netopeer2 is
based on the new generation of the NETCONF and YANG libraries -
libyang and libnetconf2. The Netopeer server uses sysrepo as a
NETCONF datastore implementation.
- GitHub page: https://github.com/CESNET/Netopeer2
Clixon
^^^^^^
Clixon is an automatic configuration manager where you generate
interactive CLI, NETCONF, RESTCONF and embedded databases with
transaction support from a YANG specification.
- GitHub page: https://github.com/clicon/clixon
- Project page: http://www.clicon.org/

21
doc/developer/northbound/northbound.rst Normal file
View File

@ -0,0 +1,21 @@
.. _northbound:
**************
Northbound API
**************
.. toctree::
:maxdepth: 2
advanced-topics
architecture
demos
links
operational-data-rpcs-and-notifications
plugins-sysrepo
ppr-basic-test-topology
ppr-mpls-basic-test-topology
retrofitting-configuration-commands
transactional-cli
yang-module-translator
yang-tools

565
doc/developer/northbound/operational-data-rpcs-and-notifications.rst Normal file
View File

@ -0,0 +1,565 @@
Operational data
~~~~~~~~~~~~~~~~
Writing API-agnostic code for YANG-modeled operational data is
challenging. ConfD and Sysrepo, for instance, have completely different
APIs to fetch operational data. So how can we write API-agnostic
callbacks that can be used by both the ConfD and Sysrepo plugins, and
any other northbound client that might be written in the future?
As an additional requirement, the callbacks must be designed in a way
that makes in-place XPath filtering possible. As an example, a
management client might want to retrieve only a subset of a large YANG
list (e.g. a BGP table), and for optimal performance it should be
possible to filter out the unwanted elements locally in the managed
devices instead of returning all elements and performing the filtering
on the management application.
To meet all these requirements, the four callbacks below were introduced
in the northbound architecture:
.. code:: c
/*
* Operational data callback.
*
* The callback function should return the value of a specific leaf or
* inform if a typeless value (presence containers or leafs of type
* empty) exists or not.
*
* xpath
* YANG data path of the data we want to get
*
* list_entry
* pointer to list entry
*
* Returns:
* pointer to newly created yang_data structure, or NULL to indicate
* the absence of data
*/
struct yang_data *(*get_elem)(const char *xpath, void *list_entry);
/*
* Operational data callback for YANG lists.
*
* The callback function should return the next entry in the list. The
* 'list_entry' parameter will be NULL on the first invocation.
*
* list_entry
* pointer to a list entry
*
* Returns:
* pointer to the next entry in the list, or NULL to signal that the
* end of the list was reached
*/
void *(*get_next)(void *list_entry);
/*
* Operational data callback for YANG lists.
*
* The callback function should fill the 'keys' parameter based on the
* given list_entry.
*
* list_entry
* pointer to a list entry
*
* keys
* structure to be filled based on the attributes of the provided
* list entry
*
* Returns:
* NB_OK on success, NB_ERR otherwise
*/
int (*get_keys)(void *list_entry, struct yang_list_keys *keys);
/*
* Operational data callback for YANG lists.
*
* The callback function should return a list entry based on the list
* keys given as a parameter.
*
* keys
* structure containing the keys of the list entry
*
* Returns:
* a pointer to the list entry if found, or NULL if not found
*/
void *(*lookup_entry)(struct yang_list_keys *keys);
These callbacks were designed to provide maximum flexibility, and borrow
a lot of ideas from the ConfD API. Each callback does one and only one
task, they are indivisible primitives that can be combined in several
different ways to iterate over operational data. The extra flexibility
certainly has a performance cost, but it’s the price to pay if we want
to expose FRR operational data using several different management
interfaces (e.g. NETCONF via either ConfD or Sysrepo+Netopeer2). In the
future it might be possible to introduce optional callbacks that do
things like returning multiple objects at once. They would provide
enhanced performance when iterating over large lists, but their use
would be limited by the northbound plugins that can be integrated with
them.
NOTE: using the northbound callbacks as a base, the ConfD plugin can
provide up to 100 objects between each round trip between FRR and the
*confd* daemon. Preliminary tests showed FRR taking ~7 seconds
(asynchronously, without blocking the main pthread) to return a RIP
table containing 100k routes to a NETCONF client connected to *confd*
(JSON was used as the encoding format). Work needs to be done to find
the bottlenecks and optimize this operation.
The [[Plugins - Writing Your Own]] page explains how the northbound
plugins can fetch operational data using the aforementioned northbound
callbacks, and how in-place XPath filtering can be implemented.
Example
^^^^^^^
Now let’s move to an example to show how these callbacks are implemented
in practice. The following YANG container is part of the *ietf-rip*
module and contains operational data about RIP neighbors:
.. code:: yang
container neighbors {
description
"Neighbor information.";
list neighbor {
key "address";
description
"A RIP neighbor.";
leaf address {
type inet:ipv4-address;
description
"IP address that a RIP neighbor is using as its
source address.";
}
leaf last-update {
type yang:date-and-time;
description
"The time when the most recent RIP update was
received from this neighbor.";
}
leaf bad-packets-rcvd {
type yang:counter32;
description
"The number of RIP invalid packets received from
this neighbor which were subsequently discarded
for any reason (e.g. a version 0 packet, or an
unknown command type).";
}
leaf bad-routes-rcvd {
type yang:counter32;
description
"The number of routes received from this neighbor,
in valid RIP packets, which were ignored for any
reason (e.g. unknown address family, or invalid
metric).";
}
}
}
We know that this is operational data because the ``neighbors``
container is within the ``state`` container, which has the
``config false;`` property (which is applied recursively).
As expected, the ``gen_northbound_callbacks`` tool also generates
skeleton callbacks for nodes that represent operational data:
.. code:: c
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor",
.cbs.get_next = ripd_state_neighbors_neighbor_get_next,
.cbs.get_keys = ripd_state_neighbors_neighbor_get_keys,
.cbs.lookup_entry = ripd_state_neighbors_neighbor_lookup_entry,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/address",
.cbs.get_elem = ripd_state_neighbors_neighbor_address_get_elem,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/last-update",
.cbs.get_elem = ripd_state_neighbors_neighbor_last_update_get_elem,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd",
.cbs.get_elem = ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd",
.cbs.get_elem = ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem,
},
The ``/frr-ripd:ripd/state/neighbors/neighbor`` list within the
``neighbors`` container has three different callbacks that need to be
implemented. Let’s start with the first one, the ``get_next`` callback:
.. code:: c
static void *ripd_state_neighbors_neighbor_get_next(void *list_entry)
{
struct listnode *node;
if (list_entry == NULL)
node = listhead(peer_list);
else
node = listnextnode((struct listnode *)list_entry);
return node;
}
Given a list entry, the job of this callback is to find the next element
from the list. When the ``list_entry`` parameter is NULL, then the first
element of the list should be returned.
*ripd* uses the ``rip_peer`` structure to represent RIP neighbors, and
the ``peer_list`` global variable (linked list) is used to store all RIP
neighbors.
In order to be able to iterate over the list of RIP neighbors, the
callback returns a ``listnode`` variable instead of a ``rip_peer``
variable. The ``listnextnode`` macro can then be used to find the next
element from the linked list.
Now the second callback, ``get_keys``:
.. code:: c
static int ripd_state_neighbors_neighbor_get_keys(void *list_entry,
struct yang_list_keys *keys)
{
struct listnode *node = list_entry;
struct rip_peer *peer = listgetdata(node);
keys->num = 1;
(void)inet_ntop(AF_INET, &peer->addr, keys->key[0].value,
sizeof(keys->key[0].value));
return NB_OK;
}
This one is easy. First, we obtain the RIP neighbor from the
``listnode`` structure. Then, we fill the ``keys`` parameter according
to the attributes of the RIP neighbor. In this case, the ``neighbor``
YANG list has only one key: the neighbor IP address. We then use the
``inet_ntop()`` function to transform this binary IP address into a
string (the lingua franca of the FRR northbound).
The last callback for the ``neighbor`` YANG list is the ``lookup_entry``
callback:
.. code:: c
static void *
ripd_state_neighbors_neighbor_lookup_entry(struct yang_list_keys *keys)
{
struct in_addr address;
yang_str2ipv4(keys->key[0].value, &address);
return rip_peer_lookup(&address);
}
This callback is the counterpart of the ``get_keys`` callback: given an
array of list keys, the associated list entry should be returned. The
``yang_str2ipv4()`` function is used to convert the list key (an IP
address) from a string to an ``in_addr`` structure. Then the
``rip_peer_lookup()`` function is used to find the list entry.
Finally, each YANG leaf inside the ``neighbor`` list has its associated
``get_elem`` callback:
.. code:: c
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/address
*/
static struct yang_data *
ripd_state_neighbors_neighbor_address_get_elem(const char *xpath,
void *list_entry)
{
struct rip_peer *peer = list_entry;
return yang_data_new_ipv4(xpath, &peer->addr);
}
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/last-update
*/
static struct yang_data *
ripd_state_neighbors_neighbor_last_update_get_elem(const char *xpath,
void *list_entry)
{
/* TODO: yang:date-and-time is tricky */
return NULL;
}
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd
*/
static struct yang_data *
ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem(const char *xpath,
void *list_entry)
{
struct rip_peer *peer = list_entry;
return yang_data_new_uint32(xpath, peer->recv_badpackets);
}
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd
*/
static struct yang_data *
ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem(const char *xpath,
void *list_entry)
{
struct rip_peer *peer = list_entry;
return yang_data_new_uint32(xpath, peer->recv_badroutes);
}
These callbacks receive the list entry as parameter and return the
corresponding data using the ``yang_data_new_*()`` wrapper functions.
Not much to explain here.
Iterating over operational data without blocking the main pthread
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
One of the problems we have in FRR is that some “show” commands in the
CLI can take too long, potentially long enough to the point of
triggering some protocol timeouts and bringing sessions down.
To avoid this kind of problem, northbound clients are encouraged to do
one of the following: \* Create a separate pthread for handling requests
to fetch operational data. \* Iterate over YANG lists and leaf-lists
asynchronously, returning a maximum number of elements per time instead
of returning all elements in one shot.
In order to handle both cases correctly, the ``get_next`` callbacks need
to use locks to prevent the YANG lists from being modified while they
are being iterated over. If that is not done, the list entry returned by
this callback can become a dangling pointer when used in another
callback.
Currently the ConfD and Sysrepo plugins run only in the main pthread.
The plan in the short-term is to introduce a separate pthread only for
handling operational data, and use the main pthread only for handling
configuration changes, RPCs and notifications.
RPCs and Actions
~~~~~~~~~~~~~~~~
The FRR northbound supports YANG RPCs and Actions through the ``rpc()``
callback, which is documented as follows in the *lib/northbound.h* file:
.. code:: c
/*
* RPC and action callback.
*
* Both 'input' and 'output' are lists of 'yang_data' structures. The
* callback should fetch all the input parameters from the 'input' list,
* and add output parameters to the 'output' list if necessary.
*
* xpath
* xpath of the YANG RPC or action
*
* input
* read-only list of input parameters
*
* output
* list of output parameters to be populated by the callback
*
* Returns:
* NB_OK on success, NB_ERR otherwise
*/
int (*rpc)(const char *xpath, const struct list *input,
struct list *output);
Note that the same callback is used for both RPCs and actions, which are
essentially the same thing. In the case of YANG actions, the ``xpath``
parameter can be consulted to find the data node associated to the
operation.
As part of the northbound retrofitting process, it’s suggested to model
some EXEC-level commands using YANG so that their functionality is
exposed to other management interfaces other than the CLI. As an
example, if the ``clear bgp`` command is modeled using a YANG RPC, and a
corresponding ``rpc`` callback is written, then it should be possible to
clear BGP neighbors using NETCONF and RESTCONF with that RPC (the ConfD
and Sysrepo plugins have full support for YANG RPCs and actions).
Here’s an example of a very simple RPC modeled using YANG:
.. code:: yang
rpc clear-rip-route {
description
"Clears RIP routes from the IP routing table and routes
redistributed into the RIP protocol.";
}
This RPC doesn’t have any input or output parameters. Below we can see
the implementation of the corresponding ``rpc`` callback, whose skeleton
was automatically generated by the ``gen_northbound_callbacks`` tool:
.. code:: c
/*
* XPath: /frr-ripd:clear-rip-route
*/
static int clear_rip_route_rpc(const char *xpath, const struct list *input,
struct list *output)
{
struct route_node *rp;
struct rip_info *rinfo;
struct list *list;
struct listnode *listnode;
/* Clear received RIP routes */
for (rp = route_top(rip->table); rp; rp = route_next(rp)) {
list = rp->info;
if (list == NULL)
continue;
for (ALL_LIST_ELEMENTS_RO(list, listnode, rinfo)) {
if (!rip_route_rte(rinfo))
continue;
if (CHECK_FLAG(rinfo->flags, RIP_RTF_FIB))
rip_zebra_ipv4_delete(rp);
break;
}
if (rinfo) {
RIP_TIMER_OFF(rinfo->t_timeout);
RIP_TIMER_OFF(rinfo->t_garbage_collect);
listnode_delete(list, rinfo);
rip_info_free(rinfo);
}
if (list_isempty(list)) {
list_delete_and_null(&list);
rp->info = NULL;
route_unlock_node(rp);
}
}
return NB_OK;
}
If the ``clear-rip-route`` RPC had any input parameters, they would be
available in the ``input`` list given as a parameter to the callback.
Similarly, the ``output`` list can be used to append output parameters
generated by the RPC, if any are defined in the YANG model.
The northbound clients (CLI and northbound plugins) have the
responsibility to create and delete the ``input`` and ``output`` lists.
However, in the cases where the RPC or action doesn’t have any input or
output parameters, the northbound client can pass NULL pointers to the
``rpc`` callback to avoid creating linked lists unnecessarily. We can
see this happening in the example below:
.. code:: c
/*
* XPath: /frr-ripd:clear-rip-route
*/
DEFPY (clear_ip_rip,
clear_ip_rip_cmd,
"clear ip rip",
CLEAR_STR
IP_STR
"Clear IP RIP database\n")
{
return nb_cli_rpc("/frr-ripd:clear-rip-route", NULL, NULL);
}
``nb_cli_rpc()`` is a helper function that merely finds the appropriate
``rpc`` callback based on the XPath provided in the first argument, and
map the northbound error code from the ``rpc`` callback to a vty error
code (e.g. ``CMD_SUCCESS``, ``CMD_WARNING``). The second and third
arguments provided to the function refer to the ``input`` and ``output``
lists. In this case, both arguments are set to NULL since the YANG RPC
in question doesn’t have any input/output parameters.
Notifications
~~~~~~~~~~~~~
YANG notifations are sent using the ``nb_notification_send()`` function,
documented in the *lib/northbound.h* file as follows:
.. code:: c
/*
* Send a YANG notification. This is a no-op unless the 'nb_notification_send'
* hook was registered by a northbound plugin.
*
* xpath
* xpath of the YANG notification
*
* arguments
* linked list containing the arguments that should be sent. This list is
* deleted after being used.
*
* Returns:
* NB_OK on success, NB_ERR otherwise
*/
extern int nb_notification_send(const char *xpath, struct list *arguments);
The northbound doesn’t use callbacks for notifications because
notifications are generated locally and sent to the northbound clients.
This way, whenever a notification needs to be sent, it’s possible to
call the appropriate function directly instead of finding a callback
based on the XPath of the YANG notification.
As an example, the *ietf-rip* module contains the following
notification:
.. code:: yang
notification authentication-failure {
description
"This notification is sent when the system
receives a PDU with the wrong authentication
information.";
leaf interface-name {
type string;
description
"Describes the name of the RIP interface.";
}
}
The following convenience function was implemented in *ripd* to send
*authentication-failure* YANG notifications:
.. code:: c
/*
* XPath: /frr-ripd:authentication-failure
*/
void ripd_notif_send_auth_failure(const char *ifname)
{
const char *xpath = "/frr-ripd:authentication-failure";
struct list *arguments;
char xpath_arg[XPATH_MAXLEN];
struct yang_data *data;
arguments = yang_data_list_new();
snprintf(xpath_arg, sizeof(xpath_arg), "%s/interface-name", xpath);
data = yang_data_new_string(xpath_arg, ifname);
listnode_add(arguments, data);
nb_notification_send(xpath, arguments);
}
Now sending the *authentication-failure* YANG notification should be as
simple as calling the above function and provide the appropriate
interface name. The notification will be processed by all northbound
plugins that subscribed a callback to the ``nb_notification_send`` hook.
The ConfD and Sysrepo plugins, for instance, use this hook to relay the
notifications to the *confd*/*sysrepod* daemons, which can generate
NETCONF notifications to subscribed clients. When no northbound plugin
is loaded, ``nb_notification_send()`` doesn’t do anything and the
notifications are ignored.

137
doc/developer/northbound/plugins-sysrepo.rst Normal file
View File

@ -0,0 +1,137 @@
Installation
------------
Required dependencies
^^^^^^^^^^^^^^^^^^^^^
::
# apt-get install git cmake build-essential bison flex libpcre3-dev libev-dev \
libavl-dev libprotobuf-c-dev protobuf-c-compiler libcmocka0 \
libcmocka-dev doxygen libssl-dev libssl-dev libssh-dev
libyang
^^^^^^^
::
# apt-get install libyang0.16 libyang-dev
Sysrepo
^^^^^^^
::
$ git clone https://github.com/sysrepo/sysrepo.git
$ cd sysrepo/
$ mkdir build; cd build
$ cmake -DCMAKE_BUILD_TYPE=Release -DGEN_LANGUAGE_BINDINGS=OFF .. && make
# make install
libnetconf2
^^^^^^^^^^^
::
$ git clone https://github.com/CESNET/libnetconf2.git
$ cd libnetconf2/
$ mkdir build; cd build
$ cmake .. && make
# make install
netopeer2
^^^^^^^^^
::
$ git clone https://github.com/CESNET/Netopeer2.git
$ cd Netopeer2
$ cd server
$ mkdir build; cd build
$ cmake .. && make
# make install
**Note:** If ``make install`` fails as it can’t find
``libsysrepo.so.0.7``, then run ``ldconfig`` and try again as it might
not have updated the lib search path
FRR
^^^
Build and install FRR using the ``--enable-sysrepo`` configure-time
option.
Initialization
--------------
Install the FRR YANG modules in the Sysrepo datastore:
::
# sysrepoctl --install /usr/local/share/yang/ietf-interfaces@2018-01-09.yang
# sysrepoctl --install /usr/local/share/yang/frr-vrf.yang
# sysrepoctl --install /usr/local/share/yang/frr-interface.yang
# sysrepoctl --install /usr/local/share/yang/frr-route-types.yang
# sysrepoctl --install /usr/local/share/yang/frr-filter.yang
# sysrepoctl --install /usr/local/share/yang/frr-route-map.yang
# sysrepoctl --install /usr/local/share/yang/frr-isisd.yang
# sysrepoctl --install /usr/local/share/yang/frr-ripd.yang
# sysrepoctl --install /usr/local/share/yang/frr-ripngd.yang
# sysrepoctl -c frr-vrf --owner frr --group frr
# sysrepoctl -c frr-interface --owner frr --group frr
# sysrepoctl -c frr-route-types --owner frr --group frr
# sysrepoctl -c frr-filter --owner frr --group frr
# sysrepoctl -c frr-route-map --owner frr --group frr
# sysrepoctl -c frr-isisd --owner frr --group frr
# sysrepoctl -c frr-ripd --owner frr --group frr
# sysrepoctl -c frr-ripngd --owner frr --group frr
Start netopeer2-server:
::
# netopeer2-server -d &
Start the FRR daemons with the sysrepo module:
::
# isisd -M sysrepo --log=stdout
Managing the configuration
--------------------------
The following NETCONF scripts can be used to show and edit the FRR
configuration:
https://github.com/rzalamena/ietf-hackathon-brazil-201907/tree/master/netconf-scripts
Example:
::
# ./netconf-edit.py 127.0.0.1
# ./netconf-get-config.py 127.0.0.1
<?xml version="1.0" encoding="UTF-8"?><data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><isis xmlns="http://frrouting.org/yang/isisd"><instance><area-tag>testnet</area-tag><is-type>level-1</is-type></instance></isis></data>
..
NOTE: the ncclient library needs to be installed first:
``apt install -y python3-ncclient``
The *sysrepocfg* tool can also be used to show/edit the FRR
configuration. Example:
::
# sysrepocfg --format=json --import=frr-isisd.json --datastore=running frr-isisd
# sysrepocfg --format=json --export --datastore=running frr-isisd
{
"frr-isisd:isis": {
"instance": [
{
"area-tag": "testnet",
"is-type": "level-1"
}
]
}
}

1632
doc/developer/northbound/ppr-basic-test-topology.rst Normal file
View File

File diff suppressed because it is too large Load Diff

1991
doc/developer/northbound/ppr-mpls-basic-test-topology.rst Normal file
View File

File diff suppressed because it is too large Load Diff

1916
doc/developer/northbound/retrofitting-configuration-commands.rst Normal file
View File

File diff suppressed because it is too large Load Diff

244
doc/developer/northbound/transactional-cli.rst Normal file
View File

@ -0,0 +1,244 @@
Table of Contents
-----------------
- `Introduction <#introduction>`__
- `Configuration modes <#config-modes>`__
- `New commands <#retrofitting-process>`__
- `commit check <#cmd1>`__
- `commit <#cmd2>`__
- `discard <#cmd3>`__
- `configuration database max-transactions <#cmd4>`__
- `configuration load <#cmd5>`__
- `rollback configuration <#cmd6>`__
- `show configuration candidate <#cmd7>`__
- `show configuration compare <#cmd8>`__
- `show configuration running <#cmd9>`__
- `show configuration transaction <#cmd10>`__
- `show yang module <#cmd11>`__
- `show yang module-translator <#cmd12>`__
- `update <#cmd13>`__
- `yang module-translator load <#cmd14>`__
- `yang module-translator unload <#cmd15>`__
Introduction
~~~~~~~~~~~~
All FRR daemons have built-in support for the CLI, which can be accessed
either through local telnet or via the vty socket (e.g. by using
*vtysh*). This will not change with the introduction of the Northbound
API. However, a new command-line option will be available for all FRR
daemons: ``--tcli``. When given, this option makes the daemon start with
a transactional CLI and configuration commands behave a bit different.
Instead of editing the running configuration, they will edit the
candidate configuration. In other words, the configuration commands
won’t be applied immediately, that has to be done on a separate step
using the new ``commit`` command.
The transactional CLI simply leverages the new capabilities provided by
the Northbound API and exposes the concept of candidate configurations
to CLI users too. When the transactional mode is not used, the
configuration commands also edit the candidate configuration, but
there’s an implicit ``commit`` after each command.
In order for the transactional CLI to work, all configuration commands
need to be converted to the new northbound model. Commands not converted
to the new northbound model will change the running configuration
directly since they bypass the FRR northbound layer. For this reason,
starting a daemon with the transactional CLI is not advisable unless all
of its commands have already been converted. When that’s not the case,
we can run into a situation like this:
::
ospfd(config)# router ospf
ospfd(config-router)# ospf router-id 1.1.1.1
[segfault in ospfd]
The segfault above can happen if ``router ospf`` edits the candidate
configuration but ``ospf router-id 1.1.1.1`` edits the running
configuration. The second command tries to set
``ospf->router_id_static`` but, since the previous ``router ospf``
command hasn’t been commited yet, the ``ospf`` global variable is set to
NULL, which leads to the crash. Besides this problem, having a set of
commands that edit the candidate configuration and others that edit the
running configuration is confusing at best. The ``--tcli`` option should
be used only by developers until the northbound retrofitting process is
complete.
Configuration modes
~~~~~~~~~~~~~~~~~~~
When using the transactional CLI (``--tcli``), FRR supports three
different forms of the ``configure`` command: \* ``configure terminal``:
in this mode, a single candidate configuration is shared by all users.
This means that one user might delete a configuration object that’s
being edited by another user, in which case the CLI will detect and
report the problem. If one user issues the ``commit`` command, all
changes done by all users are committed. \* ``configure private``: users
have a private candidate configuration that is edited separately from
the other users. The ``commit`` command commits only the changes done by
the user. \* ``configure exclusive``: similar to ``configure private``,
but also locks the running configuration to prevent other users from
changing it. The configuration lock is released when the user exits the
configuration mode.
When using ``configure terminal`` or ``configure private``, the
candidate configuration being edited might become outdated if another
user commits a different candidate configuration on another session.
TODO: show image to illustrate the problem.
New commands
~~~~~~~~~~~~
The list below contains the new CLI commands introduced by Northbound
API. The commands are available when a daemon is started using the
transactional CLI (``--tcli``). Currently ``vtysh`` doesn’t support any
of these new commands.
Please refer to the [[Demos]] page to see a demo of the transactional
CLI in action.
--------------
``commit check``
''''''''''''''''
Check if the candidate configuration is valid or not.
``commit [force] [comment LINE...]``
''''''''''''''''''''''''''''''''''''
Commit the changes done in the candidate configuration into the running
configuration.
Options: \* ``force``: commit even if the candidate configuration is
outdated. It’s usually a better option to use the ``update`` command
instead. \* ``comment LINE...``: assign a comment to the configuration
transaction. This comment is displayed when viewing the recorded
transactions in the output of the ``show configuration transaction``
command.
``discard``
'''''''''''
Discard the changes done in the candidate configuration.
``configuration database max-transactions (1-100)``
'''''''''''''''''''''''''''''''''''''''''''''''''''
Set the maximum number of transactions to store in the rollback log.
``configuration load <file [<json|xml> [translate WORD]] FILENAME|transaction (1-4294967296)> [replace]``
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Load a new configuration into the candidate configuration. When loading
the configuration from a file, it’s assumed that the configuration will
be in the form of CLI commands by default. The ``json`` and ``xml``
options can be used to load configurations in the JSON and XML formats,
respectively. It’s also possible to load a configuration from a previous
transaction by specifying the desired transaction ID
(``(1-4294967296)``).
Options: \* ``translate WORD``: translate the JSON/XML configuration
file using the YANG module translator. \* ``replace``: replace the
candidate by the loaded configuration. The default is to merge the
loaded configuration into the candidate configuration.
``rollback configuration (1-4294967296)``
'''''''''''''''''''''''''''''''''''''''''
Roll back the running configuration to a previous configuration
identified by its transaction ID (``(1-4294967296)``).
``show configuration candidate [<json|xml> [translate WORD]] [<with-defaults|changes>]``
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Show the candidate configuration.
Options: \* ``json``: show the configuration in the JSON format. \*
``xml``: show the configuration in the XML format. \*
``translate WORD``: translate the JSON/XML output using the YANG module
translator. \* ``with-defaults``: show default values that are hidden by
default. \* ``changes``: show only the changes done in the candidate
configuration.
``show configuration compare <candidate|running|transaction (1-4294967296)> <candidate|running|transaction (1-4294967296)> [<json|xml> [translate WORD]]``
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Show the difference between two different configurations.
Options: \* ``json``: show the configuration differences in the JSON
format. \* ``xml``: show the configuration differences in the XML
format. \* ``translate WORD``: translate the JSON/XML output using the
YANG module translator.
``show configuration running [<json|xml> [translate WORD]] [with-defaults]``
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Show the running configuration.
Options: \* ``json``: show the configuration in the JSON format. \*
``xml``: show the configuration in the XML format. \*
``translate WORD``: translate the JSON/XML output using the YANG module
translator. \* ``with-defaults``: show default values that are hidden by
default.
NOTE: ``show configuration running`` shows only the running
configuration as known by the northbound layer. Configuration
commands not converted to the new northbound model will not be
displayed. To show the full running configuration, the legacy
``show running-config`` command must be used.
``show configuration transaction [(1-4294967296) [<json|xml> [translate WORD]] [changes]]``
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
When a transaction ID (``(1-4294967296)``) is given, show the
configuration associated to the previously committed transaction.
When a transaction ID is not given, show all recorded transactions in
the rollback log.
Options: \* ``json``: show the configuration in the JSON format. \*
``xml``: show the configuration in the XML format. \*
``translate WORD``: translate the JSON/XML output using the YANG module
translator. \* ``with-defaults``: show default values that are hidden by
default. \* ``changes``: show changes compared to the previous
transaction.
``show yang module [module-translator WORD] [WORD <summary|tree|yang|yin>]``
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
When a YANG module is not given, show all loaded YANG modules.
Otherwise, show detailed information about the given module.
Options: \* ``module-translator WORD``: change the context to modules
loaded by the specified YANG module translator. \* ``summary``: display
summary information about the module. \* ``tree``: display module in the
tree (RFC 8340) format. \* ``yang``: display module in the YANG format.
\* ``yin``: display module in the YIN format.
``show yang module-translator``
'''''''''''''''''''''''''''''''
Show all loaded YANG module translators.
``update``
''''''''''
Rebase the candidate configuration on top of the latest running
configuration. Conflicts are resolved automatically by giving preference
to the changes done in the candidate configuration.
The candidate configuration might be outdated if the running
configuration was updated after the candidate was created.
``yang module-translator load FILENAME``
''''''''''''''''''''''''''''''''''''''''
Load a YANG module translator from the filesystem.
``yang module-translator unload WORD``
''''''''''''''''''''''''''''''''''''''
Unload a YANG module translator identified by its name.

629
doc/developer/northbound/yang-module-translator.rst Normal file
View File

@ -0,0 +1,629 @@
Table of Contents
-----------------
- `Introduction <#introduction>`__
- `Deviation Modules <#deviation-modules>`__
- `Translation Tables <#translation-tables>`__
- `CLI Demonstration <#cli-demonstration>`__
- `Implementation Details <#implementation-details>`__
Introduction
------------
One key requirement for the FRR northbound architecture is that it
should be possible to configure/monitor FRR using different sets of YANG
models. This is especially important considering that the industry
hasn’t reached a consensus to provide a single source of standard models
for network management. At this moment both the IETF and OpenConfig
models are widely implemented and are unlikely to converge, at least not
in the short term. In the ideal scenario, management applications should
be able to use either IETF or OpenConfig models to configure and monitor
FRR programatically (or even both at the same time!).
But how can FRR support multiple sets of YANG models at the same time?
There must be only a single source of truth that models the existing
implementation accurately (the native models). Writing different code
paths or callbacks for different models would be inviable, it would lead
to a lot of duplicated code and extra maintenance overhead.
In order to support different sets of YANG modules without introducing
the overhead of writing additional code, the solution is to create a
mechanism that dynamically translates YANG instance data between
non-native models to native models and vice-versa. Based on this idea,
an experimental YANG module translator was implemented within the FRR
northbound layer. The translator works by translating XPaths at runtime
using translation tables provided by the user. The translator itself is
modeled using YANG and users can create translators using simple JSON
files.
A YANG module translator consists of two components: deviation modules
and translation tables.
Deviation Modules
-----------------
The first step when writing a YANG module translator is to create a
`deviations <https://tools.ietf.org/html/rfc7950#page-131>`__ module for
each module that is going be translated. This is necessary because in
most cases it won’t be possible to create a perfect translator that
covers the non-native models on their entirety. Some non-native modules
might contain nodes that can’t be mapped to a corresponding node in the
FRR native models. This is either because the corresponding
functionality is not implemented in FRR or because it’s modeled in a
different way that is incompatible.
An an example, *ripd* doesn’t have BFD support yet, so we need to create
a YANG deviation to modify the *ietf-rip* module and remove the ``bfd``
container from it:
.. code:: yang
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:interfaces/ietf-rip:interface/ietf-rip:bfd" {
deviate not-supported;
}
In the example below, while both the *frr-ripd* and *ietf-rip* modules
support RIP authentication, they model the authentication data in
different ways, making translation not possible given the constraints of
the current module translator. A new deviation is necessary to remove
the ``authentication`` container from the *ietf-rip* module:
.. code:: yang
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:interfaces/ietf-rip:interface/ietf-rip:authentication" {
deviate not-supported;
}
..
NOTE: it should be possible to translate the
``ietf-rip:authentication`` container if the *frr-ripd* module is
modified to model the corresponding data in a compatible way. Another
option is to improve the module translator to make more complex
translations possible, instead of requiring one-to-one XPath
mappings.
Sometimes creating a mapping between nodes from the native and
non-native models is possible, but the nodes have different properties
that need to be normalized to allow the translation. In the example
below, a YANG deviation is used to change the type and the default value
from a node from the ``ietf-rip`` module.
.. code:: yang
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:timers/ietf-rip:flush-interval" {
deviate replace {
default "120";
}
deviate replace {
type uint32;
}
}
The deviation modules allow the management applications to know which
parts of the custom modules (e.g. IETF/OC) can be used to configure and
monitor FRR.
In order to facilitate the process of creating YANG deviation modules,
the *gen_yang_deviations* tool was created to automate part of the
process. This tool creates a “not-supported” deviation for all nodes
from the given non-native module. Example:
::
$ tools/gen_yang_deviations ietf-rip > yang/ietf/frr-deviations-ietf-rip.yang
$ head -n 40 yang/ietf/frr-deviations-ietf-rip.yang
deviation "/ietf-rip:clear-rip-route" {
deviate not-supported;
}
deviation "/ietf-rip:clear-rip-route/ietf-rip:input" {
deviate not-supported;
}
deviation "/ietf-rip:clear-rip-route/ietf-rip:input/ietf-rip:rip-instance" {
deviate not-supported;
}
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip" {
deviate not-supported;
}
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route" {
deviate not-supported;
}
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route/ietf-rip:enabled" {
deviate not-supported;
}
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:originate-default-route/ietf-rip:route-policy" {
deviate not-supported;
}
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:default-metric" {
deviate not-supported;
}
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:distance" {
deviate not-supported;
}
deviation "/ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol/ietf-rip:rip/ietf-rip:triggered-update-threshold" {
deviate not-supported;
}
Once all existing nodes are listed in the deviation module, it’s easy to
check the deviations that need to be removed or modified. This is more
convenient than starting with a blank deviations module and listing
manually all nodes that need to be deviated.
After removing and/or modifying the auto-generated deviations, the next
step is to write the module XPath translation table as we’ll see in the
next section. Before that, it’s possible to use the *yanglint* tool to
check how the non-native module looks like after applying the
deviations. Example:
::
$ yanglint -f tree yang/ietf/ietf-rip@2018-02-03.yang yang/ietf/frr-deviations-ietf-rip.yang
module: ietf-rip
augment /ietf-routing:routing/ietf-routing:control-plane-protocols/ietf-routing:control-plane-protocol:
+--rw rip
+--rw originate-default-route
| +--rw enabled? boolean <false>
+--rw default-metric? uint8 <1>
+--rw distance? uint8 <0>
+--rw timers
| +--rw update-interval? uint32 <30>
| +--rw holddown-interval? uint32 <180>
| +--rw flush-interval? uint32 <120>
+--rw interfaces
| +--rw interface* [interface]
| +--rw interface ietf-interfaces:interface-ref
| +--rw split-horizon? enumeration <simple>
+--ro ipv4
+--ro neighbors
| +--ro neighbor* [ipv4-address]
| +--ro ipv4-address ietf-inet-types:ipv4-address
| +--ro last-update? ietf-yang-types:date-and-time
| +--ro bad-packets-rcvd? ietf-yang-types:counter32
| +--ro bad-routes-rcvd? ietf-yang-types:counter32
+--ro routes
+--ro route* [ipv4-prefix]
+--ro ipv4-prefix ietf-inet-types:ipv4-prefix
+--ro next-hop? ietf-inet-types:ipv4-address
+--ro interface? ietf-interfaces:interface-ref
+--ro metric? uint8
rpcs:
+---x clear-rip-route
..
NOTE: the same output can be obtained using the
``show yang module module-translator ietf ietf-rip tree`` command in
FRR once the *ietf* module translator is loaded.
In the example above, it can be seen that the vast majority of the
*ietf-rip* nodes were removed because of the “not-supported” deviations.
When a module translator is loaded, FRR calculates the coverage of the
translator by dividing the number of YANG nodes before applying the
deviations by the number of YANG nodes after applying the deviations.
The calculated coverage is displayed in the output of the
``show yang module-translator`` command:
::
ripd# show yang module-translator
Family Module Deviations Coverage (%)
-----------------------------------------------------------------------
ietf ietf-interfaces frr-deviations-ietf-interfaces 3.92
ietf ietf-routing frr-deviations-ietf-routing 1.56
ietf ietf-rip frr-deviations-ietf-rip 13.60
As it can be seen in the output above, the *ietf* module translator
covers only ~13% of the original *ietf-rip* module. This is in part
because the *ietf-rip* module models both RIPv2 and RIPng. Also,
*ietf-rip.yang* contains several knobs that aren’t implemented in *ripd*
yet (e.g. BFD support, per-interface timers, statistics, etc). Work can
be done over time to increase the coverage to a more reasonable number.
Translation Tables
------------------
Below is an example of a translator for the IETF family of models:
.. code:: json
{
"frr-module-translator:frr-module-translator": {
"family": "ietf",
"module": [
{
"name": "ietf-interfaces@2018-01-09",
"deviations": "frr-deviations-ietf-interfaces",
"mappings": [
{
"custom": "/ietf-interfaces:interfaces/interface[name='KEY1']",
"native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']"
},
{
"custom": "/ietf-interfaces:interfaces/interface[name='KEY1']/description",
"native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']/description"
}
]
},
{
"name": "ietf-routing@2018-01-25",
"deviations": "frr-deviations-ietf-routing",
"mappings": [
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']",
"native": "/frr-ripd:ripd/instance"
}
]
},
{
"name": "ietf-rip@2018-02-03",
"deviations": "frr-deviations-ietf-rip",
"mappings": [
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/default-metric",
"native": "/frr-ripd:ripd/instance/default-metric"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/distance",
"native": "/frr-ripd:ripd/instance/distance/default"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/originate-default-route/enabled",
"native": "/frr-ripd:ripd/instance/default-information-originate"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/update-interval",
"native": "/frr-ripd:ripd/instance/timers/update-interval"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/holddown-interval",
"native": "/frr-ripd:ripd/instance/timers/holddown-interval"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/timers/flush-interval",
"native": "/frr-ripd:ripd/instance/timers/flush-interval"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']",
"native": "/frr-ripd:ripd/instance/interface[.='KEY1']"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']/split-horizon",
"native": "/frr-interface:lib/interface[name='KEY1'][vrf='default']/frr-ripd:rip/split-horizon"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']",
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/last-update",
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/last-update"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/bad-packets-rcvd",
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/bad-packets-rcvd"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/neighbors/neighbor[ipv4-address='KEY1']/bad-routes-rcvd",
"native": "/frr-ripd:ripd/state/neighbors/neighbor[address='KEY1']/bad-routes-rcvd"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']",
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/next-hop",
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/next-hop"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/interface",
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/interface"
},
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/ipv4/routes/route[ipv4-prefix='KEY1']/metric",
"native": "/frr-ripd:ripd/state/routes/route[prefix='KEY1']/metric"
},
{
"custom": "/ietf-rip:clear-rip-route",
"native": "/frr-ripd:clear-rip-route"
}
]
}
]
}
}
The main motivation to use YANG itself to model YANG module translators
was a practical one: leverage *libyang* to validate the structure of the
user input (JSON files) instead of doing that manually in the
*lib/yang_translator.c* file (tedious and error-prone work).
Module translators can be loaded using the following CLI command:
::
ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json
% Module translator "ietf" loaded successfully.
Module translators can also be loaded/unloaded programatically using the
``yang_translator_load()/yang_translator_unload()`` functions within the
northbound plugins. These functions are documented in the
*lib/yang_translator.h* file.
Each module translator must be assigned a “family” identifier
(e.g. IETF, OpenConfig), and can contain mappings for multiple
interrelated YANG modules. The mappings consist of pairs of
custom/native XPath expressions that should be equivalent, despite
belonging to different YANG modules.
Example:
.. code:: json
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/default-metric",
"native": "/frr-ripd:ripd/instance/default-metric"
},
The nodes pointed by the custom and native XPaths must have compatible
types. In the case of the example above, both nodes point to a YANG leaf
of type ``uint8``, so the mapping is valid.
In the example below, the “custom” XPath points to a YANG list
(typeless), and the “native” XPath points to a YANG leaf-list of
strings. In this exceptional case, the types are also considered to be
compatible.
.. code:: json
{
"custom": "/ietf-routing:routing/control-plane-protocols/control-plane-protocol[type='ietf-rip:ripv2'][name='main']/ietf-rip:rip/interfaces/interface[interface='KEY1']",
"native": "/frr-ripd:ripd/instance/interface[.='KEY1']"
},
The ``KEY1..KEY4`` values have a special meaning and are used to
preserve the list keys while performing the XPath translation.
Once a YANG module translator is loaded and validated at a syntactic
level using *libyang*, further validations are performed to check for
missing mappings (after loading the deviation modules) and incompatible
YANG types. Example:
::
ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json
% Failed to load "/usr/local/share/yang/ietf/frr-ietf-translator.json"
Please check the logs for more details.
::
2018/09/03 15:18:45 RIP: yang_translator_validate_cb: YANG types are incompatible (xpath: "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/default-metric")
2018/09/03 15:18:45 RIP: yang_translator_validate_cb: missing mapping for "/ietf-routing:routing/control-plane-protocols/control-plane-protocol/ietf-rip:rip/distance"
2018/09/03 15:18:45 RIP: yang_translator_validate: failed to validate "ietf" module translator: 2 error(s)
Overall, this translation mechanism based on XPath mappings is simple
and functional, but only to a certain extent. The native models need to
be reasonably similar to the models that are going be translated,
otherwise the translation is compromised and a good coverage can’t be
achieved. Other translation techniques must be investigated to address
this shortcoming and make it possible to create more powerful YANG
module translators.
YANG module translators can be evaluated based on the following metrics:
\* Translation potential: is it possible to make complex translations,
taking several variables into account? \* Complexity: measure of how
easy or hard it is to write a module translator. \* Speed: measure of
how fast the translation can be achieved. Translation speed is of
fundamental importance, especially for operational data. \* Robustness:
can the translator be checked for inconsistencies at load time? A module
translator based on scripts wouldn’t fare well on this metric. \*
Round-trip conversions: can the translated data be translated back to
the original format without information loss?
CLI Demonstration
-----------------
As of now the only northbound client that supports the YANG module
translator is the FRR embedded CLI. The confd and sysrepo plugins need
to be extended to support the module translator, which might be used not
only for configuration data, but also for operational data, RPCs and
notifications.
In this demonstration, we’ll use the CLI ``configuration load`` command
to load the following JSON configuration file specified using the IETF
data hierarchy:
.. code:: json
{
"ietf-interfaces:interfaces": {
"interface": [
{
"description": "Engineering",
"name": "eth0"
}
]
},
"ietf-routing:routing": {
"control-plane-protocols": {
"control-plane-protocol": [
{
"name": "main",
"type": "ietf-rip:ripv2",
"ietf-rip:rip": {
"default-metric": "2",
"distance": "80",
"interfaces": {
"interface": [
{
"interface": "eth0",
"split-horizon": "poison-reverse"
}
]
},
"originate-default-route": {
"enabled": "true"
},
"timers": {
"flush-interval": "241",
"holddown-interval": "181",
"update-interval": "31"
}
}
}
]
}
}
}
In order to load this configuration file, it’s necessary to load the
IETF module translator first. Then, when entering the
``configuration load`` command, the ``translate ietf`` parameters must
be given to specify that the input needs to be translated using the
previously loaded ``ietf`` module translator. Example:
::
ripd(config)# configuration load file json /mnt/renato/git/frr/yang/example/ietf-rip.json
% Failed to load configuration:
Unknown element "interfaces".
ripd(config)#
ripd(config)# yang module-translator load /usr/local/share/yang/ietf/frr-ietf-translator.json
% Module translator "ietf" loaded successfully.
ripd(config)#
ripd(config)# configuration load file json translate ietf /mnt/renato/git/frr/yang/example/ietf-rip.json
Now let’s check the candidate configuration to see if the configuration
file was loaded successfully:
::
ripd(config)# show configuration candidate
Configuration:
!
frr version 5.1-dev
frr defaults traditional
!
interface eth0
description Engineering
ip rip split-horizon poisoned-reverse
!
router rip
default-metric 2
distance 80
network eth0
default-information originate
timers basic 31 181 241
!
end
ripd(config)# show configuration candidate json
{
"frr-interface:lib": {
"interface": [
{
"name": "eth0",
"vrf": "default",
"description": "Engineering",
"frr-ripd:rip": {
"split-horizon": "poison-reverse"
}
}
]
},
"frr-ripd:ripd": {
"instance": {
"default-metric": 2,
"distance": {
"default": 80
},
"interface": [
"eth0"
],
"default-information-originate": true,
"timers": {
"flush-interval": 241,
"holddown-interval": 181,
"update-interval": 31
}
}
}
}
As it can be seen, the candidate configuration is identical to the one
defined in the *ietf-rip.json* file, only the structure is different.
This means that the *ietf-rip.json* file was translated successfully.
The ``ietf`` module translator can also be used to do the translation in
other direction: transform data from the native format to the IETF
format. This is shown below by altering the output of the
``show configuration candidate json`` command using the
``translate ietf`` parameter:
::
ripd(config)# show configuration candidate json translate ietf
{
"ietf-interfaces:interfaces": {
"interface": [
{
"name": "eth0",
"description": "Engineering"
}
]
},
"ietf-routing:routing": {
"control-plane-protocols": {
"control-plane-protocol": [
{
"type": "ietf-rip:ripv2",
"name": "main",
"ietf-rip:rip": {
"interfaces": {
"interface": [
{
"interface": "eth0",
"split-horizon": "poison-reverse"
}
]
},
"default-metric": 2,
"distance": 80,
"originate-default-route": {
"enabled": true
},
"timers": {
"flush-interval": 241,
"holddown-interval": 181,
"update-interval": 31
}
}
}
]
}
}
}
As expected, this output is exactly identical to the configuration
defined in the *ietf-rip.json* file. The module translator was able to
do a round-trip conversion without information loss.
Implementation Details
----------------------
A different libyang context is allocated for each YANG module
translator. This is important to avoid collisions and ensure that
non-native data can’t be instantiated in the running and candidate
configurations.

106
doc/developer/northbound/yang-tools.rst Normal file
View File

@ -0,0 +1,106 @@
yanglint cheat sheet
~~~~~~~~~~~~~~~~~~~~
libyang project includes a feature-rich tool called yanglint(1) for
validation and conversion of the schemas and YANG modeled data. The
source codes are located at /tools/lint and can be used to explore
how an application is supposed to use the libyang library.
yanglint(1) binary as well as its man page are installed together
with the library itself.
Validate a YANG module:
.. code:: sh
$ yanglint -p <yang-search-path> module.yang
Generate tree representation of a YANG module:
.. code:: sh
$ yanglint -p <yang-search-path> -f tree module.yang
Validate JSON/XML instance data:
.. code:: sh
$ yanglint -p <yang-search-path> module.yang data.{json,xml}
Convert JSON/XML instance data to another format:
.. code:: sh
$ yanglint -p <yang-search-path> -f xml module.yang data.json
$ yanglint -p <yang-search-path> -f json module.yang data.xml
*yanglint* also features an interactive mode which is very useful when
needing to validate data from multiple modules at the same time. The
*yanglint* README provides several examples:
https://github.com/CESNET/libyang/blob/master/tools/lint/examples/README.md
Man page (groff):
https://github.com/CESNET/libyang/blob/master/tools/lint/yanglint.1
pyang cheat sheet
~~~~~~~~~~~~~~~~~
pyang is a YANG validator, transformator and code generator, written
in python. It can be used to validate YANG modules for correctness,
to transform YANG modules into other formats, and to generate code
from the modules.
Obtaining and installing pyang:
.. code:: sh
$ git clone https://github.com/mbj4668/pyang.git
$ cd pyang/
$ sudo python setup.py install
Validate a YANG module:
.. code:: sh
$ pyang --ietf -p <yang-search-path> module.yang
Generate tree representation of a YANG module:
.. code:: sh
$ pyang -f tree -p <yang-search-path> module.yang
Indent a YANG file:
.. code:: sh
$ pyang -p <yang-search-path> \
--keep-comments -f yang --yang-canonical \
module.yang -o module.yang
Generate skeleton instance data: \* XML:
.. code:: sh
$ pyang -p <yang-search-path> \
-f sample-xml-skeleton --sample-xml-skeleton-defaults \
module.yang [augmented-module1.yang ...] -o module.xml
- JSON:
.. code:: sh
$ pyang -p <yang-search-path> \
-f jsonxsl module.yang -o module.xsl
$ xsltproc -o module.json module.xsl module.xml
Validate XML instance data (works only with YANG 1.0):
.. code:: sh
$ yang2dsdl -v module.xml module.yang
vim
~~~
YANG syntax highlighting for vim:
https://github.com/nathanalderson/yang.vim

13
doc/developer/subdir.am
View File

@ -67,6 +67,19 @@ dev_RSTFILES = \
doc/developer/workflow.rst \ doc/developer/workflow.rst \
doc/developer/xrefs.rst \ doc/developer/xrefs.rst \
doc/developer/zebra.rst \ doc/developer/zebra.rst \
doc/developer/northbound/advanced-topics.rst \
doc/developer/northbound/architecture.rst \
doc/developer/northbound/demos.rst \
doc/developer/northbound/links.rst \
doc/developer/northbound/northbound.rst \
doc/developer/northbound/operational-data-rpcs-and-notifications.rst \
doc/developer/northbound/plugins-sysrepo.rst \
doc/developer/northbound/ppr-basic-test-topology.rst \
doc/developer/northbound/ppr-mpls-basic-test-topology.rst \
doc/developer/northbound/retrofitting-configuration-commands.rst \
doc/developer/northbound/transactional-cli.rst \
doc/developer/northbound/yang-module-translator.rst \
doc/developer/northbound/yang-tools.rst \
# end # end
EXTRA_DIST += \ EXTRA_DIST += \