Commit Graph

117 Commits

Author SHA1 Message Date
Mark Stapp
ccda0eadac
Merge pull request #7155 from donaldsharp/TRAP
Offload/Trap
2020-09-23 16:06:37 -04:00
Stephen Worley
57d7d1948e
Merge pull request #7160 from mjstapp/fix_topo_daemons_list
tests: fix bug in handling of daemons to start
2020-09-23 14:47:13 -04:00
Mark Stapp
bb91e9c03b tests: fix bug in handling of daemons to start
Ensure the list of daemons to start is either the one specified
by a caller or the default one from the router configuration.

Signed-off-by: Mark Stapp <mjs@voltanet.io>
2020-09-23 08:48:25 -04:00
Donald Sharp
5a3cf85391 lib, zebra: Add ability to read kernel notice of TRAP/OFFLOAD
The linux kernel is getting RTM_F_TRAP and RTM_F_OFFLOAD for
kernel routes that have an underlying asic offload.  Write the
code to receive these notifications from the linux kernel and
to store that data for display about the routes.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2020-09-22 15:57:43 -04:00
Donald Sharp
622c4996ef tests: Our tests are for FRR
These tests at this point only work for FRR.  Let's cut to the chase
and admit it.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>`
2020-09-22 10:11:58 -04:00
Renato Westphal
b686742e2b
Merge pull request #6934 from Niral-Networks/niral_dev_vrf_isis_core_pr4
isisd, yang, doc : Support for different VRF in ISIS.
2020-09-04 18:44:56 -03:00
harios_niral
9375b5aa24 topotests : Topotest for different VRF in isisd
1. Topotest for isis-vrf is added for ipv4 and ipv6.
2. Test case for checking isis topology.
3. Test case for checking zebra isis routes.
4. Test case for checking linux vrf routes.
5. 2 new API's written in topotest/lib for checking vrf routes.

Co-authored-by: Kaushik <kaushik@niralnetworks.com>"
Signed-off-by: harios_niral <hari@niralnetworks.com>
2020-09-01 00:56:44 -07:00
Mark Stapp
cd79342c12 tests: small topotest improvements for python3
Avoid unnecessary use of StringIO in one place, use version-
dependent method in another. Remove a couple of other py2->py3
problems.

Signed-off-by: Mark Stapp <mjs@voltanet.io>
2020-08-28 10:10:46 -04:00
Rafael Zalamena
8e72eff7d8 topotests: bump zebra netlink socket buffer size
Use the same value as we use in production release configuration.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2020-08-24 07:29:23 -03:00
David Lamparter
f033a78a99 topotests: stop wasting time at exit
... and clean up zombie child processes

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2020-07-15 18:50:30 +02:00
David Lamparter
79f6fdeb3f topotests: unshittify backtraces
add thread info, use "bt full" to get variables and add a bit of
disassembly for good measure.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2020-07-15 18:48:57 +02:00
Mark Stapp
7172f19da4 tests: don't fail topotests if bundle file not present
Test for the existence of the support_bundle conf file before
trying to access it.

Signed-off-by: Mark Stapp <mjs@voltanet.io>
2020-07-09 12:13:55 -04:00
Kuldeep Kashyap
c39fe454e2 tests: Generate support bundle/dump data on tests failures
1. It will generate support bundle/sump data on test failures
2. It used /usr/lib/frr/generate_support_bundle.py utility to dump the data

Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
2020-07-07 16:02:22 +00:00
Mark Stapp
cb3e512d97 tests,zebra: fix more startup topotest issues
Use the right list of daemons to avoid trying to start zebra twice.
Change a zebra log message to INFO level to avoid stderr check
failure.

Signed-off-by: Mark Stapp <mjs@voltanet.io>
2020-07-04 12:25:10 -04:00
Mark Stapp
cf865d1b7c tests: wait longer when shutting down in topotests
Add a few retries during router shutdown before killing a daemon. Also
work harder to start only a single instance of daemons, esp. zebra.

Signed-off-by: Mark Stapp <mjs@voltanet.io>
2020-07-03 13:22:42 -04:00
Rafael Zalamena
9ce4f4b86b topotests: remove daemons start up sleep
Instead of waiting for daemons start with `sleep`, start them with the
`-d` parameter so they can release the terminal themselves when ready.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2020-07-02 14:52:46 -03:00
Rafael Zalamena
1455684740 topotests: start logging early
Start logging early everything (including debug) to
`/tmp/topotest/<test>/<node>/<daemon>.{out,err}`.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2020-07-02 14:52:46 -03:00
Rafael Zalamena
aa5261bf7d topotests: remove duplicated code
Handle the duplicated code with a simple conditional: if called from
specialized API use provided daemons configuration, otherwise fallback
to old `Router` own daemon settings.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2020-07-02 14:52:42 -03:00
Jakub Urbańczyk
9b7decf28e topotest: add pbr test suite
Add some basic tests for installing PBR rules into the kernel.

Signed-off-by: Jakub Urbańczyk <xthaid@gmail.com>
2020-06-10 16:29:14 +02:00
Donald Sharp
223f87f4db tests: Setup very basic pbr tests for our topology tests
Just a simple setup for pbr to prove it starts.  Once the json
code for pbr gets in we can add more.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2020-05-29 16:56:14 -04:00
Donald Sharp
a0764a368f tests: Turn on nhrpd and babeld as part of all-protocol-startup
Turn on a few more daemons to run during this test.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2020-05-23 19:44:04 -04:00
GalaxyGorilla
b3100f6ce7 tests: Fix JSON diff for nested lists
The involved piece of code is supposed to find a 'closest' match for two
JSON structures using another JSON diff. However, it can happen that
during that new diff the JSON structures are altered (elements from a
list are deleted when 'found'). This is in general ok when the deleted
element is part of the JSON structure which 'matches', but when it later
turns out that some other element of the structure doesn't fit, then the
whole structure should be recovered. This is now realized by using a
deepcopy for the besaid new JSON diff such that the original is only
altered (e.g. deleted) when the diff is clean.

Signed-off-by: GalaxyGorilla <sascha@netdef.org>
2020-05-22 22:29:35 +00:00
Donald Sharp
6109dd7e22
Merge pull request #6293 from GalaxyGorilla/json_diff
tests: introduce a proper JSON diff for topotests
2020-05-21 14:19:44 -04:00
Kuldeep Kashyap
a5a52d6608 tests: Add library support for BGP-Graceful-Restart automation
1. Adding APIs to common_config.py to support BGP-Graceful-Restart automation
2. Adding APIs to create BGP-GR config to bgp.py
3. Adding verification API for BGP-GR functionality

Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
2020-05-04 17:15:55 +00:00
Kuldeep Kashyap
c65a7e26dc tests: Enhance topotests scripts to kill/start specific Daemons
1. Added APIs to topogen.py and topotest.py to kill/start specific Daemons,
   ex - bgpd, staticd

Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
2020-05-04 17:15:55 +00:00
GalaxyGorilla
849224d4ed tests: introduce a proper JSON diff for topotests
Diff'ing JSON objects is a crucial operation in the topotests for
comparing e.g. vtysh output (formatted as JSON) with a file which
covers the expectation of the tests. The current diff functionality
is 'self-written' and intended to test a JSON object d2 on being a
subset of another JSON object d1. For mismatches a diff is generated
based on a normalized textual representation of the JSON objects.

This approach has several disadvantages:

  * the human provided JSON text might not be normalized, hence
    a diff with line numbers might be worthless since it provides
    close to zero orientation what the problem is
  * the diff contains changes like commatas which are meaningless
  * the diff might contain a lot of changes about meaningless
    content which is present in d1 but not in d2
  * there is no proper functionality to test for 'equality' of
    d1 and d2
  * it is not possible to test for order, e.g. JSON arrays are
    just tested with respect to being a subset of another array
  * it is not possible to check if a key exists without also
    checking the value of that particular key

This commit attempts to solve these issues. An error report is
generated which includes the "JSON Path" to the problematic JSON
elements and also hints on what the actual problem is (e.g. missing
key, mismatch in dict values etc.).

A special parameter 'exact' was introduced such that equality can be
tested. Also there was a convention that absence of keys can be
tested using the key in question with value 'None'. This convention
is still honored such that full backwards compatiiblity is in
place.

Further order can be tested using the new tag '__ordered__' in
lists (as first element). Example:

    d1 = [1, 2, 3]
    d2 = ['__ordered__', 1, 3, 2]

Tesing d1 and d2 this way will now result in an error.

Key existence can now be tested using an asterisk '*'. Example:

    d1 = [1, 2, 3]
    d2 = [1, '*', 3]

    d1 = {'a': 1, 'b': 2}
    d2 = {'a': '*'}

Both cases will result now in a clean diff for d1 and d2.

Signed-off-by: GalaxyGorilla <sascha@netdef.org>
2020-04-30 09:33:32 +00:00
reformat
787e762445 tests: Run python formatter (black) for topotests
Mostly ' => ", whitespace changes.

Using https://github.com/psf/black

Signed-off-by: reformat <reformat@nobody.nobody>
2020-04-03 19:41:28 +03:00
Rafael Zalamena
a6fd124a2a topotest: add new run_and_expect variant
The new `run_and_expect` variant - called `run_and_expect_type` - tests
the return value type of the test function and optionally the return
value.

Now we can implement tests from test functions that return different
return types.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2019-07-23 10:28:56 -03:00
Stephen Worley
4aaea8fb49 tests: Make checkRouterRunning() show logging
Adding in the command `show log-filter` made `show log`
ambiguous. Change the checkRouterRunning() test to do
full `show logging` so it works again.

Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
2019-06-19 17:20:24 -04:00
Rafael Zalamena
04ce2b970e topotests: backward compatibility fix
Allow old topologies to use default parameters from `pytest.ini`.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2019-06-11 14:05:48 -03:00
Mark Stapp
e394d9aa5f tests: Add ip6_route_zebra lib function
Add a common function to retrieve and canonicalize
'show ipv6 route' output for use in topotests. Use that in
the test_ospf6_topo1 test; update the corresponding 'expected'
results files to match the lib function.

Replace some 'print' with 'logger' statements in that test also.

Signed-off-by: Mark Stapp <mjs@voltanet.io>
2019-06-06 13:22:04 -04:00
Rafael Zalamena
0d5e41c628 topotests/lib: fallback topology logdir if none was configured
`param.get` always evaluates the second argument and it was causing
two log directories being created for topologies using Topogen.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2019-03-25 14:46:16 -03:00
Donald Sharp
4b1d6d4db5 topotests: add FreeBSDRouter abstraction
Start the addition of FreeBSDRouter class that inherits from
class Router.  At this point we do not do a whole lot.  We
just come up on FreeBSD, lots of stuff that still needs to
be fixed.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-12-17 10:43:27 -05:00
Donald Sharp
d29fb5bd52 topotests: Move linux specific config into Linux config section
As part of the class rework, move the linux specific config
into the LinuxRouter.config section instead of being in the
Router class.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-12-17 10:38:02 -05:00
Donald Sharp
7cc96035a7 topotests: Make 'LinuxRouter' a class of 'Router'
Modify the LinuxRouter code such that it inherits from the
Router class.  This is setup work for (a) pulling out linux
specific config from class Router and (b) creating a FreebsdRouter
that inherits from class 'Router'.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-12-17 10:33:37 -05:00
Donald Sharp
849d1ed9af topotests: Allow sharpd to be used in topotests
The sharp daemon did not have any ability to be used in
topotests.  Add some code to allow this.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-12-16 20:47:30 -05:00
Donald Sharp
28440fd9f8 topotests: module_present should return a True/False
The recent change to module_present accidently didn't return anything.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-12-16 20:47:30 -05:00
Donald Sharp
cc95fbd9ff topotests: Start abstraction of module_present for other platforms
Start abstraction of the module_present into module_present_linux and freebsd
To allow for multiple platforms that the topotests can be run on.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-12-15 10:54:11 -05:00
Lou Berger
8cd7c744df topotest: remove sleep on staticd startup
Signed-off-by: Lou Berger <lberger@labn.net>
2018-12-08 10:42:21 -05:00
Christian Franke
7fe06d5559 topotests: Update json_cmp_result to return readable result
Before this update, json_cmp_results which were formatted as strings
would not show the error mesage but just an object reference.

Signed-off-by: Christian Franke <chris@opensourcerouting.org>
2018-12-04 16:13:58 +01:00
Donald Sharp
d21321142a Add a tiny breadcrumb to hopefully help end user
Add a breadcrumb for people testing to hopefully allow
them to figure out what is going wrong when they
are testing different versions of FRR using
topotests and staticd is not running because
this version of staticd needs to be cleaned up

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-11-27 20:22:14 -05:00
paco
86c21ac7fc lib, lm-proxy-topo1: label manager proxy test
Label proxy manager test covering both LDP and BGP label requests.

lib/topotest.py:restartRouter() function has been fixed in order to
allow propagating options to deamons other than zebra.

Signed-off-by: F. Aragon <paco@voltanet.io>
2018-11-27 20:22:14 -05:00
Martin Winter
6bfe4b8bfc lib: checkRouterVersion to determine version
Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>
2018-11-27 20:22:14 -05:00
Rafael Zalamena
4d45d6d3b0 topogen/topotest: add support for BFDd
Allow BFDd to be configured and used.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2018-11-27 20:22:14 -05:00
Martin Winter
2f726781b6 lib: Add support for Ubuntu 18.04 with installed FRR package
Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>
2018-11-27 20:22:14 -05:00
Rafael Zalamena
0eff58207e lib: fix ip4_route_zebra header removal code
The `strip` function is actually a method of the String object.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2018-11-27 20:22:14 -05:00
Christian Franke
41077aa191 lib/ospf-topo1-vrf: don't compare headers of show ip route
Signed-off-by: Christian Franke <nobody@nowhere.ws>
2018-11-27 20:22:14 -05:00
Donald Sharp
a93eb16a51 lib: Always attempt to set mpls platform_labels
The code was not attempting to set the max platform_labels
on a per router basis, leaving us w/ situations where
mpls install would fail because platform_labels was 0.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-11-27 20:22:14 -05:00
Philippe Guibert
f561216813 lib: add zebra api to shutdown an interface
this function operates on vtysh, and performs a shutdown on an
interface.

Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>
2018-11-27 20:22:14 -05:00
Philippe Guibert
b220b3c83a lib: add ip_route_zebra function
This command permits dumping zebra routes from a vrf

Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>
2018-11-27 20:22:14 -05:00