Commit Graph

41 Commits

Author SHA1 Message Date
Donald Sharp
bee1bcfd01 tests: Add a topology that supports a large number of ecmp
Add a basic topology that allows the testing of BGP and zebra
at scale.  I built this to help me find and fix problems with
a large number of bgp peers.  Since I plan to keep using this
and as I understand it there are future plans to take this
higher, I would like to add this as a test that people can invoke
with this command:

sudo -E python3 -m pytest -s -vv --topology-only

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2024-11-05 10:19:39 -05:00
Christian Hopps
df2133f803 tests: add asyncio default config param to quiet warning
Update the topotest doc to the latest pytest* pkg versions
known to work together

Signed-off-by: Christian Hopps <chopps@labn.net>
2024-09-09 16:35:28 -04:00
Christian Hopps
08253740ef tests: fix munet native topotest runs
Enabe/fix using a munet.yaml config file for topology configuration.
Easier test writing.

This also uses the standard `frrinit.sh` to launch and teardown
FRR, so we actually test what most users use.

Signed-off-by: Christian Hopps <chopps@labn.net>
2024-09-03 00:45:48 -04:00
Donald Sharp
c12f88cf66 tests: Add a pytest marker for the fpm
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2024-03-04 11:06:35 -05:00
Donald Sharp
b1a39549df tests: Add mgmtd to the official pytest.mark list
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2024-01-08 07:58:14 -05:00
Louis Scalbert
c040637e97 topotests: use exabgp 4.2.11 in ci
Use exabgp 4.2.11 in CI instead.

Signed-off-by: Louis Scalbert <louis.scalbert@6wind.com>
2023-12-14 10:57:29 +01:00
Christian Hopps
fd743a6a8e tests: backtraces/cores now fail tests
Previously we just raised an error for the test file with possibly all tests
passing. Now we fail any test that produces new backtraces or cores. This
just works a lot better with analysis and even CI.

Also be less verbose to the console after failure runs, just show error
level and above logs. The log files still capture all levels (DEBUG).

Signed-off-by: Christian Hopps <chopps@labn.net>
2023-07-15 00:26:03 -04:00
Christian Hopps
f3525b0b17 tests: fix implicit config file and recently added logic error
- Restore default of looking for a daemon config underneath the router directory
if no config file was specified.
- Recent change for adding unified config support had a logic bug, fix
- Update the one test that conflicted with this default
- comment out asyncio option causing warnings if asyncio wasn't installed.

Signed-off-by: Christian Hopps <chopps@labn.net>
2023-05-17 16:05:53 -04:00
Christian Hopps
a929d94ff5 tests: add an example of using munet native config
Signed-off-by: Christian Hopps <chopps@labn.net>
2023-04-23 15:02:46 -04:00
Christian Hopps
0def198cd3 tests: enable using munet native configs
Signed-off-by: Christian Hopps <chopps@labn.net>
2023-04-23 09:52:39 -04:00
Christian Hopps
60e037780e tests: switch to munet
Signed-off-by: Christian Hopps <chopps@labn.net>
2023-04-15 13:29:38 -04:00
Kuldeep Kashyap
3748e8d030 tests: Add pim6d marker to pytest.ini
Added pim6d marker to pytest.ini file,
to run tests pim6d marker based, if added
to scripts.

Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
2022-11-07 02:19:23 -08:00
Christian Hopps
1f87861ecc tests: revert default enable of memleak tests
Revert the accidental enabling of the optional memleak tests that came
with the large micronet changeset.

Signed-off-by: Christian Hopps <chopps@labn.net>
2021-10-22 17:44:47 +00:00
Christian Hopps
351bc82cac tests: micronet: update defaults for results+logging
Signed-off-by: Christian Hopps <chopps@labn.net>
2021-09-04 09:04:46 -04:00
Donatas Abraitis
93a3eae793 tests: Skip example_topojson_test and example_test from being tested
Signed-off-by: Donatas Abraitis <donatas.abraitis@gmail.com>
2021-05-12 09:00:30 +03:00
Donald Sharp
b9a210f951 tests: Add pytest.mark.snmp
Add the ability to run snmp tests

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2021-03-01 09:17:53 -05:00
lynne
0883be0c5e tests: restore isis-lsp-bits-topo1 test
Signed-off-by: Lynne Morrison <lynne@voltanet.io>
2021-02-01 16:04:10 -05:00
lynne
d61731fbe3 tests: temporarily remove isis-lsp-bits-topo1 topotest
Signed-off-by: Lynne Morrison <lynne@voltanet.io>
2021-01-28 13:37:52 -05:00
David Schweizer
309638adb5
tests: pytest marker registrations for all daemons
Signed-off-by: David Schweizer <dschweizer@opensourcerouting.org>
2021-01-21 16:24:02 +01:00
Donald Sharp
3771ded194 tests: Add pytest.mark.pbr
Add a mark to topotests that mark pbr tests

`pytest -k pbr`

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2021-01-15 09:18:59 -05:00
Donald Sharp
3914e57591 tests: Add pytest.mark.isis
Add a mark to topotests that mark isis tests

`pytest -k isis`

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2021-01-15 09:17:53 -05:00
Donald Sharp
1d0c76b74d tests: Add pytest.mark.bfd
Add a mark to topotests that mark bfd tests.

`pytest -k bfd`

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2021-01-15 08:58:24 -05:00
Donald Sharp
b9f3e47f3a tests: Start the ability to mark tests
Add the ability for our topotests to take advantage of pytest `mark`ing.
This effectively allows you to tell pytest to run against certain sets
of tests.  For a demonstration purpose I've added in marks for:
babel
eigrp
ldp
ospf
pim
rip

And setup tests to run against those tests that only test those protocols.

You can run against eigrp tests by running `pytest -k eigrp`
Other combinations are also available based upon simple boolean logic.
Just read the pytest.mark documentation.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>
2021-01-14 20:29:14 -05:00
Kuldeep Kashyap
c66ca86168 tests: Enable evpn_type5_test_topo1 suite to run in CI
1. Suite: evpn_type5_test_topo1 was added to pytest.ini during triaging phase as
there was bug: https://github.com/FRRouting/frr/issues/6867, which is fixed. Enabling
suite to be run in CI.

Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
2020-10-13 14:55:14 +00:00
Donald Sharp
622c4996ef tests: Our tests are for FRR
These tests at this point only work for FRR.  Let's cut to the chase
and admit it.

Signed-off-by: Donald Sharp <sharpd@nvidia.com>`
2020-09-22 10:11:58 -04:00
Kuldeep Kashyap
e66778d007 tests: Skipping evpn_type5_test_topo1 tests from CI runs
1. evpn_type5_test_topo1 tests started failing in CI for all Ubuntu 18.04 machine,
which are having kernel version: 5.4.0-42-generic
2. We will enable these tests once issue is found and fixed.

Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
2020-07-30 14:29:25 +00:00
Donald Sharp
a19aca1671 tests: Modify bgp_rr_ibgp to test RR server correctness
Fixup this basically unused test to ensure that a RR server
passes the nexthop through correctly.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2020-03-28 14:06:18 -04:00
Martin Winter
0c7f64c5cf
Merge branch 'master' into ecmp_tests 2019-10-04 15:19:17 +02:00
Ashish Pant
819f982590 tests: Enabling bgp-ecmp-topo2 test cases
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>
2019-10-03 07:15:20 +05:30
Donald Sharp
aa2a0498f3 tests: Add a topology that shows broken ibgp behavior
In a leaf/spine topology with only IBGP connections, where
the same network is being redistributed at multiple points
in the network ( say a redistribute connected at both leaf and spines )
we end up in a state where zebra gets very confused.

eva# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

C>* 192.168.1.0/24 is directly connected, tor1-eth0, 00:00:30
C>* 192.168.2.0/24 is directly connected, tor1-eth1, 00:00:30
B   192.168.3.0/24 [200/0] via 192.168.4.2 inactive, 00:00:25
                           via 192.168.6.2 inactive, 00:00:25
B>* 192.168.4.0/24 [200/0] via 192.168.2.3, tor1-eth1, 00:00:25
  *                        via 192.168.6.2 inactive, 00:00:25
C>* 192.168.5.0/24 is directly connected, tor1-eth2, 00:00:30
B>* 192.168.6.0/24 [200/0] via 192.168.4.2 inactive, 00:00:25
  *                        via 192.168.5.4, tor1-eth2, 00:00:25

Effectively we have ibgp routes recursing through ibgp routes
and there is no metric to discern whom to listen to.

This draft:
https://tools.ietf.org/html/draft-ietf-idr-bgp-optimal-route-reflection-19

appears to address this issue.  From looking at both cisco and arista
deployments they are handling this issue by having the route reflector
prefer the localy learned routes over from their clients.

Add this topology, in a broken state, so that when we do fix this issue
it is a simple matter of touching this topology up and re-adding it
to the normal daily builds.  I also wanted to add this topology
since it is in a state of `doneness` and I wanted to move onto
my normal day job without having to remember about this test.

This topology is not configured to be run as part of the normal
topotests.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2019-10-01 07:45:38 -04:00
Martin Winter
4e5441e749 tests: Disable bgp-ecmp-topo2 topotest until proper fix is developed
Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>
2019-09-13 02:13:12 +02:00
Ashish Pant
243daff303 tests: Fix topojson static route tests
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>

Fix static route test in bgp-basic-functionality-topo1 and
example-topojson-test after lib changes

Add example-topojson-test to ignore from pytest.ini
2019-09-03 07:22:33 +05:30
Ashish Pant
bca7983774 tests: Enhance execution logs in topojson
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>

1. Add check if show running output is corrupted as frr-reload does
   not return cause of failure, just the error codes.
2. Change logger level to debug for some extra information.
3. Modify logger messages for more clear information.
4. Print configuration commands to console instead of show running
5. Print show command output to console.
6. Move show running output within flag show_router_config.
7. Add retry decorator for retyring show commands.
2019-08-22 16:24:45 +05:30
Ashish Pant
7659b8d4e4 tests: Adds basic config load and interface API
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>

Adds method to load configuration on the router. Save configuration
for resetting.
Adds method for configuring interfaces ip address on router.
Adds logdir and other pytest.ini option
2019-07-09 10:26:53 +05:30
Rafael Zalamena
1e9c095c6f docker: add support for running inside container
Based on the original @cfra idea of running topotest on docker.

Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
2018-11-29 16:51:27 +01:00
Lou Berger
df437d254a lib: add LabN testing template
Signed-off-by: Lou Berger <lberger@labn.net>
2018-11-27 20:22:12 -05:00
Martin Winter
8ee1273e44 Enable eigrp-topo1
Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>
2018-11-27 20:22:12 -05:00
Donald Sharp
1fccd8e958 Exclude eigrp test from running normally yet
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2018-11-27 20:22:12 -05:00
Rafael Zalamena
c540096e86 topogen: add memory leak report configuration
Allow memory leak to be configured from the configuration file.
2018-11-27 20:22:11 -05:00
Rafael Zalamena
edd2bdf6de topogen: support configuration file
Use a configuration file for casual settings like:

* Verbosity level (helps when debugging mininet issues)
* Custom daemon directory (in order to support running different daemon
  binaries without touching tests)
* Daemon type switch: allow running quagga without touching any test
  files

Also fix the add_router() documentation to include all options.
2018-11-27 20:22:11 -05:00
Martin Winter
309c9764c5 example-test: Add simple example pytest for documentation (and exclude it from running in pytest.ini) 2018-11-27 20:22:11 -05:00