1. Added test to verify bgp vrf dynamic route leak functionality
2. Total execution time is ~8 mins
3. Added kernel version check, these script would be run for kernel version >= 4.19
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Avoid unnecessary use of StringIO in one place, use version-
dependent method in another. Remove a couple of other py2->py3
problems.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Change the public router stop method to always do a two-phase
shutdown - once without waiting and a second time with a wait.
Ordinary callers need to use this approach when stopping routers.
Move the detailed internal details to a private method that tests
should not call directly.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
The vxlan `ip... ` command is failing because we are passing in
`no learning` and that is failing.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
When the topotest mem-leak reporting is enabled, use the same
two-step daemon stop procedure that's used in
the topogen.stop_topology path.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
add thread info, use "bt full" to get variables and add a bit of
disassembly for good measure.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
1. Increasing BGP convergence wait time to overcome Ubuntu 16.04 arm8 box, as
bgp neighorship is taking more time in this particular testbed.
2. Debugged bgp-ecmp-topo2 failures and here also it seems to be bgp convergence
issue, doing some enhancement in scripts to handle it
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Stop printing hard-coded 30 seconds in a couple of places in
bgp.py in the topojson infra - print the actual time
spent waiting.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
1. It will generate support bundle/sump data on test failures
2. It used /usr/lib/frr/generate_support_bundle.py utility to dump the data
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Another short timeout for bgp - make the
verify_bgp_convergence_from_running_config() api use the same
generous timeout as verify_bgp_convergence()
Signed-off-by: Mark Stapp <mjs@voltanet.io>
1. Added 7 test cases to verify bgp recursive nexthop and ebgp multi-hop functionality
2. Added framework support to automate these test cases
3. Total execution time is ~5 mins
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Use the right list of daemons to avoid trying to start zebra twice.
Change a zebra log message to INFO level to avoid stderr check
failure.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Add a few retries during router shutdown before killing a daemon. Also
work harder to start only a single instance of daemons, esp. zebra.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Instead of waiting for daemons start with `sleep`, start them with the
`-d` parameter so they can release the terminal themselves when ready.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Start logging early everything (including debug) to
`/tmp/topotest/<test>/<node>/<daemon>.{out,err}`.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Handle the duplicated code with a simple conditional: if called from
specialized API use provided daemons configuration, otherwise fallback
to old `Router` own daemon settings.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Include vrf name with interface name when topojson framework
generates interface configuration. This matches the output of
'show runn', and makes config reset less disruptive. Also
stop removing configured debugs and log output when re-generating
config.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Error Message seen:
2020-06-11 14:00:35,288 ERROR: assert failed at "test_ebgp_ecmp_topo2/test_ecmp_after_clear_bgp[redist_static]": Testcase test_ecmp_after_clear_bgp[redist_static] : Failed
Error: TIMEOUT!! BGP is not converged in 30 seconds for router r3
assert 'TIMEOUT!! BGP is not converged in 30 seconds for router r3' is True
if a retry for a failed connection is 120 seconds we should wait slightly
longer than a retry session, which this clear test was not doing.
Especially since we know our topotests are lossy on data under load.
Apparently I changed this earlier to 90 seconds, but a retry window
is 120. Not sure wtf I was thinking
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The convergence timeout in lib/bgp.py was something like 40 secs;
that's not really long enough for the CI. Raise the timeout to
40 x 3 secs.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Just a simple setup for pbr to prove it starts. Once the json
code for pbr gets in we can add more.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The involved piece of code is supposed to find a 'closest' match for two
JSON structures using another JSON diff. However, it can happen that
during that new diff the JSON structures are altered (elements from a
list are deleted when 'found'). This is in general ok when the deleted
element is part of the JSON structure which 'matches', but when it later
turns out that some other element of the structure doesn't fit, then the
whole structure should be recovered. This is now realized by using a
deepcopy for the besaid new JSON diff such that the original is only
altered (e.g. deleted) when the diff is clean.
Signed-off-by: GalaxyGorilla <sascha@netdef.org>
1. Test suite has 25 test cases to verify BGP-graceful-restart functionality
2. Execution time is ~21 Mins
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Adding APIs to common_config.py to support BGP-Graceful-Restart automation
2. Adding APIs to create BGP-GR config to bgp.py
3. Adding verification API for BGP-GR functionality
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Diff'ing JSON objects is a crucial operation in the topotests for
comparing e.g. vtysh output (formatted as JSON) with a file which
covers the expectation of the tests. The current diff functionality
is 'self-written' and intended to test a JSON object d2 on being a
subset of another JSON object d1. For mismatches a diff is generated
based on a normalized textual representation of the JSON objects.
This approach has several disadvantages:
* the human provided JSON text might not be normalized, hence
a diff with line numbers might be worthless since it provides
close to zero orientation what the problem is
* the diff contains changes like commatas which are meaningless
* the diff might contain a lot of changes about meaningless
content which is present in d1 but not in d2
* there is no proper functionality to test for 'equality' of
d1 and d2
* it is not possible to test for order, e.g. JSON arrays are
just tested with respect to being a subset of another array
* it is not possible to check if a key exists without also
checking the value of that particular key
This commit attempts to solve these issues. An error report is
generated which includes the "JSON Path" to the problematic JSON
elements and also hints on what the actual problem is (e.g. missing
key, mismatch in dict values etc.).
A special parameter 'exact' was introduced such that equality can be
tested. Also there was a convention that absence of keys can be
tested using the key in question with value 'None'. This convention
is still honored such that full backwards compatiiblity is in
place.
Further order can be tested using the new tag '__ordered__' in
lists (as first element). Example:
d1 = [1, 2, 3]
d2 = ['__ordered__', 1, 3, 2]
Tesing d1 and d2 this way will now result in an error.
Key existence can now be tested using an asterisk '*'. Example:
d1 = [1, 2, 3]
d2 = [1, '*', 3]
d1 = {'a': 1, 'b': 2}
d2 = {'a': '*'}
Both cases will result now in a clean diff for d1 and d2.
Signed-off-by: GalaxyGorilla <sascha@netdef.org>
Modify the import-check command to require the underlying prefix
to exist in the rib. General consensus is that this is the correct
behavior.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
1. Added 2 new test cases to bgp-basic-functionality-topo1
2. Enhanced 2 tests to run for both static routes and network advvertise command
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Added 5 test cases to verify BGP AS-allow-in behavior in FRR
2. Enhanced framework to support BGP AS-allow-in config(lib/bgp.py)
3. Added API in bgp.py to verify BGP RIB table(lib/bgp.py)
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Added 1 test case to verify NO-ADVERTISE Community functionality
2. Enhanced bgp.py to exclude routers from verification, if doesn't have bgp config
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Added 5 test cases to verify BGP AS-allow-in behavior in FRR
2. Enhanced framework to support BGP AS-allow-in config(lib/bgp.py)
3. Added API in bgp.py to verify BGP RIB table(lib/bgp.py)
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Used aggresive values to verify keepalive and holddown timers functionality
2. Modified variable name in lib/bgp.py
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
