This python fixture was way too complex for what is needed.
Eliminate gratuitous options/over-engineering:
- Change from non-deterministic `wait` and `attempts` to a single
`retry_timeout` value. This is both more deterministic, as well as
what the user should actually be thinking about.
- Use a fixed 2 second pause between executing the wrapped function
rather than a bunch of arbitrary choices of 2, 3 and 4 seconds
spread all over the test code.
- Get rid of the multiple variables for determining what "Positive" and
"Negative" results are. Instead just implement what all the user code
already wants, i.e., boolean False or a str (errormsg) means
"Negative" result otherwise it's a "Positive" result.
- As part of the above the inversion logic is much more comprehensible
in the fixture code (and more correct to boot).
Signed-off-by: Christian Hopps <chopps@labn.net>
Issue: There was an error reported by Pylint regarding "expected" keyword:
Unexpected keyword argument 'expected' in function call (unexpected-keyword-arg)
Fix: We have defined expected keyword in all topojson APIs.
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Tests were timing out in our test system due to lost packets and
flakiness of the lower end systems. Just set the timers to 3/10
and give them plenty of time to converge.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
While accidently running the topotests with version 3
I keep getting:
TypeError: `dict_values` object does not support indexing..
version 2 of python dict.values() returns a list.
version 3 does not
Write some code to allow both to be handled.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
1. Enhanced framework to
a. Verify fib active routes(lib/common_config.py).
b. Verify bgp multi path routes(lib/bgp.py).
c. Create mininet nodes with different names(lib/topojson.py).
4. 12 Test cases of static routing with ibgp.
Test suite execution time is ~30 minutes.
5. 12 Test cases of static routing with ebgp.
Test suite execution time is ~30 minutes.
Signed-off-by: naveen <nguggarigoud@vmware.com>
1. As per recent changes done in PR #7652, we have modified the auto-rd verification logic
2. Dev PR link: https://github.com/FRRouting/frr/pull/7652
Signed-off-by: kuldeepkash <kashyapk@vmware.com>
1. Added 2 tests to verify bgp route aggregation using summary-only and
as-set commands
2. Execution time is ~90 sec
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
We have a bunch of tests that wait *then* check a command for success/failure.
Modify the tests to check *first* then to wait. This reduces test
run times on my system by ~1400 seconds for a full run.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
1. Increasing BGP convergence wait time to overcome Ubuntu 16.04 arm8 box, as
bgp neighorship is taking more time in this particular testbed.
2. Debugged bgp-ecmp-topo2 failures and here also it seems to be bgp convergence
issue, doing some enhancement in scripts to handle it
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Stop printing hard-coded 30 seconds in a couple of places in
bgp.py in the topojson infra - print the actual time
spent waiting.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Another short timeout for bgp - make the
verify_bgp_convergence_from_running_config() api use the same
generous timeout as verify_bgp_convergence()
Signed-off-by: Mark Stapp <mjs@voltanet.io>
1. Added 7 test cases to verify bgp recursive nexthop and ebgp multi-hop functionality
2. Added framework support to automate these test cases
3. Total execution time is ~5 mins
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Error Message seen:
2020-06-11 14:00:35,288 ERROR: assert failed at "test_ebgp_ecmp_topo2/test_ecmp_after_clear_bgp[redist_static]": Testcase test_ecmp_after_clear_bgp[redist_static] : Failed
Error: TIMEOUT!! BGP is not converged in 30 seconds for router r3
assert 'TIMEOUT!! BGP is not converged in 30 seconds for router r3' is True
if a retry for a failed connection is 120 seconds we should wait slightly
longer than a retry session, which this clear test was not doing.
Especially since we know our topotests are lossy on data under load.
Apparently I changed this earlier to 90 seconds, but a retry window
is 120. Not sure wtf I was thinking
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The convergence timeout in lib/bgp.py was something like 40 secs;
that's not really long enough for the CI. Raise the timeout to
40 x 3 secs.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
1. Test suite has 25 test cases to verify BGP-graceful-restart functionality
2. Execution time is ~21 Mins
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Adding APIs to common_config.py to support BGP-Graceful-Restart automation
2. Adding APIs to create BGP-GR config to bgp.py
3. Adding verification API for BGP-GR functionality
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
Modify the import-check command to require the underlying prefix
to exist in the rib. General consensus is that this is the correct
behavior.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
1. Added 5 test cases to verify BGP AS-allow-in behavior in FRR
2. Enhanced framework to support BGP AS-allow-in config(lib/bgp.py)
3. Added API in bgp.py to verify BGP RIB table(lib/bgp.py)
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Added 1 test case to verify NO-ADVERTISE Community functionality
2. Enhanced bgp.py to exclude routers from verification, if doesn't have bgp config
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Added 5 test cases to verify BGP AS-allow-in behavior in FRR
2. Enhanced framework to support BGP AS-allow-in config(lib/bgp.py)
3. Added API in bgp.py to verify BGP RIB table(lib/bgp.py)
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Used aggresive values to verify keepalive and holddown timers functionality
2. Modified variable name in lib/bgp.py
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
1. Modified scripts to optimize execution time for route-map test suite
2. Removed unnecessary print statements from /lib/bgp.py
Signed-off-by: Kuldeep Kashyap <kashyapk@vmware.com>
