When our ci test system is under high load, expecting bfd to converge
in under 2 seconds is not going to happen. Modify the test suites
to just ensure that things converge. If we need actual functional
testing of bfd response times the topotests are not an appropriate place
to do this or we need to modify the test system to gather the data for
how long it takes after the tests are run.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
During a local CI run, bgp_ecmp_topo3 was failing
to properly notice the fast-convergence command
issued before the interface is shut down. As
such there exists a race condition where under
high load the zebra process can actually shut
an interface down before we have properly ensured
that fast convergence is on for ibgp.
Modify the test for in two ways:
1) Ensure that previous section makes sure
that we have properly converged for when we
bring back up the interfaces instead of
assuming that we have done so.
2) After issuing the fast-convergence command.
Ensure that bgp has fully processed it and is
ready to receive the interface down events
as triggers for shutting down the ibgp session.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
On a local CI run. The test_ldp_topo1.py showed fail to converge
on r3. r3 has 2 neighbors but only 1 was up when we got to
further steps in the test suites.
Modify the neighbor checking to `know` how many neighbors
should be operational and continue looking for them until
they are up and running.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Previously, when a valgrind memleak was discovered, would cause a
catastrophic pytest failure. Now correctly fails the current pytest as
intended.
As a result of this fix --valgrind-memleaks now works in distributed
pytest mode as well.
Signed-off-by: Christian Hopps <chopps@labn.net>
Revert the accidental enabling of the optional memleak tests that came
with the large micronet changeset.
Signed-off-by: Christian Hopps <chopps@labn.net>
Our topotests send SIGBUS 2 seconds after a SIGTERM is
initiated. This is bad because under a heavily loaded
topotest system we may have a case where the system has
not had a chance to properly shut down the daemon.
Extend the time greatly before topotests send SIGBUS.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
*_anywhere(item) returns whether an item is on _any_ container. Only
available for unsorted containers for now.
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
This provides a "is this item on this list" check, which may or may not
be faster than using *_find() for the same purpose. (If the container
has no faster way of doing it, it falls back to using *_find().)
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
Even if it doesn't matter for an unit test in general, it hides actual
leaks in the code being tested. Fix so any leaks will be actual bugs.
(Currently there aren't any, yay.)
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
This script is failing occassionally in our upstream topotests.
Where it was changing route-maps and attempting to see if
summarization was working correctly. The problem was that
the code appeared to be attempting to add route-maps to
redistribution in ospf then modifying the route-maps behavior
to affect summarization as well as the metric type of that
summarization.
The problem is of course that ospf does not appear to modify
the summary routes metric-type when the components
of that summary change it's metric-type. So the test
is testing nothing. In addition the test had messed
up the usage of the route-map generation code and all
the generated config was in different sequence numbers
but route-map processing would never get to those
new sequence numbers because of how route-maps are processed.
Let's just remove this part of the test instead of trying
to unwind it into anything meaningfull
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Several tests used the route_map_create functionality
with `metric-type` but never bothered to add the
backend code to ensure it works correctly.
Add it in so it can be used.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
We have this pattern in this test:
# Let's kill the interface on rt2 and see what happens with the RIB and BFD on rt1
tgen.gears["rt2"].link_enable("eth-rt1", enabled=False)
# By default BFD provides a recovery time of 900ms plus jitter, so let's wait
# initial 2 seconds to let the CI not suffer.
topotest.sleep(2, 'Wait for BFD down notification')
router_compare_json_output(
"rt1", "show ip route ospf json", "step3/show_ip_route_rt2_down.ref", 1, 0
)
Under a heavy CI load, interface down events and then reacting to them may not actually
happen within 2 seconds. Allow some more grace time in the test to ensure that we
react to it in an appropriate manner.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
OSPF when it is deciding on whom it should elect for DR and backup
has a process that prioritizes network stabilty over the exact
same results of who is the DR / Backups.
Essentially if we have r1 ----- r2
Let's say r1 has a higher priority, but r2 comes up first, starts
sending hello packets and then decides that it is the DR. At some
point in time in the future, r1 comes up and then connects to r2
at that point it sees that r2 has elected itself DR and it keeps
it that way.
This is by design of the system. With our tight ospf timers as
well as high load being experienced on our test systems. There
exists a bunch of ospf tests that we cannot guarantee that a
consistent DR will be elected for the test. As such let's not
even pretend that we care a bunch and just look for `Full`.
If we care about `ordering` we need to spend more time getting
the tests to actually start routers, ensure that htey are up and
running in the right order so that priority can take place.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Fix a loop in the setup phase of isis_topo1_vrf: only configure
interfaces that each router actually has.
Signed-off-by: Mark Stapp <mstapp@nvidia.com>
Ensure GR helpers have received a Grace-LSA before killing the
ospfd/ospf6d process that is undergoing a graceful restart.
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
There's no more difference between number-named and word-named access-lists.
This commit removes separate arguments for number-named ACLs from CLI.
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
BGP LU will use implicit-null in more situations now; adjust
the original LU topotest to align with that. Node R2 uses
imp-null now, while R1 continues to allocate labels.
Signed-off-by: Mark Stapp <mstapp@nvidia.com>
Add a second BGP labelled-unicast (BGP-LU) test suite, with
an additional router and some additional tests.
Signed-off-by: Mark Stapp <mstapp@nvidia.com>
2 things:
a) Each test was setting up for graceful restart with calls to
`graceful-restart prepare ip[v6] ospf`, then sleeping for
3 or 5 seconds. Then killing the ospf process. Under heavy
load there is no guarantee that zebra has received/processed
this signal. Write some code to ensure that this happens
b) Tests are issuing commands in this order:
1) issue gr prepare command
2) kill router
3) <ensure routes were still installed in zebra>
4) start router
5) <ensure routes were stil installed in zebra>
Imagine that the system is under some load and there is
a small amount of time before step 5 happens. In this
case ospf could have come up and started neighbor relations
and also started installing routes. If zebra receives
a new route before step 5 is issued then the route could
be in a state where it is not installed, because it is
being sent to the kernel for installation. This would
fail the test because it would only look 1 time. This
is fixed by giving time on restart for the routes to
be in the installed state.
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
Any command that uses `peer_lookup_in_view` crashes when "vrf all" is
used, because bgp is NULL in this case.
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
FRR should only ever use the appropriate THREAD_ON/THREAD_OFF
semantics. This is espacially true for the functions we
end up calling the thread for.
Signed-off-by: Donatas Abraitis <donatas.abraitis@gmail.com>
Remove references to the deprecated "CLI()" function; clean up
a couple of string escapes; make one test-case sensitive to
previous failures.
Signed-off-by: Mark Stapp <mstapp@nvidia.com>
Some tests had commented-out references to the old "CLI()"
function. Remove those so they're not confusing in the future,
and replace at least one with a comment that uses the
'mininet_cli()' function.
Signed-off-by: Mark Stapp <mstapp@nvidia.com>
* Add new debug directives for NSSA LSAs;
* Remove the "debug ospf6 gr helper" command since it doesn't make
sense for this test (not to mention it was renamed to "debug ospf6
graceful-restart");
* Migrate to the new interface-level command to enable OSPFv3 on
interfaces ("interface WORD area A.B.C.D" was deprecated).
Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Document the `sleep` statement so people know that we are sleeping
because we are waiting for the BFD down notification. If we don't
sleep here it is possible that we get outdated `show` command results.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Call the `show` commands less often to reduce the CPU pressure.
Also increase the wait time from 60 to 80 seconds to have spare room
for failures (4 times more). This is the latest measure wait time:
> INFO: topolog: 'router_json_cmp' succeeded after 20.08 seconds
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Reduce timers so we send hello packets more often and reduce dead
interval to converge faster.
Previous test wait amount:
> INFO: topolog: 'router_json_cmp' succeeded after 47.20 seconds
New test wait amount:
> INFO: topolog: 'router_json_cmp' succeeded after 20.08 seconds
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
FRR should only ever use the appropriate THREAD_ON/THREAD_OFF
semantics. This is espacially true for the functions we
end up calling the thread for.
Signed-off-by: Donatas Abraitis <donatas.abraitis@gmail.com>
