Attempt to run the function multiple times to make sure we got the
result we expected. This is a middle ground between big sleeps and short
test intervals.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Allow users of `luCommand` to specify wait time between commands
execution and simplify the `wait` method implementation.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Fix two main issues:
* Don't use float to figure out if we spent the time user asked;
* Don't depend on system clock to find we reached the end of time;
The fix is basically pre caculating the amount of wait cycles we are
going to peform and use a counter.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
All debug logging has been thrown to the floor in this loop.
Add it back in and additionally add a bit more debugging to
know what is going on.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
A bunch of our current tests setup data and redistribute
it across some bgp connections and then test for it
being there. A delay of 2 seconds that was initially
there to ensure that the data has propagated does not
actually work all the time when you have a loaded
virtualized environment.
Make the sleep 10 seconds. I agree this is not the ideal
solution but I would rather get the damn tests up and running
again.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>
frr-reolad.py invocation sends info message to stderror channel
Redirected stderror from frr-reload.py to stdout
Fixed verify_convergence when more than one link is present
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>
Creates logdir everytime in start_topology, removed stop_topoloy.
Using tgen.stop_topology in test case now.
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>
Adds mulitple test scenarios for prefix-lists in bgp. Verfication
API for prefix-lits
Also adds description for bgp-basic test suite
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>
Adding verify and config apis for bgp timer testcase
Adding verify and config apis for changing ASN to 4 bytes
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>
Adding mulitple methods to form BGP configuration and other
helper methods.
If "bgp" is given in JSON the configuration will be created
Signed-off-by: Ashish Pant <ashish12pant@gmail.com>
Adds method to load configuration on the router. Save configuration
for resetting.
Adds method for configuring interfaces ip address on router.
Adds logdir and other pytest.ini option
Adding in the command `show log-filter` made `show log`
ambiguous. Change the checkRouterRunning() test to do
full `show logging` so it works again.
Signed-off-by: Stephen Worley <sworley@cumulusnetworks.com>
Add a common function to retrieve and canonicalize
'show ipv6 route' output for use in topotests. Use that in
the test_ospf6_topo1 test; update the corresponding 'expected'
results files to match the lib function.
Replace some 'print' with 'logger' statements in that test also.
Signed-off-by: Mark Stapp <mjs@voltanet.io>
Change the router log output to the previous folder so it doesn't get
erased when starting the old API (unbreaks command/output logging on
Topogen).
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
`param.get` always evaluates the second argument and it was causing
two log directories being created for topologies using Topogen.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Start the addition of FreeBSDRouter class that inherits from
class Router. At this point we do not do a whole lot. We
just come up on FreeBSD, lots of stuff that still needs to
be fixed.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
As part of the class rework, move the linux specific config
into the LinuxRouter.config section instead of being in the
Router class.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Modify the LinuxRouter code such that it inherits from the
Router class. This is setup work for (a) pulling out linux
specific config from class Router and (b) creating a FreebsdRouter
that inherits from class 'Router'.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
The sharp daemon did not have any ability to be used in
topotests. Add some code to allow this.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Start abstraction of the module_present into module_present_linux and freebsd
To allow for multiple platforms that the topotests can be run on.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Start the cleanup of diagnose_env to allow the running of topotests
on platforms besides linux.
So we split up diagnose_env into linux and freebsd variants.
At this point in time freebsd doesn't have any special code.
To be determined in the future.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Before this update, json_cmp_results which were formatted as strings
would not show the error mesage but just an object reference.
Signed-off-by: Christian Franke <chris@opensourcerouting.org>
Add a breadcrumb for people testing to hopefully allow
them to figure out what is going wrong when they
are testing different versions of FRR using
topotests and staticd is not running because
this version of staticd needs to be cleaned up
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Label proxy manager test covering both LDP and BGP label requests.
lib/topotest.py:restartRouter() function has been fixed in order to
allow propagating options to deamons other than zebra.
Signed-off-by: F. Aragon <paco@voltanet.io>
The code was not attempting to set the max platform_labels
on a per router basis, leaving us w/ situations where
mpls install would fail because platform_labels was 0.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
In the case where vrf backend is based on NETNS, the link enable command
switches from context so as to perform the command.
Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>
In a effort to migrate more tests to use `run_and_expect` instead of
`sleep`s, this commit imports some common functions used to compare
router output. Retrying output comparison for N times, instead of
relying on arbitrary code `sleep`s, should help CI system tests fail
less and possibly run for less time.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
When normalizing a text also remove trailing whitespace since external
tools might add them. This commit fixes a test failure in ospf_topo1 on
Ubuntu 18.04.1.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Standardize run_and_expect to show start and end time along with maximum
estimated wait time.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Don't quit on the first match. While here add some more `json_cmp` tests
to make sure this doesn't happen again.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Allow json_cmp to compare JSON values which begin with a list instead of
a dictionary.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Prior behavior of luCommand was to convert newlines to spaces in DUT
output before pattern matching. New method operating in parallel uses
re.DOTALL to mimic same behavior and allow preserving original DUT output.
The original output is needed for some scripts that parse line-by-line.
There is also some test code to compare match results using the
old way and new way and log a message if they are different. After
some short time we can develop confidence that using this new method
will not break any existing tests.
Signed-off-by: G. Paul Ziemba <paulz@labn.net>
topotest library is being added a new parameter when preparing the
configurationof each daemon. This parameter will be used to call the
daemon with some extra parameters. The -n parameter for zebra is taken
into account.
Also, when a extra parameter is given for calling zebra, a check is done to
see if it is possible to run zebra daemon with that option. This is the
case for vrfwnetns option. If not available, an error message is sent
back.
Signed-off-by: Philippe Guibert <philippe.guibert@6wind.com>
Use the pytest assert function to be able to report to pytest failures
that happened inside the API.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Remove PID files after a SIGKILL or confirmation that the pid file no
longer contains a running PID.
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
The messages level of this commit were downgraded because some of them
happen on 'non-error' situations. This should help diminish the error
log verbosity on the CI-system run.
It was found a regression on an edge case when the second number in the
comparison was (at least) 2 numbers longer the comparison would fail
with a wrong return value. It succeeded for some cases because the
first comparison in the exception was correct, but not the second.
Standardized function that removes format spaces (or tab) and carriage
returns characters. This function is useful to allow output text
processing without breaking diff capabilities.
Output example:
*N IA 2001:db8:2::/64 :: r2-eth0 00:03:39
Becomes:
*N IA 2001:db8:2::/64 :: r2-eth0 00:03:39
If you remove 'IA' you won't have space formatting problem anymore.
Having a generic start/stop methods for TopoGear allows TopoGen to call
start/stop for all equipments. This allows us to reduce the teardown
code by removing the necessity of having to always remember to call
each equipment clean up function.
Auto configure daemon logging files to the appropriated place. This
removes the responsibility from the test developer to set this in the
daemon configuration.
TopoRouters now create a logger (which logs to /tmp/{router_name}.log)
on start to record all commands and events that it goes through. All log
messages contain timestamps that may be used in the future to:
(1) correlate commands call with events
(2) benchmark/time command speed
Allow topotest subsystems to create their own loggers. This will help
increase log organization and allow different settings to fit the
subsystems needs.
The default logger (root) is already being used by Mininet, so to allow
customizing logging output and configuring log files Topolog was
created. Topolog is no more than a thin layer abstraction to call
logging functions without using the 'root' logger.
Use a configuration file for casual settings like:
* Verbosity level (helps when debugging mininet issues)
* Custom daemon directory (in order to support running different daemon
binaries without touching tests)
* Daemon type switch: allow running quagga without touching any test
files
Also fix the add_router() documentation to include all options.
Create a specialized assert and json_cmp() result to improve the
comparison output. With this we also got a way to display all comparison
failures instead of just the first one.
Allow vtysh_cmd() to convert JSON output to Python data structures.
This feature will be used to get vtysh JSON outputs for tests
comparsions.
Usage example:
```py
router = get_topogen().gears['r1']
json_output = router.vtysh_cmd('show ip ospf json', isjson=True)
json_cmp(json_output, {'important_key': 'important_value'})
```
Implemented a JSON compare function that tells you when a specific
subset of items exist or not inside a JSON dataset.
More details can be found in the function docstring or in the test file
lib/test_json.py.
After some feedback from mwinter@, the names of equipments are now
shorter to make it easier to type them and to keep consistency with
mininet documentation. While here, update the template and make it use
optional name parameters for clarity.
This commit changes how topology links are made in order to support
discovering who and what link is the node interface connected to. After
that, the implementation of the link state change functions were trivial
as calling a command in the node shell.
The method run() was moved from TopoRouter to TopoGear so all equipment
types can benefit from this code.
Topogen (Topology Generator) is a helper that wraps around Topotest to
simplify some of the boilerplate code. This abstraction will help the
development of new tests and new APIs without breaking the existing
ones. It also makes the relation of objects clearer, since we no longer
touch the Mininet API directly, which in turn also makes us less
vulnerable to external API changes.
Implemented two functions to help setting sysctl values:
* set_sysctl: set a sysctl and return an auditable return value
* assert_sysctl: uses the previous function to assert that the sysctl
was set
Fixes the issue for topotest to fail ot end of skipped LDP test on
a system without MPLS support
Signed-off-by: Martin Winter <mwinter@opensourcerouting.org>
When we have a kernel sub version > 10 the float conversion
of the kernel version causes 4.10 to be less than 4.5
Get the kernel version in groups on <major>.<minor> and do
comparison that way
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
