Commit Graph

172 Commits

Author SHA1 Message Date
Quentin Young
e0bebc7c22 lib: mt-safe tracebacks
can't be using them statics anymore sonny

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-29 23:32:15 +00:00
Quentin Young
0693f6fc75 lib: un-static trash buffer for pipe poker
data races

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-29 23:32:15 +00:00
Quentin Young
96ade3ed77 *: use vty_outln
Saves 400 lines

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-29 17:31:28 +00:00
Quentin Young
7feb7d7e65 lib: tighten up exit semantics for thread_fetch()
* Account for the pipe poker in poll() by explicitly returning NULL when
  we have no events, timers or file descriptors to work with
* Add a comment explaining exactly what we are doing and why

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-16 19:56:58 +00:00
Quentin Young
8797240ea4 lib: mt-safe cancel, round deux
* Update pollfds copy as well as the original
* Keep array count for copy in thread_master
* Remove last remnants of POLLHUP in .events field
* Remove unused snmpcount (lolwut)
* Improve docs
* Add missing do_thread_cancel() call in thread_cancel_event()
* Change thread_fetch() to always enter poll() to avoid starving i/o
* Remember to free up cancel_req when destroying thread_master
* Fix dereference of null pointer
* Fix dead store to timeval
* Fix missing condition for condition variable :-)

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-16 19:56:58 +00:00
Quentin Young
63ccb9cb12 lib: add mt-safe thread_cancel
This patch implements an MT-safe version of thread_cancel() in
thread_cancel_async(). Behavior as follows:

* Cancellation requests are queued into a list

* Cancellation requests made from the same pthread as the thread_master
  owner are serviced immediately (thread_cancel())

* Cancellation requests made from a separate pthread are queued and the
  call blocks on a condition variable until the owning pthread services
  the request, at which point the condition variable is signaled and
  execution continues (thread_cancel_async())

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-16 19:56:58 +00:00
Quentin Young
a587d00bac *: remove THREAD_BACKGROUND
it's just an alias for a millisecond timer used in exactly nine places
and serves only to complicate

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-08 01:53:50 +00:00
David Lamparter
fa135ba008 Merge pull request #648 from qlyoung/fix-pollfd-stack-overflow
lib: use heap for pollfds
2017-06-06 17:25:06 +02:00
Quentin Young
82fc5591f4 lib: allow infinite sleep in poll()
If fd_poll() is called with no file descriptors, an incorrect check in
the function prelude causes it to return instantly; for a thread that
wishes to poll but has no file descriptors, this results in busy
waiting. Desired behavior is to block.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-06-06 14:23:42 +00:00
Quentin Young
95db01eb22 lib: use heap for pollfds
a bunch of pollfds can cause a stack overflow when using a stack
allocated buffer...silly me...

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-31 23:21:40 +00:00
Quentin Young
8c88ac94fa lib: make writing end of pipe nonblocking
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-31 17:30:53 +00:00
Quentin Young
a772d6eae6 lib: missed AWAKEN in r/w thread scheduler
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-30 10:27:10 -04:00
Quentin Young
3bf2673b30 lib: allow pthreads to poke poll()
When scheduling a task onto a thread master owned by another pthread, we
need to lock the thread master's mutex. However, if the pthread which
owns that thread master is in poll(), we could be stuck waiting for a
very long time. To solve this, we copy all data poll() needs and unlock
during poll(). To break the target pthread out of poll(), thread_master
has gained a pipe whose reading end is passed into poll(). After an event
that requires immediate action by the target pthread, a byte is written
into the pipe in order to wake it up.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
[DL: split off from select() removal]
2017-05-30 10:27:10 -04:00
Quentin Young
75bcb3558d lib: remove select()
poll() is present on every supported platform and does not have an upper
limit on file descriptors.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
[DL: split off from AWAKEN() change]
2017-05-30 10:27:10 -04:00
Quentin Young
48cdf1a9d6 lib: enforce thread_cancel() MT-unsafe invariant
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-19 19:10:21 +00:00
Lou Berger
745ad980fe Merge pull request #538 from qlyoung/fix-stack-access-2
lib: return thread * from thread_add*
2017-05-17 07:34:30 -04:00
David Lamparter
34f99549ab Merge pull request #549 from qlyoung/fix-poll-eof
lib: correctly handle EOF when using poll()
2017-05-17 11:25:30 +02:00
Quentin Young
6ccdc60d96 lib: correctly handle EOF when using poll()
Descriptor owner should handle EOF, not thread.c

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-16 18:29:51 +00:00
Quentin Young
56a94b3668 lib: return thread pointer from thread_add*
Allow some more flexibility in case callers wish to manage their own
thread pointers and don't require or don't want the thread to keep a
back reference to its holding pointer.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-16 15:59:51 +00:00
David Lamparter
896014f4bc *: make consistent & update GPLv2 file headers
The FSF's address changed, and we had a mixture of comment styles for
the GPL file header.  (The style with * at the beginning won out with
580 to 141 in existing files.)

Note: I've intentionally left intact other "variations" of the copyright
header, e.g. whether it says "Zebra", "Quagga", "FRR", or nothing.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-05-15 16:37:41 +02:00
Quentin Young
368d025eca lib: fix access to stack value
Passing stack value to thread_add_* causes thread->ref to become an
invalid pointer when the value goes out of scope

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-15 06:29:54 +00:00
Quentin Young
32d86f8b7a lib: safely check & set thread pointers
When scheduling a thread, the scheduling function returns a pointer to
the struct thread that was placed on one of the scheduling queues in the
associated thread master. This pointer is used to check whether or not
the thread is scheduled, and is passed to thread_cancel() should the
daemon need to cancel that particular task.

The thread_fetch() function is called to retrieve the next thread to
execute. However, when it returns, the aforementioned pointer is not
updated. As a result, in order for the above use cases to work, every
thread handler function must set the associated pointer to NULL. This is
bug prone, and moreover, not thread safe.

This patch changes the thread scheduling functions to return void. If
the caller needs a reference to the scheduled thread, it must pass in a
pointer to store the pointer to the thread struct in. Subsequent calls
to thread_cancel(), thread_cancel_event() or thread_fetch() will result
in that pointer being nulled before return. These operations occur
within the thread_master critical sections.

Overall this should avoid bugs introduced by thread handler funcs
forgetting to null the associated pointer, double-scheduling caused by
overwriting pointers to currently scheduled threads without performing a
nullity check, and the introduction of true kernel threads causing race
conditions within the userspace threading world.

Also removes the return value for thread_execute since it always returns
null...

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-09 20:44:22 +00:00
Quentin Young
ffa2c8986d *: remove THREAD_ON macros, add nullity check
The way thread.c is written, a caller who wishes to be able to cancel a
thread or avoid scheduling it twice must keep a reference to the thread.
Typically this is done with a long lived pointer whose value is checked
for null in order to know if the thread is currently scheduled.  The
check-and-schedule idiom is so common that several wrapper macros in
thread.h existed solely to provide it.

This patch removes those macros and adds a new parameter to all
thread_add_* functions which is a pointer to the struct thread * to
store the result of a scheduling call. If the value passed is non-null,
the thread will only be scheduled if the value is null. This helps with
consistency.

A Coccinelle spatch has been used to transform code of the form:

  if (t == NULL)
    t = thread_add_* (...)

to the form

  thread_add_* (..., &t)

The THREAD_ON macros have also been transformed to the underlying
thread.c calls.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-09 20:44:19 +00:00
Donald Sharp
6281320446 Merge remote-tracking branch 'origin/stable/3.0' 2017-05-08 16:51:21 -04:00
Quentin Young
6a5fb771a9 lib: abort if fd >= FD_SETSIZE and using select
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-05-07 04:42:07 +00:00
Donald Sharp
704ef00bb5 lib: Rename HAVE_POLL to prevent conflicting #defines
Rename HAVE_POLL to HAVE_POLL_CALL, when compiling with
snmp and poll enabled this was causing issues.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2017-05-04 10:32:41 -04:00
Quentin Young
705f21797e lib: allow nonblocking thread_fetch()
This change adds three fields to thread_master and associated code to
use them. The fields are:

 * long selectpoll_timeout

This is a millisecond value that, if nonzero, will override the
internally calculated timeout for select()/poll(). -1 indicates
nonblocking while a positive value indicates the desired timeout in
milliseconds.

 * bool spin

This indicates whether a call to thread_fetch() should result in a loop
until work is available. By default this is set to true, in order to
keep the default behavior. In this case a return value of NULL indicates
that a fatal signal was received in select() or poll(). If it is set to
false, thread_fetch() will return immediately. NULL is then an
acceptable return value if there is no work to be done.

 * bool handle_signals

This indicates whether or not the pthread that owns the thread master
is responsible for handling signals (since this is an MT-unsafe
operation, it is best to have just the root thread do it). It is set to
true by default. Non-root pthreads should set this to false.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-04-30 23:06:14 +00:00
Quentin Young
2c70efaed1 lib: additional thread.c MT-safety work
Fixes a few insufficient critical sections. Adds back locking for
thread_cancel(), since while thread_cancel() is only safe to call from
the pthread which owns the thread master due to races involving
thread_fetch() modifying thread master's ready queue, we still need
mutual exclusion here for all of the other public thread.c functions to
maintain their MT-safety.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-04-28 22:43:37 +00:00
Quentin Young
1189d95fca lib: make thread.c pthread-safe
This change introduces synchronization mechanisms to thread.c in order
to allow safe concurrent use.

Thread.c should now be threadstafe with respect to:
* struct thread
* struct thread_master

Calls into thread.c for operations upon data of this type should not
require external synchronization.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2017-04-28 22:43:36 +00:00
David Lamparter
4c0a782d47 lib: remove unused add_snmp_pollfds()
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-03-24 12:46:26 +01:00
David Lamparter
4525281af1 *: get rid of zlog(*, LOG_LEVEL, ...)
Result of running the following Coccinelle patch + fixups:
<<EOF
/* long-forms: zlog(NULL, <level>, ...)
 *    =>       zlog_level(...)
 */

@@
expression list args;
@@
- zlog(NULL, LOG_DEBUG, args)
+ zlog_debug(args)

@@
expression list args;
@@
- zlog(NULL, LOG_NOTICE, args)
+ zlog_notice(args)

@@
expression list args;
@@
- zlog(NULL, LOG_INFO, args)
+ zlog_info(args)

@@
expression list args;
@@
- zlog(NULL, LOG_WARNING, args)
+ zlog_warn(args)

@@
expression list args;
@@
- zlog(NULL, LOG_ERR, args)
+ zlog_err(args)

/* long-forms: zlog(base->log, <level>, ...)
 *    =>       zlog_level(...)
 */

@@
expression base;
expression list args;
@@
- zlog(base->log, LOG_DEBUG, args)
+ zlog_debug(args)

@@
expression base;
expression list args;
@@
- zlog(base->log, LOG_NOTICE, args)
+ zlog_notice(args)

@@
expression base;
expression list args;
@@
- zlog(base->log, LOG_INFO, args)
+ zlog_info(args)

@@
expression base;
expression list args;
@@
- zlog(base->log, LOG_WARNING, args)
+ zlog_warn(args)

@@
expression base;
expression list args;
@@
- zlog(base->log, LOG_ERR, args)
+ zlog_err(args)

EOF

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-03-08 00:15:39 +01:00
Donald Sharp
7df2e1c379 Merge remote-tracking branch 'origin/stable/2.0' 2017-02-07 13:20:12 -05:00
Donald Sharp
f0d975f752 lib: Fixup CR comment
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2017-02-07 04:54:40 -05:00
Donald Sharp
f96872f001 lib: Don't use malloc!
thread.c was using malloc.  This was found by the
assert added to memory.c.

Fixes issues #170

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2017-02-06 12:52:28 -05:00
Donald Sharp
907ca8e4d5 lib: Inadvertent assignment in ? operator
In the case where we are using select as
the operator *and* we call
funcname_thread_add_read_write *and* the
fd is already set, we would overwrite
the read/write direction to always be READ.

Clearly this was a bad idea.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2017-02-01 19:47:39 -05:00
David Lamparter
e0e2a99068 lib: cope with negative timeout in thread.c
Since time is no longer cached, if we schedule something with zero
timeout, it will automatically be negative by the time we reach the
event loop.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-01-23 18:59:15 +01:00
David Lamparter
4b185cb399 lib: time: clean out thread.c
Remove quagga_get_relative(), replace with monotime().

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-01-23 18:55:33 +01:00
David Lamparter
816c2194ca ospfd: simplify SPF time consumption calculations
monotime_since() does exactly the same thing.
... and timeval_elapsed is now private to lib/thread.c

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-01-23 18:26:33 +01:00
David Lamparter
cf672a8654 *: use monotime()
This is largely a bulk-replace made with coccinelle.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-01-23 18:26:32 +01:00
David Lamparter
d9d5c3e8bf lib: time: remove local time helpers
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-01-23 18:26:32 +01:00
David Lamparter
3dad49ca94 lib: time: remove recent_time
This isn't even used anywhere.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-01-23 18:26:32 +01:00
David Lamparter
6ced0e7f10 lib: time: remove recent_relative_time()
Replace with monotime() [which is not cached].

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2017-01-23 18:26:32 +01:00
Donald Sharp
f7c62e11d6 lib: Fix 'show thread cpu' to display active threads
Fix the display of 'show thread cpu' to keep track
of the number of active threads and to display that
information.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>

lib: Fix thread_execute_crash
2017-01-17 18:43:28 -05:00
Donald Sharp
b3f5645df3 lib: Slight Optimization of thread handling.
This commit does these things:

1) Make thread_add_unuse own the setting of THREAD_UNUSED.
2) Move thread->hist finding to to thread_get.
   We are storing the thread->hist even when the thread
   is on the unused.  This means that we check to see
   if the funcname or func have changed and we get new
   history.  Else we've probably just retrieved the last
   unused which has the same func/funcanme.  This is
   a common practice to do THREAD_OFF/THREAD_ON in
   quick succession.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com.
2017-01-17 18:40:33 -05:00
David Lamparter
0b84f29490 *: make DEFUN installations file-local
This moves all install_element calls into the file where the DEFUNs are
located.  This fixes several small related bugs:

- ospf6d wasn't installing a "no interface FOO" command
- zebra had a useless copy of "interface FOO"
- pimd's copy of "interface FOO" was not setting qobj_index, which means
  "description LINE" commands would fail with an error

The next commit will do the actual act of making "foo_cmd" static.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-12-01 17:25:56 +01:00
David Lamparter
82992fed8c Merge branch 'vtysh-grammar'
Conflicts:
	isisd/isisd.c
	lib/Makefile.am
	lib/thread.c

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-11-28 17:56:29 +01:00
David Lamparter
24f5e2fc62 build: massively remove needless checks
Since we have autoconf results from a wide swath of target platforms, we
can go remove checks that have the same result on all systems.

This also removes several "fallback" implementations of functions that,
at some point in the history, weren't available on all target platforms.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-11-15 14:23:17 +09:00
Quentin Young
e52702f29d Merge branch 'cmaster-next' into vtysh-grammar
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>

Conflicts:
	bgpd/bgp_route.c
	bgpd/bgp_routemap.c
	bgpd/bgp_vty.c
	isisd/isis_redist.c
	isisd/isis_routemap.c
	isisd/isis_vty.c
	isisd/isisd.c
	lib/command.c
	lib/distribute.c
	lib/if.c
	lib/keychain.c
	lib/routemap.c
	lib/routemap.h
	ospf6d/ospf6_asbr.c
	ospf6d/ospf6_interface.c
	ospf6d/ospf6_neighbor.c
	ospf6d/ospf6_top.c
	ospf6d/ospf6_zebra.c
	ospf6d/ospf6d.c
	ospfd/ospf_routemap.c
	ospfd/ospf_vty.c
	ripd/rip_routemap.c
	ripngd/ripng_routemap.c
	vtysh/extract.pl.in
	vtysh/vtysh.c
	zebra/interface.c
	zebra/irdp_interface.c
	zebra/rt_netlink.c
	zebra/rtadv.c
	zebra/test_main.c
	zebra/zebra_routemap.c
	zebra/zebra_vty.c
2016-10-17 23:36:21 +00:00
Daniel Walton
abddf07563 all: scrubbed some argc CHECK MEs
Signed-off-by: Daniel Walton <dwalton@cumulusnetworks.com>
2016-09-29 19:51:56 +00:00
Daniel Walton
67656e9b65 all: added CHECK ME for DEFUNs that look at argc
Signed-off-by: Daniel Walton <dwalton@cumulusnetworks.com>
2016-09-29 17:48:57 +00:00
Donald Sharp
0cf74a8392 lib: Allow more space in thread.c output
When displaying thread time for long running/busy
protocols, the space allocated may not be sufficient.

Allow the runtime to take a bit more space.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-09-26 14:06:23 -04:00
Daniel Walton
c349116d7c lib: add 'int idx_foo' argv index variables
Signed-off-by: Daniel Walton <dwalton@cumulusnetworks.com>
2016-09-23 20:17:29 +00:00
Quentin Young
1bf1b05af9 lib: argv update for thread.c
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
2016-09-22 20:32:34 +00:00
Daniel Walton
5c307a08f2 Revert "lib: Get thread.c to compile"
This reverts commit 9c5f6b578e.
2016-09-22 18:23:29 +00:00
Donald Sharp
9c5f6b578e lib: Get thread.c to compile
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-09-20 21:23:53 -04:00
David Lamparter
4a1ab8e405 *: split & distribute memtypes and stop (re|ab)using lib/ MTYPEs
This is a rather large mechanical commit that splits up the memory types
defined in lib/memtypes.c and distributes them into *_memory.[ch] files
in the individual daemons.

The zebra change is slightly annoying because there is no nice place to
put the #include "zebra_memory.h" statement.

bgpd, ospf6d, isisd and some tests were reusing MTYPEs defined in the
library for its own use.  This is bad practice and would break when the
memtype are made static.

Acked-by: Vincent JARDIN <vincent.jardin@6wind.com>
Acked-by: Donald Sharp <sharpd@cumulusnetworks.com>
[CF: rebased for cmaster-next]
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
Signed-off-by: Christian Franke <chris@opensourcerouting.org>
2016-09-19 16:31:04 -04:00
Donald Sharp
49d41a26c6 isisd, lib, vtysh: Allow extract.pl to fully work
The regular expression for finding DEFUN/ALIAS in
extract.pl looks for "DEFUN (" or "ALIAS (" if
the *.c file does not have this then it will just
silently ignore the cli.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-09-17 07:25:35 -04:00
David Lamparter
56e2c5e847 lib: AgentX: use threads instead of eventloop hack
AgentX fd/timeout handling is rather hackishly monkeyed into thread.c.
Replace with code that uses plain thread_* functions.

NB: Net-SNMP's API rivals Quagga's in terms of age and absence of
documentation.  netsnmp_check_outstanding_agent_requests() in particular
seems to be unused and is therefore untested.

The most useful documentation on this is actually the blog post Vincent
Bernat wrote when he originally integrated this into lldpd and Quagga:
https://vincent.bernat.im/en/blog/2012-snmp-event-loop.html

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-07-28 07:27:47 -04:00
David Lamparter
d03c4cbd9a lib: add thread_add_timer_tv (struct timeval)
Another zoo extension, this adds a timer scheduling function that takes
a struct timeval argument (which is actually what the wrappers boil down
to, yet it's not exposed...)

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-07-28 07:27:47 -04:00
David Lamparter
16f5949d44 lib: remove a whole bunch of unused time stuff
QUAGGA_CLK_REALTIME and QUAGGA_CLK_REALTIME_STABILISED aren't used
anywhere in the code.  Remove.  The enum is kept to avoid having to
change the calls everywhere.

Same applies to the workaround code for systems that don't have a
monotonic clock.  None of the systems Quagga works on fall into that
category;  Linux, BSD and Solaris all do clock_gettime, for OSX we have
mach_absolute_time() - that covers everything.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-07-28 07:27:47 -04:00
David Lamparter
a05d8b7ae4 ospfd: monotonic clock for lsa_refresher_started
ospf->lsa_refresher_started is only used in relative timing to itself;
replace with monotonic clock which is appropriate for this.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-07-28 07:27:47 -04:00
David Lamparter
d11f748b30 *: fixup snmp support
- HAVE_POLL is overloaded by net-snmp
- missing includes
- ospf6_snmp converted to vrf_iflist()

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2016-07-24 03:47:03 -04:00
Donald Sharp
1bba4c938b lib: Fix some poll semantics
Two Fixes:

1) When a fd has both read and write as a .events.
(POLLHUP | POLLIN | POLLOUT) and a
thread_cancel_read_write call is executed
from a protocol, the code was blindly removing
the fd from consideration at all.

2) POLLNVAL was being evaluated before POLLIN|POLLOUT
were being evaluated.  While I didn't see a case
of POLLNVAL being included with other .revent flags
I decided to move the POLLNVAL and POLLHUP handling
to the same section of code.

Additionally the function thread_cancel_read_write
was poorly named and let me to poorly implement
the poll version of it.  I've renamed the function
thread_cancel_read_or_write in an attempt to
make this problem moot in the future.

Ticket: CM-11027
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
(cherry picked from commit f6da66a913)
2016-06-10 11:18:58 -04:00
Donald Sharp
69f30024c1 lib, bgpd: Fixup some more compile errors due to warnings
Original cherry-pick done on a fedora box with a completely
different compiler.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-06-04 20:45:43 -04:00
David Lamparter
d1265948c3 lib: include thread information in backtraces
now that we know what thread we're currently executing, let's add that
information to SEGV / assert backtraces.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
(cherry picked from commit 615f9f18fc025757a255f936748fc1e86e922783)
2016-05-29 20:11:42 -04:00
David Lamparter
9c7753e41a lib: unstupidify thread debug information
the library's thread scheduling functions keep track of the thread
function's name, so far so good.  However, copying the compiler-provided
constant into a buffer inside the thread structure is plain useless.
Also, strip_funcname() was trying to support something that never
happens.

Instead, let's use some bytes here to track where threads are scheduled
from.  Another commit will print that information on crashes.

Ripping out useless stuff:  -64 bytes in the thread structure
Re-add as const ptr:         +8 bytes
Extra debug info:           +12 bytes

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
(cherry picked from commit 3493b7731b750cbc62f00be94b624a08ccccf0b2)
2016-05-29 20:08:52 -04:00
Donald Sharp
f48f65d225 lib: Clean up some small static analysis warnings
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Reviewed-by: Don Slice <dslice@cumulusnetworks.com>
Reviewed-by: Daniel Walton <dwalton@cumulusnetworks.com>
2016-05-19 10:42:17 -04:00
Christian Franke
6ac446878f lib: add function to get precise remaining time of timer thread
Signed-off-by: Christian Franke <chris@opensourcerouting.org>
Acked-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-03-29 20:26:26 -04:00
Donald Sharp
c026ca1c01 lib: Really fix handling of poll
This fix is just the actual point fix of the poll event handling.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-03-28 08:07:08 -04:00
Donald Sharp
aa037235c1 Revert "lib: Fix handling of poll"
This reverts commit cc7165b62a.

This commit was causing crashes.
2016-03-28 08:04:48 -04:00
Donald Sharp
cc7165b62a lib: Fix handling of poll
poll returns the number of revents that we need to handle
in the array.  revent is a bit field of events that need
to be handled.  thread.c was treating each sub item in the
bitfield as a separate item to handle.

As such the loop over the pollfds would quit early
sometimes.

Ticket: CM-10077
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Reviewed-by: Don Slice <dslice@cumulusnetworks.com>
2016-03-25 13:06:21 -04:00
Donald Sharp
b53e10a173 lib: Size the pollfds array once
The pollfds was being resized if the # of fds grew to
be more than the original array size.  Just size it
once.

Ticket: CM-10077
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Reviewed-by: Don Slice <dslice@cumulusnetworks.com>
2016-03-25 13:05:54 -04:00
Donald Sharp
0a95a0d0b6 lib: Add ability to use poll() instead of select
This patch originated w/ Hannes Hofer <hhofer@barracuda.com>.
I've taken the patch fixed some bugs and reworked the code
to allow both poll and select to be choosen at compile time.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-03-04 12:33:57 -05:00
Donald Sharp
5d4ccd4e85 lib: Refactore thread_process_fd
thread_process_fd is looping over the read and write
fd's separately.  There is no need to do this individually.
loop over both the read and write fdset's at the same time.
This will improve select processing performance, especially
for large data sets.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-03-04 12:33:56 -05:00
Donald Sharp
8dadcae730 lib: Refactor read/write functionality
Both the read and write functions used the same code
slightly modified for reading and writing.  Combine this
code together.

Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
2016-03-04 12:33:56 -05:00
Donald Sharp
209a72a66a lib: Abstract fd set operations
Abstract FD set operations so that we can eventually
choose what type of select/poll operation that
we want to use.

Signed-off-by: Donald Sharp <sharpd@cumulusnetowkrs.com>
2016-03-04 12:33:56 -05:00
Donald Sharp
495f0b13e1 Fix some more memory issues in Quagga
Ticket: CM-4109
Reviewed-by: CCR-3414
Testing: See bug

Fixup of these memory issues:

(A) peer->clear_node_queue was accidently removed.  Add back in.
(B) Clean up bm->process_main_queue and bm->process_rsclient_queue initialization
(C) Some memory leaks
(D) Clean up unused threads
2015-09-02 05:19:44 -07:00
Denil Vira
308d14aed9 Replace lists with arrays to store read and write threads
Ticket : CM-6300
Reviewed by : CCR-3049
Testing : Manual

With arrays, a thread corresponding to given fd is looked up in constant time
versus the linear time taken for list traversals.
2015-08-11 13:14:40 -07:00
Donald Sharp
24873f0c06 lib: lib-warnings.patch
Remove compile warnings for the lib directory
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Reviewed-by:
2015-05-19 18:04:26 -07:00
Donald Sharp
ca1f4309e6 ospf6d: ospfv3-stub-area-support.patch
Support stubby and totally stubby areas in OSPFv3

Signed-off-by: Dinesh G Dutt <ddutt at cumulusnetworks.com>
Reviewed-by: Pradosh Mohapatra <pmohapat at cumulusnetworks.com>
2015-05-19 18:03:40 -07:00
Donald Sharp
50596be0d5 Some small enhancements to thread and workqueue libraries in zebra:
- Allow work queues to specify the yield duration for corresponding background thread
- Support using specified yield duration in thread yielding
- During work queue processing, if using a single list element with a meta-queue
  (like done in Zebra), do not exit after each element is processed, instead
  update the next-node upon a WQ_REQUEUE so that the WQ processing continues
  and is terminated by the yield logic.
- Enhance work queue debug output
2015-05-19 17:58:10 -07:00
Donald Sharp
cf744958ba ospfd-spf-stats.patch
Compute and display SPF execution statistics

Detailed SPF statistics, all around time spent executing various pieces of SPF
such as the SPF algorithm itself, installing routes, pruning unreachable networks
etc.

Reason codes for firing up SPF are:
R - Router LSA, N - Network LSA, S - Summary LSA, ABR - ABR status change,
ASBR - ASBR Status Change, AS - ASBR Summary, M - MaxAge

Signed-off-by: Dinesh G Dutt <ddutt@cumulusnetworks.com>
Reviewed-by: JR Rivers <jrrivers@cumulusnetworks.com>
Reviewed-by: Scott Feldman <sfeldma@cumulusnetworks.com>
Reviewed-by: Ayan Banerjee <ayan@cumulusnetworks.com>
2015-05-19 16:36:05 -07:00
David Lamparter
6b0655a251 *: nuke ^L (page feed)
Quagga sources have inherited a slew of Page Feed (^L, \xC) characters
from ancient history.  Among other things, these break patchwork's
XML-RPC API because \xC is not a valid character in XML documents.

Nuke them from high orbit.

Patches can be adapted simply by:
	sed -e 's%^L%%' -i filename.patch
(you can type page feeds in some environments with Ctrl-V Ctrl-L)

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2014-06-04 06:58:02 +02:00
Christian Franke
4becea724c lib: use heap to manage timers
Signed-off-by: Christian Franke <chris@opensourcerouting.org>
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2014-04-22 21:17:00 +02:00
Christian Franke
514991c763 lib: remove unused thread_master_debug function
Signed-off-by: Christian Franke <chris@opensourcerouting.org>
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2014-04-22 21:16:10 +02:00
Stephen Hemminger
90645f5598 hash: force size to be a power of 2
By forcing the hash table size to be a power of 2, a potentially
expensive divide can be replaced by a mask operation. Almost all
usage of the hash table was using default size of 1024. Only places
with different size was thread library (1011) and bgp aspath.

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2013-02-24 20:42:40 +01:00
Hasso Tepper
3b96b78136 lib: Implement monotonically increasing clock for Darwin.
There is no Posix CLOCK_MONOTONIC in Darwin, but monotonically
increasing clock can be implemented using mach_absolute_time().

Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2012-11-05 15:16:53 -05:00
Vincent Bernat
d6be5fb9bc agentx: add AgentX support to Quagga.
--enable-snmp will enable AgentX support in Quagga. SMUX is still here
and can be enabled with --enable-snmp=smux. AgentX support can be
enabled with "agentx" in configuration file. As for SMUX, this command
is not understood by vtysh. It can be disabled with "no agentx",
though there is no real use of this since this command cannot be used
with vtysh.

If "agentx" and "no agentx" command were added to vtysh, it would not
be possible to disable agentx support after enabling it because
NetSNMP does not expose the appropriate methods for this.

The internals of AgentX are hidden by NetSNMP. Therefore, we don't
have a file descriptor to add to the threading system. We do not have
the timers to set either. Therefore, the event loop is modified to
make use of snmp_select_info() from NetSNMP.

Traps are not supported yet.
2012-06-25 19:03:23 +02:00
Jorge Boncompte [DTI2]
22714f99c4 lib: do not allocate/free thread funcnames
This avoids memory heap fragmentation and imposses less load on the
system memory allocator.

* thread.h: FUNCNAME_LEN defined to 64 (ISO C99 says max 63)

Signed-off-by: Jorge Boncompte [DTI2] <jorge@dti2.net>
[changed FUNCNAME_LEN to a less arbitrary value]
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2012-05-22 20:50:14 +02:00
Jorge Boncompte [DTI2]
64018324d5 lib: micro-op for thread_get()
thread_trim_head() already checks that the list is not empty.

Signed-off-by: Jorge Boncompte [DTI2] <jorge@dti2.net>
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2012-05-22 20:35:51 +02:00
Jorge Boncompte [DTI2]
41af338ecd lib: remove RUSAGE_T from struct thread
* thread.c: It's only temporarily used in thread_call() to calculate the
  diffs. Saves 80 bytes per copy.

Signed-off-by: Jorge Boncompte [DTI2] <jorge@dti2.net>
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2012-05-22 20:35:51 +02:00
Jorge Boncompte [DTI2]
1b79fcb646 lib: fix thread_cancel_event()
ospfd was crashing some times on neighbour going down. The cause was that
ospf_nsm_event() was accessing already freed memory in ospf_nbr_delete()
call from ospf_nsm_event().

  What happens is that since commit b5043aab (lib: fix incorrect thread
list...) now a thread can be on the event and ready lists but
thread_cancel_event() doesn't account for that.

* thread.c: (thread_cancel_event) loop on the ready list too to cancel
  pending events.

Signed-off-by: Jorge Boncompte [DTI2] <jorge@dti2.net>
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
2012-05-21 15:41:21 +02:00
Paul Jakma
b5043aabb0 lib: fix incorrect thread list processing loops
* thread.c: (thread_timer_process,thread_process) thread_list_delete nulls
  thread->next. Loops need to save next first, or will only process the head.
  Problem noted by Lou Berger <lberger@labn.net>.
2012-02-28 18:35:45 +00:00
Paul Jakma
f63f06da2e general: remove inline qualifiers and move in-header functions to objects
* (general) Move functions in headers into files, to be compiled into
  shared object files. Remove inline qualifier from functions. Let the
  compiler do the work.
2012-01-06 16:18:18 +00:00
Paul Jakma
e276eb8282 lib: Add a command to clear the thread CPU history data
* (general) this can be useful when investigating thread latency problems,
  when you don't want to have to restart a daemon between tests.
* thread.c: (cpu_record_(hash_)clear) wipe the stored thread cpu history
  data, according to the filter, similar to the vty print code.
  (clear_thread_cpu_cmd) new command to clear data.
* thread.h: export new command
* command.c: install it
2010-12-08 16:53:09 +00:00
Paul Jakma
2613abe64f lib: Thread scheduler should be fair and not let events starve I/O and timers
* thread.c: (thread_fetch) the current scheduler will service events
  indefinitely, ignoring I/O and timers, so long as there are events.
  In other words, events can crowd out I/O and timers.  In theory this
  shouldn't be a huge problem as events are generated only by timers
  and I/O, however in practice it means normal-load behaviour is not as
  useful a predictor of high-load behaviour as it should be.

  Fix this by considering all the kinds of threads, in every run of the
  scheduler.  For any given run, we prioritise events, however across
  runs the scheduler should be fair.

  This has been observed to give more stable inter-packet times in
  testing of ospfd (i.e.  lower std-dev).

 (thread_process) new heler to queue all the given threads onto the ready
 list
2010-12-08 16:53:09 +00:00
Paul Jakma
8526100eeb lib: thread history funcname shouldn't be constant, it's freed
* thread.h: (struct cpu_thread_history.funcname) malloc gets called on this
  so it should not be const
* thread.c: (cpu_record_print) cast the (const char *) to (char *), this
  function knows what it's doing (free wont be called on it).
2010-12-08 16:53:09 +00:00
Chris Caputo
228da42898 [bgpd] Stability fixes including bugs 397, 492
I've spent the last several weeks working on stability fixes to bgpd.
These patches fix all of the numerous crashes, assertion failures, memory
leaks and memory stomping I could find.  Valgrind was used extensively.

Added new function bgp_exit() to help catch problems.  If "debug bgp" is
configured and bgpd exits with status of 0, statistics on remaining
lib/memory.c allocations are printed to stderr.  It is my hope that other
developers will use this to stay on top of memory issues.

Example questionable exit:

  bgpd: memstats: Current memory utilization in module LIB:
  bgpd: memstats:  Link List                     :          6
  bgpd: memstats:  Link Node                     :          5
  bgpd: memstats:  Hash                          :          8
  bgpd: memstats:  Hash Bucket                   :          2
  bgpd: memstats:  Hash Index                    :          8
  bgpd: memstats:  Work queue                    :          3
  bgpd: memstats:  Work queue item               :          2
  bgpd: memstats:  Work queue name string        :          3
  bgpd: memstats: Current memory utilization in module BGP:
  bgpd: memstats:  BGP instance                  :          1
  bgpd: memstats:  BGP peer                      :          1
  bgpd: memstats:  BGP peer hostname             :          1
  bgpd: memstats:  BGP attribute                 :          1
  bgpd: memstats:  BGP extra attributes          :          1
  bgpd: memstats:  BGP aspath                    :          1
  bgpd: memstats:  BGP aspath str                :          1
  bgpd: memstats:  BGP table                     :         24
  bgpd: memstats:  BGP node                      :          1
  bgpd: memstats:  BGP route                     :          1
  bgpd: memstats:  BGP synchronise               :          8
  bgpd: memstats:  BGP Process queue             :          1
  bgpd: memstats:  BGP node clear queue          :          1
  bgpd: memstats: NOTE: If configuration exists, utilization may be expected.

Example clean exit:

  bgpd: memstats: No remaining tracked memory utilization.

This patch fixes bug #397: "Invalid free in bgp_announce_check()".

This patch fixes bug #492: "SIGBUS in bgpd/bgp_route.c:
bgp_clear_route_node()".

My apologies for not separating out these changes into individual patches.
The complexity of doing so boggled what is left of my brain.  I hope this
is all still useful to the community.

This code has been production tested, in non-route-server-client mode, on
a linux 32-bit box and a 64-bit box.

Release/reset functions, used by bgp_exit(), added to:

  bgpd/bgp_attr.c,h
  bgpd/bgp_community.c,h
  bgpd/bgp_dump.c,h
  bgpd/bgp_ecommunity.c,h
  bgpd/bgp_filter.c,h
  bgpd/bgp_nexthop.c,h
  bgpd/bgp_route.c,h
  lib/routemap.c,h

File by file analysis:

* bgpd/bgp_aspath.c: Prevent re-use of ashash after it is released.

* bgpd/bgp_attr.c: #if removed uncalled cluster_dup().

* bgpd/bgp_clist.c,h: Allow community_list_terminate() to be called from
  bgp_exit().

* bgpd/bgp_filter.c: Fix aslist->name use without allocation check, and
  also fix memory leak.

* bgpd/bgp_main.c: Created bgp_exit() exit routine.  This function frees
  allocations made as part of bgpd initialization and, to some extent,
  configuration.  If "debug bgp" is configured, memory stats are printed
  as described above.

* bgpd/bgp_nexthop.c: zclient_new() already allocates stream for
  ibuf/obuf, so bgp_scan_init() shouldn't do it too.  Also, made it so
  zlookup is global so bgp_exit() can use it.

* bgpd/bgp_packet.c: bgp_capability_msg_parse() call to bgp_clear_route()
  adjusted to use new BGP_CLEAR_ROUTE_NORMAL flag.

* bgpd/bgp_route.h: Correct reference counter "lock" to be signed.
  bgp_clear_route() now accepts a bgp_clear_route_type of either
  BGP_CLEAR_ROUTE_NORMAL or BGP_CLEAR_ROUTE_MY_RSCLIENT.

* bgpd/bgp_route.c:
  - bgp_process_rsclient(): attr was being zero'ed and then
    bgp_attr_extra_free() was being called with it, even though it was
    never filled with valid data.

  - bgp_process_rsclient(): Make sure rsclient->group is not NULL before
    use.

  - bgp_processq_del(): Add call to bgp_table_unlock().

  - bgp_process(): Add call to bgp_table_lock().

  - bgp_update_rsclient(): memset clearing of new_attr not needed since
    declarationw with "= { 0 }" does it.  memset was already commented
    out.

  - bgp_update_rsclient(): Fix screwed up misleading indentation.

  - bgp_withdraw_rsclient(): Fix screwed up misleading indentation.

  - bgp_clear_route_node(): Support BGP_CLEAR_ROUTE_MY_RSCLIENT.

  - bgp_clear_node_queue_del(): Add call to bgp_table_unlock() and also
    free struct bgp_clear_node_queue used for work item.

  - bgp_clear_node_complete(): Do peer_unlock() after BGP_EVENT_ADD() in
    case peer is released by peer_unlock() call.

  - bgp_clear_route_table(): Support BGP_CLEAR_ROUTE_MY_RSCLIENT.  Use
    struct bgp_clear_node_queue to supply data to worker.  Add call to
    bgp_table_lock().

  - bgp_clear_route(): Add support for BGP_CLEAR_ROUTE_NORMAL or
    BGP_CLEAR_ROUTE_MY_RSCLIENT.

  - bgp_clear_route_all(): Use BGP_CLEAR_ROUTE_NORMAL.

  Bug 397 fixes:

    - bgp_default_originate()
    - bgp_announce_table()

* bgpd/bgp_table.h:
  - struct bgp_table: Added reference count.  Changed type of owner to be
    "struct peer *" rather than "void *".

  - struct bgp_node: Correct reference counter "lock" to be signed.

* bgpd/bgp_table.c:
  - Added bgp_table reference counting.

  - bgp_table_free(): Fixed cleanup code.  Call peer_unlock() on owner if
    set.

  - bgp_unlock_node(): Added assertion.

  - bgp_node_get(): Added call to bgp_lock_node() to code path that it was
    missing from.

* bgpd/bgp_vty.c:
  - peer_rsclient_set_vty(): Call peer_lock() as part of peer assignment
    to owner.  Handle failure gracefully.

  - peer_rsclient_unset_vty(): Add call to bgp_clear_route() with
    BGP_CLEAR_ROUTE_MY_RSCLIENT purpose.

* bgpd/bgp_zebra.c: Made it so zclient is global so bgp_exit() can use it.

* bgpd/bgpd.c:
  - peer_lock(): Allow to be called when status is "Deleted".

  - peer_deactivate(): Supply BGP_CLEAR_ROUTE_NORMAL purpose to
    bgp_clear_route() call.

  - peer_delete(): Common variable listnode pn.  Fix bug in which rsclient
    was only dealt with if not part of a peer group.  Call
    bgp_clear_route() for rsclient, if appropriate, and do so with
    BGP_CLEAR_ROUTE_MY_RSCLIENT purpose.

  - peer_group_get(): Use XSTRDUP() instead of strdup() for conf->host.

  - peer_group_bind(): Call bgp_clear_route() for rsclient, and do so with
    BGP_CLEAR_ROUTE_MY_RSCLIENT purpose.

  - bgp_create(): Use XSTRDUP() instead of strdup() for peer_self->host.

  - bgp_delete(): Delete peers before groups, rather than after.  And then
    rather than deleting rsclients, verify that there are none at this
    point.

  - bgp_unlock(): Add assertion.

  - bgp_free(): Call bgp_table_finish() rather than doing XFREE() itself.

* lib/command.c,h: Compiler warning fixes.  Add cmd_terminate().  Fixed
  massive leak in install_element() in which cmd_make_descvec() was being
  called more than once for the same cmd->strvec/string/doc.

* lib/log.c: Make closezlog() check fp before calling fclose().

* lib/memory.c: Catch when alloc count goes negative by using signed
  counts.  Correct #endif comment.  Add log_memstats_stderr().

* lib/memory.h: Add log_memstats_stderr().

* lib/thread.c: thread->funcname was being accessed in thread_call() after
  it had been freed.  Rearranged things so that thread_call() frees
  funcname.  Also made it so thread_master_free() cleans up cpu_record.

* lib/vty.c,h: Use global command_cr.  Add vty_terminate().

* lib/zclient.c,h: Re-enable zclient_free().
2009-07-19 18:28:08 +01:00
Paul Jakma
41b2373cec [lib/cleanup] Use a typedef for the thread type
* lib/thread.{c,h}: As per subject. This will avoid head-scratching for next
  person who adds a thread-type and gets strange breakage.
2009-06-30 16:12:49 +01:00
Joakim Tjernlund
b8192765a3 [lib] Fix timer precision.
Whenever a thread adds an timer funcname_thread_add_timer_timeval() gets called
to add the timer. Before adding the timer a quagga_gettimeofday() call
is made to do some time house keeping. However quagga_gettimeofday() only
updates recent_time, not relative_time that is used to calculate
the alarm_time. Replace with quagga_get_relative (NULL)
2008-11-29 18:04:22 +00:00