mirror of
https://git.proxmox.com/git/mirror_frr
synced 2025-06-05 00:33:53 +00:00
a0b937de42
Add a default limit to the InQ for messages off the bgp peer
socket. Make the limit configurable via cli.
Adding in this limit causes the messages to be retained in the tcp
socket and allow for tcp back pressure and congestion control to kick
in.
Before this change, we allow the InQ to grow indefinitely just taking
messages off the socket and adding them to the fifo queue, never letting
the kernel know we need to slow down. We were seeing under high loads of
messages and large perf-heavy routemaps (regex matching) this queue
would cause a memory spike and BGP would get OOM killed. Modifying this
leaves the messages in the socket and distributes that load where it
should be in the socket buffers on both send/recv while we handle the
mesages.
Also, changes were made to allow the ringbuffer to hold messages and
continue to be filled by the IO pthread while we wait for the Main
pthread to handle the work on the InQ.
Memory spike seen with large numbers of routes flapping and route-maps
with dozens of regex matching:
```
Memory statistics for bgpd:
System allocator statistics:
Total heap allocated: > 2GB
Holding block headers: 516 KiB
Used small blocks: 0 bytes
Used ordinary blocks: 160 MiB
Free small blocks: 3680 bytes
Free ordinary blocks: > 2GB
Ordinary blocks: 121244
Small blocks: 83
Holding blocks: 1
```
With most of it being held by the inQ (seen from the stream datastructure info here):
```
Type : Current# Size Total Max# MaxBytes
...
...
Stream : 115543 variable 26963208 15970740 3571708768
```
With this change that memory is capped and load is left in the sockets:
RECV Side:
```
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
ESTAB 265350 0 [fe80::4080:30ff:feb0:cee3]%veth1:36950 [fe80::4c14:9cff:fe1d:5bfd]:179 users:(("bgpd",pid=1393334,fd=26))
skmem:(r403688,rb425984,t0,tb425984,f1816,w0,o0,bl0,d61)
```
SEND Side:
```
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
ESTAB 0 1275012 [fe80::4c14:9cff:fe1d:5bfd]%veth1:179 [fe80::4080:30ff:feb0:cee3]:36950 users:(("bgpd",pid=1393443,fd=27))
skmem:(r0,rb131072,t0,tb1453568,f1916,w1300612,o0,bl0,d0)
```
Signed-off-by: Stephen Worley <sworley@nvidia.com>
|
||
|---|---|---|
| .. | ||
| _static | ||
| images | ||
| .gitignore | ||
| babeld.rst | ||
| basic.rst | ||
| bfd.rst | ||
| bgp.rst | ||
| bmp.rst | ||
| bugs.rst | ||
| conf.py | ||
| eigrpd.rst | ||
| extlog.rst | ||
| fabricd.rst | ||
| filter.rst | ||
| flowspec.rst | ||
| frr-reload.rst | ||
| glossary.rst | ||
| grpc.rst | ||
| index.rst | ||
| installation.rst | ||
| ipv6.rst | ||
| isisd.rst | ||
| kernel.rst | ||
| ldpd.rst | ||
| Makefile | ||
| nhrpd.rst | ||
| ospf6d.rst | ||
| ospf_fundamentals.rst | ||
| ospfd.rst | ||
| overview.rst | ||
| packet-dumps.rst | ||
| pathd.rst | ||
| pbr.rst | ||
| pim.rst | ||
| pimv6.rst | ||
| ripd.rst | ||
| ripngd.rst | ||
| routemap.rst | ||
| routeserver.rst | ||
| rpki.rst | ||
| scripting.rst | ||
| setup.rst | ||
| sharp.rst | ||
| snmp.rst | ||
| snmptrap.rst | ||
| static.rst | ||
| subdir.am | ||
| Useful_Sysctl_Settings.md | ||
| vnc.rst | ||
| vrrp.rst | ||
| vtysh.rst | ||
| watchfrr.rst | ||
| wecmp_linkbw.rst | ||
| zebra.rst | ||