api: ceph osd: factor out getting PSS stat & improve error handling

Do not crowd the higher level API endpoint handler code directly with
some rather low level procfs parsing code, rather factor that out in a
helper. Make said helper private for now so that anybody wanting to
use cannot do so, and thus increase the chance that said dev will
actually think about if this makes sense as is as a general interface.

Avoid fatal die's for the odd case that the smaps_rollup file cannot
be opened, or the even less likely case where PSS stats cannot be
found in the content.

The former could happen due to the general TOCTOU race here, i.e., the
PID we get from systemctl service status parsing isn't guaranteed to
exist anymore when we read from procfs, and if, it's actually not
guaranteed to still be the OSD - but we cannot easily use pidfd's
here and OSD stops are not something that happens frequently, but in
anyway avoid that such a thing fails the whole API call only because a
single metric is affected.

In the long rung it might be better to add a "errors" array to the
response, so that the user can be informed about such an (odd) thing
happening.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This commit is contained in:
Thomas Lamprecht 2023-09-04 14:24:17 +02:00
parent f7b7e942a7
commit b4b39b55f8

View File

@ -70,6 +70,24 @@ my $get_osd_usage = sub {
return $osdstat;
};
my sub get_proc_pss_from_pid {
my ($pid) = @_;
return if !defined($pid) || $pid <= 1;
open (my $SMAPS_FH, '<', "/proc/$pid/smaps_rollup")
or die "failed to open PSS memory-stat from process - $!\n";
while (my $line = <$SMAPS_FH>) {
if ($line =~ m/^Pss:\s+([0-9]+) kB$/) { # using PSS avoids bias with many OSDs
close $SMAPS_FH;
return int($1) * 1024;
}
}
close $SMAPS_FH;
die "internal error: failed to find PSS memory-stat in procfs for PID $pid\n";
}
__PACKAGE__->register_method ({
name => 'index',
path => '',
@ -702,26 +720,14 @@ __PACKAGE__->register_method ({
];
run_command($cmd, errmsg => 'fetching OSD PID and memory usage failed', outfunc => $parser);
my $memory = 0;
if ($pid && $pid > 0) {
open (my $SMAPS, '<', "/proc/$pid/smaps_rollup")
or die "failed to read PSS memory-stat from process - $!\n";
while (my $line = <$SMAPS>) {
if ($line =~ m/^Pss:\s+([0-9]+) kB$/) {
$memory = $1 * 1024;
last;
}
}
close $SMAPS;
}
my $osd_pss_memory = eval { get_proc_pss_from_pid($pid) } // 0;
warn $@ if $@;
my $data = {
osd => {
hostname => $metadata->{hostname},
id => $metadata->{id},
mem_usage => $memory,
mem_usage => $osd_pss_memory,
osd_data => $metadata->{osd_data},
osd_objectstore => $metadata->{osd_objectstore},
pid => $pid,