Commit Graph

93 Commits

Author SHA1 Message Date
Leon Romanovsky
359f69f76f rdma: Add the prefix for driver attributes
There is a need to distinguish between driver vs. general exposed
attributes. The most common use case is to expose some internal
garbage under extremely common and sexy name, e.g. pi, ci e.t.c

In order to achieve that, we will add "drv_" prefix to all strings
which were received through RDMA_NLDEV_ATTR_DRIVER_* attributes.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>a
Tested-by: Lijun Ou <oulijun@huawei.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-27 08:25:47 -08:00
Leon Romanovsky
78728b7ee0 rdma: Provide and reuse filter functions
Globally replace all filter function in safer variants of those
is_filtered functions, which take into account the availability/lack
of netlink attributes.

Such conversion allowed to fix a number of places in the code, where
the previous implementation didn't honor filter requests if netlink
attribute wasn't present.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:09:24 -08:00
Leon Romanovsky
5a823593d6 rdma: Perform single .doit call to query specific objects
If user provides specific index, we can speedup query
by using .doit callback and save full dump and filtering
after that.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:09:15 -08:00
Leon Romanovsky
127ff95610 rdma: Unify netlink attribute checks prior to prints
Place check if netlink attribute available in general place,
instead of doing the same check in many paces.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:09:09 -08:00
Leon Romanovsky
6da9d2517c rdma: Move QP code to separate function
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:09:05 -08:00
Leon Romanovsky
f9a73796d1 rdma: Place PD parsing print routine into separate function
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:09:00 -08:00
Leon Romanovsky
46695227d6 rdma: Move MR code to be suitable for per-line parsing
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:07:46 -08:00
Leon Romanovsky
83ea72289e rdma: Refactor CQ prints
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:07:40 -08:00
Leon Romanovsky
7d06b31f0e rdma: Simplify CM_ID print code
Refactor our the CM_ID print code.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:07:33 -08:00
Leon Romanovsky
05846c9cd3 rdma: Simplify code to reuse existing functions
Remove duplicated functions in favour general res_print_uint() call.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:07:28 -08:00
Leon Romanovsky
835d83216b rdma: Properly mark RDMAtool license
RDMA subsystem is dual-licensed with "GPL-2.0 OR Linux-OpenIB" proper
license and Mellanox submission are supposed to have this type of license.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:07:19 -08:00
Leon Romanovsky
687daf98f9 rdma: Move resource QP logic to separate file
Logically separate resource QP logic to separate file,
in order to make PD specific logic self-contained.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:06:56 -08:00
Leon Romanovsky
438fac3a25 rdma: Move out resource CM-ID logic to separate file
Logically separate resource CM-ID logic to separate file,
in order to make CM-ID specific logic self-contained.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:06:48 -08:00
Leon Romanovsky
fcdd2e0c68 rdma: Move out resource CQ logic to separate file
Logically separate resource CQ logic to separate file,
in order to make CQ specific logic self-contained.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:06:40 -08:00
Leon Romanovsky
42ed283e4a rdma: Refactor out resource MR logic to separate file
Logically separate resource MR logic to separate file,
in order to make MR specific logic self-contained.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:06:25 -08:00
Leon Romanovsky
cc6131276c rdma: Move resource PD logic to separate file
Logically separate resource PD logic to separate file,
in order to make PD specific logic self-contained.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:06:12 -08:00
Leon Romanovsky
96f59e7fdc rdma: Provide parent context index for all objects except CM_ID
Allow users to correlate allocated object with relevant parent

[leonro@server ~]$ rdma res show pd
dev mlx5_0 users 5 pid 0 comm [ib_core] pdn 1
dev mlx5_0 users 7 pid 0 comm [ib_ipoib] pdn 2
dev mlx5_0 users 0 pid 0 comm [mlx5_ib] pdn 3
dev mlx5_0 users 2 pid 548 comm ibv_rc_pingpong ctxn 0 pdn 4

[leonro@server ~]$ rdma res show cq cqn 0-100
dev mlx5_0 cqe 2047 users 6 poll-ctx UNBOUND_WORKQUEUE pid 0 comm [ib_core] cqn 2
dev mlx5_0 cqe 255 users 2 poll-ctx SOFTIRQ pid 0 comm [mlx5_ib] cqn 3
dev mlx5_0 cqe 511 users 1 poll-ctx DIRECT pid 0 comm [ib_ipoib] cqn 4
dev mlx5_0 cqe 255 users 1 poll-ctx DIRECT pid 0 comm [ib_ipoib] cqn 5
dev mlx5_0 cqe 255 users 0 poll-ctx SOFTIRQ pid 0 comm [mlx5_ib] cqn 6
dev mlx5_0 cqe 511 users 2 pid 548 comm ibv_rc_pingpong cqn 7 ctxn 0

[leonro@server ~]$ rdma res show mr
dev mlx5_0 mrlen 4096 pid 548 comm ibv_rc_pingpong mrn 4 pdn 0

[leonro@nps-server-14-015 ~]$ /images/leonro/src/iproute2/rdma/rdma res show qp
link mlx5_0/1 lqpn 0 type SMI state RTS sq-psn 0 pid 0 comm [ib_core]
link mlx5_0/1 lqpn 1 type GSI state RTS sq-psn 0 pid 0 comm [ib_core]
link mlx5_0/1 lqpn 7 type UD state RTS sq-psn 0 pid 0 comm [ib_core]
link mlx5_0/1 lqpn 8 type UD state RTS sq-psn 0 pid 0 comm [ib_ipoib]
link mlx5_0/1 lqpn 9 pdn 4 rqpn 0 type RC state INIT rq-psn 0 sq-psn 0 path-mig-state MIGRATED pid 548 comm ibv_rc_pingpong

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:05:43 -08:00
Leon Romanovsky
1dc035865d rdma: Provide unique indexes for all visible objects
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:04:02 -08:00
Leon Romanovsky
beac6a3990 rdma: Remove duplicated print code
There is no need to keep same print functions for
uint32_t and uint64_t, unify them into one function.

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:03:58 -08:00
Leon Romanovsky
a985cc06bd rdma: update uapi headers
Update rdma_netlink.h file upto kernel commit
f2a0e45f36b0 RDMA/nldev: Don't expose number of not-visible entries

Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-02-24 07:03:14 -08:00
Leon Romanovsky
b058f969df rdma: Add unbound workqueue to list of poll context types
Kernel commit f794809a7259 ("IB/core: Add an unbound WQ type to the new CQ API")
added new CQ poll context type, reflect this change in rdmatool.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2019-01-21 08:19:44 -08:00
Stephen Hemminger
80e5ddec14 rdma: update uapi headers
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2019-01-07 11:41:39 -08:00
David Ahern
dfa2c3787f Merge branch 'iproute2-master' into iproute2-next
Conflicts:
	ip/iprule.c

Signed-off-by: David Ahern <dsahern@gmail.com>
2019-01-04 12:22:47 -08:00
Leon Romanovsky
f0cabaca38 rdma: Fix incorrectly handled NLA validation
mnl_attr_type_valid() receives maximum attribute type, which means that
we were supposed to supply the latest valid netlink attribute and not
the number of attributes. Such coding mistake caused to failures while
NLA attributes were extended.

Fixes: 74bd75c2b6 ("rdma: Add basic infrastructure for RDMA tool")
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-12-31 22:15:13 -08:00
Michael Guralnik
40fc8c2cec rdma: Add print of link CapabilityMask2 flags
CapabilityMask2 is defined in IBTA spec as a member of PortInfo.
Add translation to string of new CapabilityMask2 expansion of link caps.

The flags are concatenated to current caps print as seen in this example
printing EXT_INFO flag:

root@server-22 $ rdma -d link
1/1: mlx5_0/1: subnet_prefix fe80:0000:0000:0000 lid 2 sm_lid 2 lmc 0
	state ACTIVE physical_state LINK_UP
caps: <SM, TRAP, SL_MAP, SYS_IMAGE_GUID, CABLE_INFO, EXTENDED_SPEEDS,
	CAP_MASK2, CM, DEVICE_MGMT, VENDOR_CLASS, CAP_MASK_NOTICE,
	CLIENT_REG, OTHER_LOCAL_CHANGES, MULT_PKER_TRAP, EXT_INFO>

Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-12-27 15:41:21 -08:00
David Ahern
6065ddfaa7 Merge branch 'iproute2-master' into iproute2-next
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-12-19 12:02:17 -08:00
Leon Romanovsky
378dd31b4b rdma: Fix broken 32-bit compilation
Allow compilation of rdmatool on 32-bits platforms.

rdma
    CC       rdma.o
    CC       utils.o
    CC       dev.o
    CC       link.o
In file included from rdma.h:26:0,
                 from dev.c:12:
dev.c: In function 'dev_caps_tostr':
../include/utils.h:269:38: warning: left shift count >= width of type [-Wshift-count-overflow]
 #define BIT(nr)                 (1UL << (nr))
                                      ^
rdma.h:32:61: note: in expansion of macro 'BIT'
 #define RDMA_BITMAP_ENUM(name, bit_no) RDMA_BITMAP_##name = BIT(bit_no),
                                                             ^~~

Fixes: 40df8263a0 ("rdma: Add dev object")
Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-12-13 11:34:44 -08:00
David Ahern
fbe7da2306 Merge branch 'iproute2-master' into iproute2-next
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-12-07 13:02:08 -08:00
Stephen Hemminger
a9c49b8f8f rdma: align uapi headers with 4.20-rc5
Upstream headers were updated.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-12-07 09:25:59 -08:00
Emeric Dupont
a7a7e45017 iproute2: Installation errors without libmnl
When performing make install in iproute2 (current git master),
     if $(HAVE_MNL) is not selected, some Makefiles try to call
     install with an empty target, which causes a non-critical make error.

Signed-off-by: Emeric Dupont <emeric.dupont@zii.aero>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-12-04 14:27:08 -08:00
David Ahern
6ae54b1326 Revert "rdma: make local functions static"
This reverts commit e99c4443ae.

Patch added to iproute2-master breaks builds of -next because of a
more recent patch in -next that relies on the exports. Revert the
offending patch. Unfortunately this leaves a window where builds
break.

Signed-off-by: David Ahern <dsahern@gmail.com>
2018-11-24 07:06:17 -08:00
David Ahern
0868c8ab07 Merge branch 'iproute2-master' into iproute2-next
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-11-24 07:06:11 -08:00
Stephen Hemminger
e99c4443ae rdma: make local functions static
Several functions only used inside utils.c

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-11-19 11:42:44 -08:00
David Ahern
3a7246dce4 Merge branch 'iproute2-master' into iproute2-next
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-11-09 08:50:50 -08:00
Leon Romanovsky
4ee770eec9 rdma: Refresh help section of resource information
After commit 4060e4c0d2 ("rdma: Add PD resource tracking
information"), the resource information shows PDs and MRs,
but help pages didn't fully reflect it.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-11-05 08:36:36 -08:00
Leon Romanovsky
4443c9c6a0 rdma: Add an option to rename IB device interface
Enrich rdmatool with an option to rename IB devices,
the command interface follows Iproute2 convention:
"rdma dev set [OLD-DEVNAME] name NEW-DEVNAME"

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-11-02 09:38:56 -07:00
Leon Romanovsky
a14ceed325 rdma: Introduce command execution helper with required device name
In contradiction to various show commands, the set command explicitly
requires to use device name as an argument. Provide new command
execution helper which enforces it.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-11-02 09:38:18 -07:00
Leon Romanovsky
3fb00075d9 rdma: Update kernel include file to support IB device renaming
Bring kernel header file changes upto commit 05d940d3a3ec
("RDMA/nldev: Allow IB device rename through RDMA netlink")

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-11-02 09:38:15 -07:00
Phil Sutter
cdefe1d8e4 rdma: Don't pass garbage to rd_check_is_filtered()
Variables 'src_port' and 'dst_port' are initialized only if attributes
RDMA_NLDEV_ATTR_RES_SRC_ADDR or RDMA_NLDEV_ATTR_RES_DST_ADDR are
present. Make sure to pass them over to rd_check_is_filtered() only if
that is the case.

Fixes: 9a362cc71a ("rdma: Add CM_ID resource tracking information")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-10-22 10:05:43 -07:00
Phil Sutter
3b0070f6b1 rdma: Fix for ineffective check in add_filter()
With 'name' field defined as array in struct filters, it will always
contain a value irrespective of whether a name was assigned or not.

Fix this by turning the field into a const char pointer.

Fixes: 1174be72d1 ("rdma: Add filtering infrastructure")
Signed-off-by: Phil Sutter <phil@nwl.cc>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-10-22 10:05:18 -07:00
Leon Romanovsky
d090fbf33b rdma: Fix representation of PortInfo CapabilityMask
The port capability mask represents IBTA PortInfo specification,
but as it is written in description of kernel commit 2f944c0fbf58
("RDMA: Fix storage of PortInfo CapabilityMask in the kernel"),
the bit 26 was mistakenly overwritten.

The rdmatool followed it too and mislead users by presenting wrong
value. Since it never showed proper value, we update the whole
port_cap_mask to comply with IBTA and show real HW values.

Fixes: da990ab40a ("rdma: Add link object")
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-09-17 08:59:13 -07:00
Stephen Hemminger
27886a1241 uapi: update ib_verbs
Merge current uapi from 4.19-rc1

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-08-31 15:03:49 -07:00
Stephen Hemminger
7327f78565 rdam: uapi update ib_user_verbs.h
Merge in latest santized kernel header.
Put sanitized version of current ib_user_verbs.h.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-07-23 13:49:20 -07:00
Hoang Le
5887ff0922 rdma: sync some IP headers with glibc
In the commit 9a362cc71a, new userspace header:
  (i.e rdma/rdma_user_cm.h -> linux/in6.h)
is included before the kernel space header:
  (i.e utils.h -> resolv.h -> netinet/in.h).

This leads to unsynchronous some IP headers and compiler got failure
with error: redefinition of some structs IP.

In this commit, just reorder this including to make them in-sync.

Signed-off-by: Hoang Le <hoang.h.le@dektech.com.au>
Acked-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-06-15 11:11:51 -07:00
Steve Wise
331152752a rdma: print driver resource attributes
This enhancement allows printing rdma device-specific state, if provided
by the kernel. This is done in a generic manner, so rdma tool doesn't
need to know about the details of every type of rdma device.

Driver attributes for a rdma resource are in the form of <key,
[print_type], value> tuples, where the key is a string and the value can
be any supported driver attribute. The print_type attribute, if present,
provides a print format to use vs the standard print format for the type.
For example, the default print type for a PROVIDER_S32 value is "%d ",
but "0x%x " if the print_type of PRINT_TYPE_HEX is included inthe tuple.

Driver resources are only printed when the -dd flag is present.
If -p is present, then the output is formatted to not exceed 80 columns,
otherwise it is printed as a single row to be grep/awk friendly.

Example output:

# rdma resource show qp lqpn 1028 -dd -p
link cxgb4_0/- lqpn 1028 rqpn 0 type RC state RTS rq-psn 0 sq-psn 0 path-mig-state MIGRATED pid 0 comm [nvme_rdma]
    sqid 1028 flushed 0 memsize 123968 cidx 85 pidx 85 wq_pidx 106 flush_cidx 85 in_use 0
    size 386 flags 0x0 rqid 1029 memsize 16768 cidx 43 pidx 41 wq_pidx 171 msn 44 rqt_hwaddr 0x2a8a5d00
    rqt_size 256 in_use 128 size 130

Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-05-18 09:17:16 -07:00
Steve Wise
366d20b91f rdma: update rdma_netlink.h to get new driver attributes
Pull in the rdma_netlink.h changes from kernel
commits:

25a0ad85156a ("RDMA/nldev: Add explicit pad attribute")
da5c85078215 ("RDMA/nldev: add driver-specific resource tracking)"
0d52d803767e ("RDMA/uapi: Fix uapi breakage")

Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-05-18 09:16:22 -07:00
David Ahern
4aba7dc4f4 Merge branch 'iproute2-master' into iproute2-next
Conflicts:
	rdma/include/uapi/rdma/rdma_netlink.h

Signed-off-by: David Ahern <dsahern@gmail.com>
2018-05-09 21:04:16 -07:00
Stephen Hemminger
9b2ab68516 rdma: add ib header files
The iproute2 header files must be complete to allow builds on
other places where some of the headers are not present.

For example, iproute2 is built on Windows Services for Linux
as a test tool. With the partial addition of rdma it was broken.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-05-09 08:14:55 -07:00
Stephen Hemminger
36eece51e3 rdma: align headers with upstream
This makes rdma/include/uapi/rdma headers align with those produced
by doing make headers_install from upstream (Linus) tree.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2018-05-09 08:12:13 -07:00
Leon Romanovsky
1525942736 rdma: Print net device name and index for RDMA device
The RDMA devices are operated in RoCE and iWARP modes have net device
underneath. Present their names in regular output and their net index
in detailed mode.

[root@nps ~]# rdma link show mlx5_3/1
4/1: mlx5_3/1: state ACTIVE physical_state LINK_UP netdev ens7
[root@nps ~]# rdma link show mlx5_3/1 -d
4/1: mlx5_3/1: state ACTIVE physical_state LINK_UP netdev ens7 netdev_index 7
    caps: <CM, IP_BASED_GIDS>

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: David Ahern <dsahern@gmail.com>
2018-04-06 09:02:32 -07:00