Commit Graph

32 Commits

Author SHA1 Message Date
Vijay Sundar Selvamani
c6b012a26c crypto: qat - enable telemetry for GEN6 devices
Enable telemetry for QAT GEN6 devices by defining the firmware data
structures layouts, implementing the counters parsing logic and setting
the required properties on the adf_tl_hw_data data structure.

As for QAT GEN4, telemetry counters are exposed via debugfs using the
interface described in Documentation/ABI/testing/debugfs-driver-qat_telemetry.

Co-developed-by: George Abraham P <george.abraham.p@intel.com>
Signed-off-by: George Abraham P <george.abraham.p@intel.com>
Signed-off-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com>
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-18 20:52:00 +10:00
George Abraham P
c963ff0ec4 crypto: qat - enable power management debugfs for GEN6 devices
The QAT driver includes infrastructure to report power management (PM)
information via debugfs. Extend this support to QAT GEN6 devices
by exposing PM debug data through the `pm_status` file.

This implementation reports the current PM state, power management
hardware control and status registers (CSR), and per-domain power
status specific to the QAT GEN6 architecture.

The debug functionality is implemented in adf_gen6_pm_dbgfs.c
and initialized as part of the enable_pm() function.

Co-developed-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com>
Signed-off-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com>
Signed-off-by: George Abraham P <george.abraham.p@intel.com>
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-18 20:51:59 +10:00
George Abraham P
7c68005a46 crypto: qat - relocate power management debugfs helper APIs
Relocate the power management debugfs helper APIs in a common file
adf_pm_dbgfs_utils.h and adf_pm_dbgfs_utils.c so that it can be shared
between device generations.

When moving logic from adf_gen4_pm_debugfs.c to adf_pm_dbgfs_utils.c, the
include kernel.h has been replaced with the required include.

This does not introduce any functional change.

Signed-off-by: George Abraham P <george.abraham.p@intel.com>
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-18 20:51:59 +10:00
Małgorzata Mielnik
e9eec2916c crypto: qat - relocate bank state helper functions
The existing implementation of bank state management functions,
including saving and restoring state, is located within 4xxx device
files. However, these functions do not contain GEN4-specific code and
are applicable to other QAT generations.

Relocate the bank state management functions to a new file,
adf_bank_state.c, and rename them removing the `gen4` prefix. This change
enables the reuse of such functions across different QAT generations.

Add documentation to bank state related functions that were
moved from QAT 4xxx specific files to common files.

This does not introduce any functional change.

Signed-off-by: Małgorzata Mielnik <malgorzata.mielnik@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-07-18 20:51:59 +10:00
Suman Kumar Chakraborty
7f3401d0a5 crypto: qat - enable RAS support for GEN6 devices
Enable the reporting and handling of errors for QAT GEN6 devices.

Errors are categorized as correctable, non-fatal, or fatal. Error
handling involves reading the error source registers (ERRSOU0 to ERRSOU3)
to determine the source of the error and then decoding the actual source
reading specific registers.

The action taken depends on the error type:
   - Correctable and Non-Fatal errors. These error are logged, cleared and
     the corresponding counter is incremented.
   - Fatal errors. These errors are logged, cleared and a Function Level
     Reset (FLR) is scheduled.

This reports and handles the following errors:
   - Accelerator engine (AE) correctable errors
   - Accelerator engine (AE) uncorrectable errors
   - Chassis push-pull (CPP) errors
   - Host interface (HI) parity errors
   - Internal memory parity errors
   - Receive interface (RI) errors
   - Transmit interface (TI) errors
   - Interface for system-on-chip (SoC) fabric (IOSF) primary command
     parity errors
   - Shared RAM and slice module (SSM) errors

Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-19 13:48:19 +08:00
Laurent M Coquerel
17fd7514ae crypto: qat - add qat_6xxx driver
Add a new driver, qat_6xxx, to support QAT GEN6 devices.
QAT GEN6 devices are a follow-on generation of GEN4 devices and
differently from the previous generation, they can support all three
services (symmetric, asymmetric, and data compression) concurrently.

In order to have the qat_6xxx driver to reuse some of the GEN4 logic,
a new abstraction layer has been introduced to bridge the two
implementations. This allows to avoid code duplication and to keep the
qat_6xxx driver isolated from the GEN4 logic. This approach has been
used for the PF to VF logic and the HW CSR access logic.

Signed-off-by: Laurent M Coquerel <laurent.m.coquerel@intel.com>
Co-developed-by: George Abraham P <george.abraham.p@intel.com>
Signed-off-by: George Abraham P <george.abraham.p@intel.com>
Co-developed-by: Karthikeyan Gopal <karthikeyan.gopal@intel.com>
Signed-off-by: Karthikeyan Gopal <karthikeyan.gopal@intel.com>
Co-developed-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-05 18:20:45 +08:00
Suman Kumar Chakraborty
cd0e7160f8 crypto: qat - refactor compression template logic
The logic that generates the compression templates, which are used by to
submit compression requests to the QAT device, is very similar between
QAT devices and diverges mainly on the HW generation-specific
configuration word.

This makes the logic that generates the compression and decompression
templates common between GEN2 and GEN4 devices and abstracts the
generation-specific logic to the generation-specific implementations.

The adf_gen2_dc.c and adf_gen4_dc.c have been replaced by adf_dc.c, and
the generation-specific logic has been reduced and moved to
adf_gen2_hw_data.c and adf_gen4_hw_data.c.

This does not introduce any functional change.

Co-developed-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com>
Signed-off-by: Vijay Sundar Selvamani <vijay.sundar.selvamani@intel.com>
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-05 18:20:44 +08:00
George Abraham P
b8ca438062 crypto: qat - rename and relocate timer logic
Rename adf_gen4_timer.c to adf_timer.c and adf_gen4_timer.h to
adf_timer.h to make the files generation-agnostic. This includes
renaming the start() and stop() timer APIs and macro definitions
to be generic, allowing for reuse across different device
generations.
This does not introduce any functional changes.

Signed-off-by: George Abraham P <george.abraham.p@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-05-05 18:20:44 +08:00
Giovanni Cabiddu
3af4e7fa26 crypto: qat - reorder objects in qat_common Makefile
The objects in the qat_common Makefile are currently listed in a random
order.

Reorder the objects alphabetically to make it easier to find where to
add a new object.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22 15:56:03 +08:00
Giovanni Cabiddu
a4f95a2d28 crypto: qat - fix object goals in Makefiles
Align with kbuild documentation by using <module_name>-y instead of
<module_name>-objs, following the kernel convention for building modules
from multiple object files.

Link: https://docs.kernel.org/kbuild/makefiles.html#loadable-module-goals-obj-m
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2025-02-22 15:56:03 +08:00
Masahiro Yamada
ceb8bf2cea module: Convert default symbol namespace to string literal
Commit cdd30ebb1b ("module: Convert symbol namespace to string
literal") only converted MODULE_IMPORT_NS() and EXPORT_SYMBOL_NS(),
leaving DEFAULT_SYMBOL_NAMESPACE as a macro expansion.

This commit converts DEFAULT_SYMBOL_NAMESPACE in the same way to avoid
annoyance for the default namespace as well.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Uwe Kleine-König <u.kleine-koenig@baylibre.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2024-12-03 08:22:25 -08:00
Xin Zeng
a5d8922ab2 crypto: qat - fix linking errors when PCI_IOV is disabled
When CONFIG_PCI_IOV=n, the build of the QAT vfio pci variant driver
fails reporting the following linking errors:

    ERROR: modpost: "qat_vfmig_open" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_resume" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_save_state" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_suspend" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_load_state" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_reset" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_save_setup" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_destroy" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_close" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    ERROR: modpost: "qat_vfmig_cleanup" [drivers/vfio/pci/qat/qat_vfio_pci.ko] undefined!
    WARNING: modpost: suppressed 1 unresolved symbol warnings because there were too many)

Make live migration helpers provided by QAT PF driver always available
even if CONFIG_PCI_IOV is not selected. This does not cause any side
effect.

Reported-by: Arnd Bergmann <arnd@arndb.de>
Closes: https://lore.kernel.org/lkml/20240607153406.60355e6c.alex.williamson@redhat.com/T/
Fixes: bb208810b1 ("vfio/qat: Add vfio_pci driver for Intel QAT SR-IOV VF devices")
Signed-off-by: Xin Zeng <xin.zeng@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-06-21 21:51:51 +10:00
Xin Zeng
f0bbfc391a crypto: qat - implement interface for live migration
Add logic to implement the interface for live migration defined in
qat/qat_mig_dev.h. This is specific for QAT GEN4 Virtual Functions
(VFs).

This introduces a migration data manager which is used to handle the
device state during migration. The manager ensures that the device state
is stored in a format that can be restored in the destination node.

The VF state is organized into a hierarchical structure that includes a
preamble, a general state section, a MISC bar section and an ETR bar
section. The latter contains the state of the 4 ring pairs contained on
a VF. Here is a graphical representation of the state:

    preamble | general state section | leaf state
             | MISC bar state section| leaf state
             | ETR bar state section | bank0 state section | leaf state
                                     | bank1 state section | leaf state
                                     | bank2 state section | leaf state
                                     | bank3 state section | leaf state

In addition to the implementation of the qat_migdev_ops interface and
the state manager framework, add a mutex in pfvf to avoid pf2vf messages
during migration.

Signed-off-by: Xin Zeng <xin.zeng@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02 10:47:43 +08:00
Xin Zeng
0fce55e533 crypto: qat - add interface for live migration
Extend the driver with a new interface to be used for VF live migration.
This allows to create and destroy a qat_mig_dev object that contains
a set of methods to allow to save and restore the state of QAT VF.
This interface will be used by the qat-vfio-pci module.

Signed-off-by: Xin Zeng <xin.zeng@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02 10:47:43 +08:00
Giovanni Cabiddu
680302d191 crypto: qat - relocate CSR access code
As the common hw_data files are growing and the adf_hw_csr_ops is going
to be extended with new operations, move all logic related to ring CSRs
to the newly created adf_gen[2|4]_hw_csr_data.[c|h] files.

This does not introduce any functional change.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Xin Zeng <xin.zeng@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-04-02 10:47:43 +08:00
Damian Muszynski
e2b67859ab crypto: qat - add heartbeat error simulator
Add a mechanism that allows to inject a heartbeat error for testing
purposes.
A new attribute `inject_error` is added to debugfs for each QAT device.
Upon a write on this attribute, the driver will inject an error on the
device which can then be detected by the heartbeat feature.
Errors are breaking the device functionality thus they require a
device reset in order to be recovered.

This functionality is not compiled by default, to enable it
CRYPTO_DEV_QAT_ERROR_INJECTION must be set.

Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Markas Rapoportas <markas.rapoportas@intel.com>
Signed-off-by: Mun Chun Yep <mun.chun.yep@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2024-02-09 12:57:18 +08:00
Lucas Segarra Fernandez
69e7649f7c crypto: qat - add support for device telemetry
Expose through debugfs device telemetry data for QAT GEN4 devices.

This allows to gather metrics about the performance and the utilization
of a device. In particular, statistics on (1) the utilization of the
PCIe channel, (2) address translation, when SVA is enabled and (3) the
internal engines for crypto and data compression.

If telemetry is supported by the firmware, the driver allocates a DMA
region and a circular buffer. When telemetry is enabled, through the
`control` attribute in debugfs, the driver sends to the firmware, via
the admin interface, the `TL_START` command. This triggers the device to
periodically gather telemetry data from hardware registers and write it
into the DMA memory region. The device writes into the shared region
every second.

The driver, every 500ms, snapshots the DMA shared region into the
circular buffer. This is then used to compute basic metric
(min/max/average) on each counter, every time the `device_data` attribute
is queried.

Telemetry counters are exposed through debugfs in the folder
/sys/kernel/debug/qat_<device>_<BDF>/telemetry.

For details, refer to debugfs-driver-qat_telemetry in Documentation/ABI.

This patch is based on earlier work done by Wojciech Ziemba.

Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Damian Muszynski <damian.muszynski@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-12-29 11:25:55 +08:00
Jie Wang
de51d22364 crypto: qat - relocate portions of qat_4xxx code
Move logic that is common between QAT GEN4 accelerators to the
qat_common folder. This includes addresses of CSRs, setters and
configuration logic.
When moved, functions and defines have been renamed from 4XXX to GEN4.

Code specific to the device is moved to the file adf_gen4_hw_data.c.
Code related to configuration is moved to the newly created
adf_gen4_config.c.

This does not introduce any functional change.

Signed-off-by: Jie Wang <jie.wang@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-12-22 12:30:19 +08:00
Giovanni Cabiddu
03c76e8e7a crypto: qat - move adf_cfg_services
The file adf_cfg_services.h cannot be included in header files since it
instantiates the structure adf_cfg_services. Move that structure to its
own file and export the symbol.

This does not introduce any functional change.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Damian Muszynski <damian.muszynski@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27 18:04:28 +08:00
Ciunas Bennett
db74e16258 crypto: qat - add rate limiting sysfs interface
Add an interface for the rate limiting feature which allows to add,
remove and modify a QAT SLA (Service Level Agreement).

This adds a new sysfs attribute group, `qat_rl`, which can be accessed
from /sys/bus/pci/devices/<BUS:DEV:FUNCTION> with the following
hierarchy:
    |-+ qat_rl
      |---- id  (RW) # SLA identifier
      |---- cir (RW) # Committed Information Rate
      |---- pir (RW) # Peak Information Rate
      |---- srv (RW) # Service to be rate limited
      |---- rp  (RW) (HEX) # Ring pairs to be rate limited
      |---- cap_rem  (RW)  # Remaining capability for a service
      |---- sla_op   (WO)  # Allows to perform an operation on an SLA

The API works by setting the appropriate RW attributes and then
issuing a command through the `sla_op`. For example, to create an SLA, a
user needs to input the necessary data into the attributes cir, pir, srv
and rp and then write into `sla_op` the command `add` to execute the
operation.
The API also provides `cap_rem` attribute to get information about
the remaining device capability within a certain service which is
required when setting an SLA.

Signed-off-by: Ciunas Bennett <ciunas.bennett@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27 18:04:28 +08:00
Damian Muszynski
d9fb840837 crypto: qat - add rate limiting feature to qat_4xxx
The Rate Limiting (RL) feature allows to control the rate of requests
that can be submitted on a ring pair (RP). This allows sharing a QAT
device among multiple users while ensuring a guaranteed throughput.

The driver provides a mechanism that allows users to set policies, that
are programmed to the device. The device is then enforcing those policies.

Configuration of RL is accomplished through entities called SLAs
(Service Level Agreement). Each SLA object gets a unique identifier
and defines the limitations for a single service across up to four
ring pairs (RPs count allocated to a single VF).

The rate is determined using two fields:
  * CIR (Committed Information Rate), i.e., the guaranteed rate.
  * PIR (Peak Information Rate), i.e., the maximum rate achievable
    when the device has available resources.
The rate values are expressed in permille scale i.e. 0-1000.
Ring pair selection is achieved by providing a 64-bit mask, where
each bit corresponds to one of the ring pairs.

This adds an interface and logic that allow to add, update, retrieve
and remove an SLA.

Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27 18:04:28 +08:00
Shashank Gupta
532d7f6bc4 crypto: qat - add error counters
Introduce ras counters interface for counting QAT specific device
errors and expose them through the newly created qat_ras sysfs
group attribute.

This adds the following attributes:

- errors_correctable: number of correctable errors
- errors_nonfatal: number of uncorrectable non fatal errors
- errors_fatal: number of uncorrectable fatal errors
- reset_error_counters: resets all counters

These counters are initialized during device bring up and cleared
during device shutdown and are applicable only to QAT GEN4 devices.

Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27 18:04:27 +08:00
Shashank Gupta
93b2f7de7d crypto: qat - add infrastructure for error reporting
Add infrastructure for enabling, disabling and reporting errors in the QAT
driver. This adds a new structure, adf_ras_ops, to adf_hw_device_data that
contains the following methods:
  - enable_ras_errors(): allows to enable RAS errors at device
    initialization.
  - disable_ras_errors(): allows to disable RAS errors at device shutdown.
  - handle_interrupt(): allows to detect if there is an error and report if
    a reset is required. This is executed immediately after the error is
    reported, in the context of an ISR.

An initial, empty, implementation of the methods above is provided
for QAT GEN4.

Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-27 18:04:27 +08:00
Lucas Segarra Fernandez
d807f0240c crypto: qat - add cnv_errors debugfs file
The Compress and Verify (CnV) feature check and ensures data integrity
in the compression operation. The implementation of CnV keeps a record
of the CnV errors that have occurred since the driver was loaded.

Expose CnV error stats by providing the "cnv_errors" file under
debugfs. This includes the number of errors detected up to now and
the type of the last error. The error count is provided on a per
Acceleration Engine basis and it is reset every time the driver is loaded.

Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-13 18:31:07 +08:00
Lucas Segarra Fernandez
e079231676 crypto: qat - add pm_status debugfs file
QAT devices implement a mechanism that allows them to go autonomously
to a low power state depending on the load.

Expose power management info by providing the "pm_status" file under
debugfs. This includes PM state, PM event log, PM event counters, PM HW
CSRs, per-resource type constrain counters and per-domain power gating
status specific to the QAT device.

This information is retrieved from (1) the FW by means of
ICP_QAT_FW_PM_INFO command, (2) CSRs and (3) counters collected by the
device driver.

In addition, add logic to keep track and report power management event
interrupts and acks/nacks sent to FW to allow/prevent state transitions.

Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-13 18:31:07 +08:00
Giovanni Cabiddu
4999999ed7 crypto: qat - add namespace to driver
Create CRYPTO_QAT namespace for symbols exported by the qat_common
module and import those in the QAT drivers. It will reduce the global
namespace crowdedness and potential misuse or the API.

This does not introduce any functional change.

Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-10-13 18:27:26 +08:00
Damian Muszynski
359b84f8db crypto: qat - add heartbeat feature
Under some circumstances, firmware in the QAT devices could become
unresponsive. The Heartbeat feature provides a mechanism to detect
unresponsive devices.

The QAT FW periodically writes to memory a set of counters that allow
to detect the liveness of a device. This patch adds logic to enable
the reporting of those counters, analyze them and report if a device
is alive or not.

In particular this adds
  (1) heartbeat enabling, reading and detection logic
  (2) reporting of heartbeat status and configuration via debugfs
  (3) documentation for the newly created sysfs entries
  (4) configuration of FW settings related to heartbeat, e.g. tick period
  (5) logic to convert time in ms (provided by the user) to clock ticks

This patch introduces a new folder in debugfs called heartbeat with the
following attributes:
 - status
 - queries_sent
 - queries_failed
 - config

All attributes except config are reading only. In particular:
 - `status` file returns 0 when device is operational and -1 otherwise.
 - `queries_sent` returns the total number of heartbeat queries sent.
 - `queries_failed` returns the total number of heartbeat queries failed.
 - `config` allows to adjust the frequency at which the firmware writes
   counters to memory. This period is given in milliseconds and it is
   fixed for GEN4 devices.

Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-07-20 22:16:23 +12:00
Damian Muszynski
e2980ba57e crypto: qat - add measure clock frequency
The QAT hardware does not expose a mechanism to report its clock
frequency. This is required to implement the Heartbeat feature.

Add a clock measuring algorithm that estimates the frequency by
comparing the internal timestamp counter incremented by the firmware
with the time measured by the kernel.
The frequency value is only used internally and not exposed to
the user.

Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-07-20 22:16:11 +12:00
Damian Muszynski
fd77d8da1f crypto: qat - add internal timer for qat 4xxx
The power management feature in QAT 4xxx devices can disable clock
sources used to implement timers. Because of that, the firmware needs to
get an external reliable source of time.

Add a kernel delayed work that periodically sends an event to the
firmware. This is triggered every 200ms. At each execution, the driver
sends a sync request to the firmware reporting the current timestamp
counter value.

This is a pre-requisite for enabling the heartbeat, telemetry and
rate limiting features.

Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-07-20 22:16:01 +12:00
Lucas Segarra Fernandez
865b50fe6e crypto: qat - add fw_counters debugfs file
Expose FW counters statistics by providing the "fw_counters" file
under debugfs. Currently the statistics include the number of
requests sent to the FW and the number of responses received
from the FW for each Acceleration Engine, for all the QAT product
line.

This patch is based on earlier work done by Marco Chiappero.

Co-developed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-07-20 22:15:30 +12:00
Damian Muszynski
9260db6640 crypto: qat - move dbgfs init to separate file
Move initialization of debugfs entries to a separate file.
This simplifies the exclusion of the debugfs logic in the QAT driver
when the kernel is built with CONFIG_DEBUG_FS=n.
In addition, it will allow to consolidate the addition of debugfs
entries to a single location in the code.

This implementation adds infrastructure to create (and remove) debugfs
entries at two different stages. The first, done when a device is probed,
allows to keep debugfs entries persistent between a transition in device
state (up to down or vice versa). The second, done after the initialization
phase, allows to have debugfs entries that are accessible only when
the device is up.

In addition, move the creation of debugfs entries for configuration
to the newly created function adf_dbgfs_init() and replace symbolic
permissions with octal permissions when creating the debugfs files.
This is to resolve the following warning reported by checkpatch:

  WARNING: Symbolic permissions 'S_IRUSR' are not preferred. Consider using octal permissions '0400'.

Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-06-02 18:21:32 +08:00
Tom Zanussi
a4b16dad46 crypto: qat - Move driver to drivers/crypto/intel/qat
With the growing number of Intel crypto drivers, it makes sense to
group them all into a single drivers/crypto/intel/ directory.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2023-04-06 16:41:28 +08:00