pm_runtime_put_autosuspend(), pm_runtime_put_sync_autosuspend(),
pm_runtime_autosuspend() and pm_request_autosuspend() now include a call
to pm_runtime_mark_last_busy(). Remove the now-reduntant explicit call to
pm_runtime_mark_last_busy().
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Thierry Reding <treding@nvidia.com> (tegra-vde/h264.c)
Acked-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com> (alvium-csi2.c)
Reviewed-by: Dikshita Agarwal <quic_dikshita@quicinc.com> (iris_hfi_queue.c)
Reviewed-by: Sean Young <sean@mess.org>
Acked-by: Dave Stevenson <dave.stevenson@raspberrypi.com> (imx219.c)
Acked-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
hrtimer_setup() takes the callback function pointer as argument and
initializes the timer completely.
Replace hrtimer_init() and the open coded initialization of
hrtimer::function with the new setup mechanism.
Patch was created by using Coccinelle.
Signed-off-by: Nam Cao <namcao@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Acked-by: Zack Rusin <zack.rusin@broadcom.com>
Link: https://lore.kernel.org/all/c725ad8d0ecac3cf6bbc532af567e56d47a6b75c.1738746904.git.namcao@linutronix.de
The Wave5 521C variant does not support 10 bit decoding. When 10 bit
decoding support was added for the 515 variant, a section of the code
was removed which returned an error. This removal causes a timeout for
the 521 variant, which was discovered during HEVC 10-bit decoding tests.
Fixes: 143e7ab4d9 ("media: chips-media: wave5: support decoding HEVC Main10 profile")
Cc: stable@vger.kernel.org
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil@xs4all.nl>
While seeking, the driver calls the flush command. Before the flush
command is sent to the VPU, the driver should handle the display buffer
flags and should get all decoded information from the VPU if the VCORE
is running.
Fixes: 9707a6254a ("media: chips-media: wave5: Add the v4l2 layer")
Cc: stable@vger.kernel.org
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil@xs4all.nl>
In case of multiple active instances, new interrupts can occur as soon
as the current interrupt is cleared. If the driver reads the
instance_info after clearing the interrupt, then there is no guarantee,
that the instance_info is still valid for the current interrupt.
Read the instance_info register for each interrupt before clearing the
interrupt.
Fixes: ed7276ed2f ("media: chips-media: wave5: Add hrtimer based polling support")
Cc: stable@vger.kernel.org
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil@xs4all.nl>
When a decoder instance is created, the W5_CMD_ERR_CONCEAL register
should be initialized to 0. Otherwise, gray color is occasionally
displayed on the screen while decoding.
Fixes: 45d1a2b932 ("media: chips-media: wave5: Add vpuapi layer")
Cc: stable@vger.kernel.org
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil@xs4all.nl>
kthread_create() creates a kthread without running it yet. kthread_run()
creates a kthread and runs it.
On the other hand, kthread_create_worker() creates a kthread worker and
runs it.
This difference in behaviours is confusing. Also there is no way to
create a kthread worker and affine it using kthread_bind_mask() or
kthread_affine_preferred() before starting it.
Consolidate the behaviours and introduce kthread_run_worker[_on_cpu]()
that behaves just like kthread_run(). kthread_create_worker[_on_cpu]()
will now only create a kthread worker without starting it.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Since commit 88785982a1 ("media: vb2: use lock if wait_prepare/finish
are NULL") it is no longer needed to set the wait_prepare/finish
vb2_ops callbacks as long as the lock field in vb2_queue is set.
Since the vb2_ops_wait_prepare/finish callbacks already rely on that field,
we can safely drop these callbacks.
This simplifies the code and this is a step towards the goal of deleting
these callbacks.
Signed-off-by: Hans Verkuil <hverkuil@xs4all.nl>
Reviewed-by: Neil Armstrong <neil.armstrong@linaro.org> # for meson-ge2d
Acked-by: Andrzej Pietrasiewicz <andrzejtp2010@gmail.com>
After commit 0edb555a65 ("platform: Make platform_driver::remove()
return void") .remove() is (again) the right callback to implement for
platform drivers.
Convert all platform drivers below drivers/media to use .remove(), with
the eventual goal to drop struct platform_driver::remove_new(). As
.remove() and .remove_new() have the same prototypes, conversion is done
by just changing the structure member name in the driver initializer.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@baylibre.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add support for the YUV422P, NV16, NV61, YUV422M, NV16M,
NV61M raw pixel-formats to the Wave5 encoder.
All these formats have a chroma subsampling ratio of 4:2:2 and
therefore require a new image size calculation as the driver
previously only handled a ratio of 4:2:0.
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Use v4l2-common helper functions to calculate bytesperline and sizeimage,
instead of calculating in a wave5 driver directly.
In case of raw(YUV) v4l2_pix_format, the wave5 driver updates
v4l2_pix_format_mplane struct through v4l2_fill_pixfmt_mp() function.
Encoder and Decoder need the same bytesperline and sizeimage values for
the same v4l2_pix_format. So, wave5_update_pix_fmt function is refactored
to support both together.
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add support for runtime suspend/resume in the encoder and decoder. This is
achieved by saving the VPU state and powering it off while the VPU is idle.
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Provide a control to toggle (0 = off / 1 = on), whether the SPS and
PPS are generated for every IDR.
Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add initial support for the Wave515 multi-decoder IP. For now it is only
able to decode HEVC Main/Main10 profile videos into YUV420.
This was tested on FPGA prototype, so wave5_dt_ids[] was not expanded.
Users of the real hardware with Wave515 IP will have to
* provide firmware specific to their SoC
* add struct wave5_match_data like this:
static const struct wave5_match_data platform_name_wave515_data = {
.flags = WAVE5_IS_DEC,
.fw_name = "cnm/wave515_platform_name_fw.bin",
.sram_size = (71 * 1024),
};
* add item to wave5_dt_ids[] like this:
{
.compatible = "vendor,soc-wave515",
.data = &platform_name_wave515_data,
},
* describe new compatible in
Documentation/devicetree/bindings/media/cnm,wave521c.yaml
Signed-off-by: Ivan Bornyakov <brnkv.i1@gmail.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Move the excessive "sram-size" device-tree property to the device match
data. Also change the SRAM memory allocation strategy, instead of
allocating exactly sram_size bytes, allocate all available SRAM memory
up to sram_size. Add the placeholders wave5_vpu_dec_validate_sec_axi()
and wave5_vpu_enc_validate_sec_axi() to validate that the allocated SRAM
memory is sufficient to decode/encode bitstream with a given resolution.
Signed-off-by: Ivan Bornyakov <brnkv.i1@gmail.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Implement a separate setup routine for interrupts to reduce code
duplication. Also enable interrupts based on vpu_attr->support_encoders
and vpu_attr->support_decoders fields to facilitate support for other
Wave5xx IPs, because not all of them are both encoders and decoders.
Signed-off-by: Ivan Bornyakov <brnkv.i1@gmail.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add initial support for optional reset lines. For now, simply deassert
resets while probing the driver and assert them back when removing the
driver.
Signed-off-by: Ivan Bornyakov <brnkv.i1@gmail.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add support for decoding HEVC Main10 profile by scaling the FBC
buffer stride and size by a factor of (bitdepth / 8).
Signed-off-by: Ivan Bornyakov <brnkv.i1@gmail.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add support for starting a polling timer in case an interrupt is not
available. This helps to keep the VPU functional in SoCs such as AM62A,
where the hardware interrupt hookup may not be present due to an SoC errata
[1].
The timer is shared across all instances of encoders and decoders and is
started when the first instance of an encoder or decoder is opened and
stopped when the last instance is closed, thus avoiding per instance
polling and saving CPU bandwidth. As VPU driver manages this instance
related tracking and synchronization, the aforementioned shared timer
related polling logic is implemented within the VPU driver itself. This
scheme may also be useful in general too (even if irq is present) for
non-realtime multi-instance VPU use-cases (for e.g 32 instances of VPU
being run together) where system is running already under high interrupt
load and switching to polling may help mitigate this as the polling thread
is shared across all the VPU instances.
Hrtimer is chosen for polling here as it provides precise timing and
scheduling and the API seems better suited for periodic polling task such
as this. As a general rule of thumb,
Worst case latency with hrtimer = Actual latency (achievable with irq)
+ Polling interval
NOTE (the meaning of terms used above is as follows):
- Latency: Time taken to process one frame
- Actual Latency : Time taken by hardware to process one frame and signal
it to OS (i.e. if latency that was possible to achieve if irq line was
present)
There is a trade-off between latency and CPU usage when deciding the value
for polling interval. With aggressive polling intervals (i.e. going with
even lesser values) the CPU usage increases although worst case latencies
get better. On the contrary, with greater polling intervals worst case
latencies will increase although the CPU usage will decrease.
The 5ms offered a good balance between the two as we were able to reach
close to actual latencies (as achievable with irq) without incurring too
much of CPU as seen in below experiments and thus 5ms is chosen as default
polling interval.
- 1x 640x480@25 Encoding using different hrtimer polling intervals [2]
- 4x 1080p30 Transcode (File->decode->encode->file) irq vs polling
comparison [3]
- 1x 1080p Transcode (File->decode->encode->file) irq vs polling comparison
[4]
- 1080p60 Streaming use-case irq vs polling comparison [5]
- 1x 1080p30 sanity decode and encode tests [6]
The polling interval can also be changed using vpu_poll_interval module
param in case user want to change it as per their use-case requirement
keeping in mind above trade-off.
Parse the irq number and if not present, initialize the hrtimer and the
polling worker thread before proceeding with v4l2 device registrations.
Based on interrupt status, we use a worker thread to iterate over the
interrupt status for each instance and send completion event as being done
in irq thread function.
Move the core functionality of the irq thread function to a separate
function wave5_vpu_handle_irq so that it can be used by both the worker
thread when using polling mode and irq thread when using interrupt mode.
Protect the hrtimer access and instance list with device specific mutex
locks to avoid race conditions while different instances of encoder and
decoder are started together.
[1] https://www.ti.com/lit/pdf/spruj16
(Ref: Section 4.2.3.3 Resets, Interrupts, and Clocks)
[2] https://gist.github.com/devarsht/ee9664d3403d1212ef477a027b71896c
[3] https://gist.github.com/devarsht/3a58b4f201430dfc61697c7e224e74c2
[4] https://gist.github.com/devarsht/a6480f1f2cbdf8dd694d698309d81fb0
[5] https://gist.github.com/devarsht/44aaa4322454e85e01a8d65ac47c5edb
[6] https://gist.github.com/devarsht/2f956bcc6152dba728ce08cebdcebe1d
Signed-off-by: Devarsh Thakkar <devarsht@ti.com>
Tested-by: Jackson Lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The .remove() callback for a platform driver returns an int which makes
many driver authors wrongly assume it's possible to do error handling by
returning an error code. However the value returned is ignored (apart
from emitting a warning) and this typically results in resource leaks.
To improve here there is a quest to make the remove callback return
void. In the first step of this quest all drivers are converted to
.remove_new(), which already returns void. Eventually after all drivers
are converted, .remove_new() will be renamed to .remove().
Trivially convert this driver from always returning zero in the remove
callback to the void returning variant.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Remove unnecessary semicolons reported by Coccinelle/coccicheck and the
semantic patch at scripts/coccinelle/misc/semicolon.cocci.
Signed-off-by: Thorsten Blum <thorsten.blum@toblux.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Change compatible string to match dt bindings for TI devices. K3 family
prefix should not be included as it deviates from naming convention.
Fixes: 9707a6254a ("media: chips-media: wave5: Add the v4l2 layer")
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Closes: https://lore.kernel.org/all/CAMuHMdUYOq=q1j=d+Eac28hthOUAaNUkuvxmRu-mUN1pLKq69g@mail.gmail.com/
Signed-off-by: Brandon Brnich <b-brnich@ti.com>
Reviewed-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
This driver uses the API that is provided by GENERIC_ALLOCATOR API, so
select it to prevent build errors:
riscv32-linux-ld: drivers/media/platform/chips-media/wave5/wave5-vpu.o: in function `.L37':
wave5-vpu.c:(.text+0x468): undefined reference to `of_gen_pool_get'
riscv32-linux-ld: drivers/media/platform/chips-media/wave5/wave5-vdi.o: in function `.L116':
wave5-vdi.c:(.text+0xaac): undefined reference to `gen_pool_dma_alloc'
riscv32-linux-ld: drivers/media/platform/chips-media/wave5/wave5-vdi.o: in function `wave5_vdi_free_sram':
wave5-vdi.c:(.text+0xb60): undefined reference to `gen_pool_free_owner'
Fixes: 9707a6254a ("media: chips-media: wave5: Add the v4l2 layer")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Nas Chung <nas.chung@chipsnmedia.com>
Cc: Jackson Lee <jackson.lee@chipsnmedia.com>
Cc: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: linux-media@vger.kernel.org
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
There is a spelling mistake in a dev_dbg message. Fix it.
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
We already verified that "ret" is zero a few lines earlier. Delete this
duplicate check.
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The Chips&Media Wave 5 Series multi-standard codec IP is currently only
supported on Texas Instruments K3 architecture. Hence add a dependency on
ARCH_K3, to prevent asking the user about this driver when configuring a
kernel without Texas Instruments K3 Multicore SoC support.
Fixes: 9707a6254a ("media: chips-media: wave5: Add the v4l2 layer")
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Reviewed-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
[hverkuil: small update in commit log, and reformatted the Fixes tag]
Add the decoder and encoder implementing the v4l2
API. This patch also adds the Makefile and the VIDEO_WAVE_VPU config
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Robert Beckett <bob.beckett@collabora.com>
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add the vpuapi layer of the wave5 codec driver.
This layer is used to configure the hardware according
to the parameters.
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Signed-off-by: Robert Beckett <bob.beckett@collabora.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
