Include video metadata when using DMA-BUF to provide stride and
offset. Without this metadata, GStreamer may incorrectly interpret
the buffer layout, leading to corrupted image output—especially
when the framebuffer includes padding.
We need to convert the scanout's drm format to the correct Gstreamer
format while configuring the pipeline. This can be done using
gst_video_dma_drm_fourcc_to_format() API, which will take the drm
fourcc value and return the appropriate Gst format.
Signed-off-by: Vivek Kasireddy <vivek.kasireddy@intel.com>
Signed-off-by: Michael Scherle <michael.scherle@rz.uni-freiburg.de>
Acked-by: Frediano Ziglio <freddy77@gmail.com>
This patch adds a new function to enable the creation of Gst memory with
the dmabuf fd as the source by using a dmabuf allocator. And, it also
adds a mechanism to register and invoke any callbacks once the Gst memory
object is no longer used by the pipeline.
This patch also ensures that the source_fps value is always non-zero.
add dmabuf encoding if `drm/drm_fourcc.h` is present and
gstreamer is at least 1.24 due to
`gst_video_dma_drm_fourcc_to_format()`.
Cc: Frediano Ziglio <freddy77@gmail.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Dongwon Kim <dongwon.kim@intel.com>
Signed-off-by: Vivek Kasireddy <vivek.kasireddy@intel.com>
Signed-off-by: Michael Scherle <michael.scherle@rz.uni-freiburg.de>
Acked-by: Frediano Ziglio <freddy77@gmail.com>
Once it is determined that an Intel GPU is available/active (after
looking into udev's database), we try to see if there is a h/w
based encoder (element) available (in Gstreamer's registry cache)
for the user selected video codec. In other words, if we find that
the Intel Media SDK Gstreamer plugin (libgstmsdk.so) and associated
libraries (such as va or vaapi) are all installed properly, we add
the appropriate h/w based encoder and post-processor/converter
elements to the pipeline (along with any relevant options) instead
of the s/w based elements.
For example, if the user selects h264 as the preferred codec format,
msdkh264enc and vapostproc will be preferred instead of x264enc
and videoconvert.
Cc: Frediano Ziglio <freddy77@gmail.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Dongwon Kim <dongwon.kim@intel.com>
Signed-off-by: Vivek Kasireddy <vivek.kasireddy@intel.com>
Co-developed-by: Jin Chung Teng <jin.chung.teng@intel.com>
Co-developed-by: Hazwan Arif Mazlan <hazwan.arif.mazlan@intel.com>
Acked-by: Frediano Ziglio <freddy77@gmail.com>
If we use the x264enc encoder to encode a stream, then videoconvert
would convert the BGRx data into Y444, which is the preferred format
for x264enc. However, some decoders particularly the ones that are
h/w based cannot work with Y444 if it was the format used by the
encoder. Therefore, to address these situations, we need a way to
override the format used during the encoding stage which can be
accomplished by using the environment variable introduced in this
patch: SPICE_CONVERTER_PREFERRED_FORMAT.
For example, using NV12 as the output format for the videoconvert
element would allow us to pair a s/w based encoder (such as x264enc)
with a h/w based decoder (such as msdkh264dec) for decoding the
stream as most h/w based decoders only work with NV12 format given
its popularity.
Note that choosing an encoder format such as NV12 over Y444 would
probably result in decreased video quality although it would be
compatible with more decoders. Ideally, the client and server need
to negotiate a suitable format dynamically but the current
capabilities do not allow for such exchange.
Cc: Frediano Ziglio <freddy77@gmail.com>
Cc: Dongwon Kim <dongwon.kim@intel.com>
Based-on-patch-by: Hazwan Arif Mazlan <hazwan.arif.mazlan@intel.com>
Signed-off-by: Jin Chung Teng <jin.chung.teng@intel.com>
Signed-off-by: Vivek Kasireddy <vivek.kasireddy@intel.com>
Acked-by: Frediano Ziglio <freddy77@gmail.com>
Using Fedora 38 the compilation fails due to this warning:
In file included from /usr/include/gstreamer-1.0/gst/video/video.h:202,
from ../../server/gstreamer-encoder.c:27:
/usr/include/gstreamer-1.0/gst/video/video-sei.h:39:21: error: 'H265_MISP_NANOSECONDS' defined but not used [-Werror=unused-const-variable=]
39 | static const guint8 H265_MISP_NANOSECONDS[] = {
| ^~~~~~~~~~~~~~~~~~~~~
Ignore the warning for Gstreamer includes.
Signed-off-by: Frediano Ziglio <freddy77@gmail.com>
Using Fedora 35 the compilation fails due to this warning:
../server/gstreamer-encoder.c: In function 'create_pipeline':
../server/gstreamer-encoder.c:994:5: error: braces around scalar initializer [-Werror]
994 | GstAppSinkCallbacks appsink_cbs = {NULL, NULL, &new_sample, {NULL}};
| ^~~~~~~~~~~~~~~~~~~
../server/gstreamer-encoder.c:994:5: note: (near initialization for 'appsink_cbs.new_event')
../server/gstreamer-encoder.c:994:5: error: missing initializer for field '_gst_reserved' of 'GstAppSinkCallbacks' [-Werror=missing-field-initializers]
In file included from ../server/gstreamer-encoder.c:26:
/usr/include/gstreamer-1.0/gst/app/gstappsink.h:81:16: note: '_gst_reserved' declared here
81 | gpointer _gst_reserved[GST_PADDING - 1];
| ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
Change structure initialisation to avoid the warning.
The same syntax is already used in server/tests/test-gst.cpp.
Signed-off-by: Frediano Ziglio <freddy77@gmail.com>
Better clear the result type instead of a generic "int".
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Kevin Pouget <kpouget@redhat.com>
1. It was renamed in newer versions to 'threads'
2. The default is 1 anyway
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
Signed-off-by: Uri Lublin <uril@redhat.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
Fedora 30 / gcc 9.1.1 20190503 (Red Hat 9.1.1-1) fails to build
because of this error/warning:
> gstreamer-encoder.c: In function 'set_video_bit_rate':
> gstreamer-encoder.c:518:17: error: taking the absolute value of
> unsigned type 'uint64_t' {aka 'long unsigned int'} has no effect
> [-Werror=absolute-value]
> 518 | } else if (abs(bit_rate - encoder->video_bit_rate) > encoder->video_bit_rate * SPICE_GST_VIDEO_BITRATE_MARGIN) {
> | ^~~
> gstreamer-encoder.c:518:17: error: absolute value function 'abs'
> given an argument of type 'uint64_t' {aka 'long unsigned int'}
This patches solves these two warnings:
1) cast the substraction to a signed type (int64_t instead of
uint64_t) to preserve the operation meaning;
2) use a custom version of abs() to avoid data truncation and/or
platform-dependent type lengths (abs/labs/llabs)
Acked-by: Frediano Ziglio <fziglio@redhat.com>
It makes no sense to expect average frame sizes anywhere close to 2GB.
But then make sure to avoid arithmetic overflows.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
The source framerate is as important as the resolution when trying to
understand if the system should be fast enough to encode the video
stream in real time.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
This way all the minimum delay calculation is in one place and this
makes gstreamer's implementation closer to the mjpeg one.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
This should always be defined and including config.h is a requirement.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Victor Toso <victortoso@redhat.com>
Formatting string should be compatible with GLib.
GLib uses formatting types compatible with GNU.
For Linux this is not an issue as both systems (like a printf) and
GLib one uses the same formatting type. However on Windows they
differs potentially causing issues.
This is also make worse as GLib 2.58 changed format attribute from
__printf__ to gnu_printf (Microsoft compatibility formats like %I64d
are still supported but you'll get warnings using GCC/Clang
compilers).
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
ORC library is used internally by GStreamer to generate code
dynamically.
If ORC cannot allocate executable memory, the failure causes
an abort(3) to be called.
This happens on some SELinux configurations that disable executable
memory allocation (execmem boolean).
Check that ORC could work before attempting to use GStreamer to
avoid crashes.
While this check is done, the ORC library outputs an error which will
be well visible in Qemu output.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
This reverts commit c3d237075b.
When you call gst_buffer_add_video_meta_full GStreamer assumes that
buffer is contiguous. Specifically GStreamer assumes that the first
chunk is big enough to hold a whole frame. This results usually in
some pixel shifts in the video. The pixel shifts you can see are
artifacts due to how the guest sends the frames.
Assuming you allocate the 2 chunks with 2 malloc:
p1 = malloc(size);
...
p2 = malloc(size);
Usually the memory allocator tend to allocate linearly if there are
space adding a prefix to describe next block leading to a memory
arrangement as:
+-------------------+
| p1 prefix |
+-------------------+
| p1 buffer |
+-------------------+
| p2 prefix |
+-------------------+
| p2 buffer |
+-------------------+
now if you take p1 pointer and you assume it points to a 2 * size
buffer you will get p2 prefix in the middle, this prefix is the pixel
shifts.
Problems happens specifically in gst_video_frame_map_id.
This bug is reported in https://bugzilla.gnome.org/show_bug.cgi?id=779524.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
Since c3d237 "gstreamer: Avoid memory copy if strides are different" is
only needed when zero copy is disabled. This moves the function
definition to an already existing #ifdef block.
Acked-by: Pavel Grunt <pgrunt@redhat.com>
This structure is used to store format information for
both Gstreamer 0.10 and 1.0 however the two format uses
different fields from it.
Use a macro to filter only needed fields.
This currently also fixes a compile error using Gstreamer 0.10
(GST_VIDEO_FORMAT_RGB15 not defined as not available).
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Victor Toso <victortoso@redhat.com>
If bitmap stride and stream stride are different copy was used.
Using GStreamer 1.0 you can avoid the copy setting correctly
image offset and stride.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
This format is required to add metadata to the source buffer
using gst_buffer_add_video_meta_full.
This metadata can be used to pass strides/offsets, or
dmabuf-specific information.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
These formats were tested manually using test-gst utility
in server/tests.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Pavel Grunt <pgrunt@redhat.com>
The partial expression "MSEC_PER_SEC * size * 8" can overflow if
size is 536870 or more. This as the operation is done using
32 bit unsigned integers. Being the size potentially double of
a compressed frame size the limit can be easily reached.
As get_average_frame_size already return a 64 bit use 64 bit
even for the size to avoid the integer overflow.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Francois Gouget <fgouget@codeweavers.com>
The check to limit too low bit rates was setting encoder->bit_rate
instead of bit_rate. However after some lines bit_rate was used
to set encoder->bit_rate basically removing the lower threshold.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Francois Gouget <fgouget@codeweavers.com>
The video encoders already have sophisticated bit rate control code that
can react to both stream reports sent by the client, and server frame
drop notifications. Furthermore they can adjust both the frame rate and
the image quality to best match the network conditions.
But if the client cannot send stream reports all this is bypassed and
instead the streaming code performs its own primitive bit rate control
that can only adjust the frame rate.
So this patch removes the code duplication and lets the video encoders
do their job.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
GCond/GMutex interface is different between Glib 2.32 and
previous versions. Use pthread for compatibility.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
The verify macro used currently has some problem
as it raise a warning in RHEL6.
Use new SPICE_VERIFY macro defined in spice-common
to avoid this issue.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
When uncapped x264enc can compress the frames beyond recognition in low
bitrate situation. Beyond the set limit the gains are modest and it is
better to drop frames to reduce the bit rate further.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
This avoid a check for NULL.
Also will be used to catch invalid values when table will be extended.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Pavel Grunt <pgrunt@redhat.com>
Reduce structure length using static allocated string inside the
structure.
This will also avoid using .data.rel.ro section and relocations
reducing even more library size.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Pavel Grunt <pgrunt@redhat.com>
This allow option string to contain separator so you could set as
"field=value ! format".
This is useful as some encoders use the output format to specify
compression types (for instance with some H264 encoders you can
specify the profile to use).
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Pavel Grunt <pgrunt@redhat.com>