Note that this requires some adjustments to the encode_frame()
parameters to avoid red_worker-specific types.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
cinfo.dest is allocated in spice_jpeg_mem_dest but never freed.
Note that jpeg_destroy_compress does not free this field as is
supposed to be a buffer provided by jpeg caller.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
As the input line could be uint8_t*, uint16_t* or uint32_t*, changing
the default from uint8_t* to void* seems the correct choice to deal with
upcasting warnings.
Regarding chunks->data allocation, I quote Frediano explantion:
"Lines came from spice_bitmap_get_line. This function assume that bitmap
data is split among chunks each containing some lines
(always full lines). If chunk->data is allocated using malloc or similar
SHOULD (not 100% sure) be 4 bytes aligned so in our cases
(8, 16, 24 or 32 bit images) should be aligned enough.
All the casts unfortunately came from the fact we compute based on
pixel bytes to make it generic so we use uint8_t*."
and
"Looking at code looks like these chunks came from the virtual machine.
So the question is... why should the virtual machine give use some
not-pixel align data?
I would put a large comment to state that we assume VM send aligned
data, would be stupid for the VM to not align it!"
clang output:
jpeg_encoder.c:109:26: error: cast from 'uint8_t *'
(aka 'unsigned char *') to 'uint16_t *' (aka 'unsigned short *')
increases required alignment from 1 to 2 [-Werror,-Wcast-align]
uint16_t *src_line = (uint16_t *)line;
^~~~~~~~~~~~~~~~
jpeg_encoder.c:144:26: error: cast from 'uint8_t *'
(aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *')
increases required alignment from 1 to 4 [-Werror,-Wcast-align]
uint32_t *src_line = (uint32_t *)line;
^~~~~~~~~~~~~~~~
mjpeg_encoder.c:260:23: error: cast from 'uint8_t *'
(aka 'unsigned char *') to 'uint16_t *' (aka 'unsigned short *')
increases required alignment from 1 to 2 [-Werror,-Wcast-align]
uint16_t pixel = *(uint16_t *)src;
^~~~~~~~~~~~~~~
The checks would lead the reader to think these functions can be called
when bit rate control is off when in fact they are only called when it
is active.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
If mjpeg_encoder_reset_quality() is called with the same quality as currently
set, it will not reset last_enc_size but not reset num_recent_enc_frames,
violating some assumptions in _adjust_params_to_bit_rate(). To avoid aborting
the server, simply return early from this function.
Resolves: rhbz#1086820
gcc's some integer type definitions are different between 32/64bit system.
This causes platform dependency problem with printf function. However,
we can avoid this problem by using PRI macros that supports platform
independent printf.
When trying to start mjpeg compression mode, mjpeg_encoder_start_frame()
tests the image format as its only able to compress 24/32bpp images. On
images with lower bit depths, we return MJPEG_ENCODER_FRAME_UNSUPPORTED to
indicate this is not a format we can compress. However, this return goes
with a spice_warning("unsupported format"). As the rest of the code can
cope with this unsupported format by not doing mjpeg compression, it's
nicer to downgrade this spice_warning() to spice_debug().
This fixes https://bugzilla.redhat.com/show_bug.cgi?id=1070028
mjpeg_encoder modify the initial bit we supply it, according to the
client feedback. If it reaches a bit rate which is higher than the
initial one, we use the higher bit rate as the new bit rate estimation.
The stream starts after lossless frames were sent to the client,
and without rate control (except for pipe congestion). Thus, on the beginning
of the stream, we might observe frame drops on the client and server side which
are not necessarily related to mis-estimation of the bit rate, and we would
like to wait till the stream stabilizes.
The actual frames distribution does not necessarily fit the
condition "at least one frame every (1000/rate_contorl->fps)
milliseconds".
For keeping the average frame rate close to the defined fps, we
periodically measure the current average fps, and modify
rate_control->adjusted_fps accordingly. Then, we use
(1000/rate_control->adjusted_fps) as the interval between the
frames.
The required client playback latency is assessed based on the current
estimation of the bit rate, the network latency, and the encoding size
of the frames. When the playback delay that is reported by the client
seems too small, or when the stream parameters change, we send the
client an updated playback latency estimation.
mjpeg_encoder can receive periodic reports about the playback status on
the client side. Then, mjpeg_encoder analyses the report and can
increase or decrease the stream bit rate, depending on the report.
When the bit rate is changed, the quality and frame rate of the stream
are re-evaluated.
Previously, the mjpeg quality was always 70. The frame rate was
tuned according to the frames' congestion in the pipe.
This patch sets the quality and frame rate according to
a given bit rate and the size of the first encoded frames.
The following patches will introduce an adaptive video streaming, in which
the bit rate, the quality, and the frame rate, change in response to
different parameters.
Patches that make red_worker adopt this feature will also follow.
rhbz #813826
When playing a youtube video on Windows guest, the driver sometimes(**) sends
images which contain the video frames, but also other parts of the
screen (e.g., the youtube process bar). In order to prevent glitches, we send these
images as part of the stream, using SPICE_MSG_DISPLAY_STREAM_DATA_SIZED.
(**) It happens regularly with the you tube html5 player. With flash,
it occurs when moving the cursor in the player area.
It's not used when we use jpeg-turbo colorspaces, so it's better
to allocate it when we know we'll need it rather than always
allocating it even if it won't be used.
After the refactoring to optionally use libjpeg-turbo, some
of the functions that mjpeg-encoder used to provide are now no
longer used. This commit removes them.
When libjpeg-turbo is available, we can use the BGR and BGRX
colorspaces that it provides to avoid extra conversions of the
data we want to compress to mjpeg
This API is meant to allow us to move the pixel format conversion
into MjpegEncoder. This will allow us to be able to use the
additional pixel formats from libjpeg-turbo when available.
When encoding a frame, red_worker passes an allocated buffer to
libjpeg where it should encode the frame. When it fails, a new
bigger buffer is allocated and the encoding is restarted from
scratch. However, it's possible to use libjpeg to realloc this
buffer if it gets too small during the encoding process. Make use
of this feature, especially since it will make it easier to encore
one line at a time instead of a full frame in subsequent commits.
When using config.h, it must be the very first include in all source
files since it contains #define that may change the compilation process
(eg libc structure layout changes when it's used to enable large file
support on 32 bit x86 archs). This commit adds it at the beginning
of all .c and .cpp files
