When trying to start mjpeg compression mode, mjpeg_encoder_start_frame()
tests the image format as its only able to compress 24/32bpp images. On
images with lower bit depths, we return MJPEG_ENCODER_FRAME_UNSUPPORTED to
indicate this is not a format we can compress. However, this return goes
with a spice_warning("unsupported format"). As the rest of the code can
cope with this unsupported format by not doing mjpeg compression, it's
nicer to downgrade this spice_warning() to spice_debug().
This fixes https://bugzilla.redhat.com/show_bug.cgi?id=1070028
mjpeg_encoder modify the initial bit we supply it, according to the
client feedback. If it reaches a bit rate which is higher than the
initial one, we use the higher bit rate as the new bit rate estimation.
The stream starts after lossless frames were sent to the client,
and without rate control (except for pipe congestion). Thus, on the beginning
of the stream, we might observe frame drops on the client and server side which
are not necessarily related to mis-estimation of the bit rate, and we would
like to wait till the stream stabilizes.
The actual frames distribution does not necessarily fit the
condition "at least one frame every (1000/rate_contorl->fps)
milliseconds".
For keeping the average frame rate close to the defined fps, we
periodically measure the current average fps, and modify
rate_control->adjusted_fps accordingly. Then, we use
(1000/rate_control->adjusted_fps) as the interval between the
frames.
The required client playback latency is assessed based on the current
estimation of the bit rate, the network latency, and the encoding size
of the frames. When the playback delay that is reported by the client
seems too small, or when the stream parameters change, we send the
client an updated playback latency estimation.
mjpeg_encoder can receive periodic reports about the playback status on
the client side. Then, mjpeg_encoder analyses the report and can
increase or decrease the stream bit rate, depending on the report.
When the bit rate is changed, the quality and frame rate of the stream
are re-evaluated.
Previously, the mjpeg quality was always 70. The frame rate was
tuned according to the frames' congestion in the pipe.
This patch sets the quality and frame rate according to
a given bit rate and the size of the first encoded frames.
The following patches will introduce an adaptive video streaming, in which
the bit rate, the quality, and the frame rate, change in response to
different parameters.
Patches that make red_worker adopt this feature will also follow.
rhbz #813826
When playing a youtube video on Windows guest, the driver sometimes(**) sends
images which contain the video frames, but also other parts of the
screen (e.g., the youtube process bar). In order to prevent glitches, we send these
images as part of the stream, using SPICE_MSG_DISPLAY_STREAM_DATA_SIZED.
(**) It happens regularly with the you tube html5 player. With flash,
it occurs when moving the cursor in the player area.
It's not used when we use jpeg-turbo colorspaces, so it's better
to allocate it when we know we'll need it rather than always
allocating it even if it won't be used.
After the refactoring to optionally use libjpeg-turbo, some
of the functions that mjpeg-encoder used to provide are now no
longer used. This commit removes them.
When libjpeg-turbo is available, we can use the BGR and BGRX
colorspaces that it provides to avoid extra conversions of the
data we want to compress to mjpeg
This API is meant to allow us to move the pixel format conversion
into MjpegEncoder. This will allow us to be able to use the
additional pixel formats from libjpeg-turbo when available.
When encoding a frame, red_worker passes an allocated buffer to
libjpeg where it should encode the frame. When it fails, a new
bigger buffer is allocated and the encoding is restarted from
scratch. However, it's possible to use libjpeg to realloc this
buffer if it gets too small during the encoding process. Make use
of this feature, especially since it will make it easier to encore
one line at a time instead of a full frame in subsequent commits.
When using config.h, it must be the very first include in all source
files since it contains #define that may change the compilation process
(eg libc structure layout changes when it's used to enable large file
support on 32 bit x86 archs). This commit adds it at the beginning
of all .c and .cpp files
