Python 3 puts its .pyc files in a __pycache__ subdirectory, This commit
adds these to DISTCLEANFILES. My main motivation for doing this is so
that git.mk can then properly add them to .gitignore.
Acked-by: Victor Toso <victortoso@redhat.com>
In a comparison with current autotools build system, meson/ninja
provides a huge improvement in build speed, while keeping the same
functionalities currently available and being considered more user
friendly.
The new system coexists within the same repository with the current one,
so we can do more extensive testing of its functionality before deciding
if the old system can be removed, or for some reason, has to stay for
good.
- Meson: https://mesonbuild.com
This is the equivalent of autogen/configure step in autotools. It
generates the files that will be used by ninja to actually build the
source code.
The project has received lots of traction recently, with many GNOME
projects willing to move to this new build system. The following wiki
page has more details of the status of the many projects being ported:
https://wiki.gnome.org/Initiatives/GnomeGoals/MesonPorting
Meson has a python-like syntax, easy to read, and the documentation
on the project is very complete, with a dedicated page on how to port
from autotools, explaining how most common use cases can be
implemented using meson.
http://mesonbuild.com/Porting-from-autotools.html
Other important sources of information:
http://mesonbuild.com/howtox.htmlhttp://mesonbuild.com/Syntax.htmlhttp://mesonbuild.com/Reference-manual.html
- Ninja: https://ninja-build.org
Ninja is the equivalent of make in an autotools setup, which actually
builds the source code. It has being used by large and complex
projects such as Google Chrome, Android and LLVM. There is not much to
say about ninja (other than it is much faster than make) because we
won't interact directly with it as much, as meson does the middle man
job here. The reasoning for creating ninja in the first place is
explained on the following post:
http://neugierig.org/software/chromium/notes/2011/02/ninja.html
Also its manual provides more in-depth information about the design
principles:
https://ninja-build.org/manual.html
- Basic workflow:
Meson package is available for most if not all distros, so, taking
Fedora as an example, we only need to run:
# dnf -y install meson ninja-build.
With Meson, building in-tree is not possible at all, so we need to
pass a directory as argument to meson where we want the build to be
done. This has the advantage of creating builds with different options
under the same parent directory, e.g.:
$ meson ./build --prefix=/usr
$ meson ./build-extra -Dextra-checks=true -Dalignment-checks=true
After configuration is done, we call ninja to actually do the build.
$ ninja -C ./build
$ ninja -C ./build install
Ninja defaults to parallel builds, and this can be changed with the -j
flag.
$ ninja -j 10 -C ./build
- Hacking:
* meson.build: Mandatory for the project root and usually found under
each directory you want something to be built.
* meson_options.txt: Options that can interfere with the result of the
build.
Signed-off-by: Eduardo Lima (Etrunko) <etrunko@redhat.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Jonathon Jongsma <jjongsma@redhat.com>
Make sure there are not 2 messages with the same value in the
same channel.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
Make sure there are not 2 messages with the same name in the
same channel.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
Code that handled client and server messages check was the same, just
changed some variable names.
Instead use a class to store same information and reuse the code.
This allows easier extension of the 2 path of code.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Lukáš Hrázký <lhrazky@redhat.com>
Even though commit df4ec5c318 commented
out most of smartcard code which triggered this error, it still might
happen if a new message is added with an array member.
The reason is a missing declaration of mem_size, which is fixed simply
by checking if the attribute 'nocopy' is present.
The error log follows:
generated_server_demarshallers.c: In function ‘parse_msgc_smartcard_reader_add’:
generated_server_demarshallers.c:1985:30: error: ‘mem_size’ undeclared (first use in this function); did you mean ‘nw_size’?
data = (uint8_t *)malloc(mem_size);
^~~~~~~~
nw_size
This patch also updates test-marshallers so that this bug is triggered.
The diff between generated demarshallers with the patch applied follows:
--- tests/generated_test_demarshallers.c.old 2018-05-17 14:35:29.234056487 -0300
+++ tests/generated_test_demarshallers.c 2018-05-17 14:35:40.554031295 -0300
@@ -286,6 +286,7 @@ static uint8_t * parse_msg_main_ArrayMes
uint8_t *start = message_start;
uint8_t *data = NULL;
uint64_t nw_size;
+ uint64_t mem_size;
uint8_t *in, *end;
uint64_t name__nw_size;
uint64_t name__nelements;
@@ -298,6 +299,7 @@ static uint8_t * parse_msg_main_ArrayMes
}
nw_size = 0 + name__nw_size;
+ mem_size = sizeof(SpiceMsgMainArrayMessage);
/* Check if message fits in reported side */
if (nw_size > (uintptr_t) (message_end - start)) {
Signed-off-by: Eduardo Lima (Etrunko) <etrunko@redhat.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
Make code safe using both 32 and 64 bit machine.
Consider that this code can be compiled for machines with 32 bit.
There are some arrays length which are 32 bit.
If size_t this can cause easily an overflow. For instance message_len
sending SPICE_MSG_NOTIFY messages are 32 bit and code add a small
constant (currently 24) before doing the test for size. Now passing
(uint32_t) -20 as message_len would lead to a size of 4 after the
addition. This overflow does not happen on 64 bit machine as the length
is converted to size_t.
There are also some array length where some item are bigger than 1 byte.
For instance SPICE_MAIN_CHANNELS_LIST message have a number of channels
and each channel is composed by 2 bytes. Now the code generated try to do
length * 2 where length is still a 32 bit so if we put a value like
0x80000002u we get 4 as length. This will cause an overflow as code will
allocate very few bytes but try to fill with a huge number of elements.
This overflow happen in both 32 and 64 bit machine.
To avoid all these possible overflows this patch use only 64 bit for
nelements (number of elements), nw_size (network size) and mem_size
(memory size needed) checking the sizes to avoid other overflows
(like pointers conversions under 32 bit machines).
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe de Dinechin <dinechin@redhat.com>
When testing out some experimental protocol changes, I managed to
trigger the following error:
GEN generated_client_demarshallers.c
Traceback (most recent call last):
File "../../../spice-common/spice_codegen.py", line 267, in <module>
demarshal.write_protocol_parser(writer, proto, True)
File "/home/jjongsma/work/spice/spice-common/python_modules/demarshal.py", line 1270, in write_protocol_parser
parsers[channel.value] = (channel.channel_type, write_channel_parser(writer, channel.channel_type, is_server))
File "/home/jjongsma/work/spice/spice-common/python_modules/demarshal.py", line 1163, in write_channel_parser
func = write_msg_parser(helpers, ids[i].message_type)
File "/home/jjongsma/work/spice/spice-common/python_modules/demarshal.py", line 1061, in write_msg_parser
num_pointers = message.get_num_pointers()
File "/home/jjongsma/work/spice/spice-common/python_modules/ptypes.py", line 855, in get_num_pointers
count = count + m.get_num_pointers()
File "/home/jjongsma/work/spice/spice-common/python_modules/ptypes.py", line 662, in get_num_pointers
return self.member_type.get_num_pointers()
File "/home/jjongsma/work/spice/spice-common/python_modules/ptypes.py", line 507, in get_num_pointers
if self.is_constant_length(self):
TypeError: is_constant_length() takes exactly 1 argument (2 given)
Calling a member function will implicitly pass 'self' as the first
argument, but we were also explicitly passing it as an argument
(self.is_constant_length(self)). This resulted in the above error.
Acked-by: Lukáš Hrázký <lhrazky@redhat.com>
Instead of assuming that the system can safely do unaligned access
to memory use packed structures to allow the compiler generate
best code possible.
A packed structure tells the compiler to not leave padding inside it
and that the structure can be unaligned so any field can be unaligned
having to generate proper access code based on architecture.
For instance ARM7 can use unaligned access but not for 64 bit
numbers (currently these accesses are emulated by Linux kernel
with obvious performance consequences).
This changes the current methods from:
#ifdef WORDS_BIGENDIAN
#define read_uint32(ptr) ((uint32_t)SPICE_BYTESWAP32(*((uint32_t *)(ptr))))
#define write_uint32(ptr, val) *(uint32_t *)(ptr) = SPICE_BYTESWAP32((uint32_t)val)
#else
#define read_uint32(ptr) (*((uint32_t *)(ptr)))
#define write_uint32(ptr, val) (*((uint32_t *)(ptr))) = val
#endif
to:
#include <spice/start-packed.h>
typedef struct SPICE_ATTR_PACKED {
uint32_t v;
} uint32_unaligned_t;
#include <spice/end-packed.h>
#ifdef WORDS_BIGENDIAN
#define read_uint32(ptr) ((uint32_t)SPICE_BYTESWAP32(((uint32_unaligned_t *)(ptr))->v))
#define write_uint32(ptr, val) ((uint32_unaligned_t *)(ptr))->v = SPICE_BYTESWAP32((uint32_t)val)
#else
#define read_uint32(ptr) (((uint32_unaligned_t *)(ptr))->v)
#define write_uint32(ptr, val) (((uint32_unaligned_t *)(ptr))->v) = val
#endif
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
Python 2.6 does not have a flags parameter.
This is needed for RHEL 6.8.
Signed-off-by: Francois Gouget <fgouget@codeweavers.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
This commit adds autogeneration of a generated_client_marshallers.h
header, which is then included in client_marshallers.h
This allows to remove the SpiceMessageMarshallers struct from this file,
which has to match what the generated code expects.
Acked-by: Frediano Ziglio <fziglio@redhat.com>
Until now, the same header guard was used for all generated .h files.
Now the header guard name is based on the name of the file being
generated so that it's different for each .h file.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
Not supported by code so trigger an error to avoid invalid usages
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Acked-by: Christophe Fergeau <cfergeau@redhat.com>
This patch is a left-over from the fd passing commit 267391c8fd as
suggested by Frediano Ziglio during review.
Signed-off-by: Marc-André Lureau <marcandre.lureau@gmail.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
(cherry picked from spice-protocol commit 47076559628d71c128e14e11147ce36b92677885)
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Add a new type, "unix_fd", used to describe file descriptor sharing via
socket ancillary data (these messages are local only).
The marshaller/demarshaller can't serialize this in memory (consume_fd
implementation is empty), so it is the responsability of the marshaller
user to handle sending and receiving the handles, which are appended at
the end of the message with an extra stream byte (because some Unix
requires sending at least a byte with ancillary data).
Even if there is no fd to send (or if the fd is invalid etc), the
receiver side expects an extra byte anyway.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Frediano Ziglio <fziglio@redhat.com>
(cherry-picked from spice-protocol commit 267391c8fd7c90c067b3e4845ff0227a2580e2e2)
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
This reverts commit 7665dcf1bb.
Also revert the related build-sys changes to fix the build.
codegen generated code depends on spice-common code (marshaller,
messages etc), it makes more sense to keep the generator along
this. Otherwise a newer protocol release will fail to build older
projects.
*.proto files are required as well, since it generates code that parent
modules depend on unconditionnaly.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
This patch changes the type of 'valid_attributes' and
'attributes_with_arguments'.
Both of them are list of different strings and are kept in sets.
This was the intention of the original code, but this patch
use a specific set([ strings ]) format, instead of { strings }.
This fixes the build for me on RHEL-6 (python-2.6.6).
Build error is:
File "/home/ulublin/git/spice/spice-common/python_modules/ptypes.py",
line 67
'end',
^
SyntaxError: invalid syntax
This is to prepare to generate the wireshark dissector which uses
glib types instead of the newer C ones (for compatibility with some
compilers).
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Verify that the attribute is known. This could help for instance to
avoid some future typo mistakes.
We also now have a list of attributes that we can comment for
documentation purpose.
Signed-off-by: Frediano Ziglio <fziglio@redhat.com>
Older versions of python-six (at least 1.3.0) defined six.PY3 but not
six.PY2. six.PY2 is only used twice in straightforward tests so it's
easy to use six.PY3 instead.
This is a new version of my previous patch that does not include six.py.
It's still kind of big, but at least it's all spice-common changes now.
There are also a few other fixes that Christophe brought to my attention.
Note that six now needs to be installed on the system (python-six on
Fedora and Debian, six on PyPI).
This *should* be enough to make spice_codegen.py work on both Python 2
and Python 3. The major changes are as follows:
* cStringIO.StringIO -> io.StringIO
* str vs. unicode updates (io.StringIO doesn't like str)
* integer division
* foo.has_key(bar) -> bar in foo
* import internal_thing -> from . import internal_thing
* removed from __future__ import with_statement
(might break Python 2.5?)
* changed some lambdas to list comprehensions (done by 2to3)
* cast some_dict.keys() to list where needed (e.g. for sorting)
* use normal type names with isinstance instead of types.WhateverType
Signed-off-by: Alexander Wauck <awauck@codeweavers.com>
The return of the get_array_size() is used as a limit in a loop,
being compared with an unsigned index (indexes are always unsigned).
To avoid the -Wsign-compare warning, let's cast the return of the
get_array_size() to unsigned and make GCC happier.
Although the most part of the parameters marked as unused are actually
being used for a few functions, a bunch of warnings can be seen when
the code is compiled with "-Wall -Wextra". As adding the unused attribute
means that the variable/parameter is meant to be *possibly* unused, we're
safe adding it in the generated code, even for used variables/parameters.
Since the (de)marshallers are now generated in $builddir and not in
$srcdir, when these generated files include a file located in
$srcdir/common, the compiler will find them thanks to a -I directive, so it
makes more sense to use <> rather than "" when including them.
Now that they are created in $builddir, their includes will need to refer
to files in $srcdir, which can be different. It's cleaner to add
-I $(top_srcdir)/spice-common/ to modules using spice-common rather than
having -I $(top_srcdir)/spice-common/common which would could create header
collisions.
The wireshark protocol dissector is a bit out-of-date. Several new channel types
and enums have been added. It would be nice if these values (and the
translation between the value and the name) could be automatically generated so
that updating the dissector was a slightly less manual process. This patch adds
a commandline switch which generates both the enums and value-name lists in the
format that wireshark expects.
Fixes the resulting enums.h from the invocation of:
./spice_codegen.py --generate-enums spice.proto spice-protocol/spice/enums.h
Right now any enum will contain None as the enum members, with this fix
it will contain the real enum members, i.e. SPICE_FOO.
