All repository objects can now be created from scratch in memory using
either the git_object_new() method, or the corresponding git_XXX_new()
for each object.
So far, only git_commits can be written back to disk once created in
memory.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
All the required git_commit_set_XXX methods have been implemented; all
the attributes of a commit object can now be modified in-memory.
The new method git_object_write() automatically writes back the
in-memory changes of any object to the repository. So far it only
supports git_commit objects.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The new 'git__source_printf' does an overflow-safe printf on a source
bfufer.
The new 'git__source_write' does an overflow-safe byte write on a source
buffer.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The 'git_obj' structure is now called 'git_rawobj', since
it represents a raw object read from the ODB.
The 'git_repository_object' structure is now called 'git_object',
since it's the base object class for all objects.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
git_repository_object has now several internal methods to write back the
object information in the repository.
- git_repository__dbo_prepare_write()
Prepares the DBO object to be modified
- git_repository__dbo_write()
Writes new bytes to the DBO object
- git_repository__dbo_writeback()
Writes back the changes to the repository
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Added several methods to access:
- The ODB behind a repo
- The SHA1 id behind a generic repo object
- The type of a generic repo object
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Some compilers give linking problems when exporting 'uint32_t' as a
return type in the external API. Use generic types instead.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
A new method 'git_repository_object_free' allows to manually force the
freeing of a repository object, even though they are still automatically
managed by the repository and don't need to be freed by the user.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The interface for loading and parsing tree objects from a repository has
been completed with all the required accesor methods for attributes,
support for manipulating individual tree entries and a new unit test
t0901-readtree which tries to load and parse a tree object from a
repository.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The new 'git_index' structure is an in-memory representation
of a git index on disk; the 'git_index_entry' structures represent
each one of the file entries on the index.
The following calls for index instantiation have been added:
git_index_alloc(): instantiate a new index structure
git_index_free(): free an existing index
git_index_clear(): clear all the entires in an existing file
The following calls for index reading and writing have been added:
git_index_read(): update the contents of the index structure from
its file on disk.
Internally implemented through:
git_index__parse()
Index files are stored on disk in network byte order; all integer fields
inside them are properly converted to the machine's byte order when
loading them in memory. The parsing engine also distinguishes
between normal index entries and extended entries with 2 extra bytes
of flags.
The 'TREE' extension for index entries is also loaded into memory:
Tree caches stored in Index files are loaded into the
'git_index_tree' structure pointed by the 'tree' pointer inside
'git_index'.
'index->tree' points to the root node of the tree cache; the full tree
can be traversed through each of the node's 'tree->children'.
Index files can be written back to disk through:
git_index_write(): atomic writing of existing index objects
backed by internal method git_index__write()
The following calls for entry manipulation have been added:
git_index_add(): insert an empty entry to the index
git_index_find(): search an entry by its path name
git_index__append(): appends a new index entry to the end of the
list, resizing the entries array if required
New index entries are always inserted at the end of the array; since the
index entries must be sorted for it to be internally consistent, the
index object is only sorted once, and if required, before accessing the
whole entriea array (e.g. before writing to disk, before traversing,
etc).
git_index__remove_pos(): remove an index entry in a specific position
git_index__sort(): sort the entries in the array by path name
The entries array is sorted stably and in place using an
insertion sort, which ought to be the most efficient approach
since the entries array is always mostly-sorted.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The struct 'git_filelock' represents an atomically-locked
file, git-style.
Locked files can be modified atomically through the new file lock
interface:
int git_filelock_init(git_filelock *lock, const char *path);
int git_filelock_lock(git_filelock *lock, int append);
void git_filelock_unlock(git_filelock *lock);
int git_filelock_commit(git_filelock *lock);
int git_filelock_write(git_filelock *lock, const char *buffer, size_t length);
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The old 'git_revpool' object has been removed and
split into two distinct objects with separate
functionality, in order to have separate methods for
object management and object walking.
* A new object 'git_repository' does the high-level
management of a repository's objects (commits, trees,
tags, etc) on top of a 'git_odb'.
Eventually, it will also manage other repository
attributes (e.g. tag resolution, references, etc).
See: src/git/repository.h
* A new external method
'git_repository_lookup(repo, oid, type)'
has been added to the 'git_repository' API.
All object lookups (git_XXX_lookup()) are now
wrappers to this method, and duplicated code
has been removed. The method does automatic type
checking and returns a generic 'git_revpool_object'
that can be cast to any specific object.
See: src/git/repository.h
* The external methods for object parsing of repository
objects (git_XXX_parse()) have been removed.
Loading objects from the repository is now managed
through the 'lookup' functions. These objects are
loaded with minimal information, and the relevant
parsing is done automatically when the user requests
any of the parsed attributes through accessor methods.
An attribute has been added to 'git_repository' in
order to force the parsing of all the repository objects
immediately after lookup.
See: src/git/commit.h
See: src/git/tag.h
See: src/git/tree.h
* The previous walking functionality of the revpool
is now found in 'git_revwalk', which does the actual
revision walking on a repository; the attributes
when walking through commits in a database have been
decoupled from the actual commit objects.
This increases performance when accessing commits
during the walk and allows to have several
'git_revwalk' instances working at the same time on
top of the same repository, without having to load
commits in memory several times.
See: src/git/revwalk.h
* The old 'git_revpool_table' has been renamed to
'git_hashtable' and now works as a generic hashtable
with support for any kind of object and custom hash
functions.
See: src/hashtable.h
* All the relevant unit tests have been updated, renamed
and grouped accordingly.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Tag objects are now properly loaded from the revision pool.
New test t0801 checks for loading a parsing a series of tags, including
the tag of a tag.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The 'parse_oid' and 'parse_person' methods which were used by the commit
parser are now global so they can be used when parsing other objects.
The 'git_commit_person' struct has been changed to a generic
'git_person'.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Packed objects inside packfiles are now properly unpacked when calling
the git_odb__read_packed() method; delta'ed objects are also properly
generated when needed.
A new unit test 0204-readpack tries to read a couple hundred packed
objects from a standard packed repository.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The basic information (pointed trees and blobs) of each tree object in a
revision pool can now be parsed and queried.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The following new external methods have been added:
GIT_EXTERN(const char *) git_commit_message_short(git_commit *commit);
GIT_EXTERN(const char *) git_commit_message(git_commit *commit);
GIT_EXTERN(time_t) git_commit_time(git_commit *commit);
GIT_EXTERN(const git_commit_person *) git_commit_committer(git_commit *commit);
GIT_EXTERN(const git_commit_person *) git_commit_author(git_commit *commit);
GIT_EXTERN(const git_tree *) git_commit_tree(git_commit *commit);
A new structure, git_commit_person has been added to represent a
commit's author or committer.
The parsing of a commit has been split in two phases.
When adding a commit to the revision pool:
- the commit's ODB object is opened
- its raw contents are parsed for commit TIME, PARENTS and TREE
(the minimal amount of data required to traverse the pool)
- the commit's ODB object is closed
When querying for extended information on a commit:
- the commit's ODB object is reopened
- its raw contents are parsed for the requested information
- the commit's ODB object remains open to handle additional queries
New unit tests have been added for the new functionality:
In t0401-parse: parse_person_test
In t0402-details: query_details_test
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Commits now store pointers to their tree objects.
Tree objects now work as separate git_revpool_object
entities.
Tree objects can be loaded and parsed inedependently
from commits.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
git_revpool_object now has a type identifier for each object
type in a revpool (commits, trees, blobs, etc).
Trees can now be stored in the revision pool.
git_revpool_tableit now supports filtering objects by their
type when iterating through the object table.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Created commit objects in t0401-parse weren't being freed properly.
Updated the API documentation to note that commit objects are owned
by the revision pool and should not be freed manually.
The parents list of each commit was being freed twice after each test.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Previously the objects table was being freed, but not
the actuall commits. All git_commit objects are freed
and hence invalidated when freeing the git_rp object
they belong to.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
This fix had been delayed by Ramsay because on 32-bit systems it
highlights the fact that off_t is set to an invalid value.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
This reduces the global namespace pollution. These functions
were the only remaining external symbols (with the exception
of an PPC_SHA1 build) which did not start with 'git', and
since these are private library symbols the 'git__' prefix is
appropriate.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Given that the sha1.h header file should never be included into
any other file, since it represents an implementation detail of
hash.c, we remove the header and inline it's content.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
On Intel machines, the msvc compiler defines the CPU architecture
macros _M_IX86 and _M_X64 (equivalent to __i386__ and __x86_64__
respectively). Use these macros in the pre-processor expression
to select the "fast" definition of the {get,put}_be32() macros.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
When git_oid_to_string() was passed a buffer size larger than
GIT_OID_HEXSZ+1, the function placed the c-string NUL char at
the wrong position. Fix the code to place the NUL at the end
of the (possibly truncated) oid string.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
In order to avoid inconsistent definitions of type off_t, all
compilation units should include the "common.h" header file
before certain system headers (those which directly or indirectly
lead to the definition of off_t). The "common.h" header contains
the definition of _FILE_OFFSET_BITS to select 64-bit file offsets.
The symptom of this inconsistency, while compiling with -Wextra, is
the following warning:
In file included from src/common.h:50,
from src/commit.c:28:
src/util.h: In function git__is_sizet:
src/util.h:41: warning: comparison between signed and unsigned
In order to fix the problem, we simply remove the #include <time.h>
statement at the head of src/commit.c. Note that src/commit.h also
includes <time.h>.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
gcc (4.4.0) issues the following warning:
src/revobject.c:33: warning: dereferencing type-punned pointer \
will break strict-aliasing rules
We suppress the warning by copying the first 4 bytes from the oid
structure into an 'unsigned int' using memcpy(). This will also
fix any potential alignment issues on certain platforms.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In particular, doxygen issues the following warning:
.../src/git/revwalk.h:86: Warning: The following parameters of \
gitrp_sorting(git_revpool *pool, unsigned int sort_mode) are \
not documented:
parameter 'pool'
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In particular, sparse issues the following warnings:
src/revobject.c:29:14: warning: symbol 'max_load_factor' was \
not declared. Should it be static?
src/revobject.c:31:14: warning: symbol 'git_revpool_table__hash' was \
not declared. Should it be static?
In order to suppress these warnings, we simply declare them as
static, since they are not (currently) referenced outside of this
file.
In the case of max_load_factor, this is probably correct. However,
this may not be appropriate for git_revpool_table__hash(), given
how it is named. So, this should either be re-named to reflect it's
non-external status, or a declaration needs to be added to the
revobject.h header file.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In order to suppress this warning, we could simply replace the
constant 0 with NULL. However, in this case, replacing the
comparison with 0 by !buffer is more idiomatic.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In particular, the compiler issues the following warning:
src/revwalk.c(61) : warning C4244: '=' : conversion from \
'unsigned int' to 'unsigned char', possible loss of data
In order to suppress the warning, we change the type of the
sorting "enum" field of the git_revpool structure to be consistent
with the sort_mode parameter of the gitrp_sorting() function.
Note that if the size of the git_revpool structure is an issue,
then we could change the type of the sort_mode parameter instead.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In particular, the compiler issues the following warnings:
src/revobject.c(29) : warning C4305: 'initializing' : truncation \
from 'double' to 'const float'
src/revobject.c(56) : warning C4244: '=' : conversion from \
'const float' to 'unsigned int', possible loss of data
src/revobject.c(149) : warning C4244: '=' : conversion from \
'const float' to 'unsigned int', possible loss of data
In order to suppress the warnings we change the type of max_load_factor
to double, rather than change the initialiser to 0.65f, and cast the
result type of the expressions to 'unsigned int' as expected by the
assignment operators. Note that double should be able to represent all
unsigned int values without loss.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
These warnings are issued by both gcc (-Wextra) and msvc (-W3).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
On the msvc build, the tests t0401-parse and t0501-walk both
crash with a runtime error (ACCESS_VIOLATION). This is caused
by writing to un-allocated memory due to an under-allocation
of a git_revpool_table data structure.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
sorting ('prev' pointers in the linked list are no longer lost).
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
The GIT_RPSORT_XXX flags have been moved to the external API,
and a new method 'gitrp_sorting(...)' has been added to safely
change the sorting method of a revision pool.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
'git_commit_list_toposort()' and 'git_commit_list_timesort()' now
sort a commit list by topological and time order respectively.
Both sorts are stable and in place.
'git_commit_list_append' has been replaced by 'git_commit_list_push_back'
and 'git_commit_list_push_front'.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
Fixed issue when generating pending commits list during iteration.
The 'git_commit_lookup' function will now check the pool's cache
for commits which have been previously loaded/parsed; there can only
be a single 'git_commit' structure for each commit on the same pool.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
All the objects which will will be eventually transversable from
a revision pool (commits, trees, etc) now inherit from the
'git_revpool_object' structure which identifies them with their
own OID.
Furthermore, the 'git_revpool_table' and related functions have
been added, which allow for constant time lookup (hash table)
of the loaded revpool objects based on their OID.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
The 'gitrp_next()' method now correctly does a revision walking
of all the pushed revisions in arbritary ordering.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
git_commit_lookup() now creates commit references
without loading them from the ODB.
git_commit_parse() creates a commit reference, loads
it and parses it from the ODB.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
Basic support for iterating the revpool.
The following functions of the revwalk API have been partially
implemented:
void gitrp_reset(git_revpool *pool);
void gitrp_push(git_revpool *pool, git_commit *commit);
void gitrp_prepare_walk(git_revpool *pool);
git_commit *gitrp_next(git_revpool *pool);
Parsed commits' parents are now also parsed and stored in a
"git_commit_list" structure (linked list).
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
A few initial tests for commit parsing:
"parse_buffer_test" tests git_commit__parse_buffer() with
several malformed commit messages and a few corner cases
which should pass.
"parse_oid_test" tests git_commit__parse_oid() with several
malformed commit lines containing broken SHA1 OIDs.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
The external API function "git_commit_parse" has been renamed
to "git_commit_lookup" and has been partially implemented with
support for loading commits straight from the ODB. It still lacks
the functionality to lookup cached commits in the revpool and to
resolve tags to commits.
The following internal functions have been partially implemented:
int git_commit__parse_buffer(...);
int git_commit__parse_time(...);
int git_commit__parse_oid(...);
Commits are now fully parsed but the generated parent and tree
references are not handled yet.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In particular, using the normal (or production) compiler
warning level (-W3), msvc complains as follows:
.../sha1.c(244) : warning C4018: '<' : signed/unsigned mismatch
.../sha1.c(270) : warning C4244: 'function' : conversion from \
'unsigned __int64' to 'unsigned long', possible loss of data
.../sha1.c(271) : warning C4244: 'function' : conversion from \
'unsigned __int64' to 'unsigned long', possible loss of data
Note that gcc issues a similar complaint about line 244 when
compiling with -Wextra.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Commit 5dddf7c (Add block-sha1 in favour of the mozilla routines
2010-04-14) introduced the "bswap.h" header file which contains
an inline function (default_swab32()). The msvc compiler does
not support the inline keyword which causes the build to fail
with a syntax error.
However, msvc does support inline functions using the __inline
keyword language extension. We already have the GIT_INLINE()
macro that allows us to hide this syntatic difference. In order
to fix the build, we simply use GIT_INLINE() in the definition
of the default_swab32() function.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
* ramsay/dev:
Add a pack index 'virtual function' to fetch an index entry
Add a pack index 'virtual function' to search by file offset
Change the interface of the pack index search function
Add an 64-bit offset table index bounds check for v2 pack index
Add a minimum size check when opening an v2 pack index file
win32: Add separate MinGW and MSVC compatability header files
Makefile: Add support for custom build options in config.mak file
Fix some coding style issues
Since block-sha1 from git.git has such excellent performance, we
can also get rid of the openssl dependency. It's rather simple
to add it back later as an optional extra, but we really needn't
bother to pull in the entire ssl library and have to deal with
linking issues now that we have the portable and, performance-wise,
truly excellent block-sha1 code to fall back on.
Since this requires a slight revamp of the build rules anyway, we
take the opportunity to fix including EXTRA_OBJS in the final build
as well.
The block-sha1 code was originally implemented for git.git by
Linus Torvalds <torvalds@linux-foundation.org> and was later
polished by Nicolas Pitre <nico@cam.org>.
Signed-off-by: Andreas Ericsson <ae@op5.se>
We don't use it yet, but now we have it there at least.
All the non-trivial parts of it appears to have been written
and contributed to git.git by some anonymous genius. The original
implementation was done by Paul Mackerras <paulus@samba.org>.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Given an index entry number, the idx_get() function returns an
(version agnostic) index_entry structure containing all of the
information required to unpack the corresponding object from
the '.pack' file.
Since the v1 and v2 file formats differ in the layout of the
object records, we provide two implementations of the get
function and initialise the function pointer appropriately.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
In addition to searching the index by oid, we need to search by
'.pack' file offset, particularly when processing OBJ_OFS_DELTA
objects. Since the v1 and v2 file formats differ in the layout
of the object records, we provide two implementations of the
search function and initialise the (virtual) function pointer
appropriately.
Note that, as part of the creation of the 'offset index', we also
add a check that the offset data in the index is within the bounds
of the '.pack' file. Having sorted the file offsets, while creating
the index, we only need to check the smallest and largest values.
The offset index consists of the im_off_idx array, which contains
the index entry numbers sorted into file offset order, and the
im_off_next mapping array. The im_off_next array maps an index
entry number to the 'next' index entry in file offset order.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
In particular, on a successful search, we now return the index
entry number of the object rather than the '.pack' file offset.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
This reduces the global namespace pollution and allows for
a win32 compiler (eg. Open Watcom) to provide these routines
in a header other than <dirent.h> (eg in <io.h>).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Some win32 compilers define the SSIZE_T type, with the same
meaning and intent as ssize_t. If available, make ssize_t a
synonym of SSIZE_T.
At present, the Digital-Mars compiler is known not to define
SSIZE_T, so we provide an SSIZE_T macro to use in the typedef.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
In addition to removing the inline #define, commit 209849a also
removed a #pragma to disable msvc deprecated function warnings.
Without this #pragma, msvc currently issues 19 warnings related
to "deprecated insecure c-library functions", such as strcpy()
and 22 warnings related to "deprecated POSIX function names",
such as open().
In order to supress these warnings, re-instate the #pragma.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
No need to define inline as __inline because libgit2 code
should be using GIT_INLINE instead.
Signed-off-by: Julio Espinoza-Sokal <julioes@gmail.com>
Signed-off-by: Andreas Ericsson <ae@op5.se>
For information on FlushFileBuffers(), see the msdn document
at msdn.microsoft.com/en-us/library/aa364439(VS.85).aspx
Note that Windows 2000 is shown as the minimum windows version
to support FlushFileBuffers(), so if we wish to support Win9X
and NT4, we will need to add code to dynamically check if
kernel32.dll contains the function.
The only error return mentioned in the msdn document is
ERROR_INVALID_HANDLE, which is returned if the file/device
(eg console) is not buffered. The fsync(2) manpage says that
EINVAL is returned in errno, if "fd is bound to a special
file which does not support synchronization".
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
When setting the default value, the macro name was specified
as GIT_FLEX_ARRAY, which is inconsistent with it's earlier
usage in the file. This caused a compilation error, using the
MS Visual C/C++ compiler, when compiling the git_packlist
struct definition in src/odb.c.
In addition to changing the spelling of the FLEX_ARRAY macro
to GIT_FLEX_ARRAY, including it's use in src/odb.c, we also
rename the TYPEOF macro to GIT_TYPEOF.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
This supresses some "conversion from 'size_t' to 'unsigned int',
possible loss of data" warning messages from the MS Visual C/C++
compiler with -Wp64.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In particular, in standard C, a struct or union must have at
least one member declared (ie. structs and unions cannot be
empty). Some compilers allow empty structs as an extension
and won't even issue a warning unless asked for it (eg, gcc
requires -pedantic). Some compilers allow empty structs as
an extension and will only treat it as an error if asked for
strict checking (eg Digital-Mars with -A). Some compilers
simply treat it as an error (eg MS Visual C/C++).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
In 82324ac, the new static function exists_loose() called
object_file_name() and, in order to detect an error return,
tested for a negative value. This usage is incorrect, as
the error return is indicated by a positive return value.
(A successful call is indicated by a zero return value)
The only error return from object_file_name() relates to
insufficient buffer space and the return value gives the
required minimum buffer size (which will always be >0).
If the caller requires a dynamically allocated buffer,
this allows something like the following call sequence:
size_t len = object_file_name(NULL, 0, db->object_dir, id);
char *buf = git__malloc(len);
if (!buf)
error(...);
object_file_name(buf, len, db->object_dir,id);
...
No current callers take advantage of this capability.
Fix up the call site and change the return type of the
function, from int to size_t, which more accurately
reflects the implementation.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Andreas Ericsson <ae@op5.se>
This test assumed that it was invoked in an empty directory,
which is true when run from the Makefile, and so would fail
if run standalone. In order to allow the test to work when
run from any directory, create a sub directory "dir-walk"
and chdir() into this directory while running the tests.
Also, add some additional tests.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, the git__mmap() and git__munmap() routines provide
the interface to platform specific memory-mapped file facilities.
We provide implementations for unix and win32, which can be found
in their own sub-directories.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
On windows, unless we use the O_BINARY flag in the open()
call, the file I/O routines will perform line ending
conversion (\r\n => \n on input, \n => \r\n on output).
In addition to the performance penalty, most files in the
object database are binary and will, therefore, become
corrupted by this conversion.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, conditional expressions which contain an
assignment statement, where the expression type is not
explicitly made to be boolean, elicits the following
message:
warning 2: possible unintended assignment
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, using pointer arithmetic on void pointers,
despite being quite useful, is not legal in standard C.
Avoiding non-standard C constructs will help in porting
the library to other compilers/platforms.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Paul agreed to the GCC-exception license by email:
|
| From: Paul Kocher <paul@cryptography.com>
| Date: Sun, 15 Mar 2009 11:37:23 -0700
| Subject: Re: Adding Mozilla SHA1 implementation to libgit2
|
| Yes - that's fine.
|
| At 01:56 AM 3/5/2009, Andreas Ericsson wrote:
| > Hi Paul. We spoke earlier about this, if you remember?
| > We'd like to add the GCC-exception to the GPL license
| > for these files.
Signed-off-by: Paul Kocher <paul@cryptography.com>
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This function determines if the given object can be found
in the object database. At present, only the local object
database is searched.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, the test for z-stream input completion
(zs.avail_in != 0) logically belongs with the test for
the Z_STREAM_END stream status. This is also consistent
with the identical check in finish_inflate().
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
At present, it is sufficient to ensure that an error return
from inflateInit() is not ignored. Most error returns, like
Z_VERSION_ERROR and Z_STREAM_ERROR, indicate programming or
build errors. These errors could, perhaps, be handled with
simple asserts. However, for a Z_MEM_ERROR, we may want to
perform some further error handling in the future.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, neglecting to call inflateEnd() along various
codepaths in the inflate_tail() routine, would result in the
failure to release zlib internal state.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These routines are intended to extract the directory and
base name from a path string. Note that these routines
do not interact with any filesystem and work only on the
text of the path.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, the git__delta_apply() function has not been
declared prior to it's definition. In order to suppress the
warning, include the delta-apply.h header which provides the
public interface. This ensures that the declaration and
definition are consistent.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The git__delta_apply() function can be used to apply a Git style
delta, such as those used in pack files or in git patch files,
to recover the original object stream.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The fanout table is fairly commonly accessed, we need to read it
twice for each object we lookup in any given pack file. Most of
the processors running Git are running in little-endian mode, as
they are variants of the x86 platform, so reading the fanout is
a costly operation as we need to convert from network byte order
to local byte order. By decoding the fanout table into a malloc
obtained buffer we can save these 2 decode operations per lookup
and make search go more quickly.
This also cleans up the initialization of the search functions
by cutting out a few instructions, saving a small amount of time.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The index data is mapped into memory and then scanned using a
binary search algorithm to locate the matching entry for the
supplied git_oid. The standard fanout hash trick is applied to
reduce the search space by 8 iterations.
Since the v1 and v2 file formats differ in their search function,
due to the different layouts used for the object records, we use
two different search implementations and a virtual function pointer
to jump to the correct version of code for the current pack index.
The single function jump per-pack should be faster then computing
a branch point inside the inner loop of a common binary search.
To improve concurrency during read operations the pack lock is only
held while verifying the index is actually open, or while opening
the index for the first time. This permits multiple concurrent
readers to scan through the same index.
If an invalid index file is opened we close it and mark the
git_pack's invalid bit to true. The git_pack structure is kept
around in its parent git_packlist, but the invalid bit will cause
all future readers to skip over the pack entirely. Pruning the
invalid entries is relatively unimportant because they shouldn't
be very common, a $GIT_DIRECTORY/objects/pack directory tends to
only have valid pack files.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Win32 has a variant of mmap that is harder to use than POSIX, but
to run natively and efficiently on Win32 we need some form of it.
gitfo_map_ro() provides a basic mmap function for use in locations
where we need read-only random data access to large ranges of a file,
such as a pack-*.idx.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Using an atomic reference counter is difficult to make
cross-platform, as the reference count implementations
are generally processor specific. Its also hard to do
a proper multi-read/single-write implementation.
We now use a simple mutex around the reference count for the list
of packs. Readers grab the mutex and either build the list, or
increment the existing one's reference count. When the reader is
done with the list, the reference count is decremented. In this way
parallel readers are able to operate on the list without worrying
about it being deallocated out from under them.
Individual pack structures are held by reference counts, but we
only care about the list the pack structure is held in. There is
no need to increment/decrement the pack reference counts as we
scan through them during a read operation, the caller holds the
git_packlist and that is sufficient to hold the packs it references.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
As far as gcc is concerned, the "z size specifier" is available as
an extension to the language, which is available with or without any
-std= switch. (I think you have to go back to 2.95 for a version
of gcc which doesn't work.) Many other compilers have this as an
extension as well (ie without the equivalent of -std=c99).
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These headers aren't always available; they typically come from the
Linux kernel, but aren't supposed to be exported into the userspace
/usr/include. Modern kernels won't install these and some distros
rm -rf the directory post kernel header install.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Currently we only catalog the available pack files into a table,
storing their path names relative to the pack directory.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When scanning the pack directory we need to see if the path
name is present for ".idx" when we discover a ".pack" file.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Checking the return value of snprintf is a pain, as it must be
>= 0 and < sizeof(buffer). git__fmt is a simple wrapper to
perform these checks.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Our fileops API is currently private. We aren't planning on supplying
a cross-platform file API to applications that link to us. If we did,
we'd probably whole-sale publish fileops, not just the dirent code.
By moving it to be private we can also change the call signature to
permit the buffer to be passed down through the call chain. This is
very helpful when we are doing a recursive scan as we can reuse just
one buffer in all stack frames, reducing the impact the recursion has
on the stack frames in the data cache.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We grab the lock while accessing the alternates list, ensuring that
we only initialize it once for the given git_odb.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These abstractions can be used to implement an efficient resource
reference counter and simple mutual exclusion. On pthreads we use
pthread_mutex_t, except when we are also on glibc and can directly
use its asm/atomic.h definitions.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If we are using threads we need to make sure pthread.h comes
in before just about anything else. Some platforms enable
macros that alter what other headers define.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This error code indicates the OS error code has a better value
describing the last error, as it is likely a network or local
file IO problem identified by a C library function call.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We now forbid direct use of malloc, strdup or calloc within the
library and instead use wrapper functions git__malloc, etc. to
invoke the underlying library malloc and set git_errno to a no
memory error code if the allocation fails.
In the future once we have pack objects in memory we are likely
to enhance these routines with garbage collection logic to purge
cached pack data when allocations fail. Because the size of the
function will grow somewhat large, we don't want to mark them for
inline as gcc tends to aggressively inline, creating larger than
expected executables.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We're likely to add additional path data, like the path of the
refs or the path to the config file into the git_odb structure,
as it may grow into the repository wrapper. Changing the name
of the objects directory reference makes it more clear should
we later add something else.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We didn't search for the object, so we cannot possibly promise it
to the caller of git_odb_read().
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This is the correct C99 format code for the size_t type when passed
as an argument to the *printf family. If the platform doesn't
define it, we assume %lu and just cross our fingers that its the
proper setting for a size_t on this system. On most sane platforms,
"unsigned long" is the underlying type of "size_t".
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The Mach-O format does not permit gcc to implement the __thread
TLS specification, so we must instead emulate it using a single
int cell allocated from memory and stored inside of the thread
specific data associated with the current pthread.
What makes this tricky is git_errno must be a valid lvalue, so
we really need to return a pointer to the caller and deference it
as part of the git_errno macro.
The GCC-specific __attribute__((constructor)) extension is used
to ensure the pthread_key_t is allocated before any Git functions
are executed in the library, as this is necessary to access our
thread specific storage.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
[sp: Changed signature for output to use git_oid, and added
a test case to verify an allocated git_hash_ctx can be
reinitialized and reused.]
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, when asked to read an empty file, this function
calls malloc() with a zero size allocation request. Standard C
says that the behaviour of malloc() in this case is implementation
defined.
[C99, 7.20.3 says "... If the size of the space requested is zero,
the behavior is implementation-defined: either a null pointer is
returned, or the behavior is as if the size were some nonzero
value, except that the returned pointer shall not be used to
access an object."]
Finesse the issue by over-allocating by one byte. Setting the extra
byte to '\0' may also provide a useful sentinel for text files.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, the gitfo_read_file() routine can be used to slurp
the complete file contents into an gitfo_buf structure. The buffer
content will be allocated by malloc() and may be released by the
gitfo_free_buf() routine. The io buffer type can be initialised
on the stack with the GITFO_BUF_INIT macro.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In particular, the warning relates to malloc(), which is
declared in <stdlib.h>. This header is now included,
indirectly, via the "common.h" header.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The PATH_MAX symbol is often, but not always, defined
in the <limits.h> header. In particular, on cygwin you
need to include this header to avoid a compilation error.
However, some systems define PATH_MAX to be something as
small as 256, which POSIX is happy to allow, while others
allow much larger values. In general it can vary from
one filesystem to another.
In order to avoid the vagaries of different systems, define
our own symbol.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
[sp: Credit for some of this implementation goes to Pieter, I
started off a patch he proposed for libgit2 but reworked
enough of it that I don't want to blame him for any bugs.]
Suggested-by: Pieter de Bie <pdebie@ai.rug.nl>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
commit dff79e27d3 renamed
the (small object) "git_sobj" to a plain "git_obj", but
neglected to update some of the documentation to reflect
that change.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since at least MS have something like GetFirstDirEnt() and
GetNextDirEnt() (presumably with superior performance), we
can let MS hackers add support for a dirent walker using
that API instead, while we stick with the posix-style
readdir() calls.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The idea is taken from Junio's work in read-cache.c, where
it's used for writing out the index without tap-dancing on
the poor harddrive. Since it's almost certainly useful for
cached writing of packfiles too, we turn it into a generic
API, making it perfectly simple to reuse it later.
gitfo_write_cached() has the same contract as gitfo_write(), it
returns GIT_SUCCESS if all bytes are successfully written (or were
at least buffered for later writing), and <0 if an error occurs
during buffer writing.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since it doesn't make sense to make the disk access stuff
portable *AND* public (that's a job for each application
imo), we can take a shortcut and just support unixy stuff
for now and get away with coding most of it as macros.
Since we go with an internal API for starters and only
provide higher-level API's to the libgit users, we'll be
ok with this approach.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since it's being added when we install the headers anyway,
we might as well get rid of it. If anything, we should point
coders to the COPYING file in the project's root directory
instead of duplicating the same (large-ish) text everywhere.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This adds the per-thread global variable git_errno to the
system, which callers can examine to get information about
an error.
Two helper functions are added to reduce LoC-count for the
library code itself.
Also, some exceptions are made for running sparse on GIT_TLS
definitions, since it doesn't grok thread-local variables at
all.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
ARRAY_SIZE() et al go in util.h, included from common.h
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This one pulls in compiler compatibility macros, some
common header files, and also the public common.h header.
C source files are modified to use the private common.h
in favour of the public one.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Otherwise their prototypes don't match their declarations.
Detected by 'sparse', which is obviously good to run
before each commit.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Using it in the first place means something's wrong.
This patch replaces it with an internal header which
carries the previously "protected" code instead.
Internal source-files simply include "commit.h" and
they're done. The internal header includes the public
one to make sure we always use the proper prototype.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
It doesn't cover all cases, but we can work on those as
we go along. For now, gcc, MSVC++, Intel C/C++, IBM XL C/C++,
Sun Studio C/C++ and Borland C++ Builder are the supported
compilers (although we boldly assume that they all are of
a recent enough version to support thread-local storage).
This is intended to be used in upcoming patches that implement
graceful (but TLS-dependant) error-handling in the library.
As an added bonus, we also bring the online_cpus() function
from git.git to detect the number of usable cpu's.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
It's arguably smoother to keep them close to the source,
as that's where one's working when modifying them. More
importantly, though, is the ability to use private headers
in the src/ dir that simply include "git/$samename.h" to
get to the public API at the same time.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
git_revp is something I personally can't stop pronouncing
"rev pointer". I'm sure others would suffer the same
problem.
Also, rename the git_revp_ sub-api "gitrp_". This is the
first of many such renames, primarily done to prevent
extreme inflation in the "git_" namespace, which we'd like
to reserve for a higher-level API.
While we're at it, we remove the noise-char "c" from a lot
of functions. Since revision walking is all about commits,
the common case should be that we're dealing with commits.
Exceptions can get a more mnemonic description as needed.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The 's' never really made sense, since it's not a "small"
object at all, but rather a plain object. As such, it should
have a "plain" object name.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We never want to accept a short read or a short write when
transferring data to or from a local file.
Either the entire read (or write) completes or the operation
failed and we will not recover gracefully from it.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These are easily built off the standard C library functions memcpy
and memcmp. By marking these inline we stand a good chance of
the C compiler replacing the entire thing with tight machine code,
because many compilers will actually inline a memcmp or memcpy when
the 3rd argument (the size) is a constant value.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This way we can start to write IO code to read and write files in the
Git object database, but provide a hook to inject native Win32 APIs
instead so libgit2 can be ported to run natively on that platform.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This isn't the best idea I've head. Pierre Habouzit was suggesting
a technique of assigning a unique integer to each commit and then
allocating storage out of auxiliary pools, using the commit's unique
integer to index into any auxiliary pool in constant time. This way
both applications and the library can efficiently attach arbitrary
data onto a commit, such as rewritten parents, or flags, and have
them disconnected from the main object hash table.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This way only structures we ask the caller to allocate on their
call stack or which we want to allow them to use members from
are shown in the API docs.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Most read calls will use the small object format, as the
majority of the content within the database is very small
objects (under 20 KB when inflated).
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
