This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to
the new style of error handling. Also got the unix and win32
versions of map.c. There are some minor changes to other
files but no others were completely converted.
This also contains an update to filebuf so that a zeroed out
filebuf will not think that the fd (== 0) is actually open
(and inadvertently call close() on fd 0 if cleaned up).
Lastly, this was built and tested on win32 and contains a
bunch of fixes for the win32 build which was pretty broken.
It turns out that commit 31e9cfc4cbcaf1b38cdd3dbe3282a8f57e5366a5
did not fix the GIT_USUSED behavior on all platforms. This commit
walks through and really cleans things up more thoroughly, getting
rid of the unnecessary stuff.
To remove the use of some GIT_UNUSED, I ended up adding a couple
of new iterators for hashtables that allow you to iterator just
over keys or just over values.
In making this change, I found a bug in the clar tests (where we
were doing *count++ but meant to do (*count)++ to increment the
value). I fixed that but then found the test failing because it
was not really using an empty repo. So, I took some of the code
that I wrote for iterator testing and moved it to clar_helpers.c,
then made use of that to make it easier to open fixtures on a
per test basis even within a single test file.
This makes so much sense that I can't believe it hasn't been done
before. Kill the old `git_fbuffer` and read files straight into
`git_buf` objects.
Also: In order to fully support 4GB files in 32-bit systems, the
`git_buf` implementation has been changed from using `ssize_t` for
storage and storing negative values on allocation failure, to using
`size_t` and changing the buffer pointer to a magical pointer on
allocation failure.
Hopefully this won't break anything.
This takes all of the functions that look up simple data about
paths (such as `git_futils_isdir`) and moves them over to path.h
(becoming `git_path_isdir`). This leaves fileops.h just with
functions that actually manipulate the filesystem or look at
the file contents in some way.
As part of this, the dir.h header which is really just for win32
support was moved into win32 (with some minor changes).
This adds APIs for querying git attributes. In addition to
the new API in include/git2/attr.h, most of the action is in
src/attr_file.[hc] which contains utilities for dealing with
a single attributes file, and src/attr.[hc] which contains
the implementation of the APIs that merge all applicable
attributes files.
This converts virtually all of the places that allocate GIT_PATH_MAX
buffers on the stack for manipulating paths to use git_buf objects
instead. The patch is pretty careful not to touch the public API
for libgit2, so there are a few places that still use GIT_PATH_MAX.
This extends and changes some details of the git_buf implementation
to add a couple of extra functions and to make error handling easier.
This includes serious alterations to all the path.c functions, and
several of the fileops.c ones, too. Also, there are a number of new
functions that parallel existing ones except that use a git_buf
instead of a stack-based buffer (such as git_config_find_global_r
that exists alongsize git_config_find_global).
This also modifies the win32 version of p_realpath to allocate whatever
buffer size is needed to accommodate the realpath instead of hardcoding
a GIT_PATH_MAX limit, but that change needs to be tested still.
The ownership semantics have been changed all over the library to be
consistent. There are no more "borrowed" or duplicated references.
Main changes:
- `git_repository_open2` and `3` have been dropped.
- Added setters and getters to hotswap all the repository owned
objects:
`git_repository_index`
`git_repository_set_index`
`git_repository_odb`
`git_repository_set_odb`
`git_repository_config`
`git_repository_set_config`
`git_repository_workdir`
`git_repository_set_workdir`
Now working directories/index files/ODBs and so on can be
hot-swapped after creating a repository and between operations.
- All these objects now have proper ownership semantics with
refcounting: they all require freeing after they are no longer
needed (the repository always keeps its internal reference).
- Repository open and initialization has been updated to keep in
mind the configuration files. Bare repositories are now always
detected, and a default config file is created on init.
- All the tests affected by these changes have been dropped from the
old test suite and ported to the new one.
Update all stack allocations of git_filebuf to use GIT_FILEBUF_INIT
and make git_filebuf_open and git_filebuf_cleanup safe to be called
multiple times on the same buffer.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
reference_rename used to delete an old reflog file when renaming a
reference to not confuse git.git. Don't do this anymore but let the user
take care of writing a reflog entry.
Signed-off-by: schu <schu-github@schulog.org>
This new version of the references code is significantly faster and
hopefully easier to read.
External API stays the same. A new method `git_reference_reload()` has
been added to force updating a memory reference from disk. In-memory
references are no longer updated automagically -- this was killing us.
If a reference is deleted externally and the user doesn't reload the
memory object, nothing critical happens: any functions using that
reference should fail gracefully (e.g. deletion, renaming, and so on).
All generated references from the API are read only and must be free'd
by the user. There is no reference counting and no traces of generated
references are kept in the library.
There is no longer an internal representation for references. There is
only one reference struct `git_reference`, and symbolic/oid targets are
stored inside an union.
Packfile references are stored using an optimized struct with flex array
for reference names. This should significantly reduce the memory cost of
loading the packfile from disk.
git_reference_rename() didn't properly cleanup old references given by
the user to not break some ugly old tests. Since references don't point
to libgit's internal cache anymore we can cleanup git_reference_rename()
to be somewhat less messy.
Signed-off-by: schu <schu-github@schulog.org>
Currently libgit2 shares pointers to its internal reference cache with
the user. This leads to several problems like invalidation of reference
pointers when reordering the cache or manipulation of the cache from
user side.
Give each user its own git_reference instead of leaking the internal
representation (struct reference).
Add the following new API functions:
* git_reference_free
* git_reference_is_packed
Signed-off-by: schu <schu-github@schulog.org>
The following files now have 0444 permissions:
- loose objects
- pack indexes
- pack files
- packs downloaded by fetch
- packs downloaded by the HTTP transport
And the following files now have 0666 permissions:
- config files
- repository indexes
- reflogs
- refs
This brings libgit2 more in line with Git.
Note that git_filebuf_commit() and git_filebuf_commit_at() have both
gained a new mode parameter.
The latter change fixes an important issue where filebufs created with
GIT_FILEBUF_TEMPORARY received 0600 permissions (due to mkstemp(3)
usage). Now we chmod() the file before renaming it into place.
Tests have been added to confirm that new commit, tag, and tree
objects are created with the right permissions. I don't have access to
Windows, so for now I've guarded the tests with "#ifndef GIT_WIN32".
6c8b458 removed an "unused" variable needed for git_hashtable_insert2(),
causing a segfault in reference_rename(). Instead, use
git_hashtable_insert().
Signed-off-by: schu <schu-github@schulog.org>
reference_rename() recently failed when renaming an existing reference
refs/heads/foo/bar -> refs/heads/foo because of a change in the
underlying functions / error codes. Fixes#412.
Signed-off-by: schu <schu-github@schulog.org>
There were quite a few places were spaces were being used instead of
tabs. Try to catch them all. This should hopefully not break anything.
Except for `git blame`. Oh well.
1. The license header is technically not valid if it doesn't have a
copyright signature.
2. The COPYING file has been updated with the different licenses used in
the project.
3. The full GPLv2 header in each file annoys me.
We need to do an unsigned comparison, as otherwise UTF-8 characters
might look like they have the sign bit set and the check will fail.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
/home/kas/git/public/libgit2/src/refs.c: In function ‘normalize_name’:
/home/kas/git/public/libgit2/src/refs.c:1681:12: warning: cast discards ‘__attribute__((const))’ qualifier from pointer target type [-Wcast-qual]
Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
In reference_read we stat a file and then call futils which stats it
again. Use git_futils_readbuffer_updated to avoid the extra stat
call. This introduces another parameter which is used to tell the
caller whether the file was read or not.
Modify the callers to take advantage of this new feature. This change
removes ~140 stat calls from the test suite.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Current implementation of git_reference_rename() removes 'ref' from
loose cache, but not frees it. In result 'ref' is not reachable any more
and we have got memory leak.
Let's re-add 'ref' with corrected name to loose cache instead of
'new_ref' and free 'new_ref' properly.
'rollback' path seems leak too. git_reference_rename() need to be rewritten
for proper resource management.
Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
Drop the GLibc implementation of Merge Sort and replace it with Timsort.
The algorithm has been tuned to work on arrays of pointers (void **),
so there's no longer a need to abstract the byte-width of each element
in the array.
All the comparison callbacks now take pointers-to-elements, not
pointers-to-pointers, so there's now one less level of dereferencing.
E.g.
int index_cmp(const void *a, const void *b)
{
- const git_index_entry *entry_a = *(const git_index_entry **)(a);
+ const git_index_entry *entry_a = (const git_index_entry *)(a);
The result is up to a 40% speed-up when sorting vectors. Memory usage
remains lineal.
A new `bsearch` implementation has been added, whose callback also
supplies pointer-to-elements, to uniform the Vector API again.
`git_futils_rmdir_r`: rename, clean up.
`git_reference_rename`: cleanup. Do not use 3x4096 buffers on the stack
or things will get ugly very fast. We can reuse the same buffer.
So far libgit2 didn't handle the following scenarios:
* Rename of reference m -> m/m
* Rename of reference n/n -> n
Fixed.
Since we don't write reflogs, we have to delete any old reflog for the
renamed reference. Otherwise git.git will possibly fail when it finds
invalid logs.
Reported-by: nulltoken <emeric.fermas@gmail.com>
Signed-off-by: schu <schu-github@schulog.org>
Cleaned up the structure of the whole OS-abstraction layer.
fileops.c now contains a set of utility methods for file management used
by the library. These are abstractions on top of the original POSIX
calls.
There's a new file called `posix.c` that contains
emulations/reimplementations of all the POSIX calls the library uses.
These are prefixed with `p_`. There's a specific posix file for each
platform (win32 and unix).
All the path-related methods have been moved from `utils.c` to `path.c`
and have their own prefix.
The assertion in line 360 was there to check that only loose refs were
being written as loose, but there are times when we need to re-write a
packed reference as loose.
A bunch of redundant methods have been removed from the external API.
- All the reference/tag creation methods with `_f` are gone. The force
flag is now passed as an argument to the normal create methods.
- All the different commit creation methods are gone; commit creation
now always requires a `git_commit` pointer for parents and a `git_tree`
pointer for tree, to ensure that corrupted commits cannot be generated.
- All the different tag creation methods are gone; tag creation now
always requires a `git_object` pointer to ensure that tags are not
created to inexisting objects.
There is no need to store the format outselves, as the library
provides git_filebuf_printf which takes care of the formatting itself.
Also get rid of an use of strcat + strcpy which is always a nice
thing.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Ok, this is the real deal. Hopefully. Here's how it's going to work:
- One main method, called `git__throw`, that sets the error
code and error message when an error happens.
This method must be called in every single place where an error
code was being returned previously, setting an error message
instead.
Example, instead of:
return GIT_EOBJCORRUPTED;
Use:
return git__throw(GIT_EOBJCORRUPTED,
"The object is missing a finalizing line feed");
And instead of:
[...] {
error = GIT_EOBJCORRUPTED;
goto cleanup;
}
Use:
[...] {
error = git__throw(GIT_EOBJCORRUPTED, "What an error!");
goto cleanup;
}
The **only** exception to this are the allocation methods, which
return NULL on failure but already set the message manually.
/* only place where an error code can be returned directly,
because the error message has already been set by the wrapper */
if (foo == NULL)
return GIT_ENOMEM;
- One secondary method, called `git__rethrow`, which can be used to
fine-grain an error message and build an error stack.
Example, instead of:
if ((error = foobar(baz)) < GIT_SUCCESS)
return error;
You can now do:
if ((error = foobar(baz)) < GIT_SUCCESS)
return git__rethrow(error, "Failed to do a major operation");
The return of the `git_lasterror` method will be a string in the
shape of:
"Failed to do a major operation. (Failed to do an internal
operation)"
E.g.
"Failed to open the index. (Not enough permissions to access
'/path/to/index')."
NOTE: do not abuse this method. Try to write all `git__throw`
messages in a descriptive manner, to avoid having to rethrow them to
clarify their meaning.
This method should only be used in the places where the original
error message set by a subroutine is not specific enough.
It is encouraged to continue using this style as much possible to
enforce error propagation:
if ((error = foobar(baz)) < GIT_SUCCESS)
return error; /* `foobar` has set an error message, and
we are just propagating it */
The error handling revamp will take place in two phases:
- Phase 1: Replace all pieces of code that return direct error codes
with calls to `git__throw`. This can be done semi-automatically
using `ack` to locate all the error codes that must be replaced.
- Phase 2: Add some `git__rethrow` calls in those cases where the
original error messages are not specific enough.
Phase 1 is the main goal. A minor libgit2 release will be shipped once
Phase 1 is ready, and the work will start on gradually improving the
error handling mechanism by refining specific error messages.
OTHER NOTES:
- When writing error messages, please refrain from using weasel
words. They add verbosity to the message without giving any real
information. (<3 Emeric)
E.g.
"The reference file appears to be missing a carriage return"
Nope.
"The reference file is missing a carriage return"
Yes.
- When calling `git__throw`, please try to use more generic error
codes so we can eventually reduce the list of error codes to
something more reasonable. Feel free to add new, more generic error
codes if these are going to replace several of the old ones.
E.g.
return GIT_EREFCORRUPTED;
Can be turned into:
return git__throw(GIT_EOBJCORRUPTED,
"The reference is corrupted");
Typical use is git_reference_resolve(&ref, ref). Currently, if there is
an error, ref will point to NULL, causing the user to lose that
reference.
Always update resolved_ref instead of just on finding an OID ref,
storing the last valid reference in it.
This change helps simplify the code for allowing root commits.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Allow any well-formed reference name to live under refs/ removing the
condition that they be under refs/{heads,tags,remotes}/ as was the
design of git.
An exception is made for HEAD which is allowed to contain an OID
reference in detached HEAD state.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Add internal reference create and rename functions which take a force
parameter, telling them to overwrite an existing reference if it
exists.
These functions try to update the reference if it's of the same type
as the one it's going to be replaced by. Otherwise the old reference
becomes invalid.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
List all the references in the repository, calling a custom
callback for each one.
The listed references may be filtered by type, or using
a bitwise OR of several types. Use the magic value
`GIT_REF_LISTALL` to obtain all references, including
packed ones.
The `callback` function will be called for each of the references
in the repository, and will receive the name of the reference and
the `payload` value passed to this method.
It's no longer retarded. All object interdependencies are stored as OIDs
instead of actual objects. This should be hundreds of times faster,
specially on big repositories. Heck, who knows, maye it doesn't even
segfault -- wouldn't that be awesome?
What has changed on the API?
`git_commit_parent`, `git_commit_tree`, `git_tag_target` now return
their values through a pointer-to-pointer, and have an error code.
`git_commit_set_tree` and `git_tag_set_target` now return an error
code and may fail.
`git_repository_free__no_gc` has been deprecated because it's
stupid. Since there are no longer any interdependencies between
objects, we don't need internal reference counting, and GC
never fails or double-free's pointers.
`git_object_close` now does a very sane thing: marks an object
as unused. Closed objects will be eventually free'd from the
object cache based on LRU. Please use `git_object_close` from
the garbage collector `destroy` method on your bindings. It's
100% safe.
`git_repository_gc` is a new method that forces a garbage collector
pass through the repo, to free as many LRU objects as possible.
This is useful if we are running out of memory.
We now store only one sorting callback that does entry comparison. This
is used when sorting the entries using a quicksort, and when looking for
a specific entry with the new search methods.
The following search methods now exist:
git_vector_search(vector, entry)
git_vector_search2(vector, custom_search_callback, key)
git_vector_bsearch(vector, entry)
git_vector_bsearch2(vector, custom_search_callback, key)
The sorting state of the vector is now stored internally.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The methods previously known as
git_repository_lookup
git_repository_newobject
git_repository_lookup_ref
are now part of their respective namespaces:
git_object_lookup
git_object_new
git_reference_lookup
This makes the API more consistent with the new references API.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The following methods have been implemented:
git_reference_packall
git_reference_rename
git_reference_delete
The library now has full support for packed references, including
partial and total writing. Internal documentation has been updated with
the details.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
These two reference types are now stored separately to eventually allow
the removal/renaming of loose references and rewriting of the refs
packfile.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The old hash table with chained buckets has been replaced by a new one
using Cuckoo hashing, which offers guaranteed constant lookup times.
This should improve speeds on most use cases, since hash tables in
libgit2 are usually used as caches where the objects are stored once and
queried several times.
The Cuckoo hash implementation is based off the one in the Basekit
library [1] for the IO language, but rewritten to support an arbritrary
number of hashes. We currently use 3 to maximize the usage of the nodes pool.
[1]: https://github.com/stevedekorte/basekit/blob/master/source/CHash.c
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The new `git_filebuf` structure provides atomic high-performance writes
to disk by using a write cache, and optionally a double-buffered scheme
through a worker thread (not enabled yet).
Writes can be done 3-layered, like in git.git (user code -> write cache
-> disk), or 2-layered, by writing directly on the cache. This makes
index writing considerably faster.
The `git_filebuf` structure contains all the old functionality of
`git_filelock` for atomic file writes and reads. The `git_filelock`
structure has been removed.
Additionally, the `git_filebuf` API allows to automatically hash (SHA1)
all the data as it is written to disk (hashing is done smartly on big
chunks to improve performance).
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The `git__joinpath` function has been changed to use a statically
allocated buffer; we assume the buffer to be 4096 bytes, because fuck
you.
The new method also supports an arbritrary number of paths to join,
which may come in handy in the future.
Some methods which were manually joining paths with `strcpy` now use the
new function, namely those in `index.c` and `refs.c`.
Based on Emeric Fermas' original patch, which was using the old
`git__joinpath` because I'm stupid. Thanks!
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Several changes have been committed to allow the user to create
in-memory references and write back to disk. Peeling of symbolic
references has been made explicit. Added getter and setter methods for
all attributes on a reference. Added corresponding documentation.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
All the commits have been squashed into a single one before refactoring
the final code, to keep everything tidy.
Individual commit messages are as follows:
Added repository reference looking up functionality placeholder.
Added basic reference database definition and caching infrastructure.
Removed useless constant.
Added GIT_EINVALIDREFNAME error and description. Added missing description for GIT_EBAREINDEX.
Added GIT_EREFCORRUPTED error and description.
Added GIT_ETOONESTEDSYMREF error and description.
Added resolving of direct and symbolic references.
Prepared the packed-refs parsing.
Added parsing of the packed-refs file content.
When no loose reference has been found, the full content of the packed-refs file is parsed. All of the new (i.e. not previously parsed as a loose reference) references are eagerly stored in the cached references storage.
The method packed_reference_file__parse() is in deer need of some refactoring. :-)
Extracted to a method the parsing of the peeled target of a tag.
Extracted to a method the parsing of a standard packed ref.
Fixed leaky removal of the cached references.
Ensured that a previously parsed packed reference isn't returned if a more up-to-date loose reference exists.
Enhanced documentation of git_repository_reference_lookup().
Moved some refs related constants from repository.c to refs.h.
Made parsing of a packed tag reference more robust.
Updated git_repository_reference_lookup() documentation.
Added some references to the test repository.
Added some tests covering tag references looking up.
Added some tests covering symbolic and head references looking up.
Added some tests covering packed references looking up.
