When encoding varints to a buffer, we want to remain sure that the
required buffer space does not exceed what is actually available. Our
current check does not do the right thing, though, in that it does not
honor that our `pos` variable counts the position down instead of up. As
such, we will require too much memory for small varints and not enough
memory for big varints.
Fix the issue by correctly calculating the required size as
`(sizeof(varint) - pos)`. Add a test which failed before.
Upstream git.git has changed the way how packfiles are named.
Previously, they were using a hash of the contained object's OIDs, which
has then been changed to use the hash of the complete packfile instead.
See 1190a1acf (pack-objects: name pack files after trailer hash,
2013-12-05) in the git.git repository for more information on this
change.
This commit changes our logic to match the behavior of core git.
To determine if a repository is a worktree or not, we currently check
for the existence of a "gitdir" file inside of the repository's gitdir.
While this is sufficient for non-broken repositories, we have at least
one case of a subtly broken repository where there exists a gitdir file
inside of a gitmodule. This will cause us to misidentify the submodule
as a worktree.
While this is not really a fault of ours, we can do better here by
observing that a repository can only ever be a worktree iff its common
directory and dotgit directory are different. This allows us to make our
check whether a repo is a worktree or not more strict by doing a simple
string comparison of these two directories. This will also allow us to
do the right thing in the above case of a broken repository, as for
submodules these directories will be the same. At the same time, this
allows us to skip the `stat` check for the "gitdir" file for most
repositories.
The check whether a repository is a worktree or not is currently done
inside of `git_repository_open_ext`. As we want to extend this function
later on, pull it out into its own function `repo_is_worktree` to ease
working on it.
The out-parameters of `find_repo` containing found paths of a repository
are a tad confusing, as they are not as obvious as they could be. Rename
them like following to ease reading the code:
- `repo_path` -> `gitdir_path`
- `parent_path` -> `workdir_path`
- `link_path` -> `gitlink_path`
- `common_path` -> `commondir_path`
The `path` out-parameter of `find_repo` is being sanitized initially
such that we do not try to append to existing content. The sanitization
is done via `git_buf_free`, though, which forces us to needlessly
reallocate the buffer later in the function. Fix this by using
`git_buf_clear` instead.
The fields `declared_size` and `received_bytes` of the `git_odb_stream`
are both of type `git_off_t` which is defined as a signed integer. When
passing these values to a printf-style string in
`git_odb_stream__invalid_length`, though, we format these as PRIuZ,
which is unsigned.
Fix the issue by using PRIdZ instead, silencing warnings on macOS.
The `error` variable is used as a return value in the out-section of
both `odb_read_1` and `read_prefix_1`. While the value will actually
always be initialized inside of this section, GCC fails to realize this
due to interactions with the `found` variable: if `found` is set, the
error will always be initialized. If it is not, we return early without
reaching the out-statements.
Shut up the warnings by initializing the error variable, even though it
is unnecessary.
The current signature of `git_worktree_prune` accepts a flags field to
alter its behavior. This is not as flexible as we'd like it to be when
we want to enable passing additional options in the future. As the
function has not been part of any release yet, we are still free to
alter its current signature. This commit does so by using our usual
pattern of an options structure, which is easily extendable without
breaking the API.
When creating a new worktree, we do have a potential race with us
creating the worktree and another process trying to delete the same
worktree as it is being created. As such, the upstream git project has
introduced a flag `git worktree add --locked`, which will cause the
newly created worktree to be locked immediately after its creation. This
mitigates the race condition.
We want to be able to mirror the same behavior. As such, a new flag
`locked` is added to the options structure of `git_worktree_add` which
allows the user to enable this behavior.
After calling `libssh2_init`, we need to clean up after the library by
executing `libssh2_exit` as soon as we exit. Register a shutdown handler
to do so which simply calls `libssh2_exit`. This fixes several memory
leaks.
We unconditionally return success when initializing libssh2, regardless
of whether `libgssh2_init` signals success or an error. Fix this by
checking its return code.
The `git_worktree_add` function currently accepts only a path and name
for the new work tree. As we may want to expand these parameters in
future versions without adding additional parameters to the function for
every option, this commit introduces our typical pattern of an options
struct. Right now, this structure is still empty, which will change with
the next commit.
While the function reading an object from the complete OID already
verifies OIDs, we do not yet do so for reading objects from a partial
OID. Do so when strict OID verification is enabled.
The read_prefix_1 function has several return statements springled
throughout the code. As we have to free memory upon getting an error,
the free code has to be repeated at every single retrun -- which it is
not, so we have a memory leak here.
Refactor the code to use the typical `goto out` pattern, which will free
data when an error has occurred. While we're at it, we can also improve
the error message thrown when multiple ambiguous prefixes are found. It
will now include the colliding prefixes.
Verifying hashsums of objects we are reading from the ODB may be costly
as we have to perform an additional hashsum calculation on the object.
Especially when reading large objects, the penalty can be as high as
35%, as can be seen when executing the equivalent of `git cat-file` with
and without verification enabled. To mitigate for this, we add a global
option for libgit2 which enables the developer to turn off the
verification, e.g. when he can be reasonably sure that the objects on
disk won't be corrupted.
The upstream git.git project verifies objects when looking them up from
disk. This avoids scenarios where objects have somehow become corrupt on
disk, e.g. due to hardware failures or bit flips. While our mantra is
usually to follow upstream behavior, we do not do so in this case, as we
never check hashes of objects we have just read from disk.
To fix this, we create a new error class `GIT_EMISMATCH` which denotes
that we have looked up an object with a hashsum mismatch. `odb_read_1`
will then, after having read the object from its backend, hash the
object and compare the resulting hash to the expected hash. If hashes do
not match, it will return an error.
This obviously introduces another computation of checksums and could
potentially impact performance. Note though that we usually perform I/O
operations directly before doing this computation, and as such the
actual overhead should be drowned out by I/O. Running our test suite
seems to confirm this guess. On a Linux system with best-of-five
timings, we had 21.592s with the check enabled and 21.590s with the
ckeck disabled. Note though that our test suite mostly contains very
small blobs only. It is expected that repositories with bigger blobs may
notice an increased hit by this check.
In addition to a new test, we also had to change the
odb::backend::nonrefreshing test suite, which now triggers a hashsum
mismatch when looking up the commit "deadbeef...". This is expected, as
the fake backend allocated inside of the test will return an empty
object for the OID "deadbeef...", which will obviously not hash back to
"deadbeef..." again. We can simply adjust the hash to equal the hash of
the empty object here to fix this test.
When connecting to a remote via socket stream, we first use getaddrinfo
to obtain the possible connection methods followed by creating and
connecting the socket. But when creating the socket, we error out as
soon as we get an invalid socket instead of trying out other address
hints returned by addrinfo.
Fix this by continuing on invalid socket instead of returning an error.
This fixes connection establishment with musl libc.
Configuration backends have a readonly-flag which is currently used to
distinguish configuration snapshots. But somewhat unexpectedly, we do
not use the flag to prevent writing to a readonly backend but happily
proceed to do so.
This commit modifies logic to also honor the readonly flag for
configuration setters. We will now traverse through all backends and
pick the first one which is not marked as read-only whenever we want to
write new configuration.
The config_file.h header provides some inline declarations accessing the
`git_config_backend`, but misses its declaration. Add the missing
include for "git2/sys/config.h" to add it.
In e7330016a (diff_parse: check return value of `git_diff_init_options`,
2017-03-20), we've introduced an error check whether we're able to
correctly initialize the diff options. This simple commit actually
introduced a segfault in that we now try to free the pointer to the
allocated diff in an error case, instead of the allocated diff itself.
This commit fixes the issue.
The `remediation` function is run in the retry loop in order to attempt
to fix any problems that the prior run encountered. There is nothing
"cleaned up". Clarify the name.
As with the callbacks, third-party implementations of smart subtransports cannot
reach into the opaque struct and thus cannot know what options the user set.
Add a getter for these options to copy the proxy options into something external
implementors can use.
When the `git_futils_mmap_ro_file` function encounters an error after
the file has been opened, it will do a simple returns. Instead, we
should close the opened file descriptor to avoid a leak. This commit
fixes the issue.
Previous to OpenSSL version 1.1, the user had to initialize at least the error
strings as well as the SSL algorithms by himself. OpenSSL version 1.1 instead
provides a new function `OPENSSL_init_ssl`, which handles initialization of all
subsystems. As the new API call will by default load error strings and
initialize the SSL algorithms, we can safely replace these calls when compiling
against version 1.1 or later.
This fixes a compiler error when compiling against OpenSSL version 1.1 which has
been built without stubs for deprecated syntax.
Up to version 1.0, OpenSSL required us to provide a callback which implements
a locking mechanism. Due to problems in the API design though this mechanism was
inherently broken, especially regarding that the locking callback cannot report
errors in an obvious way. Due to this shortcoming, the locking initialization
has been completely removed in OpenSSL version 1.1. As the library has also been
refactored to not make any use of these callback functions, we can safely remove
all initialization of the locking subsystem if compiling against OpenSSL version
1.1 or higher.
This fixes a compilation error when compiling against OpenSSL version 1.1 which
has been built without stubs for deprecated syntax.
In the function `git_filter_list_stream_data`, we initialize, write and
subesquently close the stream which should receive content processed by
the filter. While we skip writing to the stream if its initialization
failed, we still try to close it unconditionally -- even if the
initialization failed, where the stream might not be set at all, leading
us to segfault.
Semantics in this code is not really clear. The function handling the
same logic for files instead of data seems to do the right thing here in
only closing the stream when initialization succeeded. When stepping
back a bit, this is only reasonable: if a stream cannot be initialized,
the caller would not expect it to be closed again. So actually, both
callers of `stream_list_init` fail to do so. The data streaming function
will always close the stream and the file streaming function will not
close the stream if writing to it has failed.
The fix is thus two-fold:
- callers of `stream_list_init` now close the stream iff it has been
initialized
- `stream_list_init` now closes the lastly initialized stream if
the current stream in the chain failed to initialize
Add a test which segfaulted previous to these changes.
Whenever we rename a branch, we update the repository's symbolic HEAD
reference if it currently points to the branch that is to be renamed.
But with the introduction of worktrees, we also have to iterate over all
HEADs of linked worktrees to adjust them. Do so.
While we already provide functions to get the current repository's HEAD,
it is quite involved to iterate over HEADs of both the repository and
all linked work trees. This commit implements a function
`git_repository_foreach_head`, which accepts a callback which is then
called for all HEAD files.
The functions `git_repository_head_for_worktree` and
`git_repository_detached_head_for_worktree` both implement their
own logic to read the HEAD reference file. Use the new function
`git_reference__read_head` instead to unify the code paths.
The function `read_worktree_head` has the logic embedded to construct
the path to `HEAD` in the work tree's git directory, which is quite
useful for other callers. Extract the logic into its own function to
make it reusable by others.
If trying to set the HEAD of a repository to another reference, we have
to check whether this reference is already checked out in another linked
work tree. If it is, we will refuse setting the HEAD and return an
error, but do not set a meaningful error message. Add one.
Currently, we only provide functions to read references directly from a
repository's reference store via e.g. `git_reference_lookup`. But in
some cases, we may want to read files not connected to the current
repository, e.g. when looking up HEAD of connected work trees. This
commit implements `git_reference__read_head`, which will read out and
allocate a reference at an arbitrary path.
When executing `git_futils_mmap_ro_file`, we first try to guess whether
the file is mmapable at all. Part of this check is whether the file is
too large to be mmaped, which can be true on systems with 32 bit
`size_t` types.
The check is performed by first getting the file size wtih
`git_futils_filesize` and then checking whether the returned size can be
represented as `size_t`, returning an error if so. While this test also
catches the case where the function returned an error (as `-1` is not
representable by `size_t`), we will set the misleading error message
"file too large to mmap". But in fact, a negative return value from
`git_futils_filesize` will be caused by the inability to fstat the file.
Fix the error message by handling negative return values separately and
not overwriting the error message in that case.
Short-circuit the call to `git_path_resolve_relative` in case
`git_buf_joinpath` returns an error. While this does not fix any
immediate errors, the resulting code is easier to read and handles
potential new error conditions raised by `git_buf_joinpath`.
In the `_check_dir_contents` function, we first allocate memory for
joining the directory and subdirectory together and afterwards use
`git_buf_joinpath`. While this function in fact should not fail as
memory is already allocated, err on the safe side and check for returned
errors.
The current code in `parse_section_header_ext` is only prepared to
properly handle out-of-memory conditions for the `git_buf` structure.
While very unlikely and probably caused by a programming error, it is
also possible to run into error conditions other than out-of-memory
previous to reaching the actual parsing loop. In these cases, we will
run into undefined behavior as the `rpos` variable is only initialized
after these triggerable errors, but we use it in the cleanup-routine.
Fix the issue by unifying the function's cleanup code with an
`end_error` section, which will not use the `rpos` variable.
POSIX emulation retries should be configurable so that tests can disable
them. In particular, maniacally threading tests may end up trying to
open locked files and need retries, which will slow continuous
integration tests significantly.
This can prevent FILE_SHARED_VIOLATIONS when used in tools such as TortoiseGit TGitCache and FILE_SHARE_DELETE, because files can be opened w/o being locked any more.
Signed-off-by: Sven Strickroth <email@cs-ware.de>
When `git_repository_set_head` is provided a remote reference, update
the reflog with the tag name, like we do with a branch. This helps
consumers match the semantics of `git checkout remote`.
Provide a macro that will allow us to run a function with posix-like
return values multiple times in a retry loop, with an optional cleanup
function called between invocations.
Introduce mapping from windows error codes to errno values. This
allows us to replace our calls to the Windows posix emulation functions
with calls to the Win32 APIs for more fine-grained control over the
emulation.
These mappings match the Windows CRT's mappings for its posix emulation
as they were described to me.
While writing the tree inside of a buffer, we check whether the buffer
runs out of memory after each tree entry. While we set the error code as
soon as we detect the OOM situation, we happily proceed iterating over
the entries. This is not useful at all, as we will try to write into the
buffer repeatedly, which cannot work.
Fix this by exiting as soon as we are OOM.
The `git_tree_entry *entry` variable is defined twice inside of this
function. While this is not a problem currently, remove the shadowing
variable to avoid future confusion.
While we detect errors in `git_treebuilder_write_with_buffer`, we just
exit directly instead of freeing allocated memory. Fix this by
remembering error codes and skipping forward to the function's cleanup
code.
The recent addition of an error code to `pass_whole_blame` in ff8d2eb15
(blame_git: check return value of object lookup, 2017-03-20) introduced
a spurious goto. Remove it.
The config and attrcache file reading code would attempt to load a file
in a home directory by expanding the `~` and looking for the file, using
`git_sysdir_find_global_file`. If the file was not found, the error
handling would look for the literal path, eg `~/filename.txt`.
Use the new `git_config_expand_global_file` instead, which allows us to
get the path to the file separately, when the path is prefixed with
`~/`, and fail with a not found error without falling back to looking
for the literal path.
Provide a mechanism for callers to expand the full path of a file in the
global configuration directory (that is to say, the home directory) even
if the file doesn't necessarily exist. This lets callers use their own
logic for building paths separate from handling file existence.
`O_EXCL` and `O_TRUNC` are mutually exclusive flags to open(2); you can't
truncate a file if you're asserting that it can't exist in the first place.
Drop `O_TRUNC`.
When `git_repository_set_head` is provided a tag reference, update the
reflog with the tag name, like we do with a branch. This helps
consumers match the semantics of `git checkout tag`.
While parsing patch header lines, we iterate over each line and check if
the line has trailing garbage. What we do not check though is that the
line is actually a line ending with a trailing newline.
Fix this by checking the return code of `parse_advance_expected_str`.
