When fsync'ing files, fsync the parent directory in the case where we
rename a file into place, or create a new file, to ensure that the
directory entry is flushed correctly.
Add a custom `O_FSYNC` bit (if it's not been defined by the operating
system`) so that `git_futils_writebuffer` can optionally do an `fsync`
when it's done writing.
We call `fsync` ourselves, even on systems that define `O_FSYNC` because
its definition is no guarantee of its actual support. Mac, for
instance, defines it but doesn't support it in an `open(2)` call.
Introduce a simple counter that `p_fsync` implements. This is useful
for ensuring that `p_fsync` is called when we expect it to be, for
example when we have enabled an odb backend to perform `fsync`s when
writing objects.
Fixes a regression from #4092. This is a crash on 32-bit and I assume that
it doesn't do the right thing on 64-bit either. MSVC emits a warning for this,
but of course, it's easy to get lost among all of the similar 'possible loss
of data' warnings.
Provide a descriptive error message when compiling THREADSAFE on gcc
versions < 4.1. We require the atomic primitives (eg
`__sync_synchronize`) that were introduced in that version.
(Note, clang setes `__GNUC__` but appears to set its version > 4.1.)
Remove useless indirection from `git_attr_cache__init` to
`git_attr_cache__do_init`. The difference is that the
`git_attr_cache__init` macro first checks if the cache is already
initialized and, if so, not call `git_attr_cache__do_init`. But
actually, `git_attr_cache__do_init` already does the same thing and
returns immediately if the cache is already initialized.
Remove the indirection.
When doing an upsert of a file, we used to use `git__compare_and_swap`,
comparing the entry's file which is to be replaced with itself. This can
be more easily formulated by using `git__swap`, which unconditionally
replaces the value.
`snprintf` requires a _format_ but does not require _arguments_ to the
format. eg: `snprintf(buf, 42, "hi")` is perfectly legal. Expand the
macro to match.
Without this, `p_sprintf(buf, 42, "hi")` errors with:
```
error: expected expression
p_snprintf(msg, 42, "hi");
^
src/unix/posix.h:53:34: note: expanded from macro 'p_snprintf'
^
/usr/include/secure/_stdio.h:57:73: note: expanded from macro 'snprintf'
__builtin___snprintf_chk (str, len, 0, __darwin_obsz(str),
__VA_ARGS__)
```
The upstream git.git project currently identifies all references inside
of `refs/bisect/` as well as `HEAD` as per-worktree references. This is
already incorrect and is currently being fixed by an in-flight topic
[1]. The new behavior will be to match all pseudo-references outside of
the `refs/` hierarchy as well as `refs/bisect/`.
Our current behavior is to mark a selection of pseudo-references as
per-worktree, only. This matches more pseudo-references than current
git, but forgets about `refs/bisect/`. Adjust behavior to match the
in-flight topic, that is classify the following references as
per-worktree:
- everything outside of `refs/`
- everything inside of `refs/bisect/`
[1]: <20170213152011.12050-1-pclouds@gmail.com>
When extracting a commit's signature, we first free the object and only
afterwards put its signature contents into the result buffer. This works
in most cases - the free'd object will normally be cached anyway, so we
only end up decrementing its reference count without actually freeing
its contents. But in some more exotic setups, where caching is disabled,
this can definitly be a problem, as we might be the only instance
currently holding a reference to this object.
Fix this issue by first extracting the contents and freeing the object
afterwards only.
The functions `git_commit_header_field` and
`git_commit_extract_signature` both receive buffers used to hand back
the results to the user. While these functions called `git_buf_sanitize`
on these buffers, this is not the right thing to do, as it will simply
initialize or zero-terminate passed buffers. As we want to overwrite
contents, we instead have to call `git_buf_clear` to completely reset
them.
When `git_buf_sanitize` gets called, it converts a buffer with NULL
content to be correctly initialized. This is done by pointing it to
`git_buf__initbuf`. While the method's documentation states this
clearly, it may also lead to the conclusion that it will do the same to
buffers which do _not_ have NULL contents.
Clarify behavior when passing a buffer with non-NULL contents, where
`git_buf_sanitize` will ensure that the contents are `\0`-terminated.
When opening a worktree via the gitdir of its parent repository
we fail to correctly set up the worktree's working directory. The
problem here is two-fold: we first fail to see that the gitdir
actually is a gitdir of a working tree and then subsequently
fail to determine the working tree location from the gitdir.
The first problem of not noticing a gitdir belongs to a worktree
can be solved by checking for the existence of a `gitdir` file in
the gitdir. This file points back to the gitlink file located in
the working tree's working directory. As this file only exists
for worktrees, it should be sufficient indication of the gitdir
belonging to a worktree.
The second problem, that is determining the location of the
worktree's working directory, can then be solved by reading the
`gitdir` file in the working directory's gitdir. When we now
resolve relative paths and strip the final `.git` component, we
have the actual worktree's working directory location.
The `path_repository` variable is actually confusing to think
about, as it is not always clear what the repository actually is.
It may either be the path to the folder containing worktree and
.git directory, the path to .git itself, a worktree or something
entirely different. Actually, the intent of the variable is to
hold the path to the gitdir, which is either the .git directory
or the bare repository.
Rename the variable to `gitdir` to avoid confusion. While at it,
also rename `path_gitlink` to `gitlink` to improve consistency.
If a branch is already checked out in a working tree we are not
allowed to check out that branch in another repository. Introduce
this restriction when setting a repository's HEAD.
Implement a new function that is able to determine if a branch is
checked out in any repository connected to the current
repository. In particular, this is required to check if for a
given repository and branch, there exists any working tree
connected to that repository that is referencing this branch.
Implement `git_repository_head_for_worktree` and
`git_repository_head_detached_for_worktree` for directly accessing a
worktree's HEAD without opening it as a `git_repository` first.
Implement the `git_worktree_prune` function. This function can be
used to delete working trees from a repository. According to the
flags passed to it, it can either delete the working tree's
gitdir only or both gitdir and the working directory.
Working trees support locking by creating a file `locked` inside
the tree's gitdir with an optional reason inside. Support this
feature by adding functions to get and set the locking status.
Add a new function that checks wether a given `struct
git_worktree` is valid. The validation includes checking if the
gitdir, parent directory and common directory are present.
Introduce a new `struct git_worktree`, which holds information
about a possible working tree connected to a repository.
Introduce functions to allow opening working trees for a
repository.
A repository's configuartion file can always be found in the
GIT_COMMON_DIR, which has been newly introduced. For normal
repositories this does change nothing, but for working trees this
change allows to access the shared configuration file.
The refdb_fs_backend is not aware of the git commondir, which
stores common objects like the o bject database and packed/loose
refereensces when worktrees are used.
Make refdb_fs_backend aware of the common directory by
introducing a new commonpath variable that points to the actual
common path of the database and using it instead of the gitdir
for the mentioned objects.
The variable '.path' of the refdb_fs_backend struct becomes
confusing regarding the introduction of the git commondir. It
does not immediatly become obvious what it should point to.
Fix this problem by renaming the variable to `gitpath`,
clarifying that it acutally points to the `.git` directory of the
repository, in contrast to the commonpath directory, which points
to the directory containing shared objects like references and
the object store.
The recent introduction of the commondir variable of a repository
requires callers to distinguish whether their files are part of
the dot-git directory or the common directory shared between
multpile worktrees. In order to take the burden from callers and
unify knowledge on which files reside where, the
`git_repository_item_path` function has been introduced which
encapsulate this knowledge.
Modify most existing callers of `git_repository_path` to use
`git_repository_item_path` instead, thus making them implicitly
aware of the common directory.
The commondir variable stores the path to the common directory.
The common directory is used to store objects and references
shared across multiple repositories. A current use case is the
newly introduced `git worktree` feature, which sets up a separate
working copy, where the backing git object store and references
are pointed to by the common directory.
When calling `git_path_dirname_r` on a Win32 prefix, e.g. a drive
or network share prefix, we always want to return the trailing
'/'. This does not work currently when passing in a path like
'C:', where the '/' would not be appended correctly.
Fix this by appending a '/' if we try to normalize a Win32 prefix
and there is no trailing '/'.
Getting the dirname of a filesystem root should return the filesystem
root itself. E.g. the dirname of "/" is always "/". On Windows, we
emulate this behavior and as such, we should return e.g. "C:/" if
calling dirname on "C:/". But we currently fail to do so and instead
return ".", as we do not check if we actually have a Windows prefix
before stripping off the last directory component.
Fix this by calling out to `win32_prefix_length` immediately after
stripping trailing slashes, returning early if we have a prefix.
Extract code which determines if a path is at a Windows system's root.
This incluses drive prefixes (e.g. "C:\") as well as network computer
names (e.g. "//computername/").
The code reversing a vector initially determines the rear-pointer by
simply subtracting 1 from the vector's length. Obviously, this fails if
the vector is empty, in which case we have an integer overflow.
Fix the issue by returning early if the vector is empty.
Fix the following warning emitted by clang:
[ 16%] Building C object CMakeFiles/libgit2_clar.dir/src/submodule.c.o
/Users/mplough/devel/external/libgit2/src/submodule.c:408:6: warning: variable 'i' is used uninitialized whenever 'if' condition is true
[-Wsometimes-uninitialized]
if ((error = load_submodule_names(names, cfg)))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mplough/devel/external/libgit2/src/submodule.c:448:20: note: uninitialized use occurs here
git_iterator_free(i);
^
/Users/mplough/devel/external/libgit2/src/submodule.c:408:2: note: remove the 'if' if its condition is always false
if ((error = load_submodule_names(names, cfg)))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/mplough/devel/external/libgit2/src/submodule.c:404:17: note: initialize the variable 'i' to silence this warning
git_iterator *i;
^
= NULL
1 warning generated.
Set up a WinHTTP status callback; inspect the WinHTTP status for
WINHTTP_CALLBACK_STATUS_SECURE_FAILURE, and convert the status code
to a useful message for callers.
`git_submodule_status` is very slow, bottlenecked on
`git_repository_head_tree`, which it uses through `submodule_update_head`. If
the user has requested submodule caching, assume that they want this status
cached too and skip it.
Signed-off-by: David Turner <dturner@twosigma.com>
Added `git_repository_submodule_cache_all` to initialze a cache of
submodules on the repository so that operations looking up N
submodules are O(N) and not O(N^2). Added a
`git_repository_submodule_cache_clear` function to remove the cache.
Also optimized the function that loads all submodules as it was itself
O(N^2) w.r.t the number of submodules, having to loop through the
`.gitmodules` file once per submodule. I changed it to process the
`.gitmodules` file once, into a map.
Signed-off-by: David Turner <dturner@twosigma.com>
For username/password credentials, support NTLM or Basic (in that order
of priority). Use the WinHTTP built-in authentication support for both,
and maintain a bitfield of the supported mechanisms from the response.
The Git protocol does not specify what should happen in the case
of an empty packet line (that is a packet line "0004"). We
currently indicate success, but do not return a packet in the
case where we hit an empty line. The smart protocol was not
prepared to handle such packets in all cases, though, resulting
in a `NULL` pointer dereference.
Fix the issue by returning an error instead. As such kind of
packets is not even specified by upstream, this is the right
thing to do.
Each packet line in the Git protocol is prefixed by a four-byte
length of how much data will follow, which we parse in
`git_pkt_parse_line`. The transmitted length can either be equal
to zero in case of a flush packet or has to be at least of length
four, as it also includes the encoded length itself. Not
checking this may result in a buffer overflow as we directly pass
the length to functions which accept a `size_t` length as
parameter.
Fix the issue by verifying that non-flush packets have at least a
length of `PKT_LEN_SIZE`.
git_checkout_tree() sets up its working directory iterator to respect the
pathlist if GIT_CHECKOUT_DISABLE_PATHSPEC_MATCH is present, which is great.
What's not so great is that this iterator is then used side-by-side with
an iterator created by git_checkout_iterator(), which did not set up its
pathlist appropriately (although the iterator mirrors all other iterator
options).
This could cause git_checkout_tree() to delete working tree files which
were not specified in the pathlist when GIT_CHECKOUT_DISABLE_PATHSPEC_MATCH
was used, as the unsynchronized iterators causes git_checkout_tree() to think
that files have been deleted between the two trees. Oops.
And added a test which fails without this fix (specifically, the final check
for "testrepo/README" to still be present fails).
Error messages should be sentence fragments, and therefore:
1. Should not begin with a capital letter,
2. Should not conclude with punctuation, and
3. Should not end a sentence and begin a new one
We want to keep the git UA in order for services to recognise that we're
a Git client and not a browser. But in order to stop dumb HTTP some
services have blocked UAs that claim to be pre-1.6.6 git.
Thread these needles by using the "git/2.0" prefix which is still close
enough to git's yet distinct enough that you can tell it's us.
This partially reverts bdec62dce1 which activates
the transport code-paths which allow you to use a custom TLS implementation
without having to have one at build-time.
However the capabilities describe how libgit2 was built, not what it could
potentially support, bring back the ifdefs so we only say we support HTTPS if
libgit2 was itself built with a TLS implementation.
The function to write trees allocates a new buffer for each tree.
This causes problems with performance when performing a lot
of actions involving writing trees, e.g. when doing many merges.
Fix the issue by instead handing in a shared buffer, which is then
re-used across the calls without having to re-allocate between
calls.
When trying to uncompress deltas in a packfile's delta chain, we try to
add object bases to the packfile cache, subsequently decrementing its
reference count if it has been added successfully. This may lead to a
mismatched reference count in the case where we exit the loop early due
to an encountered error.
Fix the issue by decrementing the reference count in error cleanup.
git_rebase_finish relies on head_detached being set, but
rebase_init_merge was only setting it when branch->ref_name was unset.
But branch->ref_name would be set to "HEAD" in the case of detached
HEAD being either implicitly (NULL) or explicitly passed to
git_rebase_init.
Introduce `git_thread_exit`, which will allow threads to terminate at an
arbitrary time, returning a `void *`. On Windows, this means that we
need to store the current `git_thread` in TLS, so that we can set its
`return` value when terminating.
We cannot simply use `ExitThread`, since Win32 returns `DWORD`s from
threads; we return `void *`.
`giterr_set()` is used when it is required to format a string, and since
we don't really require it for this case, it is better to stick to
`giterr_set_str()`.
This also suppresses a warning(-Wformat-security) raised by the compiler.
Signed-off-by: Pranit Bauva <pranit.bauva@gmail.com>
The `CURLINFO_LASTSOCKET` information has been deprecated since
curl version 7.45.0 as it may result in an overflow in the
returned socket on certain systems, most importantly on 64 bit
Windows. Instead, a new call `CURLINFO_ACTIVESOCKET` has been
added which instead returns a `curl_socket_t`, which is always
sufficiently long to store a socket.
As we need to provide backwards compatibility with curl versions
smaller than 7.45.0, alias CURLINFO_ACTIVESOCKET to
CURLINFO_LASTSOCKET on platforms without CURLINFO_ACTIVESOCKET.
We look at whether we're trying to replace a blob with a tree during the
update phase, but we fail to look at whether we've just inserted a blob
where we're now trying to insert a tree.
Update the check to look at both places. The test for this was
previously succeeding due to the bu where we did not look at the sorted
output.
On Windows we can find locked files even when reading a reference or the
packed-refs file. Bubble up the error in this case as well to allow
callers on Windows to retry more intelligently.
It does not help us to check whether the file exists before trying to
unlink it since it might be gone by the time unlink is called.
Instead try to remove it and handle the resulting error if it did not
exist.
Checking the size before we open the file descriptor can lead to the
file being replaced from under us when renames aren't quite atomic, so
we can end up reading too little of the file, leading to us thinking the
file is corrupted.
There might be a few threads or processes working with references
concurrently, so fortify the code to ignore errors which come from
concurrent access which do not stop us from continuing the work.
This includes ignoring an unlinking error. Either someone else removed
it or we leave the file around. In the former case the job is done, and
in the latter case, the ref is still in a valid state.
We need to save the errno, lest we clobber it in the giterr_set()
call. Also add code for reporting that a path component is missing,
which is a distinct failure mode.
In order not to undo concurrent modifications to references, we must
make sure that we only delete a loose reference if it still has the same
value as when we packed it.
This means we need to lock it and then compare the value with the one we
put in the packed file.
When trying to find a discovery, we walk up the directory
structure checking if there is a ".git" file or directory and, if
so, check its validity. But in the case that we've got a ".git"
file, we do not want to unconditionally assume that the file is
in fact a ".git" file and treat it as such, as we would error out
if it is not.
Fix the issue by only treating a file as a gitlink file if it
ends with "/.git". This allows users of the function to discover
a repository by handing in any path contained inside of a git
repository.
The existing code would set a namespace of "" (empty string) with
GIT_NAMESPACE unset. In a repository where refs/heads/namespaces/
exists, that can produce incorrect results. Detect that case and avoid
setting the namespace at all.
Since that makes the last assignment to error conditional, and the
previous assignment can potentially get GIT_ENOTFOUND, set error to 0
explicitly to prevent the call from incorrectly failing with
GIT_ENOTFOUND.
We're recently trying to upgrade to the current master of libgit2
in Cargo but we're unfortunately hitting a segfault in one of our
tests. This particular test is just a small smoke test that https
works (e.g. it's configured in libgit2). It attempts to clone
from a URL which simply immediately drops connections after
they're accepted (e.g. terminate abnormally). We expect to see a
standard error from libgit2 but unfortunately we're seeing a
segfault.
This segfault is happening inside of the `wait_for` function of
`curl_stream.c` at the line `FD_SET(fd, &errfd)` because `fd` is
-1. This ends up doing an out-of-bounds array access that faults
the program. I tracked back to where this -1 came from to the
line here (returned by `CURLINFO_LASTSOCKET`) and added a check
to return an error.
The code correctly detects that forced creation of a branch on a
nonbare repo should not be able to overwrite a branch which is
the HEAD reference. But there's no reason to prevent this on
a bare repo, and in fact, git allows this. I.e.,
git branch -f master new_sha
works on a bare repo with HEAD set to master. This change fixes
that problem, and updates tests so that, for this case, both the
bare and nonbare cases are checked for correct behavior.
We need to include the initialisation and construction functions in all
backend, so we include this header when building against SecureTransport
and WinHTTP as well.
