This is useful to send to the client while we're performing the work.
The reporting function has a force parameter which makes sure that we
do send out the message of 100% completed, even if this comes before the
next udpate window.
Instead of copying each object individually, as we'd been doing, use the
packbuilder which should be faster and give us some feedback.
While performing this change, we can hook up the packbuilder's writing
to the push progress so the caller knows how far along we are.
We currently first look in the loose object dir and then in the packs
for objects. When performing operations on recent history this has a
higher likelihood of hitting, but when we deal with operations which
look further back into the past, we start spending a large amount of
time getting ENOTENT from `access`.
Reversing the priorities means that long-running operations can get to
their objects faster, as we can look at the index data we have in memory
(or rather mapped) to figure out whether we have an object, which is
faster than going out to the filesystem.
The packed backend already implements an optimistic read algorithm by
first looking at the packs we know about and only going out to disk to
referesh if the object is not found which means that in the case where
we do have the object (which will be in the majority for anything that
traverses the graph) we can avoid going to to disk entirely to determine
whether an object exists.
Operations which look at recent history may take a slight impact, but
these would be operations which look a lot less at object and thus take
less time regardless.
The base refspecs changing can be a cause of confusion as to what is the
current base refspec set and complicate saving the remote's
configuration.
Change `git_remote_add_{fetch,push}()` to update the configuration
instead of an instance.
This finally makes `git_remote_save()` a no-op, it will be removed in a
later commit.
While this will rarely be different from the default, having it in the
remote adds yet another setting it has to keep around and can affect its
behaviour. Move it to the options.
Instead of having it set in a different place from every other callback,
put it the main structure. This removes some state from the remote and
makes it behave more like clone, where the constructors are passed via
the options.
As a first step in removing the repository-saving logic, don't allow
chaning the url or push url from a remote object, but change the
configuration on the configuration immediately.
Having the setting be different from calling its actions was not a great
idea and made for the sake of the wrong convenience.
Instead of that, accept either fetch options, push options or the
callbacks when dealing with the remote. The fetch options are currently
only the callbacks, but more options will be moved from setters and
getters on the remote to the options.
This does mean passing the same struct along the different functions but
the typical use-case will only call git_remote_fetch() or
git_remote_push() and so won't notice much difference.
The push object knows which remote it's associated with, and therefore
does not need to keep its own copy of the callbacks stored in the
remote.
Remove the copy and simply access the callbacks struct within the
remote.
Restricting files to size_t is a silly limitation. The loose backend
writes to a file directly, so there is no issue in using 63 bits for the
size.
We still assume that the header is going to fit in 64 bytes, which does
mean quite a bit smaller files due to the run-length encoding, but it's
still a much larger size than you would want Git to handle.
When handling attr matching, simply compare the directory path where the
attribute file resides to the path being matched. Skip over commonality
to allow us to compare the contents of the attribute file to the remainder
of the path.
This allows us to more easily compare the pattern directly to the path,
instead of trying to guess whether we want to compare the path's basename
or the full path based on whether the match was inside a containing
directory or not.
This also allows us to do fewer translations on the pattern (trying to
re-prefix it.)
When determining whether some file matches an attr pattern, do
not try to truncate the path to pass to fnmatch. When there is
no containing directory for an item (eg, from a .gitignore in the
root) this will cause us to truncate our path, which means that
we cannot do meaningful comparisons on it and we may have false
positives when trying to determine whether a given file is actually
a file or a folder (as we have lost the path's base information.)
This mangling was to allow fnmatch to compare a directory on disk to
the name of a directory, but it is unnecessary as our fnmatch accepts
FNM_LEADING_DIR.
We use a blocking socket and set the mode to AUTO_RETRY which means that
`SSL_write` and `SSL_read` will only return once the read or write has
been completed. We therefore don't need to handle partial writes or
re-try read due to a regenotiation.
While here, consider that a zero also indicates an error condition.
When writing a configuration file, we want to take a lock on the
new file (eg, `config.lock`) before opening the configuration file
(`config`) for reading so that we can prevent somebody from changing
the contents underneath us.
When updating a configuration file, we want to copy the old data
from the file to preserve comments and funny whitespace, instead
of writing it in some "canonical" format. Thus, we keep a
pointer to the start of the line and the line length to preserve
these things we don't care to rewrite.
Previously we would try to be clever when writing the configuration
file and try to stop parsing (and simply copy the rest of the old
file) when we either found the value we were trying to write,
or when we left the section that value was in, the assumption being
that there was no more work to do.
Regrettably, you can have another section with the same name later
in the file, and we must cope with that gracefully, thus we read the
whole file in order to write a new file.
Now, writing a file looks even more than reading. Pull the config
parsing out into its own function that can be used by both reading
and writing the configuration.
When checking out with a case-insensitive working directory, we
want to change the case of items in the working directory to
reflect changes that occured in the checkout target. Diff now
has an option to break case-changing renames into delete/add.
Using FindFirstFile and FindNextFile in win32 allows us to
use the directory information that is returned, instead of
us having to get the file attributes all over again, which
is a distinct cost savings on win32.
The _next method shouldn't take a path pointer (and a path_len
pointer) as 100% of current users use the full path and ignore
the filename.
Plus let's add some docs and a unit test.
Changed win32/path_w32.c to utilize NTFS' FindFirst..FindNext data instead of doing an lstat per file. Avoiding unnecessary directory opens and file scans reduces IO, improving overall performance. Effect is magnified due to NTFS being a kernel mode file system (as opposed to user mode).
In checkout.c and filter.c we were casting a sub struct
to a parent struct which breaks the strict aliasing rules
in C. However we can use .parent or .base to access the
parent struct to avoid the build warnings.
In remote.c the local variable error was not initialized
or updated in some cases. For unintialized error a build
warning will be generated. So always keep error variable
up-to-date.
Do not automatically fail on a bad certificate, but let the caller
decide. This means we don't need our switch on errors anymore but can
return a string representation from the security framework.
If git_config_delete is to work properly in the presence of duplicate section
headers, it cannot stop searching at the end of the first matching section, as
there may be another matching section later.
When config_write is used for deletion (value = NULL), it may only terminate
when the desired key is found or there are no sections left to parse.
The idea...sometimes, a filemode is user-specified via an
explicit git_index_entry. In this case, believe the user, always.
Sometimes, it is instead built up by statting the file system. In
those cases, go with the existing logic we have to determine
whether the file system supports all filemodes and symlinks, and
make the best guess.
On file systems which have full filemode and symlink support, this
commit should make no difference. On others (most notably Windows),
this will fix problems things like:
* git_index_add and git_index_add_frombuffer() should be believed.
* As a consequence, git_checkout_tree should make the filemodes in
the index match the ones in the tree.
* And diffs with GIT_DIFF_UPDATE_INDEX don't write the wrong filemodes.
* And merges, and probably other downstream stuff now fixed, too.
This makes my previous changes to checkout.c unnecessary,
so they are now reverted.
Also, added a test for index_entry permissions from git_index_add
and git_index_add_frombuffer, both of which failed before these changes.
In `git_rebase_operation_current()`, indicate when a rebase has not
started (with `GIT_REBASE_NO_OPERATION`) rather than conflating that
with the first operation being in-progress.
It can be useful for the caller to know which update commands will be
sent to the server before the packfile is pushed up. git does this via
the pre-push hook.
We don't have hooks, but as it adds introspection into what is
happening, we can add a callback which performs the same function.
In the prior implementation, enabling GIT_DIFF_UPDATE_INDEX would overwrite
entries in the index with the ones generated from scanning the working if the
OID was the same.
Because this OID comparison ignores file modes, this means an file in the
workdir with only an exec bit difference with the one in the index would end
up being overwritten, resulting in the exec bit being loss. There might be
other related bugs but the fix of comparing OIDs and file modes should
address them all.
When walking backwards and marking parents uninteresting, make sure we
detect when the list of commits we have left has run out of
uninteresting commits so we can stop marking commits as
uninteresting. Failing to do so can mean that we walk the whole history
marking everything uninteresting, which eats up time, CPU and IO for
with useless work.
While pre-marking does look for this, we still need to check during the
main traversal as there are setups for which pre-marking does not leave
enough information in the commits. This can happen if we push a commit
and hide its parent.
The regcomp function returns a non-zero value if compilation of
a regular expression fails. In most places we only check for
negative values, but positive values indicate an error, as well.
Fix this tree-wide, fixing a segmentation fault when calling
git_config_iterator_glob_new with an invalid regexp.
When a commit is first set as unintersting and then pushed, we must take
care that we do not put it into the commit list as that makes us return
at least that commit (but maybe more) as we've inserted it into the list
because we have the assumption that we want anything in the commit list.
When no reference names could be found we did error out when trying to describe
a commit. This is wrong, though, when the option to fall back to a commit's
object ID is set.
git_checkout_tree() has some fallback behaviors for file systems
which don't have full support of filemodes. Generally works fine,
but if a given file had a change of type from a 0644 to 0755 (i.e.,
you add executable permissions), the fallback behavior incorrectly
triggers when writing hte updated index.
This would cause a git_checkout_tree() command, even with the
GIT_CHECKOUT_FORCE option set, to leave a dirty index on Windows.
Also added checks to an existing test to catch this case.
When diffs are generated, the value for the 'nfiles' field of 'git_diff_delta'
will be consistent with the value in the 'status' field. Merging diffs can
modify the 'status' field of some deltas and the 'nfiles' field needs to be
updated accordingly.
When we insert e.g. a tag or tagged object into the packfile, we must
make sure to insert any referenced objects as well, or we will have
broken links.
Use the recursive version of packfile insertion to make sure we send
over not just the tagged object but also the objects it references.
This function recursively inserts the given object and any referenced
ones. It can be thought of as a more general version of the functions to
insert a commit or tree.
When the user has a certificate check callback set, we still have to
check whether the stream we're using is even capable of providing a
certificate.
In the case of an unencrypted certificate, do not ask for it from the
stream, and do not call the callback.
This extra constructor will be useful for the annotated versions of
ref-modifying functions, as it allows us to create a commit with the
extended sha syntax which was used to retrieve it.
We do not always want to put the id directly into the reflog, but we
want to speicfy what a user typed. For this use-case we provide
annotated version of a few functions which let the caller specify what
user-friendly name was used when asking for the operation.
It turns out that erroring out on duplicate commits is the right thing
to do, but git was not hitting the bug on the server-side.
Bring back a descriptive error message in case of duplicate entries and
error out.
If a packfile includes duplicate objects, we can choose to use the
secon copy instead of the first by using the same logic as if it were
the first.
Change the error condition from 0 to -1, which indicates a bad resize,
and set the OOM message in that case.
This does mean we will leak the first copy of the object. We can deal
with that later, but making fetches work is more important.
While this is not even close to a fix, we can at least set an error
message so we know which error we are facing. Up to know we just
returned an error without a message.
Currently git_submodule_sync writes the submodule's URL to the
key 'branch.<REMOTE_NAME>.remote' while the reference
implementation of `git submodule sync` writes to
'remote.<REMOTE_NAME>.url', which is the intended behavior
according to git-submodule(1).
The current implementation does not set 'fire_callback' back to 0 for failed updates so the callback still fires.
Instead of adding yet another condition check to set 'fire_callback' to 0 if needed, considering this function should be a no-op for failed updates anyway, the best fix is to simplify its logic to check upfront if the update is a failed one.
The default behaviour for the packbuilder is to perform the work in a
single thread, which is fine for the public API, but we currently have
no way for a user to determine the number of threads to use when
creating the packfile, which makes our clone behaviour over the
filesystem quite a bit slower than what git offers.
This is a very particular scenario, in which we avoid spawning git by
being ourselves the server-side, so it's probably ok to auto-set the
threading, as the upload-pack process would do if we were talking to
git.
Currently we use the most naïve and inefficient method for figuring out
which objects to send to the remote whereby we end up trying to insert
subdirs which have not changed multiple times.
Instead, make use of the packbuilder's built-in more efficient method
which uses the walk to feed the object list and avoids inserting an
object and its descendants.
Most use-cases for the object packer communicate in terms of commits
which each side has. We already have an object to specify this
relationship between commits, namely git_revwalk.
By knowing which commits we want to pack and which the other side
already has, we can perform similar optimisations to git, by marking
each tree as interesting or uninteresting only once, and not sending
those trees which we know the other side has.
Keep the definitions in the headers, while putting the declarations in
the C files. Putting the function definitions in headers causes
them to be duplicated if you include two headers with them.
If a refspec could not be parsed, the git_refspec__parse function would
return an error value but would not provide additional error information
for the callers. This commit amends that function to set a more useful
error message.
When we rename a reference, we want the old and new ids to be the same
one (as we did not change it). The normal code path looks up the old id
from the current value of the brtanch, but by the time we look it up, it
does not exist anymore and thus we write a zero id.
Pass the old id explicitly instead.
Clear the error message on git_libgit2_shutdown for all versions of
the library (no threads and Win32 threads). Drop the giterr_clear
in clar, as that shouldn't be necessary.
This changes the get_entry() method to return a refcounted version of
the config entry, which you have to free when you're done.
This allows us to avoid freeing the memory in which the entry is stored
on a refresh, which may happen at any time for a live config.
For this reason, get_string() has been forbidden on live configs and a
new function get_string_buf() has been added, which stores the string in
a git_buf which the user then owns.
The functions which parse the string value takea advantage of the
borrowing to parse safely and then release the entry.
The user may decide to return any type of credential, including ones we
did not say we support. Add a check to make sure the user returned an
object of the right type and error out if not.
We want to use the "checkout: moving from ..." message in order to let
git know when a change of branch has happened. Make the convenience
functions for this goal write this message.
This namespace is about behaving like git's branch command, so let's do
exactly that instead of taking a reflog message.
This override is still available via the reference namespace.
The signature for the reflog is not something which changes
dynamically. Almost all uses will be NULL, since we want for the
repository's default identity to be used, making it noise.
In order to allow for changing the identity, we instead provide
git_repository_set_ident() and git_repository_ident() which allow a user
to override the choice of signature.
Win32 DLLs have four fields for the version number (major, minor,
teeny, patch). If a consumer wants to build a custom DLL, it may
be useful to set the patchlevel version number in the DLL.
This value only affects the DLL version number, it does not affect
the resultant "version number", which remains major.minor.teeny.
Windows headers #define some names that openssl uses too. Openssl
headers #undef the offending names before reusing them. But if those
offending Windows headers get included after the openssl headers the
namespace is polluted and nothing good happens.
Fixes issue #2850.
When the repository does not contain an index, emulate git's behavior
and upgrade to `SAFE_CREATE`. This allows us to check out repositories
created with `git clone --no-checkout`.
A repository can have multiple "reserved names" now, not just
a single "short name" for the repository folder itself. Refactor
to include a git_repository__reserved_names that returns all the
reserved names for a repository.
git_index_add_frombuffer enables now to store a memory buffer in the odb
and to store an entry in the index directly if the index is attached to a
repository.
Provide a convenience function that creates a buffer that can be provided
to callers but will not be freed via `git_buf_free`, so the buffer
creator maintains the allocation lifecycle of the buffer's contents.
