There is a bug in building the linked list of line records in the
diff iterator and also an off by one element error in the hunk
counts. This fixes both of these, adds some test data with more
complex sets of hunk and line diffs to exercise this code better.
This reduces the rate of syscalls for the common case of sequences of
object reads from the same pack.
Best of 5 timings for libgit2_clar before this patch:
real 0m5.375s
user 0m0.392s
sys 0m3.564s
After applying this patch:
real 0m5.285s
user 0m0.356s
sys 0m3.544s
0.6% improvement in system time.
9.2% improvement in user time.
1.7% improvement in elapsed time.
Confirmed a 0.6% reduction in number of system calls with strace.
Expect greater improvement for graph-traversal with large packs.
Fixed some minor `git_repository_hashfile` issues:
- Fixed incorrect doc (saying that repo could be NULL)
- Added checking of object type value to acceptable ones
- Added more tests for various parameter permutations
The existing `git_odb_hashfile` does not apply text filtering
rules because it doesn't have a repository context to evaluate
the correct rules to apply. This adds a new hashfile function
that will apply repository-specific filters (based on config,
attributes, and filename) before calculating the hash.
In the process of adding tests for the max file size threshold
(which treats files over a certain size as binary) there seem to
be a number of problems in the new code with detecting binaries.
This should fix those up, as well as add a test for the file
size threshold stuff.
Also, this un-deprecates `GIT_DIFF_LINE_ADD_EOFNL`, since I
finally found a legitimate situation where it would be returned.
Example: a cached node is owned only by the cache (refcount == 1).
Thread A holds the lock and determines that the entry which should get
cached equals the node (git_oid_cmp(&node->oid, &entry->oid) == 0).
It frees the given entry to instead return the cached node to the user
(entry = node). Now, before Thread A happens to increment the refcount
of the node *outside* the cache lock, Thread B tries to store another
entry and hits the slot of the node before, decrements its refcount and
frees it *before* Thread A gets a chance to increment for the user.
git_cached_obj_incref(entry);
git_mutex_lock(&cache->lock);
{
git_cached_obj *node = cache->nodes[hash & cache->size_mask];
if (node == NULL) {
cache->nodes[hash & cache->size_mask] = entry;
} else if (git_oid_cmp(&node->oid, &entry->oid) == 0) {
git_cached_obj_decref(entry, cache->free_obj);
entry = node;
} else {
git_cached_obj_decref(node, cache->free_obj);
// Thread B is here
cache->nodes[hash & cache->size_mask] = entry;
}
}
git_mutex_unlock(&cache->lock);
// Thread A is here
/* increase the refcount again, because we are
* returning it to the user */
git_cached_obj_incref(entry);
Often `git_odb_read_header` will "fail" and have to read the
entire object into memory instead of just the header. When this
happens, the object is loaded and then disposed of immediately,
which makes it difficult to efficiently use the header information
to decide if the object should be loaded (since attempting to do
so will often result in loading the object twice).
This commit takes the existing code and reorganizes it to have
two new functions:
- `git_odb__read_header_or_object` which acts just like the old
read header function except that it returns the object, too, if
it was forced to load the whole thing. It then becomes the
callers responsibility to free the `git_odb_object`.
- `git_object__from_odb_object` which was extracted from the old
`git_object_lookup` and creates a subclass of `git_object` from
an existing `git_odb_object` (separating the ODB lookup from the
`git_object` creation). This allows you to use the first header
reading function efficiently without instantiating the
`git_odb_object` twice.
There is no net change to the behavior of any of the existing
functions, but this allows internal code to tap into the ODB
lookup and object creation to be more efficient.
This commit adds a max_size value in the public `git_diff_options`
structure so that the user can automatically flag blobs over a
certain size as binary regardless of other properties.
Also, and perhaps more importantly, this moves binary detection
to be as early as possible in the diff traversal inner loop and
makes sure that we stop loading objects as soon as we decide that
they are binary.
The `git_diff_iterator_num_files` API was problematic, since we
don't actually know the exact number of files to be iterated over
until we load those files into memory. This replaces it with a
new `git_diff_iterator_progress` API that goes from 0 to 1, and
moves and renamed the old API for the internal places that can
tolerate a max value instead of an exact value.
Previously when diffing blobs, the diff code just ran with a NULL
repository object. Of course, that's not necessary and the test
for a NULL repo was confusing. This makes the blob diff run with
the repo that contains the blobs and clarifies the test that it
is possible to be diffing data where the path is unknown.
This adds support to diff and status for running filters (a la crlf)
on blobs in the workdir before computing SHAs and before generating
text diffs. This ended up being a bit more code change than I had
thought since I had to reorganize some of the diff logic to minimize
peak memory use when filtering blobs in a diff.
This also adds a cap on the maximum size of data that will be loaded
to diff. I set it at 512Mb which should match core git. Right now
it is a #define in src/diff.h but it could be moved into the public
API if desired.
This refactors the diff output code so that an iterator object
can be used to traverse and generate the diffs, instead of just
the `foreach()` style with callbacks. The code has been rearranged
so that the two styles can still share most functions.
This also replaces `GIT_REVWALKOVER` with `GIT_ITEROVER` and uses
that as a common error code for marking the end of iteration when
using a iterator style of object.
SSL_get_error() allows to receive a result code for various SSL
operations. Depending on the return value (see man (3) SSL_get_error)
there might be additional information in the OpenSSL error queue. Return
the queued message if available, otherwise set an error message
corresponding to the return code.
There is a better and less fragile way to calculate time offsets. Let
the OS take care of dealing with DST and simply take the the offset
between the local time and UTC that it gives us.
Passing SSL_VERIFY_PEER makes OpenSSL shut down the connection if the
certificate is invalid, without giving us a chance to ignore that
error. Pass SSL_VERIFY_NONE and call SSL_get_verify_result if the user
wanted us to check.
When no CNs match, we used to jump to on_error which gave a bogus
error as that's for OpenSSL errors. Jump to cert_fail so we tell the
user that the error came from checking the certificate.
This expands the types of peeling that `git_object_peel` knows
how to do to include TAG -> BLOB peeling, and makes the errors
slightly more consistent depending on the situation. It also
adds a new special behavior where peeling to ANY will peel until
the object type changes (e.g. chases TAGs to a non-TAG).
Using this expanded peeling, this replaces peeling code that was
embedded in `git_tag_peel` and `git_reset`.
It's not really needed with the current code as we have EOS and the
sideband's flush to tell us we're done.
Keep the distinction between processed and received objects.
Just clean up valgrind warnings about uninitialized memory
and also clear out errno in some cases where it results in
a false error message being generated at a later point.
This is a big redesign of the git_submodule_status API and the
implementation of the redesigned API. It also fixes a number of
bugs that I found in other parts of the submodule API while
writing the tests for the status part.
This also fixes a couple of bugs in the iterators that had not
been noticed before - one with iterating when there is a gitlink
(i.e. separate-work-dir) and one where I was treating anything
even vaguely submodule-like as a submodule, more aggressively
than core git does.
This fixes up a number of problems flagged by valgrind and also
cleans up the internal `git_submodule` allocation handling
overall with a simpler model.
This cleans up a number of items suggested during code review
with @vmg, including:
* renaming "outside repo" config API to `git_config_open_default`
* killing the `git_config_open_global` API
* removing the `git_` prefix from the static functions in fileops
* removing some unnecessary functionality from the "cp" command
This extends git_repository_init_ext further with support for
initializing the repository from an external template directory
and with support for the "create shared" type flags that make a
set GID repository directory.
This also adds tests for much of the new functionality to the
existing `repo/init.c` test suite.
Also, this adds a bunch of new utility functions including a
very general purpose `git_futils_mkdir` (with the ability to
make paths and to chmod the paths post-creation) and a file
tree copying function `git_futils_cp_r`. Also, this includes
some new path functions that were useful to keep the code
simple.
The extended version of repository init adds support for many
of the things that you can do with `git init` and sets up
structures that will make it easier to extend further in the
future.
In looking at PR #878, I found a few small bugs in the diff code,
mostly related to work that can be avoided when processing tree-
to-tree diffs that was always being carried out. This commit has
some small fixes in it.
This creates a public API for adding to the internal ignores
list, which already existing but was not accessible.
This adds the new default value for core.excludesfile also.
Up to now, the idea was that the user would do all the operations for
one repository in the same thread. Thus we could have the
memory-mapped window information thread-local and avoid any locking.
This is not practical in a few environments, such as Apple's GCD which
allocates threads arbitrarily or the .NET CLR, where the OS-level
thread can change at any moment.
Make the control structure global and protect it with a mutex so we
don't depend on the thread currently executing the code.
If you want to be absolutely safe with git_message_prettify, you
can now pass a NULL pointer for the buffer and get back the number
of bytes that would be copied into the buffer.
This means that an error is a non-negative return code and a
success will be greater than zero from this function.
Returning a negative cancels the walk, and returning a positive one
causes us to skip an entry, which was previously done by a negative
value.
This allows us to stay consistent with the rest of the functions that
take a callback and keeps the skipping functionality.
There is a little cleanup necessary from PR #843. Since the
new callbacks return `GIT_EUSER` we have to be a little careful
about return values when they are used internally to the library.
Also, callbacks should be checked for non-zero return values,
not just less than zero.
This updates all the `foreach()` type functions across the library
that take callbacks from the user to have a consistent behavior.
The rules are:
* A callback terminates the loop by returning any non-zero value
* Once the callback returns non-zero, it will not be called again
(i.e. the loop stops all iteration regardless of state)
* If the callback returns non-zero, the parent fn returns GIT_EUSER
* Although the parent returns GIT_EUSER, no error will be set in
the library and `giterr_last()` will return NULL if called.
This commit makes those changes across the library and adds tests
for most of the iteration APIs to make sure that they follow the
above rules.
Fixes#824
Exporting variables in a dynamic library is a PITA. Let's keep
these values internally and wrap them through a helper method.
This doesn't break the external API. @arrbee, aren't you glad I turned
the `GIT_ATTR_` macros into function macros? ✨
The 'git revert/cherry-pick/merge -n' commands leave .git/MERGE_MSG
behind so that git-commit can find it. As we don't yet support these
operations, users who are shelling out to let git perform these
operations haven't had a convenient way to get this message.
These functions allow the user to retrieve the message and remove it
when she's created the commit.
Instad of each transport having its own function and logic to get to
its refs, store them directly in transport.
Leverage the new gitno_buffer to make the parsing and storing of the
refs use common code and get rid of the git_protocol struct.
This allows us to add capabilitites to both at the same time, keeps
them in sync and removes a lot of code.
gitno_buffer now uses a callback to fill its buffer, allowing us to
use the same interface for git and http (which uses callbacks).
For the transition, http is going to keep its own logic until the
git/common code catches up with the implied multi_ack that http
has. This also has the side-effect of making the code cleaner and more
correct regardingt he protocol.
git.git uses an inlined hashcmp function instead of memcmp, since it
performes much better when comparing hashes (most hashes compared
diverge within the first byte).
Measurements and rationale for the curious reader:
http://thread.gmane.org/gmane.comp.version-control.git/172286
Renamed git_checkout_index to what it really was,
and removed duplicate code from clone.c. Added
git_checkout_ref, which updates HEAD and hands off
to git_checkout_head.
Added tests for the options the caller can pass to
git_checkout_*.
This makes sure that an error code returned by the callback function
of `git_tree_walk` will stop the iteration and get propagated back
to the caller verbatim.
Also, this adds a minor helper function `git_tree_entry_byoid` that
searches a `git_tree` for an entry with the given OID. This isn't
a fast function, but it's easier than writing the loop yourself as
an external user of the library.
A diff that is created with a NULL options parameter could result
in a NULL prefix string, but diff merge was unconditionally
strdup'ing it. I added a test to replicate the issue and then a
new method that does the right thing with NULL values.
Josh Triplett noticed libgit2 actually does preorder entries in
tree_walk_post instead of postorder. Also, we continued walking even
when an error occured in the callback.
Fix#773; also, allow both pre- and postorder walking.
Also removes the unnecessary check for filter
length, since git_filters_apply does the right
thing when there are none, and it's more efficient
than this.
Not all delta bases are available on the first try. By delaying
resolving all deltas until the end, we avoid decompressing some of the
data twice or even more times, saving effort and time.
The correct way to advertise out capabilities is by appending them to
the first 'want' line, using SP as separator, instead of NUL as the
server does. Inconsistent documentation lead to the use of NUL in
libgit2.
Fix this so we can request much more efficient packs from the
remote which reduces the indexing time considerably.
passing 0 to git_strol(32|64) let the implementation guess if it's
dealing with an octal number or a decimal one.
Let's make it safe and ensure that both 'HEAD@{010}' and 'HEAD@{10}'
point at the same commit.
This added a flag to the `git_repository_set_workdir()` function
that enables generation of a `.git` gitlink file that links the
new workdir to the parent repository. Essentially, the flag tells
the function to write out the changes to disk to permanently set
the workdir of the repository to the new path.
If you pass this flag as true, then setting the workdir to something
other than the default workdir (i.e. the parent of the .git repo
directory), will create a plain file named ".git" with the standard
gitlink contents "gitdir: <repo-path>", and also update the
"core.worktree" and "core.bare" config values.
Setting the workdir to the default repo workdir will clear the
core.worktree flag (but still permanently set core.bare to false).
BTW, the libgit2 API does not currently provide a function for
clearing the workdir and converting a non-bare repo into a bare one.
Adding a new config iteration function that let's you iterate
over just the config entries that match a particular regular
expression. The old foreach becomes a simple use of this with
an empty pattern.
This also fixes an apparent bug in the existing `git_config_foreach`
where returning a non-zero value from the iteration callback was
not correctly aborting the iteration and the returned value was
not being propogated back to the caller of foreach.
Added to tests to cover all these changes.
This makes it easy to take a buffer containing a path with relative
references (i.e. .. or . path segments) and resolve all of those
into a clean path. This can be applied to URLs as well as file
paths which can be useful.
As part of this, I made the drive-letter detection apply on all
platforms, not just windows. If you give a path that looks like
"c:/..." on any platform, it seems like we might as well detect
that as a rooted path. I suppose if you create a directory named
"x:" on another platform and want to use that as the beginning
of a relative path under the root directory of your repo, this
could cause a problem, but then it seems like you're asking for
trouble.
* `git_buf_rfind` (with tests and tests for `git_buf_rfind_next`)
* `git_buf_puts_escaped` and `git_buf_puts_escaped_regex` (with tests)
to copy strings into a buffer while injecting an escape sequence
(e.g. '\') in front of particular characters.
passing 0 to git_strol(32|64) let the implementation guess if it's
dealing with an octal number or a decimal one.
Let's make it safe and ensure that both 'HEAD@{010}' and 'HEAD@{10}'
point at the same commit.
On GNU, the d_name field of the dirent structure is defined as "char d_name[1]",
so we must allocate more than sizeof(struct dirent) bytes, just like on Sun.
Once a file is registered, there is no way to deregister it, even
after the structure that contains it is no longer needed and has been
freed. This may be the source of #624.
Allow and use the deregister function to remove our file from the
global list.
Currently, the first call of git_indexer_stream_add adds the data to the
underlying pack file and opens it for later use, but doesn't start
parsing the already available data.
This means, git_indexer_stream_finalize only works if
git_indexer_stream_add was called at least twice. Kill this limitation
by parsing available data immediately.
When the repository was reinitialized, every configuration change in repo_init_config() was directly performed against the file on the filesystem. However, a previous version of the configuration had previously been loaded in memory and attached to the repository, in repo_init_reinit().
The repository was unaware of the change and the stale cached version of the configuration never refreshed.
So far they only create a repo, setup the "origin"
remote, and fetch. The API probably needs work as
well; there's no way to get progress information
at this point.
Also uncovered a shortcoming; git_remote_download
doesn't fetch over local transport.
1. compile warning:
D:\libgit2.git\src\win32\posix_w32.c: In function 'p_open':
D:\libgit2.git\src\win32\posix_w32.c:235:10: warning: 'mode_t' is promoted to 'int' when passed through '...' [enabled by default]
D:\libgit2.git\src\win32\posix_w32.c:235:10: note: (so you should pass 'int' not 'mode_t' to 'va_arg')
D:\libgit2.git\src\win32\posix_w32.c:235:10: note: if this code is reached, the program will abort
2. test crash.
3. the above two issues are same root cause. please see http://www.eskimo.com/~scs/cclass/int/sx11c.html
The call to repo_init_reinit already takes care of opening the
repository and giving us a git_repository object to give to the
caller. There is no need to call git_repository_open again.
If we find several objects with the same prefix, we need to free the
memory where we stored the earlier object. Keep track of the raw.data
pointer across read_prefix calls and free it if we find another
object.
oid_for_tree_path may not always find the path in the tree, in which
case we need to return an error. The current code doesn't do this and
results in undefined behavior.
This fixes git_index_add and git_index_append to behave more like
core git, preserving old filemode data in the index when adding
and/or appending with core.filemode = false.
This also has placeholder support for core.symlinks and
core.ignorecase, but those flags are not implemented (well,
symlinks has partial support for preserving mode information in
the same way that git does, but it isn't tested).
git_commit() and git_tag() no longer prettify the
message by default. This has to be taken care of
by the caller.
This has the nice side effect of putting the
caller in position to actually choose to strip
the comments or not.
Needs AmigaOS.cmake now from CMake package at OS4Depot, or contents below:
--8<--
SET(AMIGA 1)
SET(CMAKE_SHARED_LIBRARY_C_FLAGS "-fPIC")
SET(CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS "-shared")
--8<--
When a configuration option is set, we didn't check to see whether
there was any escaping needed. Escape the available characters so we
can unescape them correctly when we read them.
On RAM: the .idx and .pack files become links to a .lock and the original download respectively.
Assume some feature (such as record locking) supported by SFS but not JXFS or RAM: is required.
When checking for a drive letter on windows, instead of using
isalpha(), it is better to just check for a..z and A..Z, I think,
particularly because the MS isalpha implementation appears to
assert when given an 0xFF byte.
There are three actual changes in this commit:
1. When the trailing newline of a file is removed in a diff, the
change will now be reported with `GIT_DIFF_LINE_DEL_EOFNL` passed
to the callback. Previously, the `ADD_EOFNL` constant was given
which was just an error in my understanding of when the various
circumstances arose. `GIT_DIFF_LINE_ADD_EOFNL` is deprecated and
should never be generated. A new newline is simply an `ADD`.
2. Rewrote the `diff_delta__merge_like_cgit` function that contains
the core logic of the `git_diff_merge` implementation. The new
version doesn't actually have significantly different behavior,
but the logic should be much more obvious, I think.
3. Fixed a bug in `git_diff_merge` where it freed a string pool
while some of the string data was still in use. This led to
`git_diff_print_patch` accessing memory that had been freed.
The rest of this commit contains improved documentation in `diff.h`
to make the behavior and the equivalencies with core git clearer,
and a bunch of new tests to cover the various cases, oh and a minor
simplification of `examples/diff.c`.
File modes were both not being ignored properly on platforms
where they should be ignored, nor be diffed consistently on
platforms where they are supported.
This change adds a number of diff and status filemode change
tests. This also makes sure that filemode-only changes are
included in the diff output when they occur and that filemode
changes are ignored successfully when core.filemode is false.
There is no code that automatically toggles core.filemode
based on the capabilities of the current platform, so the user
still needs to be careful in their .git/config file.
- Do not create new levels of fanout when creating notes from libgit2
- Insert a note in an existing matching fanout
- Remove a note from an existing fanout
- Cleanup git_note_read, git_note_remove, git_note_foreach, git_note_create methods in order use tree structures instead of tree_oids
git_status_file would always return GIT_ENOTFOUND for these files.
The underlying bug was that git__strcmp_cb, which is used by
git_path_with_stat_cmp to sort entries in the working directory,
compares strings based on unsigned chars (this is confirmed by the
strcmp(3) manpage), while git__prefixcmp, which is used by
workdir_iterator__entry_cmp to search for a path in the working
directory, compares strings based on char. So the sort puts this path at
the end of the list, while the search expects it to be at the beginning.
The fix was simply to make git__prefixcmp compare using unsigned chars,
just like strcmp(3). The rest of the change is just adding/updating
tests.
Converted an internal utility to return an oid,
rather than a tree entry (whose lifetime is tied
to the parent tree, which was freed before
returning).
The error codes from failed lookups of system and global files
on Windows were not consistent with the codes returned on other
platforms. This makes the error detection patterns match and
adds a unit test for the various errors.
This fixes two bugs:
* Issue #728 where git_status_file was not working for files
that contain spaces. This was caused by reusing the "fnmatch"
parsing code from ignore and attribute files to interpret the
"pathspec" that constrained the files to apply the status to.
In that code, unescaped whitespace was considered terminal to
the pattern, so a file with internal whitespace was excluded
from the matched files. The fix was to add a mode to that code
that allows spaces and tabs inside patterns. This mode only
comes into play when parsing in-memory strings.
* The other issue was undetected, but it was in the recently
added code to reload gitattributes / gitignores when they were
changed on disk. That code was not clearing out the old values
from the cached file content before reparsing which meant that
newly added patterns would be read in, but deleted patterns
would not be removed. The fix was to clear the vector of
patterns in a cached file before reparsing the file.
The function to convert UTF-16 to UTF-8 was only allocating a
buffer of wcslen(utf16str) bytes for the UTF-8 string, but that
is not sufficient if you have multibyte characters, and so when
those occured, the conversion was failing. This updates the
conversion functions to use the Win APIs to calculate the correct
buffer lengths.
Also fixes a comparison in the unit tests that would fail if
you did not have a particular environment variable set.
We used to consider a missing core.bare option to mean that the
repository was corrupt. This is too strict. Consider it a non-bare
repository if it's not set.
On Windows, we are having problems with home directories
that have non-ascii characters in them. This rewrites the
relevant code to fetch environment variables as UTF-16 and
then explicitly map then into UTF-8 for our internal usage.
If it's not available, an error saying so will be returned when trying
to use a https:// URL.
This also unifies a lot of the network code to use git_transport in
many places instead of an socket descriptor.
Local fetch isn't implemented yet. Don't segfault on call, but set a
dummy for negotiate_fetch and terminate gracefully.
Reported-by: Brad Harder <bch@methodlogic.net>
Creating a workdir iterator on a directory with absolutely
no files was returning an error (GIT_ENOTFOUND) instead of
an iterator for nothing. This fixes that and includes two
new tests that cover that case.
GProf shows `git_text_gather_stats` as the most expensive call
in large diffs. The function calculates a lot of information
that is not actually used and does not do so in a optimal
order. This introduces a tuned `git_buf_is_binary` function
that executes the same algorithm in a fraction of the time.
There was a bug where tracked files inside directories that were
inside ignored directories where not being found by status. To
make that a little clearer, if you have a .gitignore with:
ignore/
And then have the following files:
ignore/dir/tracked <-- actually a tracked file
ignore/dir/untracked <-- should be ignored
Then we would show the tracked file as being removed (because
when we got the to contained item "dir/" inside the ignored
directory, we decided it was safe to skip -- bzzt, wrong!).
This update is much more careful about checking that we are
not skipping over any prefix of a tracked item, regardless of
whether it is ignored or not.
As documented in diff.c, this commit does create behavior that
still differs from core git with regards to the handling of
untracked files contained inside ignored directories. With
libgit2, those files will just not show up in status or diff.
With core git, those files don't show up in status or diff
either *unless* they are explicitly ignored by a .gitignore
pattern in which case they show up as ignored files.
Needless to say, this is a local behavior difference only, so
it should not be important and (to me) the libgit2 behavior
seems more consistent.
Ported the win32 implementations of gmtime_r,
localtime_r, and gettimeofday to be part of the
posix compatibility layer, and fixed
git_signature_now to use them.
The goal of this work is to rewrite git_status_file to use the
same underlying code as git_status_foreach.
This is done in 3 phases:
1. Extend iterators to allow ranged iteration with start and
end prefixes for the range of file names to be covered.
2. Improve diff so that when there is a pathspec and there is
a common non-wildcard prefix of the pathspec, it will use
ranged iterators to minimize excess iteration.
3. Rewrite git_status_file to call git_status_foreach_ext
with a pathspec that covers just the one file being checked.
Since ranged iterators underlie the status & diff implementation,
this is actually fairly efficient. The workdir iterator does
end up loading the contents of all the directories down to the
single file, which should ideally be avoided, but it is pretty
good.
From the description of git_revwalk_reset in revwalk.h the function should
clear all pushed and hidden commits, and leave the walker in a blank state (just like at creation).
Apparently everything gets reseted appart of pushed commits (walk->one and walk->twos)
This fix should reset the walker properly.
On Solaris, struct dirent is defined differently than Linux. The field
containing the path name is of size 0, rather than NAME_MAX. So, we need to
use a properly sized buffer on Solaris to avoid a stack overflow.
Also fix some DIR* leaks on cleanup.
This fix complements cb0ce16bbe and cover the following additional use cases
- retrieving an object which has been previously searched, found and cached
- retrieving an object through an non ambiguous abbreviated id
This makes the git attributes and git ignores cache check
stat information before using the file contents from the
cache. For cached files from the index, it checks the SHA
of the file instead. This should reduce the need to ever
call `git_attr_cache_flush()` in most situations.
This commit also fixes the `git_status_should_ignore` API
to use the libgit2 standard parameter ordering.
Since Solaris does not support some of the same flags as glibc fnmatch(),
we just use the implementation we have for Windows.
Now that it's no longer a windows-specific thing, I moved it into compat/
instead of win32/
This adds a bunch of template files to the initialization for
hooks, info/exclude, and description. This makes our initialized
repo look more like core gits.
These objects aren't considered as being advertised, so asking for
them will cause the remote end to close the connection. This makes the
checking in update_tips() unnecessary, because they don't get inserted
in the list.
When a repo is first created, there is no HEAD yet and attempting
to diff files in the index was showing nothing because a tree
iterator could not be constructed. This adds an "empty" iterator
and falls back on that when the head cannot be looked up.
The fix to support attrs on bare repos went a little too far
in trying to avoid using the working directory and ended up
not processing the input path quite correctly.
This has the nice side effect of making test_attr_repo__staging_properly_normalizes_line_endings_according_to_gitattributes_directives() test pass again on Windows. This test started to fail after commit 674a198 was applied.
'git commit' and 'git tag -a' enforce some conventions, like cleaning up excess whitespace and making sure that the last line ends with a '\n'. This fix replicates this behavior.
Fixlibgit2/libgit2sharp#117
Previously, it was defined in netops.c, but it's also needed in one of the
clar tests, so I figured we might as well just make it global for the
whole project.
Without it, the mingw32 linker won't resolve GetProcessId() (called from
the core/errors.c clar test) because of some conditionals in windows.h.
gitno_connect() can return an error or socket, which is fine on most
platforms where sockets are file descriptors (signed int), but on Windows,
SOCKET is an unsigned type, which is problematic when we are trying to
test if the socket was actually a negative error code.
This fix seperates the error code and socket in gitno_connect(), and fixes
the error handling in do_connect() functions to compensate. It appears
that git_connect() and the git-transport do_connect() functions had bugs
in the non-windows cases too (leaking sockets, and not properly reporting
connection error, respectively) so I went ahead and fixed those too.
There are three changes here:
- correctly propogate error code from failed object lookups
- make zlib inflate use our allocators
- add OID to notfound error in ODB lookups
Depending on the operation, we need to consider gitattributes
in both the work dir and the index. This adds a parameter to
all of the gitattributes related functions that allows user
control of attribute reading behavior (i.e. prefer workdir,
prefer index, only use index).
This fix also covers allowing us to check attributes (and
hence do diff and status) on bare repositories.
This was a somewhat larger change that I hoped because it had
to change the cache key used for gitattributes files.
Since strnlen is not supported on all platforms and since we
now have the shiny new git_text_is_binary in the filtering
code, let's convert diff binary detection to use the new stuff.
Currently, git_remote_disconnect not only closes the connection but also
frees the underlying transport object, making it impossible to write
code like
// fetch stuff
git_remote_download()
// close connection
git_remote_disconnect()
// call user provided callback for each ref
git_remote_update_tips(remote, callback)
because remote->refs points to references owned by the transport object.
This means, we have an idling connection while running the callback for
each reference.
Instead, allow immediate disconnect and free the transport later in
git_remote_free().
The recent 64-bit Windows fixes changed the return code in
git_pkt_parse_line() so it wouldn't signal a short buffer, breaking
the network code. Bring it back.
We were not following the git behavior for leading slashes
in path names when matching git ignores and git attribute
file patterns. This should fix issue #638.
This renamed `git_khash_str` to `git_strmap`, `git_hash_oid` to
`git_oidmap`, and deletes `git_hashtable` from the tree, plus
adds unit tests for `git_strmap`.
This updates khash.h with some extra features (like error checking
on allocations, ability to use wrapped malloc, foreach calls, etc),
creates two high-level wrappers around khash: `git_khash_str` and
`git_khash_oid` for string-to-void-ptr and oid-to-void-ptr tables,
then converts all of the old usage of `git_hashtable` over to use
these new hashtables.
For `git_khash_str`, I've tried to create a set of macros that
yield an API not too unlike the old `git_hashtable` API. Since
the oid hashtable is only used in one file, I haven't bother to
set up all those macros and just use the khash APIs directly for
now.
This converts the git attr related code (including ignores) and
the git diff related code (and implicitly the status code) to use
`git_pools` for storing strings. This reduces the number of small
blocks allocated dramatically.
This adds a `git_pool` object that can do simple paged memory
allocation with free for the entire pool at once. Using this,
you can replace many small allocations with large blocks that
can then cheaply be doled out in small pieces. This is best
used when you plan to free the small blocks all at once - for
example, if they represent the parsed state from a file or data
stream that are either all kept or all discarded.
There are two real patterns of usage for `git_pools`: either
for "string" allocation, where the item size is a single byte
and you end up just packing the allocations in together, or for
"fixed size" allocation where you are allocating a large object
(e.g. a `git_oid`) and you generally just allocation single
objects that can be tightly packed. Of course, you can use it
for other things, but those two cases are the easiest.
This allows the caller to update an internal structure or update the
user output with the tips that were updated.
While in the area, only try to update the ref if the value is
different from its old one.
Trying to send every single line immediately won't give us any speed
improvement and duplicates the code we need for other transports. Make
the git transport use the same buffer functions as HTTP.
This changes the git_remote_download() API, but the existing one is
silly, so you don't get to complain.
The new API allows to know how much data has been downloaded, how many
objects we expect in total and how many we've processed.
The code used to assume that there had to be data after the newline in
a tree cache extension entry. This isn't true for a childless
invalidated entry if it's the last one, as there won't be any children
nor a hash to take up space.
Adapt the off-by-one comparison to also work in this case. Fixes#633.
git_repository_free() calls git_odb_free() if the owned odb is not null.
According to the doc, when setting a new odb through git_repository_set_odb() the caller has to take care of releasing the odb by himself.
This fixes a possible compilation issue (when GIT_WIN32 was not set) which was introduced in revision 69a4bc1988.
Signed-off-by: Sven Strickroth <email@cs-ware.de>
The code used to assume that there had to be data after the newline in
a tree cache extension entry. This isn't true for a childless
invalidated entry if it's the last one, as there won't be any children
nor a hash to take up space.
Adapt the off-by-one comparison to also work in this case. Fixes#633.
Adds a new public reference function `git_reference_lookup_oid`
that directly resolved a reference name to an OID without returning
the intermediate `git_reference` object (hence, no free needed).
Internally, this adds a `git_reference_lookup_resolved` function
that combines looking up and resolving a reference. This allows
us to be more efficient with memory reallocation.
The existing `git_reference_lookup` and `git_reference_resolve`
are reimplmented on top of the new utility and a few places in the
code are changed to use one of the two new functions.
git_repository_free() calls git_index_free() if the owned index is not null.
According to the doc, when setting a new index through git_repository_set_index() the caller has still to take care of releasing the index by itself.
In order to cope with this, this fix makes sure the index refcount is incremented when a new repository is being plugged a new index.
This adds preliminary support for pathspecs to diff and status.
The implementation is not very optimized (it still looks at
every single file and evaluated the the pathspec match against
them), but it works.
This will allow us to index a packfile as soon as we receive it from
the network as well as storing it with its final name so we don't need
to pass temporary file names around.
As parents are older than their children, we're appending to the
commit list most of the time, which makes an ordered linked list quite
inefficient.
While we're there, don't sort the results list in the main loop, as
we're sorting them afterwards and it creates extra work.
There is no need walk down the parents of a merge base to mark them as
uninteresting because we'll never see them. Calculate the merge bases
in prepare_walk() so mark_uninteresting() can stop at a merge base
instead of walking all the way to the root.
It's implemented in revwalk.c so it has access to the revision
walker's commit cache and related functions. The algorithm is the one
used by git, modified so it fits better with the library's functions.
The code was already there, so factor it out and let users push an OID
by giving it a reference name. Only refs to commits are
supported. Annotated tags will throw an error.
Add a new command `git_repository_open_ext` with extended options
that control how searching for a repository will be done. The
existing `git_repository_open` and `git_repository_discover` are
reimplemented on top of it. We may want to change the default
behavior of `git_repository_open` but this commit does not do that.
Improve support for "gitdir" files where the work dir is separate
from the repo and support for the "separate-git-dir" config. Also,
add support for opening repos created with `git-new-workdir` script
(although I have only confirmed that they can be opened, not that
all functions work correctly).
There are also a few minor changes that came up:
- Fix `git_path_prettify` to allow in-place prettifying.
- Fix `git_path_root` to support backslashes on Win32. This fix
should help many repo open/discover scenarios - it is the one
function called when opening before prettifying the path.
- Tweak `git_config_get_string` to set the "out" pointer to NULL
if the config value is not found. Allows some other cleanup.
- Fix a couple places that should have been calling
`git_repository_config__weakptr` and were not.
- Fix `cl_git_sandbox_init` clar helper to support bare repos.
Looking through the open windows to check whether we can re-use an
open window should take into account whether both `offset` and `offset
+ extra` are contained within the same window. Failure to do so can
lead to invalid memory accesses. This closes#614.
While we're in the area remove an outdated assert.
There was a bug in git_buf_join_n when the contents of the
original buffer were joined into itself and the realloc
moved the pointer to the original buffer.
This adds support for a bunch of core.* settings that affect
diff and status, plus fixes up some incorrect implementations
of those settings from before. Also, this cleans up the
handling of config settings in the new submodules code and
in the old attrs/ignore code.
When processing status for a newly checked out repo, it is
possible that there will be submodules that have not yet been
initialized. The only way to distinguish these from untracked
directories is to have some knowledge of submodules. This
commit adds a new submodule API which, given a name or path,
can determine if it appears to be a submodule and can give
information about the submodule.
I decided that the COITERATE macro was, in the end causing
more confusion that it would save and decided just to write
out the loops that I needed for parallel diff list iteration.
It is not that much code and this just feels less obfuscated.
There was an error in the tree iterator where it would
delete two tree levels instead of just one when popping
up a tree level. Unfortunately the test data for the
tree iterator did not have any deep trees with subtrees
in the middle of the tree items, so this problem went
unnoticed. This contains the 1-line fix plus new test
data and tests that reveal the issue.
This gives `git_status_foreach()` back its old behavior of
emulating the "--untracked=all" behavior of git. You can
get any of the various --untracked options by passing flags
to `git_status_foreach_ext()` but the basic version will
keep the behavior it has always had.
This fixes the bug that @nulltoken found (thank you!) where
if there were untracked directories alphabetically after the
last tracked item, the diff implementation would deref a NULL
pointer.
The fix involved the code which decides if it is necessary
to recurse into a directory in the working dir, so it was
easy to add a new option `GIT_STATUS_OPT_RECURSE_UNTRACKED_DIRS`
to control if the contents of untracked directories should be
included in status.
This adds support for roughly-right tracking of submodules
(although it does not recurse into submodules to detect
internal modifications a la core git), and it adds support
for including unmodified files in diff iteration if requested.
This includes a few cleanups that came up while converting
these files.
This commit introduces a could new git error classes, including
the catchall class: GITERR_INVALID which I'm using as the class
for invalid and out of range values which are detected at too low
a level of library to use a higher level classification. For
example, an overflow error in parsing an integer or a bad letter
in parsing an OID string would generate an error in this class.
This converts blob.c, fileops.c, and all of the win32 files.
Also, various minor cleanups throughout the code. Plus, in
testing the win32 build, I cleaned up a bunch (although not
all) of the warnings with the 64-bit build.
This continues to add other files to the new error handling
style. I think the only real concerns here are that there are
a couple of error return cases that I have converted to asserts,
but I think that it was the correct thing to do given the new
error style.
This converts the map validation function into a macro, tweaks
the GITERR_OS system error automatic appending, and adds a
tentative new error access API and some quick unit tests for
both the old and new error APIs.
This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to
the new style of error handling. Also got the unix and win32
versions of map.c. There are some minor changes to other
files but no others were completely converted.
This also contains an update to filebuf so that a zeroed out
filebuf will not think that the fd (== 0) is actually open
(and inadvertently call close() on fd 0 if cleaned up).
Lastly, this was built and tested on win32 and contains a
bunch of fixes for the win32 build which was pretty broken.
write_section() mistakenly treated is input as the whole variable name
instead of simply the section (and possibly subsection) and would
confuse "section.subsection" as a section plus variable name and
produce a wrong section header.
Fix this and include a test for writing "section.subsection.var" and
reading it from the file.
Includes:
- Proper error reporting when encountering syntax errors in a
config file (file, line number, column).
- Rewritten `config_write`, now with 99% less goto-spaghetti
- Error state in `git_filebuf`: filebuf write functions no longer
need to be checked for error returns. If any of the writes performed
on a buffer fail, the last call to `git_filebuf_commit` or
`git_filebuf_hash` will fail accordingly and set the appropiate error
message. Baller!
This also includes droping `git_buf_lasterror` because it makes no sense
in the new system. Note that in most of the places were it has been
dropped, the code needs cleanup. I.e. GIT_ENOMEM is going away, so
instead it should return a generic `-1` and obviously not throw
anything.
Since strnlen is not supported on all platforms and since we
now have the shiny new git_text_is_binary in the filtering
code, let's convert diff binary detection to use the new stuff.
This reverts the changes to the GIT_STATUS constants and adds a
new enumeration to describe the type of change in a git_diff_delta.
I don't love this solution, but it should prevent strange errors
from occurring for now. Eventually, I would like to unify the
various status constants, but it needs a larger plan and I just
wanted to eliminate this breakage quickly.
It turns out that commit 31e9cfc4cbcaf1b38cdd3dbe3282a8f57e5366a5
did not fix the GIT_USUSED behavior on all platforms. This commit
walks through and really cleans things up more thoroughly, getting
rid of the unnecessary stuff.
To remove the use of some GIT_UNUSED, I ended up adding a couple
of new iterators for hashtables that allow you to iterator just
over keys or just over values.
In making this change, I found a bug in the clar tests (where we
were doing *count++ but meant to do (*count)++ to increment the
value). I fixed that but then found the test failing because it
was not really using an empty repo. So, I took some of the code
that I wrote for iterator testing and moved it to clar_helpers.c,
then made use of that to make it easier to open fixtures on a
per test basis even within a single test file.
This is a major reorganization of the diff code. This changes
the diff functions to use the iterators for traversing the
content. This allowed a lot of code to be simplified. Also,
this moved the functions relating to outputting a diff into a
new file (diff_output.c).
This includes a number of other changes - adding utility
functions, extending iterators, etc. plus more tests for the
diff code. This also takes the example diff.c program much
further in terms of emulating git-diff command line options.
This is an initial version of git_diff_workdir_to_index. It
also includes renaming some structures and some refactoring
of the existing code so that it could be shared better with
the new function.
This is not complete since it needs a rebase to get some
new odb functions from the upstream branch.
Once I added tests for the whitespace handling options of
diff, I realized that there were some bugs. This fixes
those and adds the new tests into the test suite.
* Implemented git_diff_index_to_tree
* Reworked git_diff_options structure to handle more options
* Made most of the options in git_diff_options actually work
* Reorganized code a bit to remove some redundancy
* Added option parsing to examples/diff.c to test most options
File mode flags are not all defined on WIN32, but since git
is so rigid in how it uses file modes, there is no reason not
to hard code a particular value. Also, this is only used in
the git_diff_print_compact helper function, so it is really
really not important.
This fixes several bugs, updates tests and docs, eliminates the
FILE* assumption in favor of printing callbacks for the diff patch
formatter helpers, and adds a "diff" example function that can
perform a diff from the command line.
This reworks the diff API to separate the steps of producing
a diff descriptions from formatting the diff. This will allow
us to share diff output code with the various diff creation
scenarios and will allow us to implement rename detection as
an optional pass that can be run on a diff list.
This gets the basic plumbing in place for git_diff_blob.
There is a known issue where additional parameters like
the number of lines of context to display on the diff
are not working correctly (which leads one of the new
unit tests to fail).
Since casting to void works to eliminate errors with unused
parameters on all platforms, avoid the various special cases.
Over time, it will make sense to eliminate the GIT_UNUSED
macro completely and just have GIT_UNUSED_ARG.
