This adds support to diff and status for running filters (a la crlf)
on blobs in the workdir before computing SHAs and before generating
text diffs. This ended up being a bit more code change than I had
thought since I had to reorganize some of the diff logic to minimize
peak memory use when filtering blobs in a diff.
This also adds a cap on the maximum size of data that will be loaded
to diff. I set it at 512Mb which should match core git. Right now
it is a #define in src/diff.h but it could be moved into the public
API if desired.
This refactors the diff output code so that an iterator object
can be used to traverse and generate the diffs, instead of just
the `foreach()` style with callbacks. The code has been rearranged
so that the two styles can still share most functions.
This also replaces `GIT_REVWALKOVER` with `GIT_ITEROVER` and uses
that as a common error code for marking the end of iteration when
using a iterator style of object.
In looking at PR #878, I found a few small bugs in the diff code,
mostly related to work that can be avoided when processing tree-
to-tree diffs that was always being carried out. This commit has
some small fixes in it.
A diff that is created with a NULL options parameter could result
in a NULL prefix string, but diff merge was unconditionally
strdup'ing it. I added a test to replicate the issue and then a
new method that does the right thing with NULL values.
This fixes git_index_add and git_index_append to behave more like
core git, preserving old filemode data in the index when adding
and/or appending with core.filemode = false.
This also has placeholder support for core.symlinks and
core.ignorecase, but those flags are not implemented (well,
symlinks has partial support for preserving mode information in
the same way that git does, but it isn't tested).
There are three actual changes in this commit:
1. When the trailing newline of a file is removed in a diff, the
change will now be reported with `GIT_DIFF_LINE_DEL_EOFNL` passed
to the callback. Previously, the `ADD_EOFNL` constant was given
which was just an error in my understanding of when the various
circumstances arose. `GIT_DIFF_LINE_ADD_EOFNL` is deprecated and
should never be generated. A new newline is simply an `ADD`.
2. Rewrote the `diff_delta__merge_like_cgit` function that contains
the core logic of the `git_diff_merge` implementation. The new
version doesn't actually have significantly different behavior,
but the logic should be much more obvious, I think.
3. Fixed a bug in `git_diff_merge` where it freed a string pool
while some of the string data was still in use. This led to
`git_diff_print_patch` accessing memory that had been freed.
The rest of this commit contains improved documentation in `diff.h`
to make the behavior and the equivalencies with core git clearer,
and a bunch of new tests to cover the various cases, oh and a minor
simplification of `examples/diff.c`.
File modes were both not being ignored properly on platforms
where they should be ignored, nor be diffed consistently on
platforms where they are supported.
This change adds a number of diff and status filemode change
tests. This also makes sure that filemode-only changes are
included in the diff output when they occur and that filemode
changes are ignored successfully when core.filemode is false.
There is no code that automatically toggles core.filemode
based on the capabilities of the current platform, so the user
still needs to be careful in their .git/config file.
This fixes two bugs:
* Issue #728 where git_status_file was not working for files
that contain spaces. This was caused by reusing the "fnmatch"
parsing code from ignore and attribute files to interpret the
"pathspec" that constrained the files to apply the status to.
In that code, unescaped whitespace was considered terminal to
the pattern, so a file with internal whitespace was excluded
from the matched files. The fix was to add a mode to that code
that allows spaces and tabs inside patterns. This mode only
comes into play when parsing in-memory strings.
* The other issue was undetected, but it was in the recently
added code to reload gitattributes / gitignores when they were
changed on disk. That code was not clearing out the old values
from the cached file content before reparsing which meant that
newly added patterns would be read in, but deleted patterns
would not be removed. The fix was to clear the vector of
patterns in a cached file before reparsing the file.
There was a bug where tracked files inside directories that were
inside ignored directories where not being found by status. To
make that a little clearer, if you have a .gitignore with:
ignore/
And then have the following files:
ignore/dir/tracked <-- actually a tracked file
ignore/dir/untracked <-- should be ignored
Then we would show the tracked file as being removed (because
when we got the to contained item "dir/" inside the ignored
directory, we decided it was safe to skip -- bzzt, wrong!).
This update is much more careful about checking that we are
not skipping over any prefix of a tracked item, regardless of
whether it is ignored or not.
As documented in diff.c, this commit does create behavior that
still differs from core git with regards to the handling of
untracked files contained inside ignored directories. With
libgit2, those files will just not show up in status or diff.
With core git, those files don't show up in status or diff
either *unless* they are explicitly ignored by a .gitignore
pattern in which case they show up as ignored files.
Needless to say, this is a local behavior difference only, so
it should not be important and (to me) the libgit2 behavior
seems more consistent.
The goal of this work is to rewrite git_status_file to use the
same underlying code as git_status_foreach.
This is done in 3 phases:
1. Extend iterators to allow ranged iteration with start and
end prefixes for the range of file names to be covered.
2. Improve diff so that when there is a pathspec and there is
a common non-wildcard prefix of the pathspec, it will use
ranged iterators to minimize excess iteration.
3. Rewrite git_status_file to call git_status_foreach_ext
with a pathspec that covers just the one file being checked.
Since ranged iterators underlie the status & diff implementation,
this is actually fairly efficient. The workdir iterator does
end up loading the contents of all the directories down to the
single file, which should ideally be avoided, but it is pretty
good.
When a repo is first created, there is no HEAD yet and attempting
to diff files in the index was showing nothing because a tree
iterator could not be constructed. This adds an "empty" iterator
and falls back on that when the head cannot be looked up.
This converts the git attr related code (including ignores) and
the git diff related code (and implicitly the status code) to use
`git_pools` for storing strings. This reduces the number of small
blocks allocated dramatically.
This adds preliminary support for pathspecs to diff and status.
The implementation is not very optimized (it still looks at
every single file and evaluated the the pathspec match against
them), but it works.
Add a new command `git_repository_open_ext` with extended options
that control how searching for a repository will be done. The
existing `git_repository_open` and `git_repository_discover` are
reimplemented on top of it. We may want to change the default
behavior of `git_repository_open` but this commit does not do that.
Improve support for "gitdir" files where the work dir is separate
from the repo and support for the "separate-git-dir" config. Also,
add support for opening repos created with `git-new-workdir` script
(although I have only confirmed that they can be opened, not that
all functions work correctly).
There are also a few minor changes that came up:
- Fix `git_path_prettify` to allow in-place prettifying.
- Fix `git_path_root` to support backslashes on Win32. This fix
should help many repo open/discover scenarios - it is the one
function called when opening before prettifying the path.
- Tweak `git_config_get_string` to set the "out" pointer to NULL
if the config value is not found. Allows some other cleanup.
- Fix a couple places that should have been calling
`git_repository_config__weakptr` and were not.
- Fix `cl_git_sandbox_init` clar helper to support bare repos.
This adds support for a bunch of core.* settings that affect
diff and status, plus fixes up some incorrect implementations
of those settings from before. Also, this cleans up the
handling of config settings in the new submodules code and
in the old attrs/ignore code.
I decided that the COITERATE macro was, in the end causing
more confusion that it would save and decided just to write
out the loops that I needed for parallel diff list iteration.
It is not that much code and this just feels less obfuscated.
This fixes the bug that @nulltoken found (thank you!) where
if there were untracked directories alphabetically after the
last tracked item, the diff implementation would deref a NULL
pointer.
The fix involved the code which decides if it is necessary
to recurse into a directory in the working dir, so it was
easy to add a new option `GIT_STATUS_OPT_RECURSE_UNTRACKED_DIRS`
to control if the contents of untracked directories should be
included in status.
This adds support for roughly-right tracking of submodules
(although it does not recurse into submodules to detect
internal modifications a la core git), and it adds support
for including unmodified files in diff iteration if requested.
This reverts the changes to the GIT_STATUS constants and adds a
new enumeration to describe the type of change in a git_diff_delta.
I don't love this solution, but it should prevent strange errors
from occurring for now. Eventually, I would like to unify the
various status constants, but it needs a larger plan and I just
wanted to eliminate this breakage quickly.
It turns out that commit 31e9cfc4cbcaf1b38cdd3dbe3282a8f57e5366a5
did not fix the GIT_USUSED behavior on all platforms. This commit
walks through and really cleans things up more thoroughly, getting
rid of the unnecessary stuff.
To remove the use of some GIT_UNUSED, I ended up adding a couple
of new iterators for hashtables that allow you to iterator just
over keys or just over values.
In making this change, I found a bug in the clar tests (where we
were doing *count++ but meant to do (*count)++ to increment the
value). I fixed that but then found the test failing because it
was not really using an empty repo. So, I took some of the code
that I wrote for iterator testing and moved it to clar_helpers.c,
then made use of that to make it easier to open fixtures on a
per test basis even within a single test file.
This is a major reorganization of the diff code. This changes
the diff functions to use the iterators for traversing the
content. This allowed a lot of code to be simplified. Also,
this moved the functions relating to outputting a diff into a
new file (diff_output.c).
This includes a number of other changes - adding utility
functions, extending iterators, etc. plus more tests for the
diff code. This also takes the example diff.c program much
further in terms of emulating git-diff command line options.
This is an initial version of git_diff_workdir_to_index. It
also includes renaming some structures and some refactoring
of the existing code so that it could be shared better with
the new function.
This is not complete since it needs a rebase to get some
new odb functions from the upstream branch.
Once I added tests for the whitespace handling options of
diff, I realized that there were some bugs. This fixes
those and adds the new tests into the test suite.
* Implemented git_diff_index_to_tree
* Reworked git_diff_options structure to handle more options
* Made most of the options in git_diff_options actually work
* Reorganized code a bit to remove some redundancy
* Added option parsing to examples/diff.c to test most options
File mode flags are not all defined on WIN32, but since git
is so rigid in how it uses file modes, there is no reason not
to hard code a particular value. Also, this is only used in
the git_diff_print_compact helper function, so it is really
really not important.
This fixes several bugs, updates tests and docs, eliminates the
FILE* assumption in favor of printing callbacks for the diff patch
formatter helpers, and adds a "diff" example function that can
perform a diff from the command line.
This reworks the diff API to separate the steps of producing
a diff descriptions from formatting the diff. This will allow
us to share diff output code with the various diff creation
scenarios and will allow us to implement rename detection as
an optional pass that can be run on a diff list.
This gets the basic plumbing in place for git_diff_blob.
There is a known issue where additional parameters like
the number of lines of context to display on the diff
are not working correctly (which leads one of the new
unit tests to fail).
