Not all delta bases are available on the first try. By delaying
resolving all deltas until the end, we avoid decompressing some of the
data twice or even more times, saving effort and time.
The correct way to advertise out capabilities is by appending them to
the first 'want' line, using SP as separator, instead of NUL as the
server does. Inconsistent documentation lead to the use of NUL in
libgit2.
Fix this so we can request much more efficient packs from the
remote which reduces the indexing time considerably.
passing 0 to git_strol(32|64) let the implementation guess if it's
dealing with an octal number or a decimal one.
Let's make it safe and ensure that both 'HEAD@{010}' and 'HEAD@{10}'
point at the same commit.
This added a flag to the `git_repository_set_workdir()` function
that enables generation of a `.git` gitlink file that links the
new workdir to the parent repository. Essentially, the flag tells
the function to write out the changes to disk to permanently set
the workdir of the repository to the new path.
If you pass this flag as true, then setting the workdir to something
other than the default workdir (i.e. the parent of the .git repo
directory), will create a plain file named ".git" with the standard
gitlink contents "gitdir: <repo-path>", and also update the
"core.worktree" and "core.bare" config values.
Setting the workdir to the default repo workdir will clear the
core.worktree flag (but still permanently set core.bare to false).
BTW, the libgit2 API does not currently provide a function for
clearing the workdir and converting a non-bare repo into a bare one.
Adding a new config iteration function that let's you iterate
over just the config entries that match a particular regular
expression. The old foreach becomes a simple use of this with
an empty pattern.
This also fixes an apparent bug in the existing `git_config_foreach`
where returning a non-zero value from the iteration callback was
not correctly aborting the iteration and the returned value was
not being propogated back to the caller of foreach.
Added to tests to cover all these changes.
This makes it easy to take a buffer containing a path with relative
references (i.e. .. or . path segments) and resolve all of those
into a clean path. This can be applied to URLs as well as file
paths which can be useful.
As part of this, I made the drive-letter detection apply on all
platforms, not just windows. If you give a path that looks like
"c:/..." on any platform, it seems like we might as well detect
that as a rooted path. I suppose if you create a directory named
"x:" on another platform and want to use that as the beginning
of a relative path under the root directory of your repo, this
could cause a problem, but then it seems like you're asking for
trouble.
* `git_buf_rfind` (with tests and tests for `git_buf_rfind_next`)
* `git_buf_puts_escaped` and `git_buf_puts_escaped_regex` (with tests)
to copy strings into a buffer while injecting an escape sequence
(e.g. '\') in front of particular characters.
passing 0 to git_strol(32|64) let the implementation guess if it's
dealing with an octal number or a decimal one.
Let's make it safe and ensure that both 'HEAD@{010}' and 'HEAD@{10}'
point at the same commit.
On GNU, the d_name field of the dirent structure is defined as "char d_name[1]",
so we must allocate more than sizeof(struct dirent) bytes, just like on Sun.
Once a file is registered, there is no way to deregister it, even
after the structure that contains it is no longer needed and has been
freed. This may be the source of #624.
Allow and use the deregister function to remove our file from the
global list.
Currently, the first call of git_indexer_stream_add adds the data to the
underlying pack file and opens it for later use, but doesn't start
parsing the already available data.
This means, git_indexer_stream_finalize only works if
git_indexer_stream_add was called at least twice. Kill this limitation
by parsing available data immediately.
When the repository was reinitialized, every configuration change in repo_init_config() was directly performed against the file on the filesystem. However, a previous version of the configuration had previously been loaded in memory and attached to the repository, in repo_init_reinit().
The repository was unaware of the change and the stale cached version of the configuration never refreshed.
So far they only create a repo, setup the "origin"
remote, and fetch. The API probably needs work as
well; there's no way to get progress information
at this point.
Also uncovered a shortcoming; git_remote_download
doesn't fetch over local transport.
1. compile warning:
D:\libgit2.git\src\win32\posix_w32.c: In function 'p_open':
D:\libgit2.git\src\win32\posix_w32.c:235:10: warning: 'mode_t' is promoted to 'int' when passed through '...' [enabled by default]
D:\libgit2.git\src\win32\posix_w32.c:235:10: note: (so you should pass 'int' not 'mode_t' to 'va_arg')
D:\libgit2.git\src\win32\posix_w32.c:235:10: note: if this code is reached, the program will abort
2. test crash.
3. the above two issues are same root cause. please see http://www.eskimo.com/~scs/cclass/int/sx11c.html
The call to repo_init_reinit already takes care of opening the
repository and giving us a git_repository object to give to the
caller. There is no need to call git_repository_open again.
If we find several objects with the same prefix, we need to free the
memory where we stored the earlier object. Keep track of the raw.data
pointer across read_prefix calls and free it if we find another
object.
oid_for_tree_path may not always find the path in the tree, in which
case we need to return an error. The current code doesn't do this and
results in undefined behavior.
This fixes git_index_add and git_index_append to behave more like
core git, preserving old filemode data in the index when adding
and/or appending with core.filemode = false.
This also has placeholder support for core.symlinks and
core.ignorecase, but those flags are not implemented (well,
symlinks has partial support for preserving mode information in
the same way that git does, but it isn't tested).
git_commit() and git_tag() no longer prettify the
message by default. This has to be taken care of
by the caller.
This has the nice side effect of putting the
caller in position to actually choose to strip
the comments or not.
Needs AmigaOS.cmake now from CMake package at OS4Depot, or contents below:
--8<--
SET(AMIGA 1)
SET(CMAKE_SHARED_LIBRARY_C_FLAGS "-fPIC")
SET(CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS "-shared")
--8<--
When a configuration option is set, we didn't check to see whether
there was any escaping needed. Escape the available characters so we
can unescape them correctly when we read them.
On RAM: the .idx and .pack files become links to a .lock and the original download respectively.
Assume some feature (such as record locking) supported by SFS but not JXFS or RAM: is required.
When checking for a drive letter on windows, instead of using
isalpha(), it is better to just check for a..z and A..Z, I think,
particularly because the MS isalpha implementation appears to
assert when given an 0xFF byte.
There are three actual changes in this commit:
1. When the trailing newline of a file is removed in a diff, the
change will now be reported with `GIT_DIFF_LINE_DEL_EOFNL` passed
to the callback. Previously, the `ADD_EOFNL` constant was given
which was just an error in my understanding of when the various
circumstances arose. `GIT_DIFF_LINE_ADD_EOFNL` is deprecated and
should never be generated. A new newline is simply an `ADD`.
2. Rewrote the `diff_delta__merge_like_cgit` function that contains
the core logic of the `git_diff_merge` implementation. The new
version doesn't actually have significantly different behavior,
but the logic should be much more obvious, I think.
3. Fixed a bug in `git_diff_merge` where it freed a string pool
while some of the string data was still in use. This led to
`git_diff_print_patch` accessing memory that had been freed.
The rest of this commit contains improved documentation in `diff.h`
to make the behavior and the equivalencies with core git clearer,
and a bunch of new tests to cover the various cases, oh and a minor
simplification of `examples/diff.c`.
File modes were both not being ignored properly on platforms
where they should be ignored, nor be diffed consistently on
platforms where they are supported.
This change adds a number of diff and status filemode change
tests. This also makes sure that filemode-only changes are
included in the diff output when they occur and that filemode
changes are ignored successfully when core.filemode is false.
There is no code that automatically toggles core.filemode
based on the capabilities of the current platform, so the user
still needs to be careful in their .git/config file.
- Do not create new levels of fanout when creating notes from libgit2
- Insert a note in an existing matching fanout
- Remove a note from an existing fanout
- Cleanup git_note_read, git_note_remove, git_note_foreach, git_note_create methods in order use tree structures instead of tree_oids
git_status_file would always return GIT_ENOTFOUND for these files.
The underlying bug was that git__strcmp_cb, which is used by
git_path_with_stat_cmp to sort entries in the working directory,
compares strings based on unsigned chars (this is confirmed by the
strcmp(3) manpage), while git__prefixcmp, which is used by
workdir_iterator__entry_cmp to search for a path in the working
directory, compares strings based on char. So the sort puts this path at
the end of the list, while the search expects it to be at the beginning.
The fix was simply to make git__prefixcmp compare using unsigned chars,
just like strcmp(3). The rest of the change is just adding/updating
tests.
Converted an internal utility to return an oid,
rather than a tree entry (whose lifetime is tied
to the parent tree, which was freed before
returning).
The error codes from failed lookups of system and global files
on Windows were not consistent with the codes returned on other
platforms. This makes the error detection patterns match and
adds a unit test for the various errors.
This fixes two bugs:
* Issue #728 where git_status_file was not working for files
that contain spaces. This was caused by reusing the "fnmatch"
parsing code from ignore and attribute files to interpret the
"pathspec" that constrained the files to apply the status to.
In that code, unescaped whitespace was considered terminal to
the pattern, so a file with internal whitespace was excluded
from the matched files. The fix was to add a mode to that code
that allows spaces and tabs inside patterns. This mode only
comes into play when parsing in-memory strings.
* The other issue was undetected, but it was in the recently
added code to reload gitattributes / gitignores when they were
changed on disk. That code was not clearing out the old values
from the cached file content before reparsing which meant that
newly added patterns would be read in, but deleted patterns
would not be removed. The fix was to clear the vector of
patterns in a cached file before reparsing the file.
The function to convert UTF-16 to UTF-8 was only allocating a
buffer of wcslen(utf16str) bytes for the UTF-8 string, but that
is not sufficient if you have multibyte characters, and so when
those occured, the conversion was failing. This updates the
conversion functions to use the Win APIs to calculate the correct
buffer lengths.
Also fixes a comparison in the unit tests that would fail if
you did not have a particular environment variable set.
We used to consider a missing core.bare option to mean that the
repository was corrupt. This is too strict. Consider it a non-bare
repository if it's not set.
On Windows, we are having problems with home directories
that have non-ascii characters in them. This rewrites the
relevant code to fetch environment variables as UTF-16 and
then explicitly map then into UTF-8 for our internal usage.
If it's not available, an error saying so will be returned when trying
to use a https:// URL.
This also unifies a lot of the network code to use git_transport in
many places instead of an socket descriptor.
Local fetch isn't implemented yet. Don't segfault on call, but set a
dummy for negotiate_fetch and terminate gracefully.
Reported-by: Brad Harder <bch@methodlogic.net>
Creating a workdir iterator on a directory with absolutely
no files was returning an error (GIT_ENOTFOUND) instead of
an iterator for nothing. This fixes that and includes two
new tests that cover that case.
GProf shows `git_text_gather_stats` as the most expensive call
in large diffs. The function calculates a lot of information
that is not actually used and does not do so in a optimal
order. This introduces a tuned `git_buf_is_binary` function
that executes the same algorithm in a fraction of the time.
There was a bug where tracked files inside directories that were
inside ignored directories where not being found by status. To
make that a little clearer, if you have a .gitignore with:
ignore/
And then have the following files:
ignore/dir/tracked <-- actually a tracked file
ignore/dir/untracked <-- should be ignored
Then we would show the tracked file as being removed (because
when we got the to contained item "dir/" inside the ignored
directory, we decided it was safe to skip -- bzzt, wrong!).
This update is much more careful about checking that we are
not skipping over any prefix of a tracked item, regardless of
whether it is ignored or not.
As documented in diff.c, this commit does create behavior that
still differs from core git with regards to the handling of
untracked files contained inside ignored directories. With
libgit2, those files will just not show up in status or diff.
With core git, those files don't show up in status or diff
either *unless* they are explicitly ignored by a .gitignore
pattern in which case they show up as ignored files.
Needless to say, this is a local behavior difference only, so
it should not be important and (to me) the libgit2 behavior
seems more consistent.
Ported the win32 implementations of gmtime_r,
localtime_r, and gettimeofday to be part of the
posix compatibility layer, and fixed
git_signature_now to use them.
The goal of this work is to rewrite git_status_file to use the
same underlying code as git_status_foreach.
This is done in 3 phases:
1. Extend iterators to allow ranged iteration with start and
end prefixes for the range of file names to be covered.
2. Improve diff so that when there is a pathspec and there is
a common non-wildcard prefix of the pathspec, it will use
ranged iterators to minimize excess iteration.
3. Rewrite git_status_file to call git_status_foreach_ext
with a pathspec that covers just the one file being checked.
Since ranged iterators underlie the status & diff implementation,
this is actually fairly efficient. The workdir iterator does
end up loading the contents of all the directories down to the
single file, which should ideally be avoided, but it is pretty
good.
From the description of git_revwalk_reset in revwalk.h the function should
clear all pushed and hidden commits, and leave the walker in a blank state (just like at creation).
Apparently everything gets reseted appart of pushed commits (walk->one and walk->twos)
This fix should reset the walker properly.
On Solaris, struct dirent is defined differently than Linux. The field
containing the path name is of size 0, rather than NAME_MAX. So, we need to
use a properly sized buffer on Solaris to avoid a stack overflow.
Also fix some DIR* leaks on cleanup.
This fix complements cb0ce16bbe and cover the following additional use cases
- retrieving an object which has been previously searched, found and cached
- retrieving an object through an non ambiguous abbreviated id
This makes the git attributes and git ignores cache check
stat information before using the file contents from the
cache. For cached files from the index, it checks the SHA
of the file instead. This should reduce the need to ever
call `git_attr_cache_flush()` in most situations.
This commit also fixes the `git_status_should_ignore` API
to use the libgit2 standard parameter ordering.
Since Solaris does not support some of the same flags as glibc fnmatch(),
we just use the implementation we have for Windows.
Now that it's no longer a windows-specific thing, I moved it into compat/
instead of win32/
This adds a bunch of template files to the initialization for
hooks, info/exclude, and description. This makes our initialized
repo look more like core gits.
These objects aren't considered as being advertised, so asking for
them will cause the remote end to close the connection. This makes the
checking in update_tips() unnecessary, because they don't get inserted
in the list.
When a repo is first created, there is no HEAD yet and attempting
to diff files in the index was showing nothing because a tree
iterator could not be constructed. This adds an "empty" iterator
and falls back on that when the head cannot be looked up.
The fix to support attrs on bare repos went a little too far
in trying to avoid using the working directory and ended up
not processing the input path quite correctly.
This has the nice side effect of making test_attr_repo__staging_properly_normalizes_line_endings_according_to_gitattributes_directives() test pass again on Windows. This test started to fail after commit 674a198 was applied.
'git commit' and 'git tag -a' enforce some conventions, like cleaning up excess whitespace and making sure that the last line ends with a '\n'. This fix replicates this behavior.
Fixlibgit2/libgit2sharp#117
Previously, it was defined in netops.c, but it's also needed in one of the
clar tests, so I figured we might as well just make it global for the
whole project.
Without it, the mingw32 linker won't resolve GetProcessId() (called from
the core/errors.c clar test) because of some conditionals in windows.h.
gitno_connect() can return an error or socket, which is fine on most
platforms where sockets are file descriptors (signed int), but on Windows,
SOCKET is an unsigned type, which is problematic when we are trying to
test if the socket was actually a negative error code.
This fix seperates the error code and socket in gitno_connect(), and fixes
the error handling in do_connect() functions to compensate. It appears
that git_connect() and the git-transport do_connect() functions had bugs
in the non-windows cases too (leaking sockets, and not properly reporting
connection error, respectively) so I went ahead and fixed those too.
There are three changes here:
- correctly propogate error code from failed object lookups
- make zlib inflate use our allocators
- add OID to notfound error in ODB lookups
Depending on the operation, we need to consider gitattributes
in both the work dir and the index. This adds a parameter to
all of the gitattributes related functions that allows user
control of attribute reading behavior (i.e. prefer workdir,
prefer index, only use index).
This fix also covers allowing us to check attributes (and
hence do diff and status) on bare repositories.
This was a somewhat larger change that I hoped because it had
to change the cache key used for gitattributes files.
Since strnlen is not supported on all platforms and since we
now have the shiny new git_text_is_binary in the filtering
code, let's convert diff binary detection to use the new stuff.
Currently, git_remote_disconnect not only closes the connection but also
frees the underlying transport object, making it impossible to write
code like
// fetch stuff
git_remote_download()
// close connection
git_remote_disconnect()
// call user provided callback for each ref
git_remote_update_tips(remote, callback)
because remote->refs points to references owned by the transport object.
This means, we have an idling connection while running the callback for
each reference.
Instead, allow immediate disconnect and free the transport later in
git_remote_free().
The recent 64-bit Windows fixes changed the return code in
git_pkt_parse_line() so it wouldn't signal a short buffer, breaking
the network code. Bring it back.
We were not following the git behavior for leading slashes
in path names when matching git ignores and git attribute
file patterns. This should fix issue #638.
This renamed `git_khash_str` to `git_strmap`, `git_hash_oid` to
`git_oidmap`, and deletes `git_hashtable` from the tree, plus
adds unit tests for `git_strmap`.
This updates khash.h with some extra features (like error checking
on allocations, ability to use wrapped malloc, foreach calls, etc),
creates two high-level wrappers around khash: `git_khash_str` and
`git_khash_oid` for string-to-void-ptr and oid-to-void-ptr tables,
then converts all of the old usage of `git_hashtable` over to use
these new hashtables.
For `git_khash_str`, I've tried to create a set of macros that
yield an API not too unlike the old `git_hashtable` API. Since
the oid hashtable is only used in one file, I haven't bother to
set up all those macros and just use the khash APIs directly for
now.
This converts the git attr related code (including ignores) and
the git diff related code (and implicitly the status code) to use
`git_pools` for storing strings. This reduces the number of small
blocks allocated dramatically.
This adds a `git_pool` object that can do simple paged memory
allocation with free for the entire pool at once. Using this,
you can replace many small allocations with large blocks that
can then cheaply be doled out in small pieces. This is best
used when you plan to free the small blocks all at once - for
example, if they represent the parsed state from a file or data
stream that are either all kept or all discarded.
There are two real patterns of usage for `git_pools`: either
for "string" allocation, where the item size is a single byte
and you end up just packing the allocations in together, or for
"fixed size" allocation where you are allocating a large object
(e.g. a `git_oid`) and you generally just allocation single
objects that can be tightly packed. Of course, you can use it
for other things, but those two cases are the easiest.
This allows the caller to update an internal structure or update the
user output with the tips that were updated.
While in the area, only try to update the ref if the value is
different from its old one.
Trying to send every single line immediately won't give us any speed
improvement and duplicates the code we need for other transports. Make
the git transport use the same buffer functions as HTTP.
This changes the git_remote_download() API, but the existing one is
silly, so you don't get to complain.
The new API allows to know how much data has been downloaded, how many
objects we expect in total and how many we've processed.
The code used to assume that there had to be data after the newline in
a tree cache extension entry. This isn't true for a childless
invalidated entry if it's the last one, as there won't be any children
nor a hash to take up space.
Adapt the off-by-one comparison to also work in this case. Fixes#633.
git_repository_free() calls git_odb_free() if the owned odb is not null.
According to the doc, when setting a new odb through git_repository_set_odb() the caller has to take care of releasing the odb by himself.
This fixes a possible compilation issue (when GIT_WIN32 was not set) which was introduced in revision 69a4bc1988.
Signed-off-by: Sven Strickroth <email@cs-ware.de>
The code used to assume that there had to be data after the newline in
a tree cache extension entry. This isn't true for a childless
invalidated entry if it's the last one, as there won't be any children
nor a hash to take up space.
Adapt the off-by-one comparison to also work in this case. Fixes#633.
Adds a new public reference function `git_reference_lookup_oid`
that directly resolved a reference name to an OID without returning
the intermediate `git_reference` object (hence, no free needed).
Internally, this adds a `git_reference_lookup_resolved` function
that combines looking up and resolving a reference. This allows
us to be more efficient with memory reallocation.
The existing `git_reference_lookup` and `git_reference_resolve`
are reimplmented on top of the new utility and a few places in the
code are changed to use one of the two new functions.
git_repository_free() calls git_index_free() if the owned index is not null.
According to the doc, when setting a new index through git_repository_set_index() the caller has still to take care of releasing the index by itself.
In order to cope with this, this fix makes sure the index refcount is incremented when a new repository is being plugged a new index.
This adds preliminary support for pathspecs to diff and status.
The implementation is not very optimized (it still looks at
every single file and evaluated the the pathspec match against
them), but it works.
This will allow us to index a packfile as soon as we receive it from
the network as well as storing it with its final name so we don't need
to pass temporary file names around.
As parents are older than their children, we're appending to the
commit list most of the time, which makes an ordered linked list quite
inefficient.
While we're there, don't sort the results list in the main loop, as
we're sorting them afterwards and it creates extra work.
There is no need walk down the parents of a merge base to mark them as
uninteresting because we'll never see them. Calculate the merge bases
in prepare_walk() so mark_uninteresting() can stop at a merge base
instead of walking all the way to the root.
It's implemented in revwalk.c so it has access to the revision
walker's commit cache and related functions. The algorithm is the one
used by git, modified so it fits better with the library's functions.
The code was already there, so factor it out and let users push an OID
by giving it a reference name. Only refs to commits are
supported. Annotated tags will throw an error.
Add a new command `git_repository_open_ext` with extended options
that control how searching for a repository will be done. The
existing `git_repository_open` and `git_repository_discover` are
reimplemented on top of it. We may want to change the default
behavior of `git_repository_open` but this commit does not do that.
Improve support for "gitdir" files where the work dir is separate
from the repo and support for the "separate-git-dir" config. Also,
add support for opening repos created with `git-new-workdir` script
(although I have only confirmed that they can be opened, not that
all functions work correctly).
There are also a few minor changes that came up:
- Fix `git_path_prettify` to allow in-place prettifying.
- Fix `git_path_root` to support backslashes on Win32. This fix
should help many repo open/discover scenarios - it is the one
function called when opening before prettifying the path.
- Tweak `git_config_get_string` to set the "out" pointer to NULL
if the config value is not found. Allows some other cleanup.
- Fix a couple places that should have been calling
`git_repository_config__weakptr` and were not.
- Fix `cl_git_sandbox_init` clar helper to support bare repos.
Looking through the open windows to check whether we can re-use an
open window should take into account whether both `offset` and `offset
+ extra` are contained within the same window. Failure to do so can
lead to invalid memory accesses. This closes#614.
While we're in the area remove an outdated assert.
There was a bug in git_buf_join_n when the contents of the
original buffer were joined into itself and the realloc
moved the pointer to the original buffer.
This adds support for a bunch of core.* settings that affect
diff and status, plus fixes up some incorrect implementations
of those settings from before. Also, this cleans up the
handling of config settings in the new submodules code and
in the old attrs/ignore code.
When processing status for a newly checked out repo, it is
possible that there will be submodules that have not yet been
initialized. The only way to distinguish these from untracked
directories is to have some knowledge of submodules. This
commit adds a new submodule API which, given a name or path,
can determine if it appears to be a submodule and can give
information about the submodule.
I decided that the COITERATE macro was, in the end causing
more confusion that it would save and decided just to write
out the loops that I needed for parallel diff list iteration.
It is not that much code and this just feels less obfuscated.
There was an error in the tree iterator where it would
delete two tree levels instead of just one when popping
up a tree level. Unfortunately the test data for the
tree iterator did not have any deep trees with subtrees
in the middle of the tree items, so this problem went
unnoticed. This contains the 1-line fix plus new test
data and tests that reveal the issue.
This gives `git_status_foreach()` back its old behavior of
emulating the "--untracked=all" behavior of git. You can
get any of the various --untracked options by passing flags
to `git_status_foreach_ext()` but the basic version will
keep the behavior it has always had.
This fixes the bug that @nulltoken found (thank you!) where
if there were untracked directories alphabetically after the
last tracked item, the diff implementation would deref a NULL
pointer.
The fix involved the code which decides if it is necessary
to recurse into a directory in the working dir, so it was
easy to add a new option `GIT_STATUS_OPT_RECURSE_UNTRACKED_DIRS`
to control if the contents of untracked directories should be
included in status.
This adds support for roughly-right tracking of submodules
(although it does not recurse into submodules to detect
internal modifications a la core git), and it adds support
for including unmodified files in diff iteration if requested.
