This adds some more macros for some standard operations on file
modes, particularly related to permissions, and then updates a
number of places around the code base to use the new macros.
Report the index being locked with its own error code in order to be
able to differentiate, as a locked index is typically the result of a
crashed process or concurrent access, both of which often require user
intervention to fix.
This reverts refactoring done in 13224ea4aa
that introduces a performance regression for NFS when reading files that
don't exist. open() forces a cache invalidation on NFS, while stat()ing a
file just uses the cache and is very quick.
To give a specific example, say you have a repo with a thousand packed
refs. Before this change, looking up every single one ould incur a thousand
slow open() calls. With this change, it's a thousand fast stat() calls.
In theory, p_stat should never return an S_ISLNK result, but due
to the current implementation on Windows with mount points it is
possible that it will. For now, work around that by allowing a
link in the path to a directory being created. If it is really a
problem, then the issue will be caught on the next iteration of
the loop, but typically this will be the right thing to do.
There are two places where git_futils_mkdir should exit early or
at least do less. The first is when using GIT_MKDIR_SKIP_LAST
and having that flag leave no directory left to create; it was
being handled previously, but the behavior was subtle. Now I put
in a clear explicit check that exits early in that case.
The second is when there is no directory to create, but there is
a valid path that should be verified. I shifted the logic a bit
so we'll be better about not entering the loop than that happens.
This makes sure that git_futils_mkdir always skips over the root
directory at a minimum, even on platforms where the root is not
simply '/'. Also, this removes the GIT_WIN32 ifdef in favor of
making EACCES as a potentially recoverable error on all platforms.
We ran into an issue where cloning a repository to a folder
directly underneath the root of a volume (e.g. 'd:\libgit2')
would fail with an access denied error. This was traced down
to a call to make a directory that is the root (e.g. 'd:') could
return an error indicated access denied instead of an error
indicating the path already exists. This change now handles
the access denied error on Win32 and checks for the existence
of the folder.
This adds a bunch more rename detection tests including checks
vs the working directory, the new exact match options, some more
whitespace variants, etc.
This also adds a git_futils_writebuffer helper function and uses
it in checkout. This is mainly added because I wanted an easy
way to write out a git_buf to disk inside my test code.
This switches the APIs for setting and getting the global/system
search paths from using git_strarray to using a simple string with
GIT_PATH_LIST_SEPARATOR delimited paths, just as the environment
PATH variable would contain. This makes it simpler to get and set
the value.
I also added code to expand "$PATH" when setting a new value to
embed the old value of the path. This means that I no longer
require separate actions to PREPEND to the value.
Implicit type conversion argument of function to size_t type
Suspicious sequence of types castings: size_t -> int -> size_t
Consider reviewing the expression of the 'A = B == C' kind. The expression is calculated as following: 'A = (B == C)'
Unsigned type is never < 0
The goal of this work is to expose the search logic for "global",
"system", and "xdg" files through the git_libgit2_opts() interface.
Behind the scenes, I changed the logic for finding files to have a
notion of a git_strarray that represents a search path and to store
a separate search path for each of the three tiers of config file.
For each tier, I implemented a function to initialize it to default
values (generally based on environment variables), and then general
interfaces to get it, set it, reset it, and prepend new directories
to it.
Next, I exposed these interfaces through the git_libgit2_opts
interface, reusing the GIT_CONFIG_LEVEL_SYSTEM, etc., constants
for the user to control which search path they were modifying.
There are alternative designs for the opts interface / argument
ordering, so I'm putting this phase out for discussion.
Additionally, I ended up doing a little bit of clean up regarding
attr.h and attr_file.h, adding a new attrcache.h so the other two
files wouldn't have to be included in so many places.
When creating files, instead of actually using GIT_FILEMODE_BLOB
and the other various constants that happen to correspond to
mode values, apparently I should be just using 0666 and 0777, and
relying on the umask to clear bits and make the value sane.
This fixes the rules for copying a template directory and fixes
the checks to match that new behavior. (Further changes to the
checkout logic to follow separately.)
When PR #1359 removed the hooks from the test resources/template
directory, it made me realize that the tests for
git_repository_init_ext using templates must be pretty shabby
because we could not have been testing if the hooks were getting
created correctly.
So, this started with me recreating a couple of hooks, including
a sample and symlink, and adding tests that they got created
correctly in the various circumstances, including with the SHARED
modes, etc. Unfortunately this uncovered some issues with how
directories and symlinks were copied and chmod'ed. Also, there
was a FIXME in the code related to the chmod behavior as well.
Going back over the directory creation logic for setting up a
repository, I found it was a little difficult to read and could
result in creating and/or chmod'ing directories that the user
almost certainly didn't intend.
So that let to this work which makes repo initialization much
more careful (and hopefully easier to follow). It required a
couple of extensions / changes to core fileops utilities, but I
also think those are for the better, at least for git_futils_cp_r
in terms of being careful about what actions it takes.
It is not legal inside our `p_mmap` function to mmap a zero length
file. This adds a test that exercises that case inside diff and
fixes the code path where we would try to do that.
The fix turns out not to be a lot of code since our default file
content is already initialized to "" which works in this case.
Fixes#1210
This flips checkout back to be driven off the changes between
the baseline and the target trees. This reinstates the complex
code for tracking the contents of the working directory, but
overall, I think the resulting logic is easier to follow.
The existing p_lstat implementation on win32 is not quite POSIX
compliant when setting errno to ENOTDIR. This adds an option to
make is be compliant so that code (such as checkout) that cares
to have separate behavior for ENOTDIR can use it portably.
This also contains a couple of other minor cleanups in the
posix_w32.c implementations to avoid unnecessary work.
This is a major reworking of checkout strategy options. The
checkout code is now sensitive to the contents of the HEAD tree
and the new options allow you to update the working tree so that
it will match the index content only when it previously matched
the contents of the HEAD. This allows you to, for example, to
distinguish between removing files that are in the HEAD but not
in the index, vs just removing all untracked files.
Because of various corner cases that arise, etc., this required
some additional capabilities in rmdir and other utility functions.
This includes the beginnings of an implementation of code to read
a partial tree into the index based on a pathspec, but that is
not enabled because of the possibility of creating conflicting
index entries.
* Rework GIT_DIRREMOVAL values to GIT_RMDIR flags, allowing
combinations of flags
* Add GIT_RMDIR_EMPTY_PARENTS flag to remove parent dirs that
are left empty after removal
* Add GIT_MKDIR_VERIFY_DIR to give an error if item is a file,
not a dir (previously an EEXISTS error was ignored, even for
files) and enable this flag for git_futils_mkpath2file call
* Improve accuracy of error messages from git_futils_mkdir
This adds a new API that allows users to reload the config if the
file has changed on disk. A new config callback function to
refresh the config was added.
The modified time and file size are used to test if the file needs
to be reloaded (and are now stored in the disk backend object).
In writing tests, just using mtime was a problem / race, so I
wanted to check file size as well. To support that, I extended
`git_futils_readbuffer_updated` to optionally check file size in
addition to mtime, and I added a new function `git_filebuf_stats`
to fetch the mtime and size for an open filebuf (so that the
config could be easily refreshed after a write).
Lastly, I moved some similar file checking code for attributes
into filebuf. It is still only being used for attrs, but it
seems potentially reusable, so I thought I'd move it over.
The new Win32 global path search was not working with the
environment variable tests. But when I fixed the test, the new
codes use of getenv() was causing more failures (presumably because
of caching on Windows ???). This fixes the global file lookup to
always go directly to the Win32 API in a predictable way.
This started as a complex new test for checkout going through the
"typechanges" test repository, but that revealed numerous issues
with checkout, including:
* complete failure with submodules
* failure to create blobs with exec bits
* problems when replacing a tree with a blob because the tree
"example/" sorts after the blob "example" so the delete was
being processed after the single file blob was created
This fixes most of those problems and includes a number of other
minor changes that made it easier to do that, including improving
the TYPECHANGE support in diff/status, etc.
Do not hardcode the installation path of msysgit, but read installation path from registry.
Also "%PROGRAMFILES%\Git\etc" won't work on x64 systems with 64-bit libgit2, because
msysgit is x86 only and located in "%ProgramFiles(x86)%\Git\etc".
Signed-off-by: Sven Strickroth <email@cs-ware.de>
On most systems %USERPROFILE% is the same as %HOMEDRIVE%\%HOMEPATH%,
however, for windows machines in an AD or domain environment this
might be different and %HOMEDRIVE%\%HOMEPATH% seems to be better.
Signed-off-by: Sven Strickroth <email@cs-ware.de>
Use %HOME% before trying to figure out the windows user directory.
Users might set this as they are used on *nix systems.
Signed-off-by: Sven Strickroth <email@cs-ware.de>
In the process of adding tests for the max file size threshold
(which treats files over a certain size as binary) there seem to
be a number of problems in the new code with detecting binaries.
This should fix those up, as well as add a test for the file
size threshold stuff.
Also, this un-deprecates `GIT_DIFF_LINE_ADD_EOFNL`, since I
finally found a legitimate situation where it would be returned.
This adds support to diff and status for running filters (a la crlf)
on blobs in the workdir before computing SHAs and before generating
text diffs. This ended up being a bit more code change than I had
thought since I had to reorganize some of the diff logic to minimize
peak memory use when filtering blobs in a diff.
This also adds a cap on the maximum size of data that will be loaded
to diff. I set it at 512Mb which should match core git. Right now
it is a #define in src/diff.h but it could be moved into the public
API if desired.
This cleans up a number of items suggested during code review
with @vmg, including:
* renaming "outside repo" config API to `git_config_open_default`
* killing the `git_config_open_global` API
* removing the `git_` prefix from the static functions in fileops
* removing some unnecessary functionality from the "cp" command
This extends git_repository_init_ext further with support for
initializing the repository from an external template directory
and with support for the "create shared" type flags that make a
set GID repository directory.
This also adds tests for much of the new functionality to the
existing `repo/init.c` test suite.
Also, this adds a bunch of new utility functions including a
very general purpose `git_futils_mkdir` (with the ability to
make paths and to chmod the paths post-creation) and a file
tree copying function `git_futils_cp_r`. Also, this includes
some new path functions that were useful to keep the code
simple.
The extended version of repository init adds support for many
of the things that you can do with `git init` and sets up
structures that will make it easier to extend further in the
future.
The error codes from failed lookups of system and global files
on Windows were not consistent with the codes returned on other
platforms. This makes the error detection patterns match and
adds a unit test for the various errors.
On Windows, we are having problems with home directories
that have non-ascii characters in them. This rewrites the
relevant code to fetch environment variables as UTF-16 and
then explicitly map then into UTF-8 for our internal usage.
This makes the git attributes and git ignores cache check
stat information before using the file contents from the
cache. For cached files from the index, it checks the SHA
of the file instead. This should reduce the need to ever
call `git_attr_cache_flush()` in most situations.
This commit also fixes the `git_status_should_ignore` API
to use the libgit2 standard parameter ordering.
This includes a few cleanups that came up while converting
these files.
This commit introduces a could new git error classes, including
the catchall class: GITERR_INVALID which I'm using as the class
for invalid and out of range values which are detected at too low
a level of library to use a higher level classification. For
example, an overflow error in parsing an integer or a bad letter
in parsing an OID string would generate an error in this class.
This converts blob.c, fileops.c, and all of the win32 files.
Also, various minor cleanups throughout the code. Plus, in
testing the win32 build, I cleaned up a bunch (although not
all) of the warnings with the 64-bit build.
This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to
the new style of error handling. Also got the unix and win32
versions of map.c. There are some minor changes to other
files but no others were completely converted.
This also contains an update to filebuf so that a zeroed out
filebuf will not think that the fd (== 0) is actually open
(and inadvertently call close() on fd 0 if cleaned up).
Lastly, this was built and tested on win32 and contains a
bunch of fixes for the win32 build which was pretty broken.
This also includes droping `git_buf_lasterror` because it makes no sense
in the new system. Note that in most of the places were it has been
dropped, the code needs cleanup. I.e. GIT_ENOMEM is going away, so
instead it should return a generic `-1` and obviously not throw
anything.
