This tells the server that we speak it, but we don't make use of its
extra information to determine if there's a better place to stop
negotiating.
In a somewhat-related change, reorder the capabilities so we ask for
them in the same order as git does.
Also take this opportunity to factor out a fairly-indented portion of
the negotiation logic.
It was there to keep it apart from the one which read in from a file on
disk. This other indexer does not exist anymore, so there is no need for
anything other than git_indexer to refer to it.
While here, rename _add() function to _append() and _finalize() to
_commit(). The former change is cosmetic, while the latter avoids
talking about "finalizing", which OO languages use to mean something
completely different.
The names from libssh2 are somewhat obtuse for us. We can simplify the
usual key/passphrase credential's name, as well as make clearer what the
custom signature function is.
This commit adds cancellation for the push operation. This work consists of:
1) Support cancellation during push operation
- During object counting phase
- During network transfer phase
- Propagate GIT_EUSER error code out to caller
2) Improve cancellation support during fetch
- Handle cancellation request during network transfer phase
- Clear error string when cancelled during indexing
3) Fix error handling in git_smart__download_pack
Cancellation during push is still only handled in the pack building and
network transfer stages of push (and not during packbuilding).
This adds the basics of progress reporting during push. While progress
for all aspects of a push operation are not reported with this change,
it lays the foundation to add these later. Push progress reporting
can be improved in the future - and consumers of the API should
just get more accurate information at that point.
The main areas where this is lacking are:
1) packbuilding progress: does not report progress during deltafication,
as this involves coordinating progress from multiple threads.
2) network progress: reports progress as objects and bytes are going
to be written to the subtransport (instead of as client gets
confirmation that they have been received by the server) and leaves
out some of the bytes that are transfered as part of the push protocol.
Basically, this reports the pack bytes that are written to the
subtransport. It does not report the bytes sent on the wire that
are received by the server. This should be a good estimate of
progress (and an improvement over no progress).
The subtransport path was relying on pointing to data owned by
the remote which meant that after a redirect, the updated path
was getting lost for future requests. This updates the http
transport to strdup the path and maintain its own lifetime.
This also pulls responsibility for parsing the URL back into the
http transport and isolates the functions that parse and free that
connection data so that they can be reused between the initial
parsing and the redirect parsing.
This is in preparation for moving the hashing to the frontend, which
requires us to handle the incoming data before passing it to the
backend's stream.
Key-based authentication also needs an username, so include it in each
one.
Also stop assuming a default username of "git" in the ssh transport
which has no business making such a decision.
The SSH error checking and reporting could still be further
improved by using the libssh2 native methods to get error info,
but at least this ensures that all error codes are checked and
translated into libgit2 error messages.
This makes all of the credential objects use the same pattern to
clear the contents and call git__memzero when done. Much of this
information is probably not sensitive, but it also seems better
to just clear consistently.
Much of the SSH credential creation API can be left enabled even
on platforms with no SSH support. We really just have to give an
error when you attempt to open the SSH connection.
Nobody should ever be using anything other than ALL at this level, so
remove the option altogether.
As part of this, git_reference_foreach_glob is now implemented in the
frontend using an iterator. Backends will later regain the ability of
doing the glob filtering in the backend.
For update and create commands where all the objects are known to
exist in the remote, we must send an empty packfile. However, if all
we issue are delete commands, no packfile must be sent.
Take this into consideration for push.
This moves some of the odb_backend stuff that is related to the
internals of an odb_backend implementation into include/git2/sys.
Some of the stuff related to streaming I left in include/git2
because it seemed like it would be reasonably needed by a normal
user who wanted to stream objects into and out of the ODB.
Also, I added APIs for traversing the list of backends so that
some of the tests would not need to access ODB internals.
Implicit type conversion argument of function to size_t type
Suspicious sequence of types castings: size_t -> int -> size_t
Consider reviewing the expression of the 'A = B == C' kind. The expression is calculated as following: 'A = (B == C)'
Unsigned type is never < 0
The cppcheck static analyzer generates warnings for a bunch of
places in the libgit2 code base. All the ones fixed in this
commit are actually false positives, but I've reorganized the
code to hopefully make it easier for static analysis tools to
correctly understand the structure. I wouldn't do this if I
felt like it was making the code harder to read or worse for
humans, but in this case, these fixes don't seem too bad and will
hopefully make it easier for better analysis tools to get at any
real issues.
* gen_pktline() in smart_protocol.c was skipping refspecs that deleted
refs that were not advertised by the server. The new behavior is to
send a delete command with an old-id of zero, which matches the behavior
of the official git client.
* Update test_network_push__delete() in reaction to above fix.
* Obviate messy logic that handles missing push_spec rrefs by canonicalizing
push_spec. After calculate_work(), loid, roid, and rref, are filled in with
exactly what is sent to the server
The fix for fetching from empty repositories (22935b06d protocol:
don't store flushes; 2012-10-07) forgot to take into account the
deletion of the flush pkt in the HTTP transport. As a result, the HEAD
ref advertisement where we detect the remote's capabilities was
deleted instead. Fix this.
Wondows has its own HTTP library. Use that one when possible instead of
our own.
As we don't depend on them anymore, remove the http-parser library from
the Windows build, as well as the search for OpenSSL.
Instad of each transport having its own function and logic to get to
its refs, store them directly in transport.
Leverage the new gitno_buffer to make the parsing and storing of the
refs use common code and get rid of the git_protocol struct.
This allows us to add capabilitites to both at the same time, keeps
them in sync and removes a lot of code.
gitno_buffer now uses a callback to fill its buffer, allowing us to
use the same interface for git and http (which uses callbacks).
For the transition, http is going to keep its own logic until the
git/common code catches up with the implied multi_ack that http
has. This also has the side-effect of making the code cleaner and more
correct regardingt he protocol.
If it's not available, an error saying so will be returned when trying
to use a https:// URL.
This also unifies a lot of the network code to use git_transport in
many places instead of an socket descriptor.
Local fetch isn't implemented yet. Don't segfault on call, but set a
dummy for negotiate_fetch and terminate gracefully.
Reported-by: Brad Harder <bch@methodlogic.net>
gitno_connect() can return an error or socket, which is fine on most
platforms where sockets are file descriptors (signed int), but on Windows,
SOCKET is an unsigned type, which is problematic when we are trying to
test if the socket was actually a negative error code.
This fix seperates the error code and socket in gitno_connect(), and fixes
the error handling in do_connect() functions to compensate. It appears
that git_connect() and the git-transport do_connect() functions had bugs
in the non-windows cases too (leaking sockets, and not properly reporting
connection error, respectively) so I went ahead and fixed those too.
Trying to send every single line immediately won't give us any speed
improvement and duplicates the code we need for other transports. Make
the git transport use the same buffer functions as HTTP.
This changes the git_remote_download() API, but the existing one is
silly, so you don't get to complain.
The new API allows to know how much data has been downloaded, how many
objects we expect in total and how many we've processed.
Adds a new public reference function `git_reference_lookup_oid`
that directly resolved a reference name to an OID without returning
the intermediate `git_reference` object (hence, no free needed).
Internally, this adds a `git_reference_lookup_resolved` function
that combines looking up and resolving a reference. This allows
us to be more efficient with memory reallocation.
The existing `git_reference_lookup` and `git_reference_resolve`
are reimplmented on top of the new utility and a few places in the
code are changed to use one of the two new functions.
This also includes droping `git_buf_lasterror` because it makes no sense
in the new system. Note that in most of the places were it has been
dropped, the code needs cleanup. I.e. GIT_ENOMEM is going away, so
instead it should return a generic `-1` and obviously not throw
anything.
It turns out that commit 31e9cfc4cbcaf1b38cdd3dbe3282a8f57e5366a5
did not fix the GIT_USUSED behavior on all platforms. This commit
walks through and really cleans things up more thoroughly, getting
rid of the unnecessary stuff.
To remove the use of some GIT_UNUSED, I ended up adding a couple
of new iterators for hashtables that allow you to iterator just
over keys or just over values.
In making this change, I found a bug in the clar tests (where we
were doing *count++ but meant to do (*count)++ to increment the
value). I fixed that but then found the test failing because it
was not really using an empty repo. So, I took some of the code
that I wrote for iterator testing and moved it to clar_helpers.c,
then made use of that to make it easier to open fixtures on a
per test basis even within a single test file.
This makes libgit2 compliant with the following scenario
$ git ls-remote file:///d:/temp/dwm%20tinou
732d790b702db4b8985f5104fc44642654f6a6b6 HEAD
732d790b702db4b8985f5104fc44642654f6a6b6 refs/heads/master
732d790b702db4b8985f5104fc44642654f6a6b6 refs/remotes/origin/HEAD
732d790b702db4b8985f5104fc44642654f6a6b6 refs/remotes/origin/master
$ mv "/d/temp/dwm tinou" /d/temp/dwm+tinou
$ git ls-remote file:///d:/temp/dwm%20tinou
fatal: 'd:/temp/dwm tinou' does not appear to be a git repository
fatal: The remote end hung up unexpectedly
$ git ls-remote file:///d:/temp/dwm+tinou
732d790b702db4b8985f5104fc44642654f6a6b6 HEAD
732d790b702db4b8985f5104fc44642654f6a6b6 refs/heads/master
732d790b702db4b8985f5104fc44642654f6a6b6 refs/remotes/origin/HEAD
732d790b702db4b8985f5104fc44642654f6a6b6 refs/remotes/origin/master
This converts virtually all of the places that allocate GIT_PATH_MAX
buffers on the stack for manipulating paths to use git_buf objects
instead. The patch is pretty careful not to touch the public API
for libgit2, so there are a few places that still use GIT_PATH_MAX.
This extends and changes some details of the git_buf implementation
to add a couple of extra functions and to make error handling easier.
This includes serious alterations to all the path.c functions, and
several of the fileops.c ones, too. Also, there are a number of new
functions that parallel existing ones except that use a git_buf
instead of a stack-based buffer (such as git_config_find_global_r
that exists alongsize git_config_find_global).
This also modifies the win32 version of p_realpath to allocate whatever
buffer size is needed to accommodate the realpath instead of hardcoding
a GIT_PATH_MAX limit, but that change needs to be tested still.
The following files now have 0444 permissions:
- loose objects
- pack indexes
- pack files
- packs downloaded by fetch
- packs downloaded by the HTTP transport
And the following files now have 0666 permissions:
- config files
- repository indexes
- reflogs
- refs
This brings libgit2 more in line with Git.
Note that git_filebuf_commit() and git_filebuf_commit_at() have both
gained a new mode parameter.
The latter change fixes an important issue where filebufs created with
GIT_FILEBUF_TEMPORARY received 0600 permissions (due to mkstemp(3)
usage). Now we chmod() the file before renaming it into place.
Tests have been added to confirm that new commit, tag, and tree
objects are created with the right permissions. I don't have access to
Windows, so for now I've guarded the tests with "#ifndef GIT_WIN32".
