There is no need to keep config file in memory until the the
configuration is freed. Free the buffer immediately after the
configuration has been parsed.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
If a variable name appears on its own in a line, it's assumed the
value is true. Store the variable name as NULL in that case.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
If a line ends at EOF there is no need to check for the newline
character and doing so will cause us to read memory beyond the
allocatd memory as we check for the Windows-style new-line, which is
two bytes long.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
If a variable value has the traditional continuation character (\) as
the last non-space character in the line, then we continue reading the
value on the next line.
Using more than two lines is also supported.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Make header and variable parse functions use their own buffers instead
of giving them the line they need to read as a parameter which they
mostly ignore.
This is in preparation for multiline configuration variables.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Streaming writes will no longer fail when writing to a backend that
doesn't support streaming writes but supports direct ones.
Now we create a fake stream on memory and then write it as a single
block using the backend `write` callback.
A variable name is stored internally with its section the way it
appeared in the configuration file in order to have the information
about what parts are case-sensitive inline.
Really implement parse_section_header_ext and move the assignment of
variables to config_parse.
The variable name matching is now done in a case-away way by
cvar_name_match and cvar_section_match. Before the user sees it, it's
normalized to the two- or three-dot version.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Such a list preserves the order the variables were first read in which
will be useful later for merging different data-sets. Furthermore,
reading and writing out the same configuration should not reorganize
the variables, which could happen when iterating through all the items
in a hash table.
A hash table is overkill for this small a data-set anyway.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
git_config_open shouldn't have to initialise variables that are only
used inside config_parse and its callees.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Config variables should be interpreted at run-time, as we don't know if a
zero means false or zero, or if yes means true or "yes".
As a variable has no intrinsic type, git_cvtype is gone and the public
API takes care of enforcing a few rules.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Allow any well-formed reference name to live under refs/ removing the
condition that they be under refs/{heads,tags,remotes}/ as was the
design of git.
An exception is made for HEAD which is allowed to contain an OID
reference in detached HEAD state.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Add internal reference create and rename functions which take a force
parameter, telling them to overwrite an existing reference if it
exists.
These functions try to update the reference if it's of the same type
as the one it's going to be replaced by. Otherwise the old reference
becomes invalid.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
These functions can be used to query or modify the variables in a
given configuration. No sanity checking is done on the variable names.
This is mostly meant as an API preview.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
If cfg_readline consumes the line, then parse_section_header will read
past it and if we read a character, parse_variable won't have the full
name.
This solution is a bit hackish, but it's the simplest way to get the
code to parse correctly.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Save the location of the name in section_out instead of returning it
as an int. Use the return code to signal success or failure.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Expose the tag parsing capabilities already present in the
library.
Exporting this function makes it possible to implement the
mktag command without duplicating this functionality.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
List all the references in the repository, calling a custom
callback for each one.
The listed references may be filtered by type, or using
a bitwise OR of several types. Use the magic value
`GIT_REF_LISTALL` to obtain all references, including
packed ones.
The `callback` function will be called for each of the references
in the repository, and will receive the name of the reference and
the `payload` value passed to this method.
The current behaviour of git_index_open{bare,inrepo}() is unexpected.
When an index is opened, an in-memory index object is created that is
linked to the index discovered by git_repository_open(). However, this
index object is empty, as the on-disk index is not read. To fully open
the on-disk index file, git_index_read() has to be called. This leads to
confusing behaviour. Consider the following code:
git_index *idx;
git_index_open_inrepo(&idx, repo);
git_index_write(idx);
You would expect this to have no effect, as the index is never
ostensibly manipulated. However, what actually happens is that the index
entries are removed from the on-disk index because the empty in-memory
index object created by open_inrepo() is written back to the disk.
This patch reads the index after opening it.
Temporary files when doing streaming writes are now stored inside the
Objects folder, to prevent issues when moving files between
disks/partitions.
Add support for block writes to the ODB again (for those backends that
cannot implement streaming).
When the system temporary folder is located on a different volume than the working directory into which libgit2 is executing, MoveFileEx() requires an additional flag.
Hey. Apologies in advance -- I broke your bindings.
This is a major commit that includes a long-overdue redesign of the
whole object-database structure. This is expected to be the last major
external API redesign of the library until the first non-alpha release.
Please get your bindings up to date with these changes. They will be
included in the next minor release. Sorry again!
Major features include:
- Real caching and refcounting on parsed objects
- Real caching and refcounting on objects read from the ODB
- Streaming writes & reads from the ODB
- Single-method writes for all object types
- The external API is now partially thread-safe
The speed increases are significant in all aspects, specially when
reading an object several times from the ODB (revwalking) and when
writing big objects to the ODB.
Here's a full changelog for the external API:
blob.h
------
- Remove `git_blob_new`
- Remove `git_blob_set_rawcontent`
- Remove `git_blob_set_rawcontent_fromfile`
- Rename `git_blob_writefile` -> `git_blob_create_fromfile`
- Change `git_blob_create_fromfile`:
The `path` argument is now relative to the repository's working dir
- Add `git_blob_create_frombuffer`
commit.h
--------
- Remove `git_commit_new`
- Remove `git_commit_add_parent`
- Remove `git_commit_set_message`
- Remove `git_commit_set_committer`
- Remove `git_commit_set_author`
- Remove `git_commit_set_tree`
- Add `git_commit_create`
- Add `git_commit_create_v`
- Add `git_commit_create_o`
- Add `git_commit_create_ov`
tag.h
-----
- Remove `git_tag_new`
- Remove `git_tag_set_target`
- Remove `git_tag_set_name`
- Remove `git_tag_set_tagger`
- Remove `git_tag_set_message`
- Add `git_tag_create`
- Add `git_tag_create_o`
tree.h
------
- Change `git_tree_entry_2object`:
New signature is `(git_object **object_out, git_repository *repo, git_tree_entry *entry)`
- Remove `git_tree_new`
- Remove `git_tree_add_entry`
- Remove `git_tree_remove_entry_byindex`
- Remove `git_tree_remove_entry_byname`
- Remove `git_tree_clearentries`
- Remove `git_tree_entry_set_id`
- Remove `git_tree_entry_set_name`
- Remove `git_tree_entry_set_attributes`
object.h
------------
- Remove `git_object_new
- Remove `git_object_write`
- Change `git_object_close`:
This method is now *mandatory*. Not closing an object causes a
memory leak.
odb.h
-----
- Remove type `git_rawobj`
- Remove `git_rawobj_close`
- Rename `git_rawobj_hash` -> `git_odb_hash`
- Change `git_odb_hash`:
New signature is `(git_oid *id, const void *data, size_t len, git_otype type)`
- Add type `git_odb_object`
- Add `git_odb_object_close`
- Change `git_odb_read`:
New signature is `(git_odb_object **out, git_odb *db, const git_oid *id)`
- Change `git_odb_read_header`:
New signature is `(size_t *len_p, git_otype *type_p, git_odb *db, const git_oid *id)`
- Remove `git_odb_write`
- Add `git_odb_open_wstream`
- Add `git_odb_open_rstream`
odb_backend.h
-------------
- Change type `git_odb_backend`:
New internal signatures are as follows
int (* read)(void **, size_t *, git_otype *, struct git_odb_backend *, const git_oid *)
int (* read_header)(size_t *, git_otype *, struct git_odb_backend *, const git_oid *)
int (* writestream)(struct git_odb_stream **, struct git_odb_backend *, size_t, git_otype)
int (* readstream)( struct git_odb_stream **, struct git_odb_backend *, const git_oid *)
- Add type `git_odb_stream`
- Add enum `git_odb_streammode`
Signed-off-by: Vicent Marti <tanoku@gmail.com>
We now depend on libpthread on all Unix platforms (should be installed
by default) and use a simple wrapper for Windows threads under Win32.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
It's no longer retarded. All object interdependencies are stored as OIDs
instead of actual objects. This should be hundreds of times faster,
specially on big repositories. Heck, who knows, maye it doesn't even
segfault -- wouldn't that be awesome?
What has changed on the API?
`git_commit_parent`, `git_commit_tree`, `git_tag_target` now return
their values through a pointer-to-pointer, and have an error code.
`git_commit_set_tree` and `git_tag_set_target` now return an error
code and may fail.
`git_repository_free__no_gc` has been deprecated because it's
stupid. Since there are no longer any interdependencies between
objects, we don't need internal reference counting, and GC
never fails or double-free's pointers.
`git_object_close` now does a very sane thing: marks an object
as unused. Closed objects will be eventually free'd from the
object cache based on LRU. Please use `git_object_close` from
the garbage collector `destroy` method on your bindings. It's
100% safe.
`git_repository_gc` is a new method that forces a garbage collector
pass through the repo, to free as many LRU objects as possible.
This is useful if we are running out of memory.
The new pack backend is an adaptation of the original git.git code in
`sha1_file.c`. It's slightly faster than the previous version and
severely less memory-hungry.
The call-stack of a normal pack backend query has been properly
documented in the top of the header for future reference. And by
properly I mean with ASCII diagrams 'n shit.
The new revision walker uses an internal Commit object storage system,
custom memory allocator and much improved topological and time sorting
algorithms. It's about 20x times faster than the previous implementation
when browsing big repositories.
The following external API calls have changed:
`git_revwalk_next` returns an OID instead of a full commit object.
The initial call to `git_revwalk_next` is no longer blocking when
iterating through a repo with a time-sorting mode.
Iterating with Topological or inverted modes still makes the initial
call blocking to preprocess the commit list, but this block should be
mostly unnoticeable on most repositories (topological preprocessing
times at 0.3s on the git.git repo).
`git_revwalk_push` and `git_revwalk_hide` now take an OID instead
of a full commit object.
Set of methods to find the minimal-length to uniquely identify every OID
in a list. Useful for GUI applications, commit logs and so on.
Includes stress test.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Set of methods to find the minimal-length to uniquely identify every OID
in a list.
Includes stress test.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
We cannot make sure that the user doesn't use the same buffer as source
and destination, so write to it using memmove.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
Disable garbage collection of cross-references to prevent
double-freeing. Internal reference management is now done
with a separate method.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
- Added several missing reference increases
- Add new destructor to the repository that does not GC the objects
Signed-off-by: Vicent Marti <tanoku@gmail.com>
All `git_object` instances looked up from the repository are reference
counted. User is expected to use the new `git_object_close` when an
object is no longer needed to force freeing it.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
We now store only one sorting callback that does entry comparison. This
is used when sorting the entries using a quicksort, and when looking for
a specific entry with the new search methods.
The following search methods now exist:
git_vector_search(vector, entry)
git_vector_search2(vector, custom_search_callback, key)
git_vector_bsearch(vector, entry)
git_vector_bsearch2(vector, custom_search_callback, key)
The sorting state of the vector is now stored internally.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The methods previously known as
git_repository_lookup
git_repository_newobject
git_repository_lookup_ref
are now part of their respective namespaces:
git_object_lookup
git_object_new
git_reference_lookup
This makes the API more consistent with the new references API.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The following methods have been implemented:
git_reference_packall
git_reference_rename
git_reference_delete
The library now has full support for packed references, including
partial and total writing. Internal documentation has been updated with
the details.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
These two reference types are now stored separately to eventually allow
the removal/renaming of loose references and rewriting of the refs
packfile.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
We now use MoveFileEx, which is not assured to be atomic but works for
always (both if the destination exists, or if it doesn't) and is
available in MinGW.
Since this is a Win32 API call, complaint about lost or overwritten files
should be forwarded at Steve Ballmer.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The `rename` call doesn't quite work on Win32: expects the destination
file to not exist. We're using a native Win32 call in those cases --
that should do the trick.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The old hash table with chained buckets has been replaced by a new one
using Cuckoo hashing, which offers guaranteed constant lookup times.
This should improve speeds on most use cases, since hash tables in
libgit2 are usually used as caches where the objects are stored once and
queried several times.
The Cuckoo hash implementation is based off the one in the Basekit
library [1] for the IO language, but rewritten to support an arbritrary
number of hashes. We currently use 3 to maximize the usage of the nodes pool.
[1]: https://github.com/stevedekorte/basekit/blob/master/source/CHash.c
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The new `git_filebuf` structure provides atomic high-performance writes
to disk by using a write cache, and optionally a double-buffered scheme
through a worker thread (not enabled yet).
Writes can be done 3-layered, like in git.git (user code -> write cache
-> disk), or 2-layered, by writing directly on the cache. This makes
index writing considerably faster.
The `git_filebuf` structure contains all the old functionality of
`git_filelock` for atomic file writes and reads. The `git_filelock`
structure has been removed.
Additionally, the `git_filebuf` API allows to automatically hash (SHA1)
all the data as it is written to disk (hashing is done smartly on big
chunks to improve performance).
Signed-off-by: Vicent Marti <tanoku@gmail.com>
The interlocking on the write threads was not being done properly (index
entries were sometimes written out of order). With proper interlocking,
the threaded write is only marginally faster on big index files, and
slower on the smaller ones because of the overhead when creating
threads.
The threaded index writing has been temporarily disabled; after more
accurate benchmarks, if might be possible to enable it again only when
writing very large index files (> 1000 entries).
Signed-off-by: Vicent Marti <tanoku@gmail.com>
64-bit types stored in memory have to be truncated into 32 bits when
writing to disk. Was causing warnings in MSVC.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
In response to issue #60 (git_index_write really slow), the write_index
function has been rewritten to improve its performance -- it should now
be in par with the performance of git.git.
On top of that, if Posix Threads are available when compiling libgit2, a
new threaded writing system will be used (3 separate threads take care
of solving byte-endianness, hashing the contents of the index and
writing to disk, respectively). For very long Index files, this method
is up to 3x times faster than git.git.
Signed-off-by: Vicent Marti <tanoku@gmail.com>
