add: use preload-index and fscache for performance

Teach "add" to use preload-index and fscache features
to improve performance on very large repositories.

During an "add", a call is made to run_diff_files()
which calls check_remove() for each index-entry.  This
calls lstat().  On Windows, the fscache code intercepts
the lstat() calls and builds a private cache using the
FindFirst/FindNext routines, which are much faster.

Somewhat independent of this, is the preload-index code
which distributes some of the start-up costs across
multiple threads.

We need to keep the call to read_cache() before parsing the
pathspecs (and hence cannot use the pathspecs to limit any preload)
because parse_pathspec() is using the index to determine whether a
pathspec is, in fact, in a submodule. If we would not read the index
first, parse_pathspec() would not error out on a path that is inside
a submodule, and t7400-submodule-basic.sh would fail with

	not ok 47 - do not add files from a submodule

We still want the nice preload performance boost, though, so we simply
call read_cache_preload(&pathspecs) after parsing the pathspecs.

Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>

Author: jeffhostetler

builtin/add.c

@@ -593,6 +593,10 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 	die_in_unpopulated_submodule(&the_index, prefix);
 	die_path_inside_submodule(&the_index, &pathspec);
 
+	enable_fscache(1);
+	/* We do not really re-read the index but update the up-to-date flags */
+	preload_index(&the_index, &pathspec, 0);
+
 	if (add_new_files) {
 		int baselen;
 
@@ -695,6 +699,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		die(_("Unable to write new index file"));
 
 	dir_clear(&dir);
+	enable_fscache(0);
 	UNLEAK(pathspec);
 	return exit_status;
 }