Auto merge of #44772 - michaelwoerister:new-graph, r=nikomatsakis

incr.comp.: Add new DepGraph implementation.

This commits does a few things:
1. It adds the new dep-graph implementation -- *in addition* to the old one. This way we can start testing the new implementation without switching all tests at once.
2. It persists the new dep-graph (which includes query result fingerprints) to the incr. comp. caching directory and also loads this data.
3. It removes support for loading fingerprints of metadata imported from other crates (except for when running autotests). This is not needed anymore with red/green. It could provide a performance advantage but that's yet to be determined. For now, as red/green is not fully implemented yet, the cross-crate incremental tests are disabled.

Note, this PR is based on top of soon-to-be-merged #44696 and only the last 4 commits are new:
```
- incr.comp.: Initial implemenation of append-only dep-graph. (c90147c)
- incr.comp.: Do some various cleanup. (8ce20c5)
- incr.comp.: Serialize and deserialize new DepGraph. (0e13c1a)
- incr.comp.: Remove support for loading metadata fingerprints. (270a134)
EDIT 2:
- incr.comp.: Make #[rustc_dirty/clean] test for fingerprint equality ... (d8f7ff9)
```
(EDIT: GH displays the commits in the wrong order for some reason)

Also note that this PR is expected to certainly result in performance regressions in the incr. comp. test cases, since we are adding quite a few things (a whole additional dep-graph, for example) without removing anything. End-to-end performance measurements will only make sense again after red/green is enabled and all the legacy tracking has been turned off.

EDIT 2: Pushed another commit that makes the `#[rustc_dirty]`/`#[rustc_clean]` based autotests compared query result fingerprints instead of testing `DepNode` existence.

Author: bors

src/librustc/dep_graph/dep_node.rs

@@ -60,7 +60,7 @@
 //! user of the `DepNode` API of having to know how to compute the expected
 //! fingerprint for a given set of node parameters.
 
-use hir::def_id::{CrateNum, DefId, DefIndex};
+use hir::def_id::{CrateNum, DefId, DefIndex, CRATE_DEF_INDEX};
 use hir::map::DefPathHash;
 use hir::{HirId, ItemLocalId};
 
@@ -420,7 +420,7 @@ define_dep_nodes!( <'tcx>
     [input] Hir(DefId),
 
     // Represents metadata from an extern crate.
-    [input] MetaData(DefId),
+    [input] CrateMetadata(CrateNum),
 
     // Represents some artifact that we save to disk. Note that these
     // do not have a def-id as part of their identifier.
@@ -678,6 +678,22 @@ impl<'a, 'gcx: 'tcx + 'a, 'tcx: 'a> DepNodeParams<'a, 'gcx, 'tcx> for (DefIndex,
     }
 }
 
+impl<'a, 'gcx: 'tcx + 'a, 'tcx: 'a> DepNodeParams<'a, 'gcx, 'tcx> for (CrateNum,) {
+    const CAN_RECONSTRUCT_QUERY_KEY: bool = true;
+
+    fn to_fingerprint(&self, tcx: TyCtxt) -> Fingerprint {
+        let def_id = DefId {
+            krate: self.0,
+            index: CRATE_DEF_INDEX,
+        };
+        tcx.def_path_hash(def_id).0
+    }
+
+    fn to_debug_str(&self, tcx: TyCtxt<'a, 'gcx, 'tcx>) -> String {
+        tcx.crate_name(self.0).as_str().to_string()
+    }
+}
+
 impl<'a, 'gcx: 'tcx + 'a, 'tcx: 'a> DepNodeParams<'a, 'gcx, 'tcx> for (DefId, DefId) {
     const CAN_RECONSTRUCT_QUERY_KEY: bool = false;
 

src/librustc/dep_graph/edges.rs

@@ -17,7 +17,7 @@ use std::mem;
 use super::{DepGraphQuery, DepKind, DepNode};
 use super::debug::EdgeFilter;
 
-pub struct DepGraphEdges {
+pub(super) struct DepGraphEdges {
     nodes: Vec<DepNode>,
     indices: FxHashMap<DepNode, DepNodeIndex>,
     edges: FxHashSet<(DepNodeIndex, DepNodeIndex)>,
@@ -31,8 +31,8 @@ pub struct DepGraphEdges {
 }
 
 #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)]
-pub struct DepNodeIndex {
-    index: u32
+pub(super) struct DepNodeIndex {
+    index: u32,
 }
 
 impl DepNodeIndex {