Add priority and TTL to `Discovery` process

[tegefaulkes]

Specification

Discovery has recently been updated in PR #451 . Further review of the discovery logic needs to be done and the core logic refactored. The processVertex logic is expansive so if possible, parts of it needs to be extracted out into protected functions to help with readability.

Priority order needs to implemented for discovering gestalts. We can make use of the TaskManager's priority system to do this. We care about gestalts in 3 tiers. Gestalts should be discovered in this order and any other gestalts we don't really care about.

1. Our own gestalt
2. Any gestalt we trust
3. Any gestalt that the user triggers manually

A TTL system needs to be applied to re-visit gestalts to check for any updates. This really just comes down to triggering discovery on the gestalt again after a delay.

We skip vertices that we have visited recently. this is currently done rather simply with verticies stored in an in-memory set. We to implement a more sophisticated method for tracking whether we should skip over a vertex. This needs to be persistent and only last for a certain amount of time. This can be done with a database entry tracking the last time a vertex was visited.

Note that for now the ability to Seek to the last processed claim for a node has been disabled. Given the level of complexity discovery is currently at we need to process a nodes whole sigchain to do re-discovery. To use the seeking feature we need to have re-discovery properly implemented.

Re-discovery will make use of already known links within the gestalt graph to populate the discovery queue. We will need to keep track of the last time a node was discovered and only re-discover nodes that haven't been checked after a given time. To this end we need to track for each node the last time it was discovered and the last claimId of its sigchain we processed.

Additional context

• Related Integrating TaskManager into Discovery #451
• Integrating TaskManager into Discovery #451 (comment)
• Integrating TaskManager into Discovery #451 (comment)

Tasks

• 1. General review of the core discovery logic
• 2. Clean up the processVertex logic, it's branching and messy.
• 3. Implement a priority system for discovery gestalts
• 4. Add a TTL system for re-visiting vertices
• 5. Implement a more complex persistent system for tracking recently visited verticies.
• 6. Track the last time a node was process, track the time and ClaimId.
• 7. Discovery should query the GestaltGraph for nodes and identities to re-discover.
• 8. When asking a node for its chain data, we should ask for claims newer that the last ClaimId we saw.

[CMCDragonkai]

@tegefaulkes please update this issue with information about #446 (comment)

[tegefaulkes]

Points 2, 3, 5, 6, 7, 8 of this have been addressed by generally and by recent changes in #696. So the only thing really left to address here is a priority system for ordering processing. That's pretty low priority at this stage.