Improve gossipsub

[sukhman-sukh]

What was wrong?

Current version of gossipsub did not have message queuing, priority message queuing, and also, message splitting based on maxMessageSize as discussed for interoperability with go-libp2p.
Issue # #891

How was it fixed?

• Added a rpc_queue which handles message sharing and separates it from pubsub.
• Added message size limit
• RPC message transfer similar to go-libp2p

Cute Animal Picture

[sukhman-sukh]

Hey @seetadev,
As per the discussion thread #719, I have shifted our gossipsub write_msg to a message queuing and splitting architecture. Please review this, and is there anything else to add to this PR.
(I am unable to tag luca).

[seetadev]

@sukhman-sukh : This is wonderful. Great work. Reviewing this in detail. Thank you so much for the contribution.

CCing @sumanjeet0012 , @lla-dane and @Winter-Soren , who are also working on different Gossipsub branches. Asking them to review this important PR with me.

@acul71 (Luca) had some github admin issues. They are now resolved and you can definitely tag Luca again :)

[lla-dane]

The urgent bool parameter has to be mentioned here also right ?

[lla-dane]

@sukhman-sukh: I had one question about the splitting logic, if an outgoing RPC is larger than maxMessageSize, it gets split into multiple smaller RPCs. On the receiving side, do we need to handle these fragments as parts of a larger message (i.e reassemble them), or is each split RPC a complete standalone message that can be processed independently?

[sukhman-sukh]

Hey @lla-dane, according to me, every RPC is standalone. We were first piggybacking multiple messages into a single RPC which might exceed it sometime, so now it is taking that big RPC and breaking it into a standalone list of RPCs.

[lla-dane]

Yeah I understand that. But suppose we break a big chunk RPC into individual RPCs, so isn't there a chamce that the content of those 2 smaller RPCs would be dependent, and since our ultimate goal was to somehow deliver the complete content of the big RPC on the receiver end, so the content of the 2 smaller RPCs need to agregated or something on the receiver end.
Is it not the case ?
@sukhman-sukh