Pass an executor through the Configuration

[tomaka]

Hopefully fixes the performance issues that Polkadot was having.

One issue that we have in master right now is that all the tasks go through this path:

• substrate/client/service/src/lib.rs

  Line 334 in a90c72e

  this.to_poll.push(Box::pin(err.into_future().compat().map(drop)));

• substrate/client/service/src/lib.rs

  Lines 339 to 341 in a90c72e

  	while let Some(pos) = this.to_poll.iter_mut().position(|t| Pin::new(t).poll(cx).is_ready()) {
  	let _ = this.to_poll.remove(pos);
  	}

In other words, everything is single-threaded (except the networking).

In this PR, we now pass a "tasks executor" through the Configuration struct.
It is set up to be a tokio runtime, or the spawn_local function of wasm_bindgen_futures, or a threads pool by default, or local polling if all else fails EDIT: removed fallbacks after review.

Important: this executor is stored within the Service, and all the components of Substrate should then use the Service (through spawn_task or spawn_task_handle or spawn_essential_task) to create tasks.

[bkchr]

Do we still need this logic? The builder creates a thread builder anyway and browser gives us a spawner as well. This sounds like we can remove the option here?

[tomaka]

If the user leaves None in the Configuration, we try to create a ThreadPool and this is the path that is reached if spawning the threads failed.

In practice, without this code we will get a panic in the browser if the user forgets to tweak the configuration. I guess it's indeed debatable whether it's worth the additional code.

[bkchr]

I think we should go the way of checking at Service creation that a task_executor is provided and otherwise an error is returned. I know that is not the best solution, but IMHO the best we can do for now.

[expenses]

Agreed with @bkchr.

[tomaka]

I'm trying of create a version of Polkadot using this branch to see if it indeed fixes the performance issue.
(that's not a blocker for merging though, I think this change is beneficial nonetheless)

[expenses]

LGTM apart from that one discussion.

[tomaka]

We deployed this PR to both a Kusama validator and a Kusama sentry node, and CPU usage seems to be mostly the same if not lower than version 0.7.18 (that includes the revert to libp2p 0.13).

[bkchr]

@tomaka service tests needs to be updated.