[ci] Add more shards (#11402)

This adds a bunch more CPU shards and moves everything to CPU-SMALL.
Some Java limitations required splitting up the logic in the templates a
bit as well.

Co-authored-by: driazati <[email protected]>

Author: driazati