Should we have a flattened route table for a VPC

[benjaminleonard]

Using the API summary in RFD 21 as a guideline, a VPC can have multiple routers – one VPC-wide and many custom routers which apply to specific subnets.

Some questions on the context of this:

1. As an individual route can already target a subnet directly, is this for convenience?
2. This looks like a deviation from GCP and AWS who seem to just have a flat route table as direct children to the VPC. Is there a specific problem we're looking to solve through our chosen API model?

From a user perspective, I'd like to avoid adding an extra layer for the user before the route CRUD. Could I add tabs for both Routers, and Route Table? The latter showing a flattened set of all routes from all routers.

[rmustacc]

Thanks for these questions, @benjaminleonard. I'll take these each in turn.

AWS / GCP Divergences

So I'll admit these interfaces are a bit different and more of a hybrid of what others are doing, but I don't think it's as far out there.

So first, I want to confirm what AWS and GCP are actually doing (mostly going back to RFD 9, but I'll include links to their documents) as my understanding of it suggests that it's not a simple flat thing:

• AWS starts with a single default route table. When Subnets are created and no other action is taken, they are added to this table. However, that is not the only option for folks. So more concretely, this default one is called the "Main Route Table" which is automatically created and contains all subnets that aren't part of a custom route table. However, AWS also has a "Custom Route Table". A VPC Subnet in AWS is only associated with one table or the other. However, there definitely already are multiple route tables. The biggest different between what I proposed and the AWS model is that AWS is an either-or situation. That is, you can have one and only one and it must have everything. Read more here. Note things get more complicated due to the use of Gateways and related and eventual automatic propagation from VPNs.

• GCP actually has a few different things. There is a global default router which generally consists of routes that are automatically managed. This is much like the AWS "Main Route Table" and what I called the "System Routing Table". However, from there, there are a bunch of custom routes that GCP can add. Importantly GCP has two different routing modes. A 'static' one which is where you end up being able to add global custom routes; however, there is also a 'dynamic' mode which is used with their 'cloud routers' with VPNs which end up becoming potentially a per-region construct. Thus you end up with multiple sources here of things. See this for more background. So I think with GCP you do get away with the notion of a flat thing until you end up having to deal with sharing BGP routes for VPNs and the outside world and thus per-region constructs.

• Azure actually uses something that's somewhat similar to this. Here they have a notion of custom route tables which can be assigned to a given Subnet as well as a global system table. I know Azure isn't necessarily a focus, but I think it's worth calling out that here and it is informative. More information here.

So I guess with all of these you really do have multiple different routing tables, even if they're not present. The biggest difference I think then is the question of how do they interact. The main reason I went with the strategy described in RFD 21 is that we didn't want a custom route table to necessitate now needing to duplicate all the primary information. I suspect that was a pain in AWS and is part of the reason you see something different in GCP and Azure here. However, GCP tries to have it both ways in terms of a single global namespace, but then ends up having to fall back to particular region categories. So ultimately this approach, which I guess is mostly the Azure one in some respects seemed to actually be a reasonable middle ground from a logical object perspective, letting you determine how they interact, but also allowing you to have some flexibility for different groups of instances.

Does that make some sense? I'm not sure I quite get the flat route table with direct children here as it seems like in each of them there are multiple different things that can be at play, though maybe it's the fact that there's an optional route table that a subnet can have which is part of what's confusing?

Subnet targeting

So when the route says the VPC Subnet is the target, this is basically a way of saying logically when traffic matches this pattern, send it over to this named VPC Subnet. I'm not quite sure I see it as convenience. Or rather what is it in convenience to? I guess it makes it a single rule that says what you logically want to do without having to go look and add two rules one for the IPv4 and one for the IPv6, but that seems a lot harder to get right and easier to mess up.

If the question is why is this here when we have the system routing table adding entries, the main reason for this is to allow overrides or do things that might not be expected. It may be that 'subnet' as a target doesn't make as much sense, but its' at least there in the system routing table as a way to logically describe where something is going without being explicit about IPs and related.

Did this answer this at all?

CRUD and Flattening

So, I'm not sure I understand the extra layers here that you're worried about. I'm not sure a single flattened view will be correct just the same way it's not really correct for all of the other cloud providers. But I'm also not sure this needs to be prominent either. But I really don't understand this very well, so it'd help to maybe talk sometime to get more information so we can better figure out how to have the flexibility we need from a customer perspective so they can build weird things, but also still have a good UI here that helps for the streamlined case.