NetworkPolicy for ipBlock in local network not honored

[j-p-k]

A Kubernetes egress network policy shall allow access from a pod to the local network such that
the pod can connect to a server outside the k8s cluster. On one of our clusters, this policy is not effective. The allowed traffic is blocked. One such server is actually a load balancer managed by a (Hcloud) Cloud Controller Manager. Here, the Network Policy for the local network does not apply.

Expected Behavior

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test
  namespace: netpoltest
spec:
  podSelector:
    matchLabels:
      netpol: test
  policyTypes:
    - Egress
  egress:
    - to:
        - ipBlock:
            cidr: 10.40.0.0/16
      ports:
        - protocol: TCP
          port: 443

A pod with label netpol:test should be allowed to connect to https://10.40.0.13:443 (the load balancer)

Current Behavior

With no netpols (allow everything), the connection works as expected. However, when selectively allowing the 10.40/16 subnet with the policy above, the connection times out.

Solution

The load balancer in question belongs to a service on the same cluster. Allow access to the pods. In our case.

  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-nginx-internal

Steps to Reproduce (for bugs)

1. Create a namespace
2. In the namespace, create the policy quoted above
3. Create a pod with a matching label and get an interactive shell
4. Test with curl. (Use IP address to avoid DNS issues)

Context

The kubernetes cluster was created with kubeone 1.10.1 in a Hetzner cloud: https://docs.kubermatic.com/kubeone/v1.10/tutorials/creating-clusters/#default-configuration

Your Environment

• Calico version: v3.29.2 with flannel:v0.24.4 (actually canal)
• Calico dataplane (iptables, windows etc.)
• Orchestrator version: Kubernetes version 1.32.6
• Operating System and version: Ubuntu 24.04 LTS

[mazdakn]

@j-p-k what's the difference between this cluster and the other 3 clusters where the policy works?

[j-p-k]

This is a new cluster. The working ones are older:

• Ubuntu 22.04
• Calico v3.26.3
• Flannel v0.21.3

[j-p-k]

I believe, I found a solution and updated the question to reflect this.

The target IP address belongs to a load balancer, i.e. a machine outside the cluster on the local network, but managed by a controller using a service with type LoadBalancer. Although the target IP address is outside the cluster, it seems that kubernetes/calico routes this internally to the pod network. The load balancer IP is treated like a service cluster IP.

In short, the network policy for 10.40.0.0/16 does not apply to 10.40.0.13 if the latter is a loadbalancer/service IP. To make this work, we have to allow the pod IPs.

Not sure if this a bug or a feature.

[caseydavenport]

Although the target IP address is outside the cluster, it seems that kubernetes/calico routes this internally to the pod network

I believe this is a "feature" of kube-proxy (not Calico) - traffic that matches a service LoadBalancer IP will be NAT'd directly to the backing endpoints. This means that traffic from within the cluster (i.e., from pods) won't exit the cluster to the external LoadBalancer just to be routed back to the cluster. But it also means that Calico network policy will be executed on the post-NAT traffic (so the LoadBalancer IP will not be seen).