DRAFT: inital egress whitelist controls

Package Merge Request

Sample values file to provide egress controls

istio:
  egressGateways:
    egress-admin:
      enabled: true
      rules:
      - host: "login.dso.mil"
        # ports:
        rules:
        - from:
          - source:
              principals:
              - "cluster.local/ns/admin/sa/default"
    egress-gateway:
      enabled: true
      k8s:
      # authorization policy rules
      rules:
      - host: "oauth2.googleapis.com"
        # ports:
        rules:
        - from:
          - source:
              principals:
              - "*"
      - host: "httpbin.org"
        rules:
        - from:
          - source:
              principals:
              - "cluster.local/ns/default/sa/default"

Work outstanding

• NetworkPolicies - When doing egress control through the egressgateway, we should add a NetworkPolicy that blocks all IP addresses so pods can't egress.
• Figure out what's going on with Metrics (See note below)
• Documentation - When using this to control https traffic (as originated from the pod) there's no AuthorizationPolicies that can be done that use path/header. We could either document this, or add some jsonschemas in here to prevent people from misconfiguring it.
• Clean up AuthPolicies
• Continue to test the system to figure out to add additional AuthorizationPolicies.
• Test adding additional egress-policy HelmRelease objects with the same host so that we know system owners can add them outside of Umbrella
• Test using different ports to make sure 443/80 are not hard coded (hint: they are)
• Test using different protocols
  • GRPC
  • TCP
  • Raw HTTP
  • MySQL
• Figure out if we can allow raw IP addresses through the egress gateway for when databases are provided via IP address
• Ensure Traces are being added to jaeger/tempo
• Ensure logs are being collected and parsed correctly in Loki/elastic
• Build dashboard in grafana to show what pods (serviceaccounts) are talking to what endpoints, and if they were allowed. May require AUDIT on the egress pod, which doesn't seem to show up until logging is increased on the egress pod.
• Create dashboard showing all istio log messages for RBAC denied network requests. Show namespace, pod, requested endpoint, etc
• Lock down the egress-policy repo, make it pass CI, tags etc

Metrics

The metrics being reported by the egressgateway pod are a small subset of the standard istio metrics that get reported by the ingressgateway pod or the sidecars. Not sure what's going on there. This was deployed using the above values file and then traffic was sent through the egressgateway.

➜  ~ k get pods -n istio-system
NAME                                     READY   STATUS    RESTARTS   AGE
public-ingressgateway-7898b84dd8-kg2fp   1/1     Running   0          23h
egress-admin-594d66dcc-cvkts             1/1     Running   0          20h
egress-gateway-57dc59d9d7-ksjgm          1/1     Running   0          20h
svclb-public-ingressgateway-hnrj9        3/3     Running   0          20h
svclb-public-ingressgateway-mtkdq        3/3     Running   0          20h
svclb-public-ingressgateway-bjvwq        3/3     Running   0          20h
svclb-public-ingressgateway-zdkbs        3/3     Running   0          20h
istiod-6498994866-5bwj6                  1/1     Running   0          16h
➜  ~ k exec -it -n istio-system public-ingressgateway-7898b84dd8-kg2fp -- curl http://localhost:15020/stats/prometheus | grep "istio" | wc -l
    2058
➜  ~ k exec -it -n istio-system egress-gateway-57dc59d9d7-ksjgm -- curl http://localhost:15020/stats/prometheus | grep "istio" | wc -l
     396