Access Control Mechanism

[birgirst]

Currently applications that expose HTTP(S) endpoints are open to the internet by default. It would be good to be able to have a way to limit their exposure to a particular pre-determined scope that would be expressed as part of the application spec. The scopes could be pre-defined sets of hostnames and/or ip-ranges that could be arranged in a ip-whitelisting set on the ingress resource or used for provisioning multiple ingress resources with different access control.

[mortenlj]

In the old issue tracker we have a long thread of discussion on this subject, so I will try to collect all the various thoughts, ideas and suggestions into a coherent discussion here.

The problem

When deploying services using fiaas-deploy-daemon, there are varying use-cases for access control. In addition, how the cluster operator wants to implement the access control will vary also, depending on where the cluster is running, what kind of infrastructure it is running on etc. There are at least a couple variations that I can think of:

1. The service should be open for public access from the internet (frontend services)
2. The service should only be accessible from select parts of the internet (ie. VPNs, legacy datacenter, specific AWS accounts, etc.)
   • Integrate with VPN solutions (which can have dynamic IPs)
   • Integrate with load balancers and firewalls for services on premise
   • Handle AWS account peerings or similar features in other clouds
   • Any number of other variations
3. The service should only be accessible internally in the cluster, and not exposed outside the cluster at all
   • We currently have support for making all services of a specific type, but we should probably be able to select type based on some configuration. A service that only needs to be accessed internally doesn't need to be a NodePort-service.
4. The service wants to be made available through a specific ingress controller in a cluster that has multiple
5. Probably some other case I haven't thought about

Generic design

We want a design that allows solutions to "all" the use-cases, without limiting the cluster operators choices too much. There are also two distinct conversations that are only slightly connected: What to do, and where to implement it. I will first focus on what to do, and towards the end I will discuss implementation.

Currently we configure some aspects of ingress handling in each namespace, via the configuration to fiaas-deploy-daemon. Instead, we should move that configuration to a separate CRD, currently named IngressZone. An IngressZone describes a set of settings for one particular "zone". We expand the ingress section in fiaas.yml to select a zone. This will apply the settings from that zone to the ingress created.

A zone contains settings such as ingress-suffixes, host-rewrite-rules, and tls settings, but also additional annotations and labels to apply to all ingresses. This will allow a zone to select an ingress class, to apply IP whitelisting rules or any number of other features that is controlled by annotations. Some settings that are implemented as annotations in Kubernetes might be worthy of their own API field in the IngressZone object.

The integration point for any third party operators would be a plain API object, no manipulating of a configmap.

Implementation ideas

The component that implements this should be responsible for all aspects of Ingress creation and updating. It should also have a watch on IngressZone objects, and automatically apply changes to all relevant applications.

Currently the easy option is to implement this functionality in fiaas-deploy-daemon, which is already responsible for creating and updating Ingress objects. This would extend the responsibility of that component, but would be easier to put into production in existing clusters.

A long term goal should be to split fiaas-deploy-daemon into smaller pieces with clear responsibilities, which would be easier to maintain. By that reasoning, implementing the access control integration should maybe not be done in fiaas-deploy-daemon, but instead we should move the entire handling of Ingress to a separate component that watches Application objects independently.

Platform vs cluster

Some of the use cases mentioned at the start are not handled by the suggested design/implementation, but that is because they are at the intersection between FIAAS as a platform, and the cluster it is running in. I argue that this is where cluster operators need to implement their own solutions, but using clear APIs from FIAAS.

In this case, the API is the IngressZone object. Integrating with VPNs with dynamically changing IPs is handled by the cluster operator creating an operator that updates the relevant IngressZone objects whenever the IPs are changed. Account peerings should be handled by an operator running in the relevant cloud and detect needed peerings to create. There are too many variations for us to make a complete solution, so this must be up to the cluster operators to implement.