Regression in behavior when setting load balancer source ranges for a Service

[elmiko]

What happened:

When setting both .spec.loadBalancerSourceRanges and the service.beta.kubernetes.io/load-balancer-source-ranges annotation on a Service object, the CCM will not reconcile the Service nor will it allow for its deletion. The following error is present:

E0612 16:22:37.193349       1 azure_loadbalancer.go:2952] "Failed to parse access control configuration for service" err="cannot set both spec.LoadBalancerSourceRanges and service annotation service.beta.kubernetes.io/azure-allowed-ip-ranges" logger="reconcileSecurityGroup" cluster="ci-op-mv04309s-04a70-6jszn" service="openshift-ingress/router-sourcerangesstatus" load-balancer="ci-op-mv04309s-04a70-6jszn" delete-lb=true

What you expected to happen:

Expected the Service to continue to be reconciled even if one of the source ranges is ignored.

How to reproduce it (as minimally and precisely as possible):

1. Create a service with both service.beta.kubernetes.io/load-balancer-source-ranges annotation and spec.LoadBalancerSourceRanges

Anything else we need to know?:

We (Red Hat) discovered this regression in our automated testing, and while the user behavior of setting both ranges may not be supported by the CCM, the error when reconciling makes it a possible upgrade blocker for users who may have (inadvertently or not) set both these range options.

There are 2 pull requests which appear to have combined to create this regression:

• Refine LoadBalancer service access control and consolidating security-rules #5164
• fix: allow space separated load balancer source ranges #5885

the effect of #5164 resulted in the following logic:

# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges OR service.beta.kubernetes.io/load-balancer-source-ranges) XOR (service.beta.kubernetes.io/azure-allowed-ip-ranges)

We do not have automation around the service.beta.kubernetes.io/azure-allowed-ip-ranges annotation, and as such we have not directly noticed this failure but we believe it exists.

when combined with #5885, the resulting logic becomes:

# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges) XOR (service.beta.kubernetes.io/load-balancer-source-ranges OR service.beta.kubernetes.io/azure-allowed-ip-ranges)

Related report in Red Hat bug tracking OCPBUGS-35757.

Environment:

This was tested on the upcoming release of OpenShift which uses Kubernetes version 1.30 and RHCOS 9.4 instances.

[elmiko]

cc @JoelSpeed @gcs278

[elmiko]

one thing that came up in discussion internally was whether we missed an announcement or release note about this change in behavior. is this just an organic regression or did we miss something about it?

[JoelSpeed]

@feiskyer @bridgetkromhout Can you please review this issue?