How to avoid duplicated array entry in machine config patches?

[stevefan1999-personal]

Hi, I'm using a GitOps based workflow with patch.yaml file, and so I find this behavior quite counterintuitive:

diff --git a/patch.yaml b/patch.yaml
index 08a65d0..768befd 100644
--- a/patch.yaml
+++ b/patch.yaml
@@ -1,6 +1,7 @@
 machine:
   kubelet:
     extraArgs:
+      cloud-provider: external
       rotate-server-certificates: true
     nodeIP:
       validSubnets:
@@ -18,7 +19,12 @@ machine:
     kubePrism:
       enabled: true
       port: 7445
-
+    kubernetesTalosAPIAccess:
+      enabled: true
+      allowedRoles:
+      - os:reader
+      allowedKubernetesNamespaces:
+      - kube-system
 cluster:
   discovery:
     enabled: true
@@ -39,3 +45,7 @@ cluster:
   extraManifests:
   - https://raw.githubusercontent.com/alex1989hu/kubelet-serving-cert-approver/main/deploy/standalone-install.yaml
   - https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
+  externalCloudProvider:
+    enabled: true
+    manifests:
+    - https://raw.githubusercontent.com/siderolabs/talos-cloud-controller-manager/main/docs/deploy/cloud-controller-manager.yml

So I just added some entries in the patch.yaml and I want to refresh the existing controlplane.yaml and worker.yaml for fast node bootstrap access. So I run talosctl machineconfig patch controlplane.yaml --patch "@patch.yaml" -o controlplane.yaml, but I noticed something is off:

    extraManifests:
        - https://raw.githubusercontent.com/alex1989hu/kubelet-serving-cert-approver/main/deploy/standalone-install.yaml
        - https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
        - https://raw.githubusercontent.com/alex1989hu/kubelet-serving-cert-approver/main/deploy/standalone-install.yaml
        - https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

As you can see, those manifests are duplicated, when they intuitively shouldn't, because from a human standpoint, the original array and the new array is naturally the same, so there's no need to intervene it, but from the machine standpoint, or rather the current array merge strategy in talosctl, is to just append the array by default. This would still happen if I output this to a new file, and then thoroughly replacing the old with the new by forcefully mving it. So there is currently no workaround regarding this unless I use JSON patch.

I don't want to bother with JSON patches, because they are quite a mess to work with, plus I have Git to manage all the config differences, so I would like to use the patch.yaml as an overlay of the original files (or should I say over the files that are currently committed). I rather want to see if I can have retained the original array unless there really are some differences.

So what I, a normal human would think of this situation, is to treat the array as a hash set, and compare if there are set differences from the patch and the existing entries (heck this may even look like it comes from a machine mindset).

[smira]

You can use other tools to manage YAML files, but the supported Talos Linux flow for managing machine configuration assumes that a set of patches is applied to the base configuration. Patching with the same patch is not always idempotent.

• Recommended flow
• Merge semantics

[stevefan1999-personal]

That's why I said it is not very intuitive to typical users