Issues Migrating to Plugin Sidecars: plugin sidecar failed. couldn't find cmp-server plugin with name argocd-vault-plugin-helm supporting the given repository

[kemra102]

In our project we are currently using the argocd-vault-plugin plugin and as we want to move to the latest ArgoCD versions we need to move from the old "ConfigMap" method of installing plugins, to the new sidecar pattern.

However after following the docs to update our config we are getting the following error on every application deployment that uses this plugin (more or less all of them):

plugin sidecar failed. couldn't find cmp-server plugin with name argocd-vault-plugin-helm supporting the given repository

Presumably there is something in the docs we've missed or not properly understood.

Versions:

• ArgoCD: v2.6.7
• ArgoCD Helm Chart: 5.19.12
• Helm: 3.13.3

ConfigManagementPlugin config:

---
apiVersion: argoproj.io/v1alpha1
kind: ConfigManagementPlugin
metadata:
  name: argocd-vault-plugin-helm
spec:
  discover:
    find:
      command: ['echo', 'hi']
  init:
    command: ['sh', '-c']
    args: ['helm dependency build']
  generate:
    command: ['bash', '-c']
    args: ['helm template -n $ARGOCD_APP_NAMESPACE $ARGOCD_APP_NAME . --values <(echo "$ARGOCD_ENV_helm_values") | argocd-vault-plugin generate -']
---
apiVersion: argoproj.io/v1alpha1
kind: ConfigManagementPlugin
metadata:
  name: argocd-vault-plugin
spec:
  generate:
    command: ['argocd-vault-plugin']
    args: ['generate', './']

Container config snippet:

repoServer:
  extraContainers:
    - name: "vault-plugin"
       image: "platform-docker-local.artifactory.corp.domain.uk/argocd:v2.6.7-2"
       command: ["/usr/local/bin/argocd-cmp-server"]
       securityContext:
         runAsNonRoot: true
         runAsUser: 999
      volumeMounts:
        - mountPath: "/home/argocd/cmp-server/plugins"
          name: "cmp-plugins"
        - mountPath: "/home/argocd/cmp-server/config/plugin.yaml"
          subPath: "plugin.yaml"
          name: "vault-plugin"
        - mountPath: "/tmp"
          name: "cmp-tmp"

Example of the plugin being called (example is the final rendered YAML from K8s, with some stuff cut out):

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "3"
  labels:
    app.kubernetes.io/instance: lob-deployment-lob-configurations
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: lob-deployment
    argocd.argoproj.io/instance: lob-configurations
    helm.sh/chart: lob-deployment-0.9.0
  name: internal-sync-api-deployments
  namespace: argocd
spec:
  generators:
  - git:
      directories:
      - path: '*'
      repoURL: https://gitlab.domain.uk/messaging/deployments/internal/sync.git
      revision: env/sandbox
  template:
    metadata:
      name: internal-{{path.basename}}
    spec:
      destination:
        namespace: internal-sync
        server: https://kubernetes.default.svc
      project: internal
      source:
        path: '{{path}}'
        plugin:
          env:
          - name: helm_values
            value: |
              global:
                hipEnvironmentInfo:
                  apiIngressClass: kong
                  apiIngressDomains:
                    instance: env.domain.uk
                    live: env.domain.uk
                  baseDomain: env.domain.uk
                  lobPathPrefix: internal
                  svcIngressClass: kong-svcgw
                  svcIngressDomains:
                    instance: svc.env.domain.uk
                    live: svc.env.domain.uk
                  wiremock:
                    enabledByDefault: true
                    namespace: internal-wiremock
          name: argocd-vault-plugin-helm
        repoURL: https://gitlab.corp.domain.uk/messaging/deployments/internal/sync.git
        targetRevision: env/sandbox
      syncPolicy:
        automated:
          allowEmpty: false
          prune: true
          selfHeal: true
        retry:
          backoff:
            duration: 5s
            factor: 2
            maxDuration: 3m
          limit: 5
        syncOptions:
        - ApplyOutOfSyncOnly=true
        - CreateNamespace=false
        - PruneLast=true
        - PrunePropagationPolicy=foreground

[devonhk]

• mountPath: "/home/argocd/cmp-server/config/plugin.yaml"
  subPath: "plugin.yaml"
  name: "vault-plugin"

Is this volumeMount coming from a volume configMap ?
I can't tell if you embedded your plugins in a ConfigMap or not.

[kemra102]

Yep it comes from our ConfigMap, which looks like this:

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: argo
    meta.helm.sh/release-namespace: argocd
  labels:
    app.kubernetes.io/managed-by: Helm
  name: vault-plugin
  namespace: argocd
data:
  plugin.yaml: |
    ---
    apiVersion: argoproj.io/v1alpha1
    kind: ConfigManagementPlugin
    metadata:
      name: argocd-vault-plugin-helm
    spec:
      discover:
        find:
          command: ['echo', 'hi']
      init:
        command: ['sh', '-c']
        args: ['helm dependency build']
      generate:
        command: ['bash', '-c']
        args: ['helm template -n $ARGOCD_APP_NAMESPACE $ARGOCD_APP_NAME . --values <(echo "$ARGOCD_ENV_helm_values") | argocd-vault-plugin generate -']
    ---
    apiVersion: argoproj.io/v1alpha1
    kind: ConfigManagementPlugin
    metadata:
      name: argocd-vault-plugin
    spec:
      generate:
        command: ['argocd-vault-plugin']
        args: ['generate', './']

[dresl]

Hello, did you solved it? I have same error here...

[kemra102]

Actually we never solved this, we decided to make the effort to migrate to the External Secrets Operator instead.

[dresl]

Alright. To make it clear, finally, I solved it. But it was problem with discover/generate keys in my custom kapitan plugin, but it was raising the same error.

[dbaltor]

For those landing here who want to figure out how to configure AVP with helm, I recommend having a look at this page. I used the InitContainer and configuration via sidecar method:

This config map worked for me:

  plugin.yaml: |
    apiVersion: argoproj.io/v1alpha1
    kind: ConfigManagementPlugin
    metadata:
      name: argocd-vault-plugin-helm
    spec:
      allowConcurrency: true
      discover:
        find:
          command:
            - sh
            - "-c"
            - "find . -name 'Chart.yaml' && find . -name 'values.yaml'"
      init:
       command:
          - bash
          - "-c"
          - |
            helm repo add bitnami https://charts.bitnami.com/bitnami
            helm dependency build
      generate:
        command:
          - bash
          - "-c"
          - |
            helm template $ARGOCD_APP_NAME -n $ARGOCD_APP_NAMESPACE -f <(echo "$ARGOCD_ENV_HELM_VALUES") . |
            argocd-vault-plugin generate -
      lockRepo: false

I used the umbrella chart pattern to config the helm deployment:

    apiVersion: argoproj.io/v1alpha1
    kind: Application
    metadata:
      name: <APP_NAME>
      namespace: argocd
      finalizers:
        - resources-finalizer.argocd.argoproj.io
    spec:
      project: default
      source:
        repoURL: https://github.com/dbaltor/argocd-test.git
        targetRevision: master
        path: <SOME_PATH>
        plugin:
          name: argocd-vault-plugin-helm
          env:
            - name: HELM_ARGS
              value: -f values.yaml
      destination:
        server: https://kubernetes.default.svc
        namespace: superset
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
          allowEmpty: false
        syncOptions:
          - Validate=true
          - CreateNamespace=false
          - PrunePropagationPolicy=foreground
          - PruneLast=true

and I stored the umbrella chart Charts.yaml and its configuration values.yaml under the <SOME_PATH> folder mentioned above.