How Argo CD treats dynamically created resource not part of git manifests

[MaharaJothi]

Hi

I am using Argo CD chart version: 10.1.3

EKS cluster 1.34 with Auto Mode enabled

I am installing wiz in my EKS cluster. Wiz creates a dynamic secret with helm hook annotation like this:

apiVersion: v1
kind: Secret
metadata:
  annotations:
     helm.sh/hook: pre-install, pre-upgrade
     helm.sh/hook-weight: "-1"
     argocd.argoproj.io/tracking-id: test-wiz-app:/Secret:wiz/test-wiz-app-connector

Can someone explain the below:
How does Argo CD treat dynamically created resources that have helm.sh/hook annotations but are not part of the desired manifests?

[initial-d]

Argo CD mainly compares two sets:

• desired resources rendered from Git/Helm/Kustomize
• live resources in the cluster that Argo can associate with the Application

For your example, the important annotation is probably not helm.sh/hook by itself, but this one:

argocd.argoproj.io/tracking-id: test-wiz-app:/Secret:wiz/test-wiz-app-connector

That annotation tells Argo CD that the live Secret belongs to test-wiz-app. If the Secret is not present in the rendered desired manifests, Argo can treat it as an extra live resource associated with the Application. Depending on your sync/prune/orphaned-resource settings, it may show as an extraneous/out-of-sync resource and may be pruned during sync if pruning is enabled.

helm.sh/hook does not automatically make an arbitrary live resource part of desired state. Argo CD can understand Helm hook annotations when they are part of the rendered manifests it is applying, but a resource dynamically created outside the desired manifest set is still outside desired state. The hook annotation is not a general “ignore this resource” marker.

So I would check these things:

1. Does argocd app manifests test-wiz-app include that Secret?
   If no, it is not desired state from Argo CD's point of view.

2. Does argocd app get test-wiz-app show the Secret as an app resource?
   If yes, the tracking annotation is causing Argo CD to associate it with the app.

3. Is automated prune enabled?
   If yes, an associated-but-not-desired Secret is at risk of being deleted on sync.

For generated/runtime Secrets, the usual fixes are:

• make the Secret part of the chart/manifests if Argo should own it;
• remove/avoid the Argo CD tracking annotation if some other controller owns it;
• or configure an explicit ignore/prune strategy for that resource if it must remain generated at runtime.

In short: a dynamically created resource with an Argo tracking id can be considered part of the Application's live set, but if it is missing from Git/Helm output, Argo still has no desired manifest for it.

[MaharaJothi]

Thank you for the response @initial-d

In my case, wiz has a Job : kubernetes-connector-create available in the manifests that create this secret file. So this secret is taking annotation from that Job.

But as you mentioned since its not present in the manifests and I have prune: false, so for every sync operation it deletes the wiz-sec-connector secret in my cluster.

how can I avoid the annotation for the secret? Since its dynamically created during runtime, I cannot patch it.

[initial-d]

If prune: false is set and the Secret is still deleted on every sync, I would first separate two possibilities:

1. Argo CD prune is deleting an extraneous tracked resource.
2. Hook/job lifecycle is deleting or recreating it during the sync.

With prune disabled, normal extraneous-resource pruning should not delete it. So if it disappears during every sync, I would suspect the hook path or the Wiz job/chart behavior more than ordinary Argo pruning.

You can confirm the actor by checking the Secret events/audit trail around sync time:

kubectl get events -n wiz --sort-by=.lastTimestamp | grep test-wiz-app-connector
kubectl get secret -n wiz test-wiz-app-connector -o yaml

If the Secret has helm.sh/hook: pre-install,pre-upgrade, it may be participating in hook lifecycle semantics. Argo CD maps many Helm hook annotations into Argo hook behavior when those resources are in the rendered manifests. Also, hook resources are often deleted/recreated around sync depending on hook delete policy / before-hook-creation behavior.

For avoiding the copied tracking annotation: if the Secret is created dynamically by the Wiz Job and the Job copies its own annotations to the Secret, Argo CD cannot prevent that with an ignoreDifferences rule. ignoreDifferences affects comparison of desired vs live fields; it does not stop a runtime job from writing annotations onto a new object.

The clean fixes are usually chart/app-level:

• check whether the Wiz chart exposes a value to stop copying pod/job annotations to the generated Secret;
• check whether it exposes a value for hook-delete-policy or whether the connector Secret can be created as a normal manifest instead of a hook-generated runtime object;
• if you control the chart, do not propagate argocd.argoproj.io/tracking-id from the Job to the Secret;
• if you control the generator Job, explicitly omit Argo CD tracking annotations when creating the Secret.

As a last-resort Argo-side workaround, you can look at orphaned/extraneous resource handling or resource exclusions, but those are broad tools and will not fix a hook/job that actively deletes and recreates the Secret. The first thing I would prove is who deletes it: Argo application-controller prune, Argo hook cleanup, or the Wiz job itself.