dagweave Connector

dagweave Connector

The dagweave Connector is a small Kubernetes operator you run in your own cluster. It bridges your Argo Workflows instances to dagweave so the app can submit workflows and collect their status, logs, and artifacts. This is the one piece a cluster operator sets up, usually once, so that workflow authors can run their workflows on your cluster. You only need it if you want dagweave to run workflows for you. If you just want the YAML, copy it from the builder and apply it yourself. Running workflows through the connector needs a plan that includes it.

How it connects

The connector is always the client. It opens a single outbound channel to dagweave and every message rides on that one connection. dagweave pushes work down the channel and receives results back up it. Your cluster never has to accept inbound traffic, so you do not open a port or expose an ingress for dagweave.

A few properties worth knowing before you install:

• It rides on Argo Workflows’ own CRDs. dagweave ships already-compiled Workflow, WorkflowTemplate, and CronWorkflow manifests, and the connector applies them as-is for the Argo Workflows controller to run. The connector’s own CRDs carry configuration only. There is no bespoke run CRD to drift from the Argo Workflows schema.
• It runs HA by default. The manager runs two replicas in active/standby with leader election, so only one replica ever dispatches. Two writers would submit duplicate runs.
• Features are toggleable, and the RBAC follows. Submit, status, logs, artifacts and sources each gate their own block of RBAC rules. Turn one off and its permissions never get installed.
• The two Secret-reading grants differ in scope. artifacts is on by default and grants get alone, so the connector can fetch a Secret it is given the name of and can never enumerate one. That get reaches every namespace until you fill in targetNamespaces, because an ArgoConnection can name a Secret anywhere. sources is off, and turning it on renders a Role in the connector’s own namespace holding list on Secrets there, never a ClusterRole. Kubernetes RBAC cannot scope a Secret read to a label, which is why sources needs list over that one namespace rather than a rule naming the Secrets it wants.
• Artifacts stay in your storage. Workflow artifacts live in your own S3 or S3-compatible store. The connector mints short-lived presigned URLs and never copies the bytes.
• Logs ship to a sink you choose. With logs on, the chart deploys a log collector as a DaemonSet that tails Argo Workflows pod logs. Point it at your own S3 bucket, your own Loki, or an HTTP endpoint and the bytes never reach dagweave. The hosted Loki is the one option that sends them to us. The chart ships with console, which only prints to the collector’s own stdout. Pick a real sink before you rely on it. Your sink also decides what shows up in the executions view.

Install with Helm

Add a connector in the dagweave UI and it hands you a ready-to-run install command with a one-time enrollment token baked in. The token is shown once, so copy the command before you dismiss it, and it stays redeemable for 24 hours. If it expires before the operator gets to it, mint another. Run it against your cluster:

helm install connector oci://ghcr.io/dagweave/charts/connector \
  --namespace connector-system --create-namespace \
  --set connectorConfig.dagweaveEndpoint=https://api.dagweave.com \
  --set-string enrollment.token=<one-time-enrollment-token>

The chart installs the operator with HA defaults, the two CRDs, per-feature RBAC, and the enrollment Secret. On first start the connector trades the one-time token for a durable identity. It presents an ed25519 key, dagweave binds that key, and every later call authenticates with it. The token is spent. Presenting it again gets you nothing.

The connector id lands on the ConnectorConfig status. A pod restart, a leader failover or a helm upgrade picks the identity straight back up from there. Enrollment only runs a second time if the keypair in the Secret is gone.

The chart pulls the connector image from GitHub Container Registry. If the pods land in ImagePullBackOff, your cluster could not authenticate to ghcr.io. Give it a pull secret and point the chart at it:

kubectl create secret docker-registry ghcr-pull \
  --namespace connector-system \
  --docker-server=ghcr.io \
  --docker-username=<github-username> \
  --docker-password=<github-token>

helm upgrade connector oci://ghcr.io/dagweave/charts/connector \
  --namespace connector-system --reuse-values \
  --set imagePullSecrets[0].name=ghcr-pull

Connect an Argo Workflows instance

Argo Workflows is often installed per namespace, so one cluster can host several instances. You model each one with an ArgoConnection. The chart does not create any, so this is a step you do after install: write one per instance you want dagweave to manage.

apiVersion: connector.dagweave.com/v1alpha1
kind: ArgoConnection
metadata:
  name: team-a
spec:
  argoInstanceID: team-a
  namespaces: [team-a]
  features:
    submit: true
    status: true
    logs: true
    artifacts: true

If no ArgoConnection matches the instance and namespace a run targets, the connector refuses the dispatch rather than guessing. Two connections that both match one target are refused the same way. Keep their namespace lists from overlapping.

Commands and status are routed and feature-gated per connection. For several clusters, install one connector in each. dagweave aggregates across them; a single connector does not route across clusters.

Run identity

By default a dispatched workflow runs as whatever ServiceAccount its namespace hands it, and dagweave has no say in what that workflow can touch. Run identity changes that. dagweave asks the connector to build the ServiceAccount a run executes as, plus a Role holding the permissions somebody on the team picked and a RoleBinding joining the two.

dagweave never sends RBAC. It sends the id of a permission template and the values for that template’s parameters, and the connector renders the rules from its own copy of your configuration. An id you did not configure comes back as a refusal that names the id and lists what is actually on offer.

A stock install has none of this. features.runIdentity is off, runIdentity.templates is empty, and the chart renders no run identity object at all. Nothing a default install puts in your cluster can create a ServiceAccount, a Role or a RoleBinding.

Writing the permission menu

runIdentity.templates is the list a dagweave user picks from. Each entry has an id, a label, the parameters its form asks for, and the PolicyRules it renders to. ${name} in a rule is replaced by the parameter of that name.

features:
  runIdentity: true
runIdentity:
  namespaces: [team-a]
  templates:
    - id: read-named-configmaps
      label: Read one ConfigMap
      params:
        - name: configMapName
          label: ConfigMap name
          required: true
      rules:
        - apiGroups: [""]
          resources: ["configmaps"]
          resourceNames: ["${configMapName}"]
          verbs: ["get"]

runIdentity.namespaces says where those permissions may be bound, and should match the spec.namespaces of the ArgoConnections you serve. Write templates with the feature off and the install stops with that as the reason. Write them with no namespaces and it stops the same way. What is installed and what is on offer cannot drift apart.

An empty menu leaves the whole thing inert however features.runIdentity is set, so turning the toggle on by itself grants nothing.

Each ArgoConnection carries its own runIdentity toggle as well, and the connector refuses to build anything for a connection that has it off. It also refuses when that connection’s spec.namespaces is empty, because a connection serving every namespace in the cluster is not a boundary to compose inside.

The ceiling you are setting

Kubernetes will not let a subject create a Role granting permissions it does not hold itself. The way around that is the escalate verb, which is a cluster takeover primitive and is not on offer here. So the connector holds the union of everything your menu can render, in the namespaces you listed, for as long as it is configured. Standing, not per run.

Once it is on, the connector holds:

• create, delete, get, list, patch and update on ServiceAccounts, Roles and RoleBindings. resourceNames cannot narrow this, because the names are minted per run and do not exist when you install.
• Every rule any template on your menu can render. A template that writes Secrets gives the connector standing Secret writes in those namespaces whether or not anybody ever picks it.

That union becomes a Role in each namespace you listed, with a RoleBinding beside it. Naming one namespace as team-a renders rel-connector-workloads in team-a and nowhere else, and the ClusterRole is left holding the connector’s own CRDs. Nothing of yours stays cluster-wide once you have named a namespace.

One kind of rule is installed wider than it reads. A rule whose resourceNames contains a ${parameter} goes into the union with no resourceNames at all, because the connector has to hold the permission for whatever value someone picks. The template above offers read the ConfigMap you name and costs the connector read any ConfigMap in these namespaces:

# --- from template "read-named-configmaps" ---
- apiGroups:
  - ""
  resources:
  - configmaps
  # resourceNames dropped: this template lets a user name the object, so the
  # connector must hold the verb for every name in these namespaces.
  verbs:
  - get

Render the ClusterRole and read it before you install. Whatever it says is the most any dagweave user can ever be granted in those namespaces.

helm template connector oci://ghcr.io/dagweave/charts/connector \
  -f your-values.yaml --set enrollment.create=false \
  --set connectorConfig.create=false \
  -s templates/runidentity-rbac.yaml

Both flags are only there because the chart refuses to render without values a real install supplies. The one-time token lives on your install command rather than in your values file, and the ConnectorConfig needs the endpoint your install command passes. Neither affects the ClusterRole you are reading. --show-only filters after rendering, so leaving either out aborts the whole render rather than skipping the file you asked for.

Some shapes never make it that far. resourceNames restricts get, update, patch and delete; Kubernetes matches nothing for list and watch, so a rule pairing those with names would grant exactly nothing and is refused when the configuration loads. Two templates sharing an id are refused there too. A menu that fails to load is reported to dagweave as empty rather than half, so a bad value leaves dagweave with nothing to offer instead of something that will not render.

Building an identity

An identity in dagweave is a name, one template off the menu, the answers to that template’s parameters, the dispatch targets it may be used on, and who runs as it: the whole team, a group, or one person. The targets have to be ones the team can already dispatch to. No Role is written by hand, and nothing outside your menu can be expressed.

A draft can be rewritten as often as you like. Submitting it freezes it, and someone who can manage the team’s members approves or rejects it. An approved identity cannot be edited. Forking one opens a fresh draft at the next version, which goes through the same decision. A team can also let whoever builds an identity approve their own, which is recorded as a self-approval rather than quietly skipped.

Submitting an identity whose template no connector behind its namespaces offers is refused by name, and the refusal says where the menu comes from. It is the same answer the cluster would give later, arriving while nothing has been applied.

Revoking an approved identity stops the next dispatch. A run already in flight keeps the identity it was given.

Create, update, delete, submit, approve, reject, revoke and the self-approval switch all go to the audit log, refusals included.

What happens at dispatch

dagweave looks for an approved identity covering the target namespace and the person dispatching: one naming them, one naming a group they belong to, or one naming the whole team. A trigger fires with nobody behind it, so only a team-wide identity applies to a triggered run. No match leaves the manifest exactly as it was and the run goes out the way it always did.

With a match, dagweave asks the connector to build the identity and waits for the answer before it submits anything. A Workflow naming a ServiceAccount that does not exist yet is rejected at admission, so a refusal, or a connector that does not answer, fails the dispatch rather than submitting a run that cannot start.

The connector writes the Role, then the RoleBinding, then the ServiceAccount. A refusal from Kubernetes therefore leaves no account for anything to run as, rather than a usable identity holding half of what was asked for.

The ServiceAccount, the Role and the RoleBinding all take one name, dagweave-run- and the run id, so one leads to the others with kubectl alone. What dagweave records is the rules the cluster actually rendered, not just the template id, because a failed run’s permissions are gone by the time anyone goes looking for them.

How long an identity lasts

Argo resolves serviceAccountName per pod, as pods are created. A retryStrategy backing off, or a suspend step waiting on an approval, can create a pod long after the run was submitted. An identity on a timer would break the run it belongs to.

So there is no timer. The ServiceAccount, Role and RoleBinding are made children of the Workflow through metadata.ownerReferences, and Kubernetes garbage collection takes them when the Workflow object goes. The identity therefore outlives the last pod: if your Argo Workflows instance keeps finished Workflows for a week, the account is there for a week with nothing left to use it. Your workflow retention decides the moment, not dagweave.

Ownership cannot cover one case. A connector that dies between building an identity and submitting the Workflow that would own it leaves objects with no owner and nothing to collect them. So the connector sweeps: it looks for the Workflow by the run-id label both carry, adopts the identity onto it if it turns up, and only deletes an identity it could find no owner for and whose deadline has passed. Repair before reap, so a live run whose ownerReference never got written is not deleted out from under itself.

Everything the connector builds is labelled dagweave.io/owner=dagweave, so you can find and remove all of it without dagweave in the loop:

kubectl get sa,role,rolebinding -A -l dagweave.io/owner=dagweave

Health and version

Each connector shows its state in the dagweave UI. A connector is pending until the agent first phones home, online while it heart-beats, and offline once dagweave has gone 300 seconds without a beat. The connector beats every 30 seconds, so ten beats may be missed before a connector is demoted. It was 90 seconds until a slow cluster tripped it on nothing; what the longer window costs is up to three and a half more minutes before a genuine outage is reported. The row also shows the running connector version and, once it heartbeats, the Argo Workflows instances it serves. dagweave never dials in to check; the state comes from the connector’s own heartbeat.

Connector logs in the UI

The connector’s own pod logs are readable in the dagweave UI. When a dispatch does not arrive, the answer is usually sitting in those logs, and reading them any other way means kubectl into the connector’s namespace, which is the cluster access dagweave otherwise spares a workflow author.

These are the connector’s logs, not the per-step logs of a workflow run. Those live in the executions view.

Reading them is permission gated rather than open to anyone who can edit a workflow, because connector logs carry target namespaces, Argo Workflows instance ids and raw cluster errors.

Self-update

Self-update is on by default (selfUpdate.enabled). When dagweave holds a target connector version and a connector reports a different one, dagweave names the target in the reply to that connector’s next heartbeat, and the connector patches the image tag on its own Deployment.

The permission behind that is narrow. The chart renders a namespaced Role and RoleBinding in the release namespace, not a cluster-wide rule: apps deployments with get and patch only, and resourceNames pinned to the connector’s own Deployment. Nothing else in the namespace is reachable, and nothing outside it is.

Two caveats before you leave it on:

• The requested tag is not checked against a signature or a digest, and nothing stops the version moving backwards.
• A helm upgrade re-applies the chart’s image.tag over whatever self-update last wrote.

If you pin connector versions yourself, install with selfUpdate.enabled=false and the Role and RoleBinding are never created. The same value is written to ConnectorConfig.spec.selfUpdate.enabled, which the connector re-reads before every patch, so you can also revoke it by editing that object without running a Helm upgrade.

Once connected

With a cluster connected, workflow authors can attach triggers to a workflow and run it from dagweave: a manual button, a cron schedule, a webhook, GitHub, or an AWS SNS notification. Each run then shows up in the executions view with its logs, metrics, and a re-run button, and run provenance is tracked, so you can see which trigger fired a given run.