> ## Documentation Index > Fetch the complete documentation index at: https://docs.textql.com/llms.txt > Use this file to discover all available pages before exploring further. # Installation > Install the TextQL Helm chart Direct installation with `helm` is available using: ```bash theme={null} helm upgrade oci://registry-1.docker.io/textqllabs/textql-charts --version --values ``` You must have access to the TextQL Docker registry. TextQL provides a PAT token that will grant you access. Before the first install, choose a size for the shared sandbox volume and plan to set its reclaim policy to `Retain` once it exists. See [Sandbox Storage](https://docs.textql.com/helm-charts/sandbox-storage.md). ## Automatic updates You can point Flux, ArgoCD or other GitOps tools to the TextQL registry for automatic releases. You can also set your own CI/CD pipeline that runs `helm install`. If you want these tools to auto install versions, we recommend doing so for **patches** only (e.g. ^1.1.X). We use Semantic Versioning ([https://semver.org/](https://semver.org/)) for our releases, so patches involve non-breaking changes and are safe to automatically install. For major and minor versions, check the [Changelog](https://docs.textql.com/helm-charts/changelog.md) page, which will provide the steps to upgrade, if needed. Contact TextQL for help with your specific setup. ## Rotate the connector encryption key Stored credentials (database connector passwords, API connector keys, OAuth tokens, identity provider secrets and similar) are encrypted with the key named by `connectorEncryptionPrimaryKey`. Rotation takes two upgrades, so that no pod ever meets a credential under a key it has not loaded. 1. Generate a new key: ```bash theme={null} openssl rand -base64 32 ``` 2. **First upgrade: load the new key, keep the current one primary.** Add the new key under a new ID and leave `connectorEncryptionPrimaryKey` unchanged: ```yaml theme={null} global: auth: connectorEncryptionKeys: "k1=,k2=" connectorEncryptionPrimaryKey: "k1" ``` Where the key ring lives depends on `global.secretsMode`: * `plaintext`: set it in your values as above. * `sealedSecrets`: seal the whole ring as one value, then set `connectorEncryptionKeys` to the sealed output. `connectorEncryptionPrimaryKey` is a key ID, not a secret, and stays unsealed. Paste the ring (`k1=,k2=`) at the hidden prompt of `read` rather than into the command itself, so the keys stay out of your shell history. After the upgrade, wait for the controller to update the `auth-secrets` Secret before restarting: ```bash theme={null} read -rs ring printf '%s' "$ring" | kubeseal --raw --namespace --scope namespace-wide \ --controller-name --controller-namespace unset ring ``` * `externalSecrets` or `nativeExternalSecrets`: update the `-connector-encryption-keys` secret in your external store, and wait for it to sync. * `none`: you manage the `auth-secrets` Kubernetes Secret yourself. Set its `connector.encryption.keys` entry to the new ring. Upgrade the chart, then restart the compute deployment. A change to the key ring alone does not restart its pods, and a pod only reads the ring when it starts: ```bash theme={null} kubectl rollout restart deployment/compute-engine -n kubectl rollout status deployment/compute-engine -n ``` Every compute pod now holds both keys. 3. **Second upgrade: switch the primary.** Set `connectorEncryptionPrimaryKey: "k2"` and upgrade. New credentials are encrypted with `k2` at once, and in the background the compute service re-encrypts every stored credential still under `k1`. It logs one line per table, plus one for cached API connector drafts (`table=api_revision_cache`): ``` Retiring-key re-encryption pass complete table= rows_moved= values_moved= errored= rows_remaining= values_unmovable= ``` 4. **Run a final pass.** Pods from before the second upgrade keep writing with `k1` until the rollout finishes. Once it has finished, restart the compute deployment and check the lines from that start: every line, including `api_revision_cache`, must report `rows_remaining=0`. ```bash theme={null} kubectl rollout restart deployment/compute-engine -n kubectl rollout status deployment/compute-engine -n ``` `values_unmovable` counts text that only looks like an encrypted value and values no loaded key can decrypt. They do not depend on `k1` and do not block the next step. 5. **Remove the old key.** Drop `k1` from the ring and upgrade: ```yaml theme={null} global: auth: connectorEncryptionKeys: "k2=" connectorEncryptionPrimaryKey: "k2" ``` Update the ring wherever your secrets mode keeps it, as in step 2, upgrade, and restart the compute deployment, waiting for the rollout to finish so every pod holds the reduced ring: ```bash theme={null} kubectl rollout restart deployment/compute-engine -n kubectl rollout status deployment/compute-engine -n ``` **Do not remove the old key while any table reports `rows_remaining` above 0.** Those credentials can only be decrypted with the old key and stop working the moment it leaves the ring. If a count stays above 0 after a restart, the log line `rows still hold values only the retiring key can decrypt` names the table; contact TextQL before removing the key. **Rolling back** after step 3 must keep `k2` in the ring: credentials already moved to `k2` cannot be decrypted without it. Captured sandbox egress payloads are diagnostics and are not re-encrypted; those captured before the rotation become unreadable once `k1` is removed. ## Spanmetrics dashboards and alerting The existing OpenTelemetry collector can generate call counts and duration histograms and send them to a dedicated metrics destination. Enable `otel.spanMetrics.enabled` and configure `otel.spanMetrics.exporter`. The chart wires the pipelines and authentication, so no collector configuration file or additional collector deployment is required. Spanmetrics export is independent of the existing OTLP export and Jaeger. Spans and native application metrics continue to use `otel.exporter`. Locally generated spanmetrics use only `otel.spanMetrics.exporter`, avoiding duplicate aggregation at the console gateway. Jaeger can also be enabled with `otel.jaeger.enabled: true`. ### CloudWatch metrics and signed console export Merge this example into your deployment's existing values. Replace the Region, deployment ID and IAM role ARN with your own values: ```yaml theme={null} compute: deploymentId: 11111111-1111-4111-8111-111111111111 otel: enabled: true replicas: 1 traces: sampler: always_on samplerArg: "1" exporter: endpoint: https://console.textql.com/otlp deploymentAuth: enabled: true privateKeySecretName: deployment-secrets spanMetrics: enabled: true exporter: mode: aws aws: region: us-west-2 serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/ ``` The named Kubernetes Secret must contain `deployment.private.key` for the identity registered with the console. Signed console export does not support `global.secretsMode: nativeExternalSecrets`. If you only need the local metrics destination, omit `otel.exporter` and `compute.deploymentId` from this example. Associate the collector's service account with a dedicated IAM role through IRSA using the annotation above, or EKS Pod Identity without that annotation. Grant the role `cloudwatch:PutMetricData` on `*`. AWS mode signs requests using SigV4 and the collector's AWS credentials. The Region defaults to `global.aws.region` when `otel.spanMetrics.exporter.aws.region` is unset. See [AWS ingestion permissions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/metrics-otel-send.html). Confirm native OTLP metrics ingestion and PromQL queries and alarms are available in your AWS Region. Allow DNS and HTTPS access to `monitoring..amazonaws.com`, plus the endpoints needed by your credential mechanism. Private clusters can use a CloudWatch interface VPC endpoint with private DNS and security groups allowing HTTPS from the collector. See [AWS private connectivity](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-and-interface-VPC.html). Signed console export also requires HTTPS access to `console.textql.com`. ### Other OTLP metrics destinations Use `mode: otlp` with any OTLP/HTTP base endpoint. The exporter appends `/v1/metrics`. For example: ```yaml theme={null} otel: enabled: true spanMetrics: enabled: true exporter: mode: otlp endpoint: https://metrics.example.com headers: Authorization: "Bearer ${env:METRICS_TOKEN}" extraEnv: - name: METRICS_TOKEN valueFrom: secretKeyRef: name: metrics-credentials key: token ``` Exporter headers are stored in a ConfigMap. Use Secret-backed environment references for credentials, as shown above. ### Sampling, aggregation and rollouts Use `otel.traces.sampler: always_on` for complete counts of emitted spans. Sampling before aggregation reduces observed counts and can affect error and latency measurements. Enabling all spans increases telemetry volume. Metrics count spans, so filter to a specific operation or span kind when measuring requests. Spanmetrics requires exactly one collector replica. The chart uses `Recreate` rollouts while it is enabled. Upgrades briefly interrupt collection, and in-memory export queues do not survive restarts. Aggregation defaults to a 30-second flush interval, a cardinality limit of 10,000, and a resource metrics cache of 1,000. Adjust these through `otel.spanMetrics.flushInterval`, `otel.spanMetrics.aggregationCardinalityLimit`, and `otel.spanMetrics.resourceMetricsCacheSize`. The metrics exporter also supports `timeout`, `queue.numConsumers`, `queue.size` and `retry.maxElapsedTime`, matching the gateway exporter settings. ### Verify dashboards and alerts After applying the values through Helm or GitOps, generate representative application traffic and check collector logs for export errors. In CloudWatch Query Studio, select PromQL and query: ```promql theme={null} {"traces.span.metrics.calls"} ``` The duration histogram is `traces.span.metrics.duration`, measured in milliseconds. Inspect emitted labels before choosing filters and include namespace or environment filters when sharing an AWS account across deployments. For example, this query shows P95 duration in milliseconds for inbound server spans in the compute service: ```promql theme={null} histogram_quantile( 0.95, sum(rate({ __name__="traces.span.metrics.duration", "service.name"="compute-engine", "span.kind"="SPAN_KIND_SERVER" }[5m])) ) ``` Save the queries as dashboard widgets. To create an alarm, add a comparison against your chosen threshold, set an evaluation interval and pending period, and connect your SNS notification topic. See [CloudWatch PromQL queries](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-PromQL-Querying.html) and [PromQL alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Create_PromQL_Alarm.html). Validate a controlled error and notification delivery before relying on the alarm. Monitor collector availability separately so an export outage is visible when application metrics stop arriving. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.