Skip to content

Pixee Enterprise Server Frequently Asked Questions

Below are list of frequently asked questions related to Pixee Enterprise Server installation, configuration and operation. If your question is not answered below, please let us know!

How do I send Pixee a support bundle?

The fastest path is the Generate Support Bundle entry under your avatar menu in the Pixee Platform UI — the bundle is generated in-cluster and uploaded to Pixee automatically. A cluster-shell fallback is available for situations where the UI itself is down. See Support Bundles for both flows.

How do I view the pods and logs?

For detailed instructions on viewing pods and logs, see the Observability - Logs & Debugging section.

How do I verify that the GitHub App is successfully sending events to pixee-enterprise-server?

To confirm that events are being properly transmitted from the GitHub App to pixee-enterprise-server, you can review the GitHub App's event log. This log can be accessed through your GitHub App's settings under the "Advanced" section.

GitHub provides detailed documentation on viewing webhook deliveries here.

The event log displays a comprehensive list of all events sent from the GitHub App, along with their respective response codes.

How do I retrieve a file out of SeaweedFS?

Pixee uses SeaweedFS to store CodeTF files and we may ask for those files to help debug issues related to Pixee.

CodeTF files are an interchange format used by the Pixee platform. They represent the results of fixes and therefore can be useful for debugging fix-related issues.

SeaweedFS ports are not exposed by default. To access the SeaweedFS web UI, first create an SSH tunnel from your local machine:

ssh -L 8888:localhost:8888 pixee@<ip>

Then in the SSH session, use the kubectl port-forward command to forward the port:

kubectl port-forward -n kotsadm svc/pixee-enterprise-server-seaweedfs-filer 8888:8888

This will forward the SeaweedFS web UI to port 8888. Remember to kill these commands once you are done. You can then access the SeaweedFS web UI by visiting http://localhost:8888 in your browser to browse and download files.

How do I update the Pixee Enterprise Server embedded cluster admin console domain name and TLS certificate?

If you have registered a domain name for your Pixee Enterprise Server and have a valid TLS certificate and want to replace the self-signed certificate used by the admin console during the initial installation, follow these steps: 1. (optional) Retrieve the TLS certificate and private key from the cluster (skip this step if you already have the cert and key available) 

# From the Pixee Enterprise Server virtual machine
sudo ./pixee shell
kubectl get secret pixee-platform-tls-requested -o jsonpath='{.data.tls\.crt}' -n kotsadm | base64 --decode > cert.pem
kubectl get secret pixee-platform-tls-requested -o jsonpath='{.data.tls\.key}' -n kotsadm | base64 --decode > key.pem
Download the retrieved TLS certificate and private key locally 
# From your local machine
scp pixee@<vm ip address>:/home/pixee/cert.pem ./ 
scp pixee@<vm ip address>:/home/pixee/key.pem ./
2. Prepare the existing admin console tls secret to be updated 
# From the Pixee Enterprise Server virtual machine
sudo ./pixee shell
kubectl -n default annotate secret kotsadm-tls acceptAnonymousUploads=1 --overwrite -n kotsadm
PROXY_SERVER=$(kubectl get pods -A | grep kurl-proxy | awk '{print $2}')
kubectl delete pods $PROXY_SERVER -n kotsadm

ℹ️ The acceptAnonymousUploads annotation will be removed after completing the update in the next step

  1. In a browser visit https://<vm ip addresss>:30000/tls and follow the prompts to set the admin console domain name to match the existing Pixee Enterprise Server and upload the TLS certificate and private key
  2. When completed you should be able to browse to the admin console at https://<domain name>:30000

Clicking "Continue" on the login page redirects to an internal address and times out

The browser is sent to something like https://10.0.90.4/authentik/application/o/authorize/?…&redirect_uri=https://10.0.90.4/… and fails with ERR_CONNECTION_TIMED_OUT.

Pixee builds every browser-facing OIDC URL from the Domain name setting, so the address in that redirect is your configured Domain name. Seeing an internal address there means Domain name was set to the cluster's own address or to the origin address your reverse proxy or CDN was configured with, rather than the public hostname browsers use. Enabling forwarded headers does not help — these URLs are fixed at deploy time, not rebuilt from X-Forwarded-*.

Fix it in two places at once:

  1. Set Domain name (Network) to the public hostname.
  2. Configure your proxy to send that same hostname in the Host header — for Azure Front Door this is the origin's Origin host header; see CDNs and Proxies That Rewrite the Host Header.

Changing only Domain name makes the gateway return 404 Not Found for every request, because it serves only the Host that matches Domain name. Changing only the proxy leaves the redirects pointing at the internal address.

If step 2 is not available to you — the proxy belongs to another team, or the product gives you no way to preserve Host — set Domain name to the public hostname anyway and add the address the proxy sends to Additional accepted hostnames, which stops the 404. Validate sign-in afterwards: see When You Cannot Change the Proxy.

After the change, confirm the platform picked it up:

kubectl -n kotsadm get deploy pixee-enterprise-server-platform -o jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_PIXEE_AUTHORIZATION_URI")].value}{"\n"}'

If that still prints the old host, the config change has not rolled out yet.

I cannot sign in to Pixee. How do I recover access?

Every login goes through the embedded Authentik identity provider, so what to do depends on the symptom: being sent to an identity provider you cannot use is a different problem from an Authentik that cannot reach its database. See Recovering Access When You Cannot Sign In, which has a tab per symptom, including a last-resort break-glass that starts the platform with authentication disabled.

How do I troubleshoot Authentik blueprint errors?

Pixee Enterprise Server uses Authentik blueprints to declaratively configure the OIDC provider, application, and brand settings. Blueprints are automatically discovered and applied by the Authentik worker pod on a periodic cycle (~60 seconds). If a blueprint fails to apply, it enters an error state and will not be retried until the issue is resolved.

Every Login Fails With 400 Invalid client identifier

This is the blueprint failing, not the reverse proxy — and it is worth checking first, because everything else looks healthy. Authentik answers its liveness and readiness endpoints, the admin console reports the app ready, and a support bundle's Authentik health check passes. But the Pixee OIDC Provider blueprint never applied, so the pixee OAuth2 client the platform authenticates against does not exist.

Two things confirm it in seconds, before you look at any proxy:

  • The login page renders stock authentik branding rather than Pixee branding, and the pixee-admins group does not exist. Both come from the same blueprint.
  • The Authentik server log shows "event": "Invalid client identifier", "client_id": "pixee".

A blueprint applies atomically, so a single bad entry rolls back every unrelated entry alongside it — the OAuth2 providers, the group, and the brand together. Find the entry that failed in the worker log:

kubectl logs -n <namespace> -l app.kubernetes.io/name=authentik,app.kubernetes.io/component=worker --tail=2000 | grep -i "blueprint validation failed"

If the failure names a host such as accounts.google.com or login.microsoftonline.com, the cluster has no egress to a seeded identity provider's discovery endpoint. Current releases seed those federation sources disabled — Authentik only fetches the discovery document for a source that is enabled — so they validate with no network call at all and this cannot happen; upgrading is the fix. On a release from before that change (6.0.11 and earlier), either allow egress to that host on port 443, or edit configmap/pixee-authentik-blueprint to set the embedded source to enabled: false — the edit unblocks sign-in immediately but is reverted by the next deployment.

Force Re-apply the Blueprint

The quickest way to check for errors and fix them in one step is to force a re-apply. This will print any errors if the blueprint fails, or silently re-apply all settings if it succeeds:

# Find the worker pod
kubectl get pods -n <namespace> -l app.kubernetes.io/name=authentik,app.kubernetes.io/component=worker

# Force re-apply — prints errors on failure, silently succeeds otherwise
kubectl exec -n <namespace> <worker-pod> -- ak apply_blueprint mounted/cm-pixee-authentik-blueprint/pixee-oidc.yaml

If the command succeeds with no error output, the blueprint has been re-applied and the configuration should be current. If it prints errors (e.g., serializer validation errors or missing references), fix the root cause, deploy the fix, and run the command again.

Stale Configuration After Failed Deployments

If a deployment partially failed — for example, the main Helm chart failed to upgrade but the embedded chart succeeded — the blueprint ConfigMap may not have been updated. In this case, even a successful re-apply will use stale content. Verify the ConfigMap has the expected values:

kubectl get configmap pixee-authentik-blueprint -n <namespace> -o jsonpath='{.data.pixee-oidc\.yaml}'

If the content is outdated, resolve the failed deployment first so the ConfigMap gets updated, then re-apply.

Checking Blueprint Status

If you need to check whether the blueprint is in an error state or when it was last applied:

kubectl exec -n <namespace> <worker-pod> -- python3 -c "
import django, os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'authentik.root.settings')
django.setup()
from authentik.blueprints.models import BlueprintInstance
for bp in BlueprintInstance.objects.all():
    print(f'{bp.name} | status={bp.status} | last_applied={bp.last_applied}')
"

Note

This command prints Authentik initialization logs before the actual output. The data you need appears at the end.

Blueprint statuses:

  • successful — Applied without errors
  • error — Failed to apply; will not be retried automatically until the status is reset
  • unknown — Not yet processed or was manually reset

If a blueprint is stuck in error, reset it and re-apply:

# Reset the status so it can be re-applied
kubectl exec -n <namespace> <worker-pod> -- python3 -c "
import django, os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'authentik.root.settings')
django.setup()
from authentik.blueprints.models import BlueprintInstance
bp = BlueprintInstance.objects.get(name='Pixee OIDC Provider')
bp.status = 'unknown'
bp.save()
print('Blueprint status reset to unknown')
"

# Then force re-apply
kubectl exec -n <namespace> <worker-pod> -- ak apply_blueprint mounted/cm-pixee-authentik-blueprint/pixee-oidc.yaml

Deeper Investigation

When the blueprint re-applies successfully but something is still wrong (broken branding, login errors, incorrect redirects), you can inspect what Authentik actually has in its database and compare against what the blueprint declares:

kubectl exec -n <namespace> <worker-pod> -- python3 -c "
import django, os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'authentik.root.settings')
django.setup()

from authentik.brands.models import Brand
print('=== Brands ===')
for b in Brand.objects.all():
    print(f'  domain={b.domain} default={b.default} logo={b.branding_logo} favicon={b.branding_favicon} title={b.branding_title}')

from authentik.providers.oauth2.models import OAuth2Provider
print('=== OAuth2 Providers ===')
for p in OAuth2Provider.objects.all():
    print(f'  name={p.name} client_id={p.client_id} redirect_uris={p.redirect_uris}')

from authentik.flows.models import Flow
print('=== Flows ===')
for f in Flow.objects.all():
    print(f'  slug={f.slug} title={f.title} designation={f.designation}')
"

Other useful checks:

# Find the server pod
kubectl get pods -n <namespace> -l app.kubernetes.io/name=authentik,app.kubernetes.io/component=server

# Check for 404 errors (missing logos, broken media references)
kubectl logs -n <namespace> <server-pod> | grep '"status":404'

# Verify logo/favicon files exist on disk
kubectl exec -n <namespace> <server-pod> -- ls -la /data/media/public/

For OIDC or flow issues, update the domain/protocol settings in the Admin Console and redeploy rather than editing the database directly.

For more details, see the Authentik blueprint documentation.

What internet access is necessary for Pixee Enterprise Server?

Pixee Enterprise Server will need access to certain resources on the internet during installation/upgrade and normal operation. All optional resources can be disabled via config/values. If you need complete air gap support, please contact us. The resources are detailed below:

During installation/update:

Domain IP Addresses (if available) Notes
images.pixee.ai
proxy.replicated.com		 see https://github.com/replicatedhq/ips/blob/main/ip_addresses.json#L52-L57 Pixee container image registry
registry.pixee.ai
registry.replicated.com		 see https://github.com/replicatedhq/ips/blob/main/ip_addresses.json#L20-L25 Pixee Helm registry
github.com see api section at https://api.github.com/meta GitHub integration (optional)
api.openai.com N/A OpenAI integration (optional)
*.api.letsencrypt.org see Letsencrypt FAQ TLS certificate requests (optional)

During normal operation:

Domain IP Addresses (if available) Notes
distribution.pixee.ai
replicated.app		 162.159.133.41
162.159.134.41
2606:4700:7::a29f:8529
2606:4700:7::a29f:8629		 Metrics reporting
sentry.io
*.us.sentry.io		 35.186.247.156/32
34.120.195.249/32		 Crash reporting (optional)
github.com see api section at https://api.github.com/meta GitHub integration (optional)
api.openai.com N/A OpenAI integration (optional)