Nutanix Enterprise AI on NKPPart 3: from the catalog to the first token

Summary: in this part Nutanix Enterprise AI (NAI) leaves the drawing board. I create a dedicated NKP cluster, enable the seven catalog dependencies, install NAI 2.8.1 with a certificate from the internal CA and apply the license on the same day, because the trial lasts only 2 days. Then I import two models from Hugging Face, one validated by Nutanix and one custom, and get the first endpoint answering on CPU, with an API key and an OpenAI-compatible API. I gathered here the four prerequisites and configuration details that make the install go smoothly, with the cause and the adjustment for each. The result: 0.5 s to first token and 4.5 tokens/s on a 2019 Xeon, and what explains that number on the lab hardware. This is part 3 of 4.

Where we are

The series follows the rollout of Nutanix Enterprise AI in a lab with NKP 2.19 and a Nutanix HCI cluster without GPUs.

Part 1 settled the design (a dedicated cluster, as in the Nutanix Validated Designs) and Part 2 delivered the nai-nfs RWX storage class. Now comes the real installation, in a single session, end to end.

The roadmap in one table

1. A cluster just for NAI

Both Nutanix NVDs for NAI place the platform on a dedicated cluster with its own workspace. I followed the same path: workspace ws-nai and a cluster with 1 control plane of 4 vCPU / 16 GiB and 3 workers of 12 vCPU / 40 GiB, all on the NKP 2.19 Ubuntu 24.04 image, which is the one in the guide.

From Create to nodes Ready took 3 min 33 s. The cluster came up with Kubernetes 1.36.2.

Two fixes every new cluster needed

Both had already shown up in the lab’s first workload cluster, and they came back here:

2. The seven dependencies

The NAI catalog app lists its dependencies in the required-dependencies annotation, and the Enable screen shows them too. On NKP 2.19 they were:

All seven were ready in about 7 minutes. NKP 2.19 compatibility is in the app’s own annotation: nkp-version-support: >=2.18 <2.20.

3. Before Enable: what I read in the chart (and what I missed)

Before clicking Enable, I pulled the catalog artifact (nutanix-ai:2.8.1) and the nai-core chart to see what the app does. It paid off:

Certificate from the internal CA

For the browser to trust NAI without warnings, I issued the certificate from the lab’s internal CA. The chart accepts a cert-manager ClusterIssuer. I created the secret with the CA certificate and key, and the issuer:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: lab-ca
spec:
  ca:
    secretName: lab-ca   # tls secret in the cert-manager namespace

A test certificate for nai.lab.example.local came out signed by the CA and passed openssl verify.

4. The Enable

In the workspace, Applications → Nutanix Enterprise AI → Enable. The form has two configuration fields: one for the whole workspace and one per cluster.

I used only the cluster field. Hostname and certificate belong to this cluster; in the workspace field they would apply to any new cluster in the workspace, and NKP enables the app automatically on clusters that join later.

The final override (with the imagePullSecrets from the prerequisite below):

global:
  storage:
    storageClassName: nutanix-volume      # RWO
    storageClassNameRWX: nai-nfs          # the default is nai-nfs-storage
  imagePullSecrets:
    - name: nai-dockerhub                 # private nutanix/nai-* repositories
gateway:
  tlsSecretName: ingress-certificate
  certManager:
    selfSigned: false
    issuerRef:
      name: lab-ca
      kind: ClusterIssuer
      group: cert-manager.io
    dnsNames:
      - nai.lab.example.local

Prerequisite: the NAI Docker Hub token

On my first try I hadn’t created this credential, and a minute after Enable the pods in nai-system were in ImagePullBackOff: pull access denied … insufficient_scope. If you see that message, there are two points to check:

1. The timing of NKP’s credential injection. NKP injects the registry credential through a Gatekeeper mutation, which only acts when the pod is created in a namespace with the inject-registry-creds label. The nai-system namespace was born at 21:51:22, the pods at 21:51:25, and the label and secret only arrived at 21:52:21. The first pods were created before the credential.
2. NAI has its own credential. Even after recreating the pods, the pull was still denied: the NKP registry user pulls NKP images, and the nutanix/nai-* repositories are private to NAI. NAI uses the NAI Docker Hub access token, generated in the portal under Downloads → Nutanix Enterprise AI → Generate Access Token (a registry user of its own).

The adjustment: create a docker-registry secret with the NAI token in two namespaces, nai-system and envoy-gateway-system (NAI’s Envoy uses a private image), and point global.imagePullSecrets at it in the override.

When the first Enable doesn’t complete: Disable + Enable

With the configuration adjusted, NAI didn’t reinstall on its own. Since the first install of the operators hadn’t completed (because of the ImagePull), Flux went into uninstall to try again. The chart has a pre-delete hook that waits for the nutanix-ai HelmRelease to go away, and that one depends on the operators. The hook waits up to 30 minutes; the uninstall times out at 15.

The fastest path was Disable → cleanup → Enable. After Disable, a Valkey PVC, a ConfigMap created outside Helm and a webhook secret were left behind, all without data. I deleted the three and kept the namespace, the registry secrets and the issuer.

The second Enable: 8 minutes

NAI’s Envoy took the next free IP from the MetalLB pool (198.51.100.131) and the certificate was ready, issued by the lab CA. From the bastion, a curl --resolve with the lab CA validated the chain and got the redirect to the login page.

5. First login: EULA, Pulse and password

With the name in DNS (in the lab, for now, in /etc/hosts), the login page opens at https://nai.lab.example.local with a valid certificate.

User admin with the default password from the guide. In 2.8.1, the screen order is: EULA (name and company), then Pulse, then the password change (the guide shows a different order).

After Pulse: sign in again

After Save and Proceed on Pulse, my session showed 403 Access Denied. In the logs, the EULA and Pulse writes returned 200 and Envoy logged no 403: the session opened before the EULA just didn’t have the updated authorization attributes.

What to do: Back to login and sign in again. Then came the password change and the dashboard.

6. The license: on day one

NAI has a trial of only 2 days after installation, and the CPU or GPU license can only be added on the first install. So it comes right after the first login.

The path:

1. In NAI, Settings → Licensing → Add License shows the Kubernetes Cluster ID. It’s the UID of the cluster’s kube-system namespace (I checked from the bastion).
2. In the Nutanix portal, Licenses → Manage Licenses → Manage NAI/NKP License → NAI: enter the cluster name and UUID, pick NAI Pro (VCPU) and the quantity.
3. Generate the key and go back to NAI.

I chose 36 vCPU, the 3 workers × 12, the ceiling an endpoint can use in this cluster.

What about the Agent Gateway?

The license screen has a third type, Agentic Gateway License: “Required for accessing Unified Endpoints and MCP Connectors”. It’s a separate entitlement, metered in access keys. A vCPU key pasted into that field gives “Meter type mismatch”.

The lab doesn’t have the Agent Gateway entitlement yet. It’s left for Part 4.

7. A node pool just for inference

After Enable, CPU requests on the three workers were at 77%, 96% and 83%, with real usage around 5%. Only ~2.7 schedulable vCPU were left on the best node. The guide confirms why: the 3 workers of 10 vCPU in the requirements table are only for NAI’s control plane, and inference resources come on top.

The solution was an nai-cpu node pool: 1 worker of 12 vCPU / 48 GiB, no taint (in NAI 2.8.1, CPU endpoints use default scheduling, with no toleration or nodeSelector).

From Add to node Ready: 1 min 40 s, with ~11.6 vCPU free.

8. Models: Hugging Face, validated and custom

A service account on Hugging Face

I created a lab account on Hugging Face instead of using my personal one: the token doesn’t depend on a person and the separation is clean. Two models to compare:

The token is of type Read. At a customer, prefer a fine-grained token with read access only to the repositories in use.

In NAI, Settings → Third Party Credentials → Add Credential, platform Hugging Face. The screen warns that there’s room for only one Hugging Face token per NAI.

The validated model

Models → Import Models → From Hugging Face Model Hub lists 52 validated models.

The status goes through Pending → Waiting for storage. Behind the scenes, NAI creates a Model object and an RWX PVC on nai-nfs, Files creates the share and a job downloads the model. Total time: ~1.5 min.

Governance: NAI starts closed (deny by default)

With Gemma ready, NAI showed the notice “The usage of this model has been restricted by the administrator”. That’s governance at work: on a fresh install, Model Access Control starts with no models allowed (0 of 52 from Hugging Face and 0 of 32 from NGC), with import by URL and manual upload off, and the administrator allows what will be used.

In the lab I opened up the catalogs and both toggles. One detail: on NGC, Allow all validated allows 27 of 32. The other 5 are in the catalog without the validated badge, and only get in through Allow specific models.

The custom model, by URL

On the same Hub screen, the Import using Model URL link accepts any Hugging Face repository. NAI asks for explicit confirmation:

Import in 59 s (the share was ready in 3 s and the processor image was already cached).

The Model object showed a difference worth knowing:

9. The first endpoint

Pick a model with a CPU profile

Local Endpoints → Create Endpoint with Gemma, and the screen showed: “No supported accelerator types available”.

The cluster was ready: NFD detected AVX-512 on every node and NAI’s inventory saw 11.6 free vCPU on nai-cpu. What defines the option is the model profile: in the 2.8.1 catalog, Gemma 3 270M is validated for GPU. With the custom Qwen3, the CPU option appears (and Intel AMX is greyed out, because the lab’s Xeon is 2nd generation).

The configuration

vLLM engine, Nutanix image v0.26.0. The screen warns that vLLM is community-supported and shows other image sources.

For a custom model, you set the size. I used:

From Pending to Active: 8.5 minutes

Behind the scenes, NAI creates a KServe InferenceService with the vLLM pod on the nai-cpu node, with requests equal to limits (8 CPU / 16 GiB).

The endpoint page shows the OpenAI-compatible URL. The model field takes the endpoint name.

10. The first token

From the UI

The Test button answers with Qwen3’s thinking mode on (the model’s default): the answer starts with the reasoning in English and then comes the reply in Portuguese. Through the API, you can turn it off with enable_thinking: false.

From the API

From the bastion, with the API key read from a prompt without echo:

curl https://nai.lab.example.local/enterpriseai/v1/chat/completions \
  -H "Authorization: Bearer $NAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-06b-cpu",
    "messages": [
      {"role": "system", "content": "Always answer in Brazilian Portuguese."},
      {"role": "user", "content": "Explain in 3 sentences what Nutanix Enterprise AI is."}
    ],
    "max_tokens": 200,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

The answer came in the requested format and generic in content, as you’d expect from a 0.6B model.

Why 4.5 tokens/s

I measured instead of guessing:

The first one can be improved with VLLM_CPU_OMP_THREADS_BIND matching the pod’s cores. In NAI 2.8.1, the endpoint’s advanced configuration offers Engine Arguments and KV Cache Aware Routing; an environment variable like this one goes on the KServe InferenceService. Part 4 shows the step by step and the gain: +40% tokens/s. The second comes from the lab hardware. For comparison, the lab’s GPU appliance serves a 30B model with NIM at ~66 tokens/s.

The numbers

What I learned

1. Read the chart before Enable. The defaults come with a self-signed certificate and a generic storage class and hostname: set them for your environment.
2. NAI has its own registry credential. Create the nai-system namespace and the secret with the NAI Docker Hub token before Enable, including in envoy-gateway-system.
3. If the first Enable doesn’t complete, Disable + cleanup + Enable. It’s the fastest path, without waiting for Flux.
4. License on day one, CPU type. The trial is 2 days, and the dialog opens on GPU.
5. Size the workers for NAI’s control plane. The default requests ask for ~21 vCPU; reserve a node pool for inference.
6. Governance from day one. Model Access Control comes as deny by default; allow models by list.
7. Check the model’s accelerator type before picking the demo model: validated for GPU doesn’t imply a CPU profile.
8. In my lab, CPU without BF16/AMX is for demos; for performance, GPU or CPU with AMX. And match vLLM’s threads to the pod limit.

Checklist: NAI 2.8.1 on NKP

• Dedicated cluster with the Ubuntu image; CoreDNS and cluster-info-configmap fixed
• RWX storage class ready (Immediate)
• The 7 catalog dependencies enabled
• ClusterIssuer for the internal CA (or accept the self-signed one)
• NAI Docker Hub token and secret in nai-system and envoy-gateway-system
• Override in the cluster field: storage classes, imagePullSecrets, issuer and dnsNames
• DNS for the NAI name pointing to the Envoy IP
• First login (EULA → Pulse → password); if a 403 shows up after Pulse, sign in again
• CPU license applied the same day
• Node pool for inference
• Hugging Face token (Read or fine-grained) and gated licenses accepted
• Model Access Control opened by list
• Endpoint with API key; test without key (401) and with key (200)

What I haven’t tested yet

• GPU endpoints in NAI. The lab cluster has no GPU, and the license applied was the CPU one. The GPU inside NKP itself is the lab’s next step.
• CPU inference with BF16 or AMX. The lab hosts are Cascade Lake, with neither AVX512_BF16 nor AMX, and vLLM runs in FP32.
• vLLM thread tuning (VLLM_CPU_OMP_THREADS_BIND): it went to Part 4, which shows the step by step and the +40% gain.
• The Agent Gateway (unified endpoints and MCP): the lab did not have that entitlement yet at this stage, and it went to Part 4.

References

Nutanix

• Nutanix Enterprise AI Guide 2.8.1 (Nutanix portal): requirements (pp. 13–17), NKP deployment, Docker Hub access token (p. 28), login (p. 132), licensing (pp. 173–178), Model Access Control (p. 194), model import (pp. 197–202), endpoints (pp. 225–246)
• Nutanix Validated Design: Nutanix Enterprise AI (NVD-2057)

Projects and community

• Cluster API: label sync between Machine and Node
• vLLM: installation and variables for CPU
• Hugging Face: User access tokens
• cert-manager: CA Issuer