Nutanix Enterprise AI on NKPPart 1: architecture, licensing and sizing

Summary: Nutanix Enterprise AI (NAI) turns a Kubernetes cluster into an inference platform with an OpenAI-compatible API, per-application keys, a model catalog and auditing. Before installing it, I read the 2.8.1 User Guide and the two Nutanix Validated Designs that cover it. Five points shaped my plan: the license is separate from NKP, with a 2-day trial (the key goes in on install day); the guide mentions NKP 2.18, and the 2.19 catalog already ships NAI 2.8.1; the control plane needs 30 vCPU and 60 GB on top of inference; the NVDs put NAI on a dedicated cluster; and with no GPU, the lab runs inference on CPUs without AMX. This is part 1 of 4.

The series

Part	Topic
1	NAI: architecture, licensing and sizing (this post)
2	Nutanix Files for Kubernetes: the RWX NAI requires
3	From the catalog to the first token: dedicated cluster, NAI 2.8.1 and license
4	CPU, GPU and MCP: NAI in a hybrid architecture

The lab: a 3-host Nutanix HCI cluster, Prism Central and AOS 7.6, NKP 2.19 with a management cluster and a services cluster. No GPU in the hosts. This part is the reading and the planning; nothing is installed yet. It is the part I wish I had read before opening an NAI project at a customer.

---

What Nutanix Enterprise AI is

NAI is a Nutanix platform for serving AI models inside Kubernetes, with governance. In practice, it delivers:

Feature	What it is
Local endpoints	A Hugging Face model or an NVIDIA NIM published as an OpenAI-compatible API, running on GPU or CPU
Validated model catalog	Models tested by Nutanix, with CPU, memory and vLLM parameters filled in automatically
API keys	One key per application, with activation, deactivation and auditing
Unified Endpoints	A single endpoint that spreads traffic across local models and external providers (OpenAI, Anthropic, Bedrock, Azure OpenAI, Vertex, Gemini, Mistral, Cohere or another NAI), with rate limiting
MCP Servers	Local or remote MCP servers for agents, with credentials and client keys
NAI Labs	Sample applications: Chat, Talk to My Data and an agent
Governance	IAM with LDAP, SAML and policies, audit events, model security scanning, usage metrics

The current version is 2.8.1, and it already shows up in the NKP 2.19 catalog, on the AI Applications tab:

How the pieces fit

Layer	Components	What it does
Ingress	Envoy Gateway (with Gateway API)	Receives HTTPS calls, validates the key, applies limits
Control plane	nai-api, UI, IAM, PostgreSQL, ClickHouse, Valkey, OpenTelemetry	Catalog, keys, users, metrics and auditing
Inference	KServe, LeaderWorkerSet, vLLM or NIM	Brings each model up as a service, with one or more replicas
Data	RWX volume (Files) and RWO volumes	Model weights and platform data

---

Licensing: separate from NKP, with a 2-day trial

Editions

Edition	What it includes
NAI Standalone (Pro)	NAI only, on any CNCF Kubernetes: NKP, EKS, AKS, GKE
NAI-NKP (Pro or Ultimate)	NAI + NKP in one bundle
NAI Full-Stack / GPT-in-a-Box (Pro or Ultimate)	NAI + NCI + NKP + Unified Storage (+ NDK and NDB)

The metric

NAI has three licenses, and you apply the one that matches your use:

License	Metric	When to use
CPU	vCPUs used by inference endpoints	Cluster without GPU (the lab’s case)
GPU	GB of GPU memory, across all GPUs	Cluster with GPUs
Agent Gateway	Active access keys on Unified Endpoints and MCP	Gateway only, no local inference

One detail that makes life easier: the CPU license counts what the endpoints use, not the node size. The NAI control plane is not counted.

The clock

Rule	What the 2.8.1 User Guide says
Trial	2 days after installation
POC license	30 days (requested from Nutanix)
After expiry	30-day grace period; then endpoints hibernate
When to apply CPU or GPU	Only on first installation, not on an upgrade
Switching	CPU → GPU with no prerequisite; to Agent Gateway, hibernate endpoints first

In the lab, licensing looks like this:

License	Status in the lab
NKP Ultimate	400 vCPU applied
NAI Pro	600 available, not applied yet (Part 3)
NUS Pro	50 TiB + 1 TiB included with NCI Ultimate (used in Part 2)
NDK	Included with NKP Ultimate, unlimited
Flow Network Security	Included with NCI Ultimate (shown as NCI Security)

---

What the 2.8.1 User Guide requires

Platform

Requirement	What the guide says	In the lab
Kubernetes	Version 1.35	1.36.2
NKP	“You can install NAI 2.8.1 only on NKP 2.18”	NKP 2.19
Node image	Ubuntu	Rocky Linux 9.8
Other platforms	EKS, AKS and GKE; air-gapped only on NKP	—
Architecture	x86 only	x86 (the Nutanix hosts)
GPU	NVIDIA, with GPU Operator and the nvidia.com/gpu:NoSchedule taint	None

The x86 requirement has a consequence in the lab: the NVIDIA GB10 GPU server we have is ARM, and cannot be an NAI node. It joins in Part 4 by another route.

Network, DNS and certificate

Requirement	Detail
LoadBalancer	External IP for Envoy (on NKP, MetalLB)
FQDN	A DNS name that resolves to that IP
Certificate	CA-signed; NAI only accepts HTTPS. For testing, there is a self-signed option through cert-manager

Storage

Item	Minimum	For
Block (RWO)	150 GiB	PostgreSQL (40), ClickHouse (50 + 10), NAI Labs (20), nai-api (20), Valkey (8), audit (1)
NFS (RWX)	2 TiB, storage class with Immediate	Models
NFSv4 export	One share	Importing custom models
S3	1 TiB (optional)	Another way to import models

NKP ships with RWO storage only. RWX is the whole subject of Part 2.

Dependencies

The guide installs them with Helm; the NKP 2.19 catalog already ships almost all of them:

Component	Version in the guide	In the NKP 2.19 catalog
Envoy Gateway + Gateway API	1.8.1	NAI Envoy Gateway 1.8.2
KServe	0.19.0 (RawDeployment mode)	0.19.0
LeaderWorkerSet	0.8.0	0.8.0
OpenTelemetry Operator	0.114.1	0.114.1 (Observability category)
CloudNativePG	NKP app	NKP app
Prometheus Monitoring	NKP app, minimal install	already present
cert-manager	ships with NKP	already present
NVIDIA GPU Operator	only with GPUs	not applicable

You also need a Docker Hub access token generated in the Nutanix portal to pull the NAI images.

---

Sizing: the control plane comes first

The guide’s minimum

The guide assumes a cluster dedicated to NAI and counts only the control plane; inference is added on top.

Nodes	Count	vCPU	Memory	Disk
Kubernetes control plane	3	4	16 GB	150 GB
NAI workers	3	10	20 GB	150 GB
Total	6	42	108 GB	—

The heaviest components:

Component	vCPU	Memory
nai-api	5.1	7.1 GiB
Envoy (ingress gateway)	5.1	10.3 GiB
ClickHouse (observability)	4	8 GiB
OpenTelemetry Collector	4	2 GiB
UI	2	2 GiB
PostgreSQL	2	2 GiB

Profiles

Profile	Capacity	Workers
Default	300 concurrent requests, 100 keys	3 × 10 vCPU / 20 GB
c1k_k200	1,000 requests, 200 keys	3 × 60 vCPU / 90 GB
c5k_k1k	5,000 requests, 1,000 keys	3 × 90 vCPU / 150 GB

Inference, on top

The guide’s reference for a 1-GPU endpoint:

Engine	CPU	Memory
NVIDIA NIM	12 cores	32 GiB
vLLM	8 cores	16 GiB (1 GPU) to 128 GiB (8 GPUs)

---

And without a GPU?

NAI supports CPU endpoints: when creating the endpoint, Acceleration Type → CPU, which shows up for models that support CPU execution. With a 4th-generation Xeon or newer, there is the Optimise endpoint to run on Intel AMX option, which uses the processor’s matrix acceleration.

The lab host:

Item	Value	Impact
CPU	2 × Xeon Silver 4210 (Cascade Lake, 2nd gen), 2.2 GHz, 20 cores per host	No AMX; has AVX-512 and VNNI. CPU inference works, at lab performance
Memory	376 GiB per host, 3 hosts	The lab’s real limit
Storage	Hybrid: 2 HDD + 1 SSD per host	Slower model loading when data lands on HDD

Candidates from the validated catalog for CPU, by size:

Model	Type	Size
ibm-granite/granite-embedding-107m-multilingual	Embeddings, 12 languages	~2 GiB
cross-encoder/ms-marco-MiniLM-L6-v2	Reranker	~4 GiB
google/gemma-3-270m-it	Text	~10 GiB
meta-llama/Llama-3.2-1B-Instruct	Text (gated on Hugging Face)	~10 GiB

Which of them show the CPU option, only the UI will tell.

---

What the Nutanix Validated Designs say

I read two: NVD-2057 (Nutanix Enterprise AI Design, August 2026) and NVD-2115 (GPT-in-a-Box with NKP, January 2026).

Validated versions × lab

Component	NVD-2057	NVD-2115	Lab
Prism Central / AOS	7.5.1.6	2024.3.0.1 / 7.0	7.6 / 7.6
NKP	2.18.0	2.14	2.19
NAI	2.7	2.0	2.8.1
Files	5.2.2.1	5.1	5.4.0.1

The lab is ahead of everything that was validated. NVD-2057 itself says it is one example of a supported configuration, not the only one.

Patterns that apply to any customer

Topic	What the NVD does
NAI cluster	Dedicated, in its own workspace. In NVD-2057, an nai-prod and an nai-stage
Control plane	CPU-only VM nodes on the management cluster; GPUs on Ubuntu 24.04 bare metal
NAI worker pool	NVD-2115: 12 vCPU / 40 GB / 100 GB
Database	External PostgreSQL, via NDB, one per NAI instance
Certificates	Private CA issuing through cert-manager, 1-year validity
Workspaces	gateway, prod, stage, qa and dev; NAI Labs only in qa and dev
High availability	2 replicas of nai-api and Envoy in gateway, prod and stage
Observability	Central stack, vLLM ServiceMonitor and Envoy dashboards
Backup	Velero to Nutanix Objects, with replication

Token as a Service

NVD-2057 describes NAI as a token platform: a single OpenAI-compatible API, with token quotas per tier (Free, Standard, Enterprise, Premium), that teams consume without seeing any GPU. In front of everything, the Agent Gateway spreads traffic between production and staging:

Model	Role	Production : staging
openai/gpt-oss-120b (MoE)	Architect: plans	75 : 25
google/gemma-4-26B-A4B-it (MoE)	Developer: writes the code	75 : 25
google/gemma-4-31B-it (dense)	Reviewer: checks	90 : 10
Embeddings, reranker, Llama Guard	RAG support and safety	50 : 50

The validated design: 16 nodes with 128 RTX PRO 6000 Blackwell GPUs, for about 1,750 concurrent users. Small models run on MIG 1g.24gb slices (4 per GPU); large ones on full GPUs.

KV cache: what limits users

Each active session reserves GPU memory for the KV cache. The NVD uses these numbers:

Usage profile	Context	KV cache per session
Chat and coding assistant	8K tokens	~0.6 GB
RAG and structured extraction	17K tokens	~1.5 GB
Summarization, research, agents	106K tokens	~8 GB

This math sets how many users fit in a replica, and it is the first question when sizing GPUs at a customer.

---

The lab’s decision: a dedicated cluster

With the guide and the NVDs in hand, I compared two options. The limit is memory: with 3 hosts of 376 GiB, I want everything to keep running if one host fails (N+1), which leaves about 752 GiB usable. The lab already uses ~591 GiB, including the Files from Part 2.

	A: node pool on the services cluster	B: dedicated cluster
Shape	+3 workers with 12 vCPU / 40 GiB	1 control plane 4/16 + 3 workers 12/40, own workspace
Memory added	120 GiB	136 GiB
Projected total	~711 GiB (94.5% of N+1)	~727 GiB (96.7% of N+1)
Follows the NVDs	No	Yes
Ubuntu nodes	No (existing cluster on Rocky)	Possible, if 2.19 offers the image
Isolation	Shares the cluster with other services	Can be deleted without affecting anything

I chose B. It costs 16 GiB more, follows the Nutanix design, isolates the experiment and allows the Ubuntu image the guide asks for. The trade-off: a single control plane (acceptable in a lab) and almost no memory headroom while NAI is up.

---

What I learned

1. NAI is a separate license, apart from NKP Ultimate; the metric is inference vCPU (CPU) or GB of GPU memory.
2. The trial is 2 days, and the CPU or GPU license only goes in on first installation. Get the key from the portal the same day.
3. The 2.8.1 guide mentions NKP 2.18. On 2.19, the path is the catalog app, compatible with >=2.18 <2.20 per the app’s own annotation.
4. The control plane needs 30 vCPU and 60 GB on 3 workers, before any model.
5. NAI needs RWX. NKP’s default storage class is RWO; RWX comes from Nutanix Files (Part 2).
6. An FQDN and a CA certificate are mandatory. Without internal DNS, that becomes a project prerequisite.
7. The NVDs put NAI on a dedicated cluster, with the control plane on VMs and the GPUs on bare metal.
8. Without a GPU, NAI is for governance, embeddings and small models. With AMX, CPU performance improves; my lab hardware does not have it.

Pre-sales checklist

• Edition defined: Standalone, NAI-NKP or Full-Stack
• Metric estimated: inference vCPU or GPU GB
• NKP version checked against the NAI app’s nkp-version-support annotation and the compatibility matrix
• Dedicated cluster sized: control plane (30 vCPU / 60 GB) + inference
• GPU: model, count, MIG or full GPU; or CPU with AMX
• KV cache estimated from the usage profile (chat, RAG or agents)
• Storage: RWO 150 GiB, RWX 2 TiB (Files), export for custom models, optional S3
• Network: LoadBalancer IP, FQDN and CA certificate
• Docker Hub token and a Hugging Face account (gated models)
• License key generated on installation day

What I haven’t tested yet

• GPU endpoints in NAI. The lab hosts have no GPU, and the GB10 GPU server is ARM, while NAI requires x86; it joins in Part 4 by another route. The GPU inside NKP itself is the lab’s next step.
• The Intel AMX-optimized endpoint option. It needs a 4th-generation Xeon or newer, and the lab hosts are Cascade Lake (2nd gen).
• The NAI cluster with 3 control planes, as in the guide’s table. In the lab, the dedicated cluster has a single control plane, which is enough for a lab.

References

Nutanix

• Nutanix Validated Design: Nutanix Enterprise AI (NVD-2057)
• Nutanix Cloud Platform: software options and licensing
• Nutanix Enterprise AI: FAQ
• Nutanix Enterprise AI Guide 2.8 (Nutanix portal), requirements, licensing and NKP deployment sections
• NVD-2115: Nutanix GPT-in-a-Box with Nutanix Kubernetes Platform Design (Nutanix portal)

Independent

• Nutanix Enterprise AI 2.8 and NKP 2.19 (StorageReview)