EKS Auto Mode: What It Actually Changes (and What It Doesn’t)

EKS Auto Mode: What It Actually Changes (and What It Doesn't)

What EKS Auto Mode is

EKS Auto Mode, generally available on December 1, 2024, shifts more Kubernetes infrastructure responsibility from you to AWS. You still run an EKS cluster in your AWS account, and your workloads still use the Kubernetes API, but AWS takes over much of the compute, storage, networking, node lifecycle, and core add-on management that platform teams usually wire together themselves.

For compute, Auto Mode uses Karpenter-style provisioning under the hood. When pods are unschedulable, Auto Mode provisions nodes that fit the workload’s requirements: instance family, size, architecture, capacity type, and availability zone. When capacity is no longer useful, it can consolidate and terminate nodes.

The important framing is this: Auto Mode is not “EKS without nodes.” It is EKS where AWS manages the node lifecycle more aggressively. You own the workloads, their scheduling requirements, their disruption behavior, and the operational consequences of those choices. AWS owns more of the infrastructure plumbing.


What it replaces

Before Auto Mode, running EKS in production usually meant choosing and operating several layers yourself:

Managed node groups: You chose instance types, defined scaling ranges, managed AMI updates, handled node draining, and configured Cluster Autoscaler or another scaling mechanism.

Self-managed Karpenter: More flexible than managed node groups, but you owned the Karpenter controller, IAM, NodePools, EC2NodeClasses, disruption settings, upgrades, and failure modes.

Fargate: AWS-managed compute per pod, with no node management, but no DaemonSets, a narrower workload compatibility envelope, and a different cost model.

EKS Auto Mode replaces a large part of that platform assembly with a managed model: declare workload intent and high-level compute constraints; AWS provisions and manages the EC2 instances behind it.


How it works in practice

You create or update an EKS cluster with Auto Mode enabled. The default setup can use AWS-managed built-in node pools. If you need more control, you create a NodeClass for Auto Mode infrastructure settings and a Karpenter NodePool for workload-facing scheduling constraints.

# NodeClass: EKS Auto Mode infrastructure settings for managed EC2 nodes.
apiVersion: eks.amazonaws.com/v1
kind: NodeClass
metadata:
  name: private-compute
spec:
  subnetSelectorTerms:
    - tags:
        kubernetes.io/role/internal-elb: "1"
  securityGroupSelectorTerms:
    - tags:
        aws:eks:cluster-name: prod-eks
  ephemeralStorage:
    size: "100Gi"
# NodePool: workload-facing constraints for nodes that Auto Mode may provision.
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    spec:
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: private-compute
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand", "spot"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: eks.amazonaws.com/instance-category
          operator: In
          values: ["c", "m", "r"]
  limits:
    cpu: "1000"
    memory: 1000Gi
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m

Those API groups are the Auto Mode-specific split documented by AWS: apiVersion: eks.amazonaws.com/v1, kind: NodeClass for Auto Mode node infrastructure, and apiVersion: karpenter.sh/v1, kind: NodePool for scheduling and capacity constraints.

The NodeClass is where you express AWS infrastructure placement and node-level defaults. The NodePool is where you express what kind of capacity is acceptable for workloads. Do not copy a self-managed Karpenter EC2NodeClass into Auto Mode; Auto Mode uses its own NodeClass API.

Auto Mode provisions nodes when pods are pending, consolidates when nodes are underutilized, and replaces nodes during maintenance or scale-down. AMI and node lifecycle updates are handled by AWS. AWS says Auto Mode AMIs are generally released weekly with CVE and security fixes, and Auto Mode nodes have a maximum lifetime of 21 days, which you can reduce. Your application still has to tolerate the disruption: a bad PodDisruptionBudget, strict affinity rule, or singleton stateful workload can still block or degrade a replacement.

Built-in components are managed differently than in a classic EKS build. AWS lists pod networking, service networking, cluster DNS, autoscaling, block storage, load balancer controller, Pod Identity agent, and node monitoring agent as Auto Mode capabilities. With Auto Mode compute, common add-ons such as Amazon VPC CNI, kube-proxy, CoreDNS, Amazon EBS CSI Driver, and EKS Pod Identity Agent become redundant for Auto Mode nodes, and the relevant controllers can run on AWS-owned infrastructure rather than as visible pods in your account. You can still install AWS Load Balancer Controller in an Auto Mode cluster when you need both models during migration, but AWS does not support directly migrating existing load balancers from AWS Load Balancer Controller to Auto Mode; use IngressClass or loadBalancerClass boundaries and plan blue-green migration. Treat this as a change in ownership, not as a reason to skip validation.

The workload support matrix is broader than Fargate, but not identical to self-managed nodes:

CapabilityAuto Mode status
EC2 SpotSupported through karpenter.sh/capacity-type requirements such as spot, on-demand, and reserved
Graviton / arm64Supported through kubernetes.io/arch: arm64 and supported Graviton instance families
GPU / acceleratorsSupported for documented accelerated families; Auto Mode manages NVIDIA, Trainium, and Inferentia drivers/device plugins for supported instance types
Windows nodesNot supported
DaemonSetsSupported as Kubernetes DaemonSets, but host-level assumptions must be validated against locked-down managed nodes

What you gain

Reduced operational surface. Node group management, AMI lifecycle, Cluster Autoscaler tuning, Karpenter controller upgrades, and a chunk of add-on wiring move out of your day-to-day scope.

Better provisioning shape by default. Dynamic provisioning is usually a better fit than fixed node group shapes. You get nodes that more closely match actual pod requirements instead of trying to pre-plan a small set of instance types.

Automatic node patching. AWS manages the node image and replacement flow. That reduces toil, but it also means your workloads need disruption policies that let AWS replace nodes safely.

Faster cluster bootstrapping. A new Auto Mode cluster can get to a usable production baseline faster than a hand-assembled EKS cluster with node groups, autoscaling, networking add-ons, storage drivers, and load balancer controllers.

Native Spot integration. Auto Mode can use Spot capacity through NodePool requirements, but you still need workload-level interruption tolerance: replicas, budgets, graceful shutdown, and queue semantics where relevant.


What you give up

Node-level access. Auto Mode nodes are intentionally locked down compared with traditional self-managed nodes. If your incident response process assumes SSH, SSM, manual package inspection, or ad hoc host changes, it needs to change.

Custom AMIs. You cannot treat the node image as your own artifact. AWS determines the operating system and AMI for Auto Mode managed instances; you cannot directly access the instance or install software on it. If your organization requires internally built, hardened, or certified AMIs, Auto Mode is likely blocked.

Unrestricted host agents. Kubernetes DaemonSets are supported, but they are the sharp edge. Some node agents work; others do not. Anything that assumes privileged host access, custom kernel modules, hostPath writes, IMDS access without hostNetwork, or low-level runtime integration needs a proof of compatibility.

Less tuning surface. You give up direct control over kubelet flags, container runtime configuration, bootstrap scripts, and arbitrary node setup. That is the point of the product, but it is also the boundary.

Different cost visibility. Managed node groups make capacity easier to reason about because you chose it up front. Auto Mode changes capacity dynamically, so cost control moves toward budgets, labels, reports, and workload-level resource hygiene.


EKS Auto Mode vs managed node groups vs Fargate

Auto ModeManaged Node GroupsFargate
Node managementAWS manages node lifecycleShared: AWS manages the group primitive, you manage capacity shape and many updatesAWS-managed per-pod compute
AMI updatesAutomatic through AWS-managed node replacementYou schedule and operate rolling updatesN/A
Instance selectionDynamic through NodePoolsYou choose instance types and scaling rangesNot exposed
Custom AMIsNo; AWS determines the AMIYesNo
DaemonSetsSupported, but validate host-access assumptionsYesNo
SSH / node accessRestrictedUsually available if you enable itNo
Spot supportYes, through capacity-type requirementsYes, with node group or Karpenter designNo; Amazon EKS does not support Fargate Spot
Cost modelEKS control plane + EC2 + EKS Auto Mode feeEKS control plane + EC2EKS control plane + Fargate pod pricing
Operational burdenLow for nodes, medium for workload compatibilityMediumLow for nodes, medium for compatibility
Right forDefault candidate for teams that do not need node customizationRegulated/custom node environments and mature platform teamsWorkloads that fit Fargate’s restrictions and want per-pod isolation
Hidden constraints / gotchasAWS controls node image and lifecycle; DaemonSets, privileged pods, hostPath, PDBs, topology rules, and node agents can block migrationYou still own AMI drift, autoscaler tuning, disruption handling, and capacity fragmentationNo DaemonSets, limited host-level integrations, different networking/storage constraints, and less flexibility for mixed workload shapes

The Auto Mode pricing

Do not model Auto Mode as “EC2 plus a generic percentage” unless you have pulled the actual rate for your region and instance mix. The official structure is:

Total EKS Auto Mode cluster cost =
  EKS control plane cost
  + normal EC2 cost for instances launched and managed by Auto Mode
  + EKS Auto Mode management fee on those managed EC2 instances
  + normal surrounding AWS costs: EBS, load balancers, data transfer, CloudWatch, etc.

EKS Auto Mode management fee =
  sum of Auto Mode-managed instance runtime
  x the regional EKS Auto Mode management rate for each EC2 instance type

The EKS Auto Mode fee is applied to the EC2 instances that Auto Mode launches and manages. It is billed in addition to the normal EC2 charge and in addition to the EKS control plane charge. AWS bills the Auto Mode fee per second with a one-minute minimum, and the charge is independent of whether the underlying EC2 capacity is On-Demand, Spot, covered by Reserved Instances, or covered by Compute Savings Plans.

Do not treat the fee as a contractual flat percentage. The official pricing page describes it as a management fee that varies by EC2 instance type, and AWS pricing data is regional. The public pricing example for US West (Oregon) shows c6a.2xlarge at $0.306/hour for EC2 plus $0.03672/hour for Auto Mode, c6a.4xlarge at $0.612/hour plus $0.07344/hour, m5a.2xlarge at $0.344/hour plus $0.04128/hour, and m5a.xlarge at $0.172/hour plus $0.02064/hour. Those examples equal 12% of the listed On-Demand EC2 rate, but the safe formula for real planning is: sum(instance-hours by instance type and region x published Auto Mode management rate).

The practical cost question is not “is there a premium?” There is. The useful question is whether the premium is lower than the engineering time, incident risk, and opportunity cost of operating node lifecycle yourself.

For small teams, the answer may be yes even if the raw bill increases. For high-scale, cost-sensitive platforms, the answer needs real data: compare current EC2 waste, bin-packing efficiency, Spot usage, interruption rate, and platform maintenance time against an Auto Mode pilot.


When to use EKS Auto Mode

Use Auto Mode if:

  • You run EKS on AWS and do not have a hard requirement to manage nodes yourself
  • You do not require custom AMIs or custom node bootstrap logic
  • You want to reduce the operations surface for node lifecycle management
  • You want Karpenter-like provisioning without operating Karpenter yourself
  • Your workloads are mostly stateless or disruption-tolerant
  • Your observability, security, and storage agents are compatible with Auto Mode

Stick with managed node groups if:

  • Your organization requires internally certified or hardened AMIs
  • You need specific kernel configuration, kubelet flags, bootstrap scripts, or host packages
  • You depend on privileged DaemonSets or host-level security tooling that Auto Mode cannot support
  • You are in a regulated environment where the node image supply chain must be owned internally
  • Your platform team already operates Karpenter well and values the extra control

Use Fargate if:

  • You specifically want per-pod compute isolation
  • Your workload does not need DaemonSets or host-level integrations
  • You accept Fargate’s scheduling, networking, storage, and observability constraints
  • You want to avoid managing EC2 capacity entirely for a narrow class of workloads

Migration from managed node groups

Migrating an existing cluster to Auto Mode is supported, but it is not a one-command operational migration. AWS supports enabling Auto Mode on existing clusters, but you must update the cluster IAM role permissions and trust policy, enable compute, block storage, and load balancing capabilities together, and meet required add-on versions when those add-ons are installed. AWS also calls out unsupported direct migrations for EBS volumes from the standard EBS CSI provisioner to the Auto Mode EBS CSI provisioner, existing load balancers from AWS Load Balancer Controller to Auto Mode, and clusters using alternative CNIs or other unsupported networking configurations. A conservative path looks like this:

  1. Enable Auto Mode on a non-production cluster running Kubernetes 1.29 or greater.
  2. Inventory workloads by scheduling assumptions: node selectors, affinities, tolerations, topology spread, PDBs, privileged mode, hostPath, local storage, and DaemonSet dependencies.
  3. Create or select the relevant Auto Mode NodeClass and NodePool resources.
  4. Move a low-risk namespace first by changing selectors, tolerations, or labels so pods land on Auto Mode nodes.
  5. Watch scheduling, replacement, load balancer behavior, persistent volume provisioning, logging, metrics, and security events.
  6. Taint old node groups to stop new scheduling once the pilot workloads are stable.
  7. Drain old nodes gradually and delete old managed node groups only after workload owners have signed off.

The hardest part is usually not enabling Auto Mode. It is discovering which workloads and platform agents quietly depended on a mutable node.


What breaks when you migrate

Auto Mode changes the node contract. The Kubernetes API still looks familiar, but the host underneath is no longer yours in the same way.

DaemonSets need a compatibility audit. Logging agents, metrics agents, service mesh node components, security scanners, CSI node plugins, and custom infrastructure daemons often assume host access. Datadog, Falco, custom CSI drivers, eBPF agents, file integrity tools, and in-house node agents should be tested explicitly rather than assumed compatible.

PodDisruptionBudgets can block AWS-managed maintenance. If every critical Deployment has maxUnavailable: 0, or singleton workloads have no safe disruption path, node replacement becomes harder. Auto Mode can manage nodes, but it cannot make an application disruption-tolerant after the fact.

nodeSelector and affinity rules can strand pods. Workloads pinned to old node group labels, instance types, capacity labels, zones, or custom AMI labels may never schedule on Auto Mode capacity. Replace legacy labels with stable requirements that Auto Mode can satisfy.

topologySpreadConstraints can become too strict. Auto Mode provisions capacity dynamically, but strict zone spreading plus narrow selectors can create unschedulable pods. Check whenUnsatisfiable, label selectors, and minimum domain assumptions.

Privileged pods and hostPath volumes are migration blockers until proven otherwise. Anything that needs /var/lib, /proc, /sys, container runtime sockets, kernel capabilities, or host networking deserves a separate test. Some patterns are fundamentally at odds with locked-down managed nodes.

Observability and security agents may lose host assumptions. Agents that expect direct node access, host package installation, kernel modules, eBPF privileges, or container runtime socket access can fail partially. The dangerous failure mode is not “pod CrashLoopBackOff”; it is silent loss of telemetry or enforcement.

Storage drivers must be reviewed. EBS integration is part of Auto Mode, but it uses the Auto Mode EBS CSI provisioner ebs.csi.eks.amazonaws.com, not the standard EBS CSI provisioner ebs.csi.aws.com. Custom CSI drivers, EFS patterns, snapshot controllers, and topology-aware storage classes should be validated. Pay particular attention to provisioner names, volume binding mode, encryption settings, and IAM assumptions.

Runbooks need rewriting. “SSH to the node and inspect X” is not a valid first response anymore. Incident procedures should move toward kubectl describe, events, logs, ephemeral debug containers where supported, cloud-side metrics, and vendor-supported diagnostics.


The honest assessment

EKS Auto Mode is a good default candidate for many teams running Kubernetes on AWS. The operational simplification is real: node provisioning, AMI updates, core add-on integration, and scaling behavior are areas where teams burn time and create incidents.

The constraints are also real. Custom AMIs, unrestricted host access from DaemonSets, privileged pods, custom CSI drivers, and strict disruption policies are the common blockers. If your platform depends on those, Auto Mode is not a free upgrade.

For teams without those constraints, Auto Mode should be evaluated early for new EKS clusters. For existing clusters, it should be treated as a migration project, not a checkbox. The right question is not whether Auto Mode is “better” than managed node groups. The right question is which operational contract your workloads can actually live with.

The pattern is the same as with managed infrastructure generally: the more your organization can treat nodes as replaceable capacity, the more value you get. The more your platform treats nodes as customized machines, the less Auto Mode fits.


1-week pilot: evaluate Auto Mode without risking production

Use a short pilot to answer compatibility and economics questions before touching production.

  1. Create a test cluster or clone a representative non-production cluster. Use the same region, Kubernetes minor version, VPC shape, IAM model, ingress pattern, and storage classes where possible.
  2. Enable Auto Mode and deploy one custom NodeClass and NodePool. Keep the first pool boring: on-demand capacity, two or three common instance families, and the same private subnet pattern as production.
  3. Select three workload types. Pick one stateless service, one stateful service with EBS, and one platform-heavy workload that uses observability or security agents.
  4. Run a scheduling audit. Check node selectors, affinity, topology spread, PDBs, tolerations, privileged mode, hostPath, and DaemonSets before migration.
  5. Force normal failure modes. Roll deployments, delete pods, scale replicas up and down, trigger node consolidation if possible, and simulate one Spot-tolerant workload if you plan to use Spot.
  6. Validate platform signals. Confirm logs, metrics, traces, runtime alerts, security events, load balancer provisioning, DNS, and persistent volume operations.
  7. Compare costs and toil. Record EC2 instance mix, the published Auto Mode management fee for each instance type and region, pod density, pending time, interruption behavior, and operator actions required.

Success criteria should be explicit:

  • 95%+ of pilot pods schedule without manual intervention
  • No silent loss of logs, metrics, traces, or security alerts
  • PDBs allow node replacement for replicated services
  • Stateful workloads survive rescheduling and volume attachment tests
  • Cost model is understood at instance-family level, not estimated from a generic percentage
  • Production migration blockers are documented with owners

If the pilot fails, that is still useful. It tells you which node assumptions are real and which workloads should stay on managed node groups.


FAQ

Does EKS Auto Mode work with existing EKS clusters?

Yes, Auto Mode can be enabled on existing clusters running Kubernetes 1.29 or greater, provided the cluster meets the IAM, add-on, and networking requirements. Existing managed node groups can continue to run while you migrate workloads gradually. Treat mixed operation as a transition state with clear scheduling boundaries.

Can I still use kubectl and standard Kubernetes tooling with Auto Mode?

Yes. From the workload API perspective, it is still Kubernetes. kubectl, Helm, Argo CD, Flux, policy engines, and CI/CD workflows should continue to work unless they depend on node-level implementation details.

What happens when a node AMI has a CVE?

AWS manages the node image and replacement flow for Auto Mode nodes. Your responsibility is to make sure workloads can be disrupted safely: replicas, PDBs, graceful shutdown, readiness probes, and topology rules all matter.

Can I use my existing Karpenter NodePools and EC2NodeClasses?

Not directly. Auto Mode uses Karpenter NodePool resources, but the AWS-specific node class is NodeClass under eks.amazonaws.com/v1, not the self-managed Karpenter EC2NodeClass. Review every field before porting anything.

Is EKS Auto Mode available in all AWS regions?

At launch, AWS announced Auto Mode in all AWS Regions where EKS was available except AWS GovCloud (US) and China Regions. That is no longer the full current picture: AWS later announced availability in both AWS GovCloud (US-East) and AWS GovCloud (US-West), and AWS China announced availability in the China (Beijing) and China (Ningxia) Regions. AWS documentation also lists Auto Mode AMI accounts across current commercial and GovCloud Regions. Still verify the target Region before rollout, because regional launches and partition-specific requirements can lag; AWS China, for example, documents Kubernetes 1.30 or later for Auto Mode.

Does Auto Mode support Windows nodes?

No. AWS currently states that EKS Auto Mode does not support Windows nodes. Windows workloads should stay on managed node groups or self-managed Windows nodes.

Does Auto Mode remove the need for HPA or KEDA?

No. Auto Mode handles node provisioning and lifecycle. It does not decide how many replicas your application should run. You still need HPA, KEDA, custom controllers, or application-level scaling logic for pod replica counts.

Is Auto Mode cheaper than managed node groups?

Not automatically. Auto Mode adds a management fee on top of EC2 and EKS control plane costs. It may still lower total cost if it improves bin packing, reduces over-provisioning, increases Spot usage safely, or saves meaningful platform engineering time. Measure it with your workload mix.

What is the biggest migration risk?

Hidden node assumptions. DaemonSets, privileged pods, hostPath, strict PDBs, old node labels, custom CSI drivers, and security agents are the areas most likely to break or degrade silently.


Sources

Amazon Managed Service for Prometheus Explained: High-Availability Monitoring on AWS

Amazon Managed Service for Prometheus Explained: High-Availability Monitoring on AWS

Learn what Amazon Managed Service for Prometheus provides and how you can benefit from it.

Monitoring is one of the hot topics when we talk about cloud-native architectures. Prometheus is a graduated Cloud Native Computing Foundation (CNCF) open-source project and one of the industry-standard solutions when it comes to monitoring your cloud-native deployment, especially when Kubernetes is involved.

Following its own philosophy of providing a managed service for some of the most used open-source projects but fully integrated with the AWS ecosystem, AWS releases a general preview (at the time of writing this article): Amazon Managed Service for Prometheus (AMP).

The first thing is to define what Amazon Managed Service for Prometheus is and what features provide. So, this is the Amazon definition of the service:

A fully managed Prometheus-compatible monitoring service that makes it easy to monitor containerized applications securely and at scale.

And I would like to spend some time on some parts of this sentence.

  • Fully managed service: So, this will be hosted and handle by Amazon, and we are just going to interact with it using API as we do with other Amazon services like EKS, RDS, MSK, SQS/SNS, and so on.
  • Prometheus-compatible: So, that means that even if this is not a pure-Prometheus installation, the API is going to be compatible. So the Prometheus clients who can use Grafana or others to get the information from Prometheus will work without changing their interfaces.
  • Service at-scale: Amazon, as part of the managed service, will take care of the solution’s scalability. You don’t need to define an instance-type or how much RAM or CPU you do need. This is going to be handled by AWS.

So, that sounds perfect. So you can think that you are going to delete your Prometheus server, and it will start using this service. Maybe you are even typing something like helm delete prom… WAIT WAIT!!

Because at this point, this is not going to replace your local Prometheus server, but it will allow the integration with it. So, that means that your Prometheus server is going to act like a scraper for the whole monitoring scalable solution that AMP is providing, something as you can see in the picture below:

Amazon Managed Service for Prometheus Explained: High-Availability Monitoring on AWS
Reference Architecture for Amazon Prometheus Service

So, you are still going to need a Prometheus server, that is right, but all the complexity are going to be avoided and leverage to the managed service: Storage configuration, High availability, API optimization, and so on is going to be just provided to you out of the box.

Ingesting information into Amazon Managed Service for Prometheus

At this moment, there is two way to ingest data into the Amazon Prometheus Service:

  • From an existing Prometheus server using the remote_write capability and configuration, so that means that each series that is scraped by the local Prometheus is going to be sent to the Amazon Prometheus Service.
  • Using AWS Distro for OpenTelemetry to integrate with this service using the Prometheus Receiver and the AWS Prometheus Remote Write Exporter components to get that.

Summary

So this is a way to provide an enterprise-grade installation leveraging on all the knowledge that AWS has hosting and managing this solution at scale and optimized in terms of performance. You can focus on the components you need to get the metrics ingested into the service.

I am sure this will not be the last movement from AWS in observability and metrics management topics. I am sure they will continue to provide more tools to the developer’s and architects’ hands to define optimized solutions as easily as possible.

📚 Want to dive deeper into Kubernetes? This article is part of our comprehensive Kubernetes Architecture Patterns guide, where you’ll find all fundamental and advanced concepts explained step by step.

EKS Hybrid Series: How to Change Namespaces for AWS Fargate Serverless Deployments

EKS Hybrid Series: How to Change Namespaces for AWS Fargate Serverless Deployments

In the previous post of these series regarding how to set up a Hybrid EKS cluster making use of both traditional EC2 machines but also serverless options using Fargate, we were able to create the EKS cluster with both deployment fashion available. If you didn’t take a look at it yet, do it now!

https://medium.com/@alexandrev/hybrid-aws-kubernetes-cluster-using-eks-ec2-and-fargate-13198d864baa

At that point, we have an empty cluster with everything ready to deploy new workloads, but we still need to configure a few things before doing the deployment. First thing is to decide which workloads are going to be deployed using the serverless option and which ones will use the traditional EC2 option.

By default, all the workloads deployed on the namespaces default and kube-system as you can see in the picture below form the AWS Console:

EKS Hybrid Series: How to Change Namespaces for AWS Fargate Serverless Deployments

So that means that all workloads from the default namespace and the kube-system namespace will be deployed in a serverless fashion. If that’s what you’d like perfect. But sometimes you’d like to start with a delimited set of namespaces where you’d like to use the serverless option and rely on the traditional deployment.

We can check that same information using eksctl and to do that we need to type the following command:

eksctl get fargateprofile --cluster cluster-test-hybrid -o yaml

The output of that command should be something similar of the information that we can see in the AWS Console:

- name: fp-default
 podExecutionRoleARN: arn:aws:iam::938784100097:role/eksctl-cluster-test-hybrid-FargatePodExecutionRole-1S12LVS5S2L62
 selectors:
 — namespace: default
 — namespace: kube-system
 subnets:
 — subnet-022f9cc3fd1180bb8
 — subnet-0aaecd5250ebcb02e
 — subnet-01b0bae6fa66ecd31

NOTE: If you don’t remember the name of your cluster you just need to type the command eksctl get clusters

So, this is what we’re going to do and to do that the first thing we need to do is to create a new namespace named “serverless” that is going to hold our serverless deployment and to do that we use a kubectl command as follows:

kubectl create namespace serverless

And now, we just need to create a new fargate profile that is going to replace the one that we have at the moment and to do that we need to make use again of eksctl to handle that job:

eksctl create fargateprofile --cluster cluster-test-hybrid --name fp-serverless-profile --namespace serverless

NOTE: We also can use not only namespace to limit the scope of our serverless deployment but also tags, so we can have in the same namespace workloads that are deployed using traditional deployment and others using serverless fashion. That will give us all the posibilities to design your cluster as you wish. To do that we will append the argument labels in a key=value fashion.

And we will get an output similar to this:

[ℹ] creating Fargate profile “fp-serverless-profile” on EKS cluster “cluster-test-hybrid”
[ℹ] created Fargate profile “fp-serverless-profile” on EKS cluster “cluster-test-hybrid”

If now we check the number of profiles that we have available we should get two profiles handling three namespaces (the ones that are managed by the default profile — default and kube-system — and the one — serverless — handled by the one we just created now)

We just will use the following command to delete the default profile:

eksctl delete fargateprofile --cluster cluster-test-hybrid fp-default

And the output of that command should be similar to this one:

[ℹ] deleted Fargate profile “fp-default” on EKS cluster “cluster-test-hybrid”

And after that, we have now ready our cluster with limited scope for serverless deployments. In the next post of the series, we will just deploy workloads on both fashions to see the difference between them. So, don’t miss the updates regarding this series making sure that you follow my posts, and if you’d like the article, or you have some doubts or comments, please leave your feedback using the comments below!

📚 Want to dive deeper into Kubernetes? This article is part of our comprehensive Kubernetes Architecture Patterns guide, where you’ll find all fundamental and advanced concepts explained step by step.

EKS Fargate Hybrid Kubernetes Cluster: Combine EC2 Nodes and Serverless Pods on AWS

EKS Fargate Hybrid Kubernetes Cluster: Combine EC2 Nodes and Serverless Pods on AWS

EKS Fargate AWS Kubernetes Cluster: Learn how to create a Kubernetes cluster that can use also all the power of serverless computing using AWS Fargate

We know that there are several movements and paradigms that are pushing us hard to change our architectures trying to leverage much more managed services and taking care of the operational level so we can focus on what’s really important for our own business: creating applications and deliver value through them.

AWS from Amazon has been a critical partner during that journey, especially in the container world. With the release of EKS some time ago were able to provide a managed Kubernetes service that everyone can use, but also introducing the CaaS solution Fargate also gives us the power to run a container workload in a serverless fashion, without needing to worry about anything else.

But you could be thinking about if those services can work together. And the short answer is yes. But even more important than that we’re seeing that also they can work in a mixed mode:

So you can have an EKS cluster that has some nodes that are Fargate services and some nodes that are normal EC2 machines for workloads that are working in a state-full fashion or fits better in a traditional EC2 approach. And everything works by the same rules and is managed by the same EKS Cluster.

So, that sounds amazing but, How we can do that? Let’s see.

eksctl

To get to that point there is a tool that we need to introduce first, and that tool is named eksctl and it is a command-line utility that helps us to do any action to interact with the EKS service and simplifies a lot the work to do and also be able to automate most of the tasks in a non-human required mode. So, the first thing we need to is to get eksctl in our platforms ready. Let’s see how we can get that.

We have here all the detailed from Amazon itself about how to install eksctl in different platforms, no matter if you’re using Windows, Linux, or MacOS X:

After doing that we can check that we have installed the eksctl software running the command:

eksctl version

And we should get an output similar to this one:

EKS Fargate Hybrid Kubernetes Cluster: Combine EC2 Nodes and Serverless Pods on AWS
eksctl version output command

So after doing that we can see that we have access to all the power behind the EKS service just typing these simple commands into our console window.

Creating the EKS Hybrid Cluster

Now, we’re going to create a mixed environment with some EC2 machines and enable the Fargate support for EKS. To do that, we will start with the following command:

eksctl create cluster --version=1.15 --name=cluster-test-hybrid --region=eu-west-1 --max-pods-per-node=1000 --fargate
[ℹ]  eksctl version 0.26.0
[ℹ]  using region eu-west-1
[ℹ]  setting availability zones to [eu-west-1c eu-west-1a eu-west-1b]
[ℹ]  subnets for eu-west-1c - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ]  subnets for eu-west-1a - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ]  subnets for eu-west-1b - public:192.168.64.0/19 private:192.168.160.0/19
[ℹ]  using Kubernetes version 1.15
[ℹ]  creating EKS cluster "cluster-test-hybrid" in "eu-west-1" region with Fargate profile
[ℹ]  if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=eu-west-1 --cluster=cluster-test-hybrid'
[ℹ]  CloudWatch logging will not be enabled for cluster "cluster-test-hybrid" in "eu-west-1"
[ℹ]  you can enable it with 'eksctl utils update-cluster-logging --region=eu-west-1 --cluster=cluster-test-hybrid'
[ℹ]  Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "cluster-test-hybrid" in "eu-west-1"
[ℹ]  2 sequential tasks: { create cluster control plane "cluster-test-hybrid", create fargate profiles }
[ℹ]  building cluster stack "eksctl-cluster-test-hybrid-cluster"
[ℹ]  deploying stack "eksctl-cluster-test-hybrid-cluster"
[ℹ]  creating Fargate profile "fp-default" on EKS cluster "cluster-test-hybrid"
[ℹ]  created Fargate profile "fp-default" on EKS cluster "cluster-test-hybrid"
[ℹ]  "coredns" is now schedulable onto Fargate
[ℹ]  "coredns" is now scheduled onto Fargate
[ℹ]  "coredns" pods are now scheduled onto Fargate
[ℹ]  waiting for the control plane availability...
[✔]  saved kubeconfig as "C:\\Users\\avazquez/.kube/config"
[ℹ]  no tasks
[✔]  all EKS cluster resources for "cluster-test-hybrid" have been created
[ℹ]  kubectl command should work with "C:\\Users\\avazquez/.kube/config", try 'kubectl get nodes'
[✔]  EKS cluster "cluster-test-hybrid" in "eu-west-1" region is ready

This command will setup the EKS cluster enabling the Fargate support.

NOTE: The first thing that we should notice is that the Fargate support for EKS is not yet available in all the AWS regions. So, depending on the region that you’re using you could get an error. At this moment this is just enabled in US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo) based on the information from AWS Announcements: https://aws.amazon.com/about-aws/whats-new/2020/04/eks-adds-fargate-support-in-frankfurt-oregon-singapore-and-sydney-aws-regions/

So, now, we should add to that cluster a Node Group. a Node Group is a set of EC2 instances that are going to be managed as part of it. And to do that we will use the following command:

eksctl create nodegroup --cluster cluster-test-hybrid --managed
[ℹ]  eksctl version 0.26.0
[ℹ]  using region eu-west-1
[ℹ]  will use version 1.15 for new nodegroup(s) based on control plane version
[ℹ]  nodegroup "ng-1262d9c0" present in the given config, but missing in the cluster
[ℹ]  1 nodegroup (ng-1262d9c0) was included (based on the include/exclude rules)
[ℹ]  will create a CloudFormation stack for each of 1 managed nodegroups in cluster "cluster-test-hybrid"
[ℹ]  2 sequential tasks: { fix cluster compatibility, 1 task: { 1 task: { create managed nodegroup "ng-1262d9c0" } } }
[ℹ]  checking cluster stack for missing resources
[ℹ]  cluster stack has all required resources
[ℹ]  building managed nodegroup stack "eksctl-cluster-test-hybrid-nodegroup-ng-1262d9c0"
[ℹ]  deploying stack "eksctl-cluster-test-hybrid-nodegroup-ng-1262d9c0"
[ℹ]  no tasks
[✔]  created 0 nodegroup(s) in cluster "cluster-test-hybrid"
[ℹ]  nodegroup "ng-1262d9c0" has 2 node(s)
[ℹ]  node "ip-192-168-69-215.eu-west-1.compute.internal" is ready
[ℹ]  node "ip-192-168-9-111.eu-west-1.compute.internal" is ready
[ℹ]  waiting for at least 2 node(s) to become ready in "ng-1262d9c0"
[ℹ]  nodegroup "ng-1262d9c0" has 2 node(s)
[ℹ]  node "ip-192-168-69-215.eu-west-1.compute.internal" is ready
[ℹ]  node "ip-192-168-9-111.eu-west-1.compute.internal" is ready
[✔]  created 1 managed nodegroup(s) in cluster "cluster-test-hybrid"
[ℹ]  checking security group configuration for all nodegroups
[ℹ]  all nodegroups have up-to-date configuration

So now we should be able to use kubectl to manage this new cluster. If you don’t have installed kubectl or you haven’t heard about it. This is the command line tool that allow us to manage your Kubernetes Cluster and you can install it based on the documentation shown here:

So, now, we should start taking a look at the infrastructure that we have. So if we type the following command to see the nodes at our disposal:

kubectl get nodes

We see an output similar to this:

NAME                                                    STATUS   ROLES    AGE   VERSION
fargate-ip-192-168-102-22.eu-west-1.compute.internal    Ready    <none>   10m   v1.15.10-eks-094994
fargate-ip-192-168-112-125.eu-west-1.compute.internal   Ready    <none>   10m   v1.15.10-eks-094994
ip-192-168-69-215.eu-west-1.compute.internal            Ready    <none>   85s   v1.15.11-eks-bf8eea
ip-192-168-9-111.eu-west-1.compute.internal             Ready    <none>   87s   v1.15.11-eks-bf8eea

As you can see we have 4 “nodes” two that start with the fargate name that are fargate nodes and two that just start with ip-… and those are the traditional EC2 instances. And after that moment that’s it, we have our mixed environment ready to use.

We can check the same cluster using the AWS EKS page to see that configuration more in detail. If we enter in the EKS page for this cluster we see in the Compute tab the following information:

EKS Fargate Hybrid Kubernetes Cluster: Combine EC2 Nodes and Serverless Pods on AWS

We see under Node Groups the data around the EC2 machines that are managed as part of this cluster and as you can see we saw 2 as the Desired Capacity and that’s why we have 2 EC2 instances in our cluster. And regarding the Fargate profile, we see the namespaces set to default and kube-system and that means that all the deployments to those namespaces are going to be deployed using Fargate Tasks.

Summary

In the following articles in these series, we will see how to progress on our Hybrid cluster, deploy workloads scale it based on the demand that we’re getting, enabling integration with other services like AWS CloudWatch, and so on. So, stay tuned, and don’t forget to follow my articles to not miss any new updates as soon as it’s available to you!

📚 Want to dive deeper into Kubernetes? This article is part of our comprehensive Kubernetes Architecture Patterns guide, where you’ll find all fundamental and advanced concepts explained step by step.