Prometheus ServiceMonitor vs PodMonitor: Key Differences and When to Use Each

Discover the differences between two of the most used CRDs from Prometheus Operator and how to use each of them.

Related reading: choosing a log storage platform.

ServiceMonitor and PodMonitor are terms that you will start to see more often when talking about using Prometheus. We have covered a lot about Prometheus in the past articles. It is one of the primary references when we talk about monitoring in a cloud-native environment and is specially focused on the Kubernetes ecosystem.

Prometheus has a new deployment model under the Kubernetes Operator Framework in recent times. That has generated several changes in terms of resources and how we configure several aspects of the monitoring of our workloads. Some of these concepts are now managed as Customer Resource Definition (CRD) that are included to simplify the system’s configuration and be more aligned with the capabilities of the Kubernetes platform itself. This is great but, at the same time, changes how we need to use this excellent monitoring tool for cloud-native workloads.

Today, we will cover two of these new CRDs, one of the most relevant ones: ServiceMonitor and PodMonitor. These are the new objects that specify the resources that will be under monitoring scope to the platform, and each of them covers a different type of object, as you can imagine: Services and Pods.

What is a ServiceMonitor?

A ServiceMonitor is a Custom Resource Definition (CRD) provided by the Prometheus Operator that tells Prometheus how to scrape a Kubernetes Service. Instead of editing prometheus.yml by hand, you declare a ServiceMonitor object that selects a Service by label; the Operator discovers the Service, walks its endpoints, and generates the scrape configuration for every pod behind it. It lives in monitoring.coreos.com/v1 and is the object most people mean when they talk about a “service monitor” in Prometheus.

What is a PodMonitor?

A PodMonitor is the sibling CRD that scrapes pods directly, by label selector, with no Service in between. You reach for a PodMonitor when the pods you need to scrape are not fronted by a Kubernetes Service — DaemonSets, sidecars, or short-lived batch pods. Same apiVersion (monitoring.coreos.com/v1), same Operator, different discovery path. If you searched for just “podmonitor” or “pod monitor”, this is it: a pod-level equivalent of the ServiceMonitor.

Each of them has its definition file with its particular fields and metadata, and to highlight them, I will present a sample for each of them below:

Service Monitor

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    serviceMonitorSelector: prometheus
  name: prometheus
  namespace: prometheus
spec:
  endpoints:
  - interval: 30s
    targetPort: 9090
    path: /metrics
  namespaceSelector:
    matchNames:
    - prometheus
  selector:
    matchLabels:
      operated-prometheus: "true"

Pod Monitor

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: front-end
  labels:
    name: front-end
spec:
  namespaceSelector:
    matchNames:
      - sock-shop
  selector:
    matchLabels:
      name: front-end
  podMetricsEndpoints:
  - targetPort: 8079

As you can see, the definitions of the components are very similar and very intuitive, focusing on the selector to detect which pods or services we should monitor and some data regarding the specific target of the monitoring, so Prometheus knows how to scrape them.

ServiceMonitor and PodMonitor spec: the key fields

Both CRDs share almost the same spec. These are the fields you will actually touch:

• selector — a label selector (matchLabels / matchExpressions) that picks the Service (ServiceMonitor) or the pods (PodMonitor) to scrape. This is the field that most often causes “my target isn’t showing up”: the labels here must match the target object exactly.
• namespaceSelector — restricts which namespaces are searched. Use matchNames: [ns1, ns2] to scope it, or any: true to scrape across every namespace. Omit it and it defaults to the CRD’s own namespace.
• endpoints (ServiceMonitor) / podMetricsEndpoints (PodMonitor) — the list of ports and per-target scrape settings: port or targetPort, path (defaults to /metrics), interval, scheme, and TLS/auth config. This is the only structural difference between the two specs — the field name and whether it points at a Service port or a pod port.

Everything else (relabeling, metric relabeling, honor labels, sample limits) is optional and shared between both CRDs. Full field-by-field docs are linked below.

If you want to take a look more in detail at any option you can configure on this CRD, I would recommend you to take a look at this URL which includes a detailed field to field documentation of the most common CRDs:

PodMonitor.monitoring.coreos.com/v1

Automatic documentation for your CustomResourceDefinitions.

These components will belong to the definition of your workloads, which means that the creation and maintenance of these objects will be from the application’s developers.

That is great because several reasons:

• It will include the Monitoring aspect of the component itself, so you will never forget the add the configuration from a specific component. That means it can be included in the duplicate YAML files or Helm Chart or a Kustomize resources as another needed resource.
  
• It will de-centralize the monitoring configuration making it more agile, and it will progress as the software components do it.
  
• It will reduce the impact on other monitored components as there is no need to act in any standard file or resource, so any different workloads will continue to work as expected.
  

Both objects are very similar in their purposes as both of them scrape all the endpoints that match the selector that we added. So, in which cases should I use one or the other?

The answer will be straightforward. By default, you will go with a ServiceMonitor because it will provide the metrics from the service itself and each of the endpoints that the service has, so each of the pods that are implementing the service will be discovered and scraped as part of this action.

So, in which cases should I use PodMonitor? Where the workload you are trying to monitor doesn’t act behind a service, so as there is no service defined, you cannot use ServiceMonitor. Do you want some examples of those? Let’s bring some!

• Services that interact using other protocols that are not HTTP-based, such as Kafka, SQS/SNS, JMS, or similar ones.
• Components such as CronJobs, DaemonSets, or non exposing any incoming connection model.
  

So I hope this article will help you understand the main difference between those objects and go a little deeper into how the new Prometheus Operator Framework resources work. We will continue covering other aspects in upcoming posts.

PodMonitor vs ServiceMonitor: Which One Should You Use?

If you searched for PodMonitor vs ServiceMonitor, here is the short answer: use a ServiceMonitor whenever your workload already sits behind a Kubernetes Service — which is the case for most long-running applications. Prometheus discovers the Service endpoints and scrapes every backing pod automatically.

Reach for a PodMonitor when there is no Service in front of the pods you need to scrape: DaemonSets exposing node-local metrics, sidecar containers, short-lived batch pods, or cases where creating a Service just for scraping would be artificial. Functionally both end up generating Prometheus scrape configs — the difference is purely the discovery path.

ServiceMonitor vs PodMonitor: side-by-side

Both CRDs end up producing equivalent Prometheus scrape configs through the Prometheus Operator — the only real difference is the discovery path. Pick by how your workload is exposed, not by performance. And once those metrics are flowing, the next decision is how you alert on them: see Alertmanager vs Grafana Alerting for routing and on-call.

📚 Want to dive deeper into Kubernetes? This article is part of our comprehensive Kubernetes Architecture Patterns guide, where you’ll find all fundamental and advanced concepts explained step by step.

Frequently Asked Questions