# Monitor Kubernetes with Nightingale Scrapes the native /metrics of the apiserver, kubelet, controller-manager, scheduler, kube-proxy, CoreDNS, kube-state-metrics and node-exporter; templates cover every layer from control plane to Pod. Last reviewed 2026-11-02 Every Kubernetes component exposes Prometheus-format `/metrics`. The recommended path is to scrape them directly with the Categraf `prometheus` input and deploy kube-state-metrics (workload state) and node-exporter (node resources). The old Categraf kubernetes input is deprecated, but the dashboards and rules in this template keep working. | Source | Typical endpoint | Templates it feeds | | ----------------------- | ------------------------------------------------------------------------ | --------------------------------------------- | | kube-apiserver | `https://:6443/metrics` | API Server | | kubelet | `https://:10250/metrics`, `/metrics/cadvisor`, `/metrics/resource` | Kubelet, Node, Pod, Container | | kube-controller-manager | `https://:10257/metrics` | Controller Manager | | kube-scheduler | `https://:10259/metrics` | Scheduler | | kube-proxy | `http://:10249/metrics` | Proxy | | kube-state-metrics | `http://:8080/metrics` | Deployment, StatefulSet, DaemonSet, Pod state | | CoreDNS | `http://:9153/metrics` | CoreDNS | | node-exporter | `http://:9100/metrics` | Node resources | This is the largest template in the catalog. Dashboards cover each control-plane component, nodes, Deployments, StatefulSets, DaemonSets, Pods, containers and CoreDNS. Rules cover crash-looping and Pending Pods, missing replicas, NotReady nodes and resource pressure, apiserver latency and error rate, etcd and scheduling failures, and expiring certificates. The metric reference explains the metrics component by component. ## What's in the template[​](#whats-in-the-template "Direct link to What's in the template") * **17 dashboards**, 470 panels in total * **154 alert rules**, each with runbook steps * **305 metric descriptions** ## How to connect[​](#how-to-connect "Direct link to How to connect") These components expose Prometheus-format `/metrics` themselves, so the Categraf `prometheus` input scrapes them without a dedicated plugin. If a Prometheus or VictoriaMetrics instance is already scraping them, [add it as a Nightingale data source](https://n9e.io/docs/data/prometheus.md) and the templates work as they are. Control-plane endpoints need a ServiceAccount token or client certificate; when Categraf runs as a DaemonSet inside the cluster, use `bearer_token_file` to read the mounted token. On distributions such as K3s, the scheduler, controller-manager and kube-proxy may listen on loopback only, so collect them on the control-plane node itself. Do not expose these unauthenticated endpoints to the internet for convenience. 1. Open the integration center in Nightingale, find Kubernetes, and [import the dashboards and alert rules](https://n9e.io/docs/data/integrations/import.md) into a business group. 2. Imported rules start disabled. [Check that the metrics are arriving](https://n9e.io/docs/data/integrations/verify.md) first, then enable the rules one by one with thresholds that fit your environment. ## Bundled dashboards[​](#bundled-dashboards "Direct link to Bundled dashboards") | Dashboard | Panels | | ------------------------------------ | ------ | | Kubernetes / API Server | 13 | | Kubernetes / Controller Manager | 22 | | Kubernetes / Deployment / Container | 44 | | Kubernetes / Kube State Metrics | 21 | | Kubernetes / Kubelet Metrics | 13 | | Kubernetes / Pod | 48 | | Kubernetes / Scheduler | 20 | | Kubernetes / Statefulset / Container | 44 | | Kubernetes / CoreDNS | 11 | | Kubernetes / Daemonset | 10 | | Kubernetes / Deployment | 33 | | Kubernetes / Etcd | 20 | | Kubernetes / Container by categraf | 5 | | Kubernetes / Node | 93 | | Kubernetes / Cluster Overview | 35 | | Kubernetes / Proxy | 11 | | Kubernetes / StatefulSet | 27 | ## Bundled alert rules[​](#bundled-alert-rules "Direct link to Bundled alert rules") 154 rules, disabled on import so you can enable them one by one with your own thresholds. The PromQL condition and runbook steps for every rule are in the alert rule library: [Kubernetes alert rules](https://n9e.io/alert-rules/kubernetes.md), [Prometheus alert rules](https://n9e.io/alert-rules/prometheus.md), [Linux alert rules](https://n9e.io/alert-rules/linux.md). | Rule | Severity | For | | ------------------------------------------------------------- | ----------- | ---- | | Deployment Pod availability below 70% (replicas above 9 pods) | S1 Critical | 1m | | Job failed | S3 Info | 1m | | Pod startup timeout | S2 Warning | 1m | | Pod state abnormal | S1 Critical | 1m | | Pod restarting frequently | S2 Warning | 1m | | Container CPU usage exceeds 80% | S3 Info | 1m | | Container memory usage exceeds 80% | S3 Info | 1m | | Node CPU usage exceeds 90% | S3 Info | 1m | | Node memory usage exceeds 90% | S2 Warning | 1m | | Node not ready | S1 Critical | 1m | | Node disk usage exceeds 90% | S3 Info | 1m | | KubeClientCertificateExpiration-S2 | S2 Warning | — | | KubeClientCertificateExpiration-S1 | S1 Critical | — | | AggregatedAPIErrors | S2 Warning | — | | AggregatedAPIDown | S2 Warning | 5m | | KubeAPIDown | S1 Critical | 15m | | KubeAPIErrorBudgetBurn-S1-120s | S1 Critical | 2m | | KubeAPIErrorBudgetBurn-S1-900s | S1 Critical | 15m | | KubeAPIErrorBudgetBurn-S2-3600s | S2 Warning | 60m | | KubeAPIErrorBudgetBurn-S2-10800s | S2 Warning | 180m | | Deployment replicas mismatch | S2 Warning | 10m | | StatefulSet replicas mismatch | S2 Warning | 10m | | DaemonSet has unready Pods | S2 Warning | 10m | | HPA scaling failed | S2 Warning | 5m | | Pod in Failed state | S2 Warning | 5m | | Job failed | S2 Warning | 5m | | CPUThrottlingHigh | S3 Info | 15m | | KubeAggregatedAPIDown | S2 Warning | 5m | | KubeAggregatedAPIErrors | S2 Warning | — | | KubeAPIDown | S1 Critical | 15m | | KubeAPIErrorBudgetBurn | S1 Critical | 2m | | KubeAPITerminatedRequests | S2 Warning | 5m | | KubeClientCertificateExpiration | S2 Warning | 5m | | KubeClientErrors | S2 Warning | 15m | | KubeContainerWaiting | S2 Warning | 60m | | KubeControllerManagerDown | S1 Critical | 15m | | KubeCPUOvercommit | S2 Warning | 10m | | KubeCPUQuotaOvercommit | S2 Warning | 5m | | KubeDaemonSetMisScheduled | S2 Warning | 15m | | KubeDaemonSetNotScheduled | S2 Warning | 10m | | KubeDaemonSetRolloutStuck | S2 Warning | 15m | | KubeDeploymentGenerationMismatch | S2 Warning | 15m | | KubeDeploymentReplicasMismatch | S2 Warning | 15m | | KubeDeploymentRolloutStuck | S2 Warning | 15m | | KubeHpaMaxedOut | S2 Warning | 15m | | KubeHpaReplicasMismatch | S2 Warning | 15m | | KubeJobFailed | S2 Warning | 15m | | KubeJobNotCompleted | S2 Warning | — | | KubeletClientCertificateExpiration | S2 Warning | — | | KubeletClientCertificateRenewalErrors | S2 Warning | 15m | | KubeletDown | S1 Critical | 15m | | KubeletPlegDurationHigh | S2 Warning | 5m | | KubeletPodStartUpLatencyHigh | S2 Warning | 15m | | KubeletServerCertificateExpiration | S2 Warning | — | | KubeletServerCertificateRenewalErrors | S2 Warning | 15m | | KubeletTooManyPods | S3 Info | 15m | | KubeMemoryOvercommit | S2 Warning | 10m | | KubeMemoryQuotaOvercommit | S2 Warning | 5m | | KubeNodeNotReady | S2 Warning | 15m | | KubeNodeReadinessFlapping | S2 Warning | 15m | | KubeNodeUnreachable | S2 Warning | 15m | | KubePersistentVolumeErrors | S1 Critical | 5m | | KubePersistentVolumeFillingUp | S1 Critical | 1m | | KubePersistentVolumeInodesFillingUp | S1 Critical | 1m | | KubePodCrashLooping | S2 Warning | 15m | | KubePodNotReady | S2 Warning | 15m | | KubeQuotaAlmostFull | S3 Info | 15m | | KubeQuotaExceeded | S2 Warning | 15m | | KubeQuotaFullyUsed | S3 Info | 15m | | KubeSchedulerDown | S1 Critical | 15m | | KubeStatefulSetGenerationMismatch | S2 Warning | 15m | | KubeStatefulSetReplicasMismatch | S2 Warning | 15m | | KubeStatefulSetUpdateNotRolledOut | S2 Warning | 15m | | KubeVersionMismatch | S2 Warning | 15m | | Node not ready | S2 Warning | 15m | | Node unreachable | S2 Warning | 15m | | Node running too many Pods | S2 Warning | 15m | | Node readiness flapping | S2 Warning | 15m | | PLEG relisting duration high | S2 Warning | 5m | | Pod startup duration high | S2 Warning | 15m | | Client certificate expiring - S2 | S2 Warning | — | | Client certificate expiring - S1 | S1 Critical | — | | Server certificate expiring - S2 | S2 Warning | — | | Server certificate expiring - S1 | S1 Critical | — | | Client certificate renewal errors | S2 Warning | 15m | | Server certificate renewal errors | S2 Warning | 15m | | Kubelet down | S1 Critical | 15m | | Host out of memory | S2 Warning | 2m | | Host under memory pressure | S2 Warning | 2m | | Host inbound network throughput abnormal | S2 Warning | 5m | | Host outbound network throughput abnormal | S2 Warning | 5m | | Host disk read rate abnormal | S2 Warning | 5m | | Host disk write rate abnormal | S2 Warning | 2m | | Host out of disk space | S2 Warning | 2m | | Host disk will fill up within 24 hours | S2 Warning | 2m | | Filesystem inode usage high | S2 Warning | 2m | | Filesystem inodes will fill up within 24 hours | S2 Warning | 2m | | Host disk read latency abnormal | S2 Warning | 2m | | Host disk write latency abnormal | S2 Warning | 2m | | Node CPU load high | S2 Warning | — | | CPU stolen by other cloud guests | S2 Warning | — | | Host context switching abnormal | S2 Warning | — | | Host swap almost full | S2 Warning | 2m | | Host system service crashed | S2 Warning | — | | Host kernel version deviation | S2 Warning | 360m | | HostOomKill detected | S2 Warning | — | | Host network receive errors | S2 Warning | 2m | | Host network transmit errors | S2 Warning | 2m | | Host network interface saturated | S2 Warning | 1m | | Conntrack entries approaching the limit | S2 Warning | 5m | | Host clock skew | S2 Warning | 2m | | Host clock not synchronising | S2 Warning | 2m | | Node filesystem filling up - S2 | S2 Warning | 60m | | Node filesystem filling up - S1 | S1 Critical | 60m | | Node disk almost full - S2 - space - 5 | S2 Warning | 60m | | Node disk almost full - S1 - space - 3 | S1 Critical | 60m | | Node disk almost full - S2 - inodes - 40 | S2 Warning | 60m | | Node disk almost full - S1 - inodes - 20 | S1 Critical | 60m | | Node disk almost full - S2 - inodes - 5 | S2 Warning | 60m | | Node disk almost full - S1 - inodes - 3 | S1 Critical | 60m | | Node conntrack entries usage high | S2 Warning | 5m | | Node textfile collector scrape errors | S2 Warning | — | | Node network interface flapping | S2 Warning | 2m | | ConfigReloaderSidecarErrors | S2 Warning | 10m | | PrometheusOperatorListErrors | S2 Warning | 15m | | PrometheusOperatorNodeLookupErrors | S2 Warning | 10m | | PrometheusOperatorNotReady | S2 Warning | 5m | | PrometheusOperatorReconcileErrors | S2 Warning | 10m | | PrometheusOperatorRejectedResources | S2 Warning | 5m | | PrometheusOperatorSyncFailed | S2 Warning | 10m | | PrometheusOperatorWatchErrors | S2 Warning | 15m | | ConfigReloaderSidecarErrors | S2 Warning | 10m | | PrometheusBadConfig | S1 Critical | 10m | | PrometheusDuplicateTimestamps | S2 Warning | 10m | | PrometheusErrorSendingAlertsToAnyAlertmanager | S1 Critical | 15m | | PrometheusErrorSendingAlertsToSomeAlertmanagers | S2 Warning | 15m | | PrometheusHighQueryLoad | S2 Warning | 15m | | PrometheusLabelLimitHit | S2 Warning | 15m | | PrometheusMissingRuleEvaluations | S2 Warning | 15m | | PrometheusNotConnectedToAlertmanagers | S2 Warning | 10m | | PrometheusNotificationQueueRunningFull | S2 Warning | 15m | | PrometheusNotIngestingSamples | S2 Warning | 10m | | PrometheusOutOfOrderTimestamps | S2 Warning | 10m | | PrometheusRemoteStorageFailures | S1 Critical | 15m | | PrometheusRemoteWriteBehind | S1 Critical | 15m | | PrometheusRemoteWriteDesiredShards | S2 Warning | 15m | | PrometheusRuleFailures | S1 Critical | 15m | | PrometheusScrapeBodySizeLimitHit | S2 Warning | 15m | | PrometheusScrapeSampleLimitHit | S2 Warning | 15m | | PrometheusSDRefreshFailure | S2 Warning | 20m | | PrometheusTargetLimitHit | S2 Warning | 15m | | PrometheusTargetSyncFailure | S1 Critical | 5m | | PrometheusTSDBCompactionsFailing | S2 Warning | 240m | | PrometheusTSDBReloadsFailing | S2 Warning | 240m | ## Metric reference[​](#metric-reference "Direct link to Metric reference") 305 metrics, the ones the dashboards and rules above are built on. | Metric | Meaning | Unit | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---- | | `sum(rate(coredns_dns_requests_total{}[5m])) by (proto,instance)` | CoreDNS Request Rate — Metric exposed by CoreDNS itself | | | `sum(rate(coredns_dns_requests_total{}[5m])) by (type,instance)` | CoreDNS Request Rate (by record type) — Metric exposed by CoreDNS itself | | | `sum(rate(coredns_dns_do_requests_total{}[5m])) by (instance)` | CoreDNS Request Rate (DO flag) — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.99,sum(rate(coredns_dns_request_size_bytes_bucket{proto="udp"}[5m])) by(le,proto,instance))` | CoreDNS UDP Request Packet Size P99 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.90,sum(rate(coredns_dns_request_size_bytes_bucket{proto="udp"}[5m])) by(le,proto,instance))` | CoreDNS UDP Request Packet Size P90 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.50,sum(rate(coredns_dns_request_size_bytes_bucket{proto="udp"}[5m])) by(le,proto,instance))` | CoreDNS UDP Request Packet Size P50 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.99,sum(rate(coredns_dns_request_size_bytes_bucket{proto="tcp"}[5m])) by(le,proto,instance))` | CoreDNS TCP Request Packet Size P99 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.90,sum(rate(coredns_dns_request_size_bytes_bucket{proto="tcp"}[5m])) by(le,proto,instance))` | CoreDNS TCP Request Packet Size P90 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.50,sum(rate(coredns_dns_request_size_bytes_bucket{proto="tcp"}[5m])) by(le,proto,instance))` | CoreDNS TCP Request Packet Size P50 — Metric exposed by CoreDNS itself | | | `sum(rate(coredns_dns_responses_total{}[5m])) by(rcode,instance)` | CoreDNS Response Rate (by response code) — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.99,sum(rate(coredns_dns_request_duration_seconds_bucket{}[5m])) by(le,job,instance))` | CoreDNS Response Latency P99 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.90,sum(rate(coredns_dns_request_duration_seconds_bucket{}[5m])) by(le,job,instance))` | CoreDNS Response Latency P90 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.50,sum(rate(coredns_dns_request_duration_seconds_bucket{}[5m])) by(le,job,instance))` | CoreDNS Response Latency P50 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.99,sum(rate(coredns_dns_response_size_bytes_bucket{proto="udp"}[5m])) by(le,proto,instance))` | CoreDNS UDP Response Packet Size P99 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.90,sum(rate(coredns_dns_response_size_bytes_bucket{proto="udp"}[5m])) by(le,proto,instance))` | CoreDNS UDP Response Packet Size P90 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.50,sum(rate(coredns_dns_response_size_bytes_bucket{proto="udp"}[5m])) by(le,proto,instance))` | CoreDNS UDP Response Packet Size P50 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.99,sum(rate(coredns_dns_response_size_bytes_bucket{proto="tcp"}[5m])) by(le,proto,instance))` | CoreDNS TCP Response Packet Size P99 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.90,sum(rate(coredns_dns_response_size_bytes_bucket{proto="tcp"}[5m])) by(le,proto,instance))` | CoreDNS TCP Response Packet Size P90 — Metric exposed by CoreDNS itself | | | `histogram_quantile(0.50,sum(rate(coredns_dns_response_size_bytes_bucket{proto="tcp"}[5m])) by(le,proto,instance))` | CoreDNS TCP Response Packet Size P50 — Metric exposed by CoreDNS itself | | | `sum (coredns_cache_entries{}) by(type,instance)` | CoreDNS Cached DNS Records — Metric exposed by CoreDNS itself | | | `sum (rate(coredns_cache_hits_total{}[5m])) by (type,instance)` | CoreDNS Cache Hit Rate — Metric exposed by CoreDNS itself | | | `sum (rate(coredns_cache_misses_total{}[5m])) by (type,instance)` | CoreDNS Cache Miss Rate — Metric exposed by CoreDNS itself | | | `kube_cronjob_info` | CronJob Information — CronJob metric from kube-state-metrics | | | `kube_cronjob_labels` | Kubernetes Labels Converted to Prometheus Labels — CronJob metric from kube-state-metrics | | | `kube_cronjob_created` | Creation Unix Timestamp — CronJob metric from kube-state-metrics | | | `kube_cronjob_next_schedule_time` | Next Schedule Time — CronJob metric from kube-state-metrics | | | `kube_cronjob_status_active` | Currently Active Jobs — CronJob metric from kube-state-metrics | | | `kube_cronjob_status_last_schedule_time` | Last Successful Schedule Time — CronJob metric from kube-state-metrics | | | `kube_cronjob_spec_suspend` | Suspend Flag — CronJob metric from kube-state-metrics | | | `kube_cronjob_spec_starting_deadline_seconds` | Job Starting Deadline (seconds) — CronJob metric from kube-state-metrics | | | `apiserver_request_duration_seconds` | Request Duration Distribution by URL and Verb — Metric exposed by the apiserver itself Type: histogram | | | `apiserver_request_total` | Total API Server Requests — Metric exposed by the apiserver itself Type: counter | | | `apiserver_current_inflight_requests` | Current Inflight Requests — Metric exposed by the apiserver itself Type: gauge | | | `apiserver_response_sizes` | Response Size Distribution (bytes) — Metric exposed by the apiserver itself Type: histogram | | | `watch_cache_capacity` | Watch Cache Capacity by Resource Type — Metric exposed by the apiserver itself Type: gauge | | | `process_cpu_seconds_total` | Total Process CPU Time (seconds) — Metric exposed by the apiserver itself Type: counter | | | `process_resident_memory_bytes` | Resident Memory Usage (bytes) — Metric exposed by the apiserver itself Type: gauge | | | `workqueue_adds_total` | Total Workqueue Additions — Metric exposed by the apiserver itself Type: counter | | | `workqueue_depth` | Workqueue Depth — Metric exposed by the apiserver itself Type: gauge | | | `kube_configmap_info` | ConfigMap Information — ConfigMap metric from kube-state-metrics | | | `kube_configmap_created` | Creation Unix Timestamp — ConfigMap metric from kube-state-metrics | | | `kube_configmap_metadata_resource_version` | ConfigMap Resource Version — ConfigMap metric from kube-state-metrics | | | `container_cpu_cfs_periods_total` | Total CFS Scheduling Periods — Container metric exposed by cAdvisor Type: counter | | | `container_cpu_cfs_throttled_periods_total` | Total Throttled CFS Periods — Container metric exposed by cAdvisor Type: counter | | | `container_cpu_cfs_throttled_seconds_total` | Total CPU Throttled Time — Container metric exposed by cAdvisor Type: counter | | | `container_file_descriptors` | Open File Descriptors — Container metric exposed by cAdvisor Type: gauge | | | `container_memory_usage_bytes` | Container Memory Usage (bytes) — Container metric exposed by cAdvisor Type: gauge | | | `container_network_receive_bytes_total` | Total Network Receive Bytes — Container metric exposed by cAdvisor Type: counter | | | `container_network_transmit_bytes_total` | Total Network Transmit Bytes — Container metric exposed by cAdvisor Type: counter | | | `container_spec_cpu_period` | Container CPU Scheduling Period — Container metric exposed by cAdvisor Type: gauge | | | `container_spec_cpu_quota` | Container CPU Quota — Container metric exposed by cAdvisor Type: gauge | | | `container_spec_memory_limit_bytes` | Container Memory Limit (bytes) — Container metric exposed by cAdvisor Type: gauge | | | `container_threads` | Current Thread Count — Container metric exposed by cAdvisor Type: gauge | | | `container_threads_max` | Maximum Thread Count — Container metric exposed by cAdvisor Type: gauge | | | `rest_client_request_duration_seconds` | API Server Request Latency Distribution (by URL and Verb) — Metric of the controller-manager service itself. Type: histogram | | | `cronjob_controller_cronjob_job_creation_skew_duration_seconds` | CronJob Creation-to-Run Time Distribution — Metric of the controller-manager service itself. Type: histogram | | | `leader_election_master_status` | Controller Leader Election Status (0: Backup, 1: Master) — Metric of the controller-manager service itself. Type: gauge | | | `node_collector_zone_health` | Healthy Node Ratio per Zone — Metric of the controller-manager service itself. Type: gauge | | | `node_collector_zone_size` | Node Count per Zone — Metric of the controller-manager service itself. Type: gauge | | | `process_cpu_seconds_total` | CPU Usage (Can Be Used as CPU Utilization) — Metric of the controller-manager service itself. Type: counter | | | `process_open_fds` | Open File Descriptors of Controller — Metric of the controller-manager service itself. Type: gauge | | | `pv_collector_bound_pv_count` | Currently Bound PV Count — Metric of the controller-manager service itself. Type: gauge | | | `pv_collector_unbound_pvc_count` | Currently Unbound PVC Count — Metric of the controller-manager service itself. Type: gauge | | | `pv_collector_bound_pvc_count` | Currently Bound PVC Count — Metric of the controller-manager service itself. Type: gauge | | | `pv_collector_total_pv_count` | Total PV Count — Metric of the controller-manager service itself. Type: gauge | | | `workqueue_adds_total` | Total Tasks Accepted by Each Controller — Metric of the controller-manager service itself. Type: counter | | | `workqueue_depth` | Workqueue Depth per Controller (Number of Tasks in a Controller) — Metric of the controller-manager service itself. Type: gauge | | | `workqueue_queue_duration_seconds` | Task Wait Time in Queue (by Controller) — Metric of the controller-manager service itself. Type: histogram | | | `workqueue_work_duration_seconds` | Task Processing Time From Dequeue to Completion (by Controller) — Metric of the controller-manager service itself. Type: histogram | | | `workqueue_retries_total` | Total Task Retries in Workqueue — Metric of the controller-manager service itself. Type: counter | | | `workqueue_longest_running_processor_seconds` | Longest Running Task Processing Time — Metric of the controller-manager service itself. Type: gauge | | | `endpoint_slice_controller_syncs` | EndpointSlice Sync Count (1.20+) — Metric of the controller-manager service itself. Type: counter | | | `get_token_fail_count` | Token Fetch Failure Count — Metric of the controller-manager service itself. Type: counter | | | `go_memstats_gc_cpu_fraction` | GC CPU Fraction of Controller — Metric of the controller-manager service itself. Type: gauge | | | `kube_daemonset_created` | Creation Unix Timestamp — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_status_current_number_scheduled` | Nodes Running at Least One Daemon Pod — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_status_desired_number_scheduled` | Nodes That Should Run Daemon Pods — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_status_number_available` | Nodes with Available Daemon Pods — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_status_number_misscheduled` | Nodes Running Daemon Pods That Should Not — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_status_number_ready` | Nodes with Ready Daemon Pods — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_status_number_unavailable` | Nodes with No Available Daemon Pods — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_updated_number_scheduled` | Nodes Running Updated Daemon Pods — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_metadata_generation` | Desired State Generation Sequence Number — DaemonSet metric from kube-state-metrics | | | `kube_daemonset_labels` | Kubernetes Labels Converted to Prometheus Labels — DaemonSet metric from kube-state-metrics | | | `kube_deployment_status_replicas` | Replicas per Deployment — Deployment metric from kube-state-metrics. | | | `kube_deployment_status_replicas_available` | Available Replicas per Deployment — Deployment metric from kube-state-metrics. | | | `kube_deployment_status_replicas_unavailable` | Unavailable Replicas per Deployment — Deployment metric from kube-state-metrics. | | | `kube_deployment_status_replicas_updated` | Updated Replicas per Deployment — Deployment metric from kube-state-metrics. | | | `kube_deployment_status_observed_generation` | Generation Observed by the Deployment Controller — Deployment metric from kube-state-metrics. | | | `kube_deployment_status_condition` | Current Status Condition of Deployment — Deployment metric from kube-state-metrics. | | | `kube_deployment_spec_replicas` | Desired Pods for Deployment — Deployment metric from kube-state-metrics. | | | `kube_deployment_spec_paused` | Deployment Paused (Not Processed by the Deployment Controller) — Deployment metric from kube-state-metrics. | | | `kube_deployment_spec_strategy_rollingupdate_max_unavailable` | Max Unavailable Replicas During Deployment Rolling Update — Deployment metric from kube-state-metrics. | | | `kube_deployment_spec_strategy_rollingupdate_max_surge` | Max Surge Replicas During Deployment Rolling Update — Deployment metric from kube-state-metrics. | | | `kube_deployment_metadata_generation` | Sequence Number Representing a Specific Generation of the Desired State — Deployment metric from kube-state-metrics. | | | `kube_deployment_labels` | Kubernetes Labels Converted to Prometheus Labels — Deployment metric from kube-state-metrics. | | | `kube_deployment_created` | Creation Unix Timestamp — Deployment metric from kube-state-metrics. | | | `kube_endpoint_address_not_ready` | Not Ready Addresses in Endpoint — Endpoint metric from kube-state-metrics | | | `kube_endpoint_address_available` | Available Addresses in Endpoint — Endpoint metric from kube-state-metrics | | | `kube_endpoint_info` | Endpoint Information — Endpoint metric from kube-state-metrics | | | `kube_endpoint_labels` | Kubernetes Labels Converted to Prometheus Labels — Endpoint metric from kube-state-metrics | | | `kube_endpoint_created` | Creation Unix Timestamp — Endpoint metric from kube-state-metrics | | | `etcd_server_is_leader` | Leader Indicator (1: leader, 0: learner) — Metric exposed by the etcd service itself Type: gauge | | | `etcd_server_health_success` | Health Check Successes — Metric exposed by the etcd service itself Type: counter | | | `etcd_server_health_failures` | Health Check Failures — Metric exposed by the etcd service itself Type: counter | | | `etcd_disk_defrag_inflight` | Defragmentation In Progress (1: defragmenting, 0: not started) — Metric exposed by the etcd service itself Type: gauge | | | `etcd_server_snapshot_apply_in_progress_total` | Snapshot Apply In Progress (1: in progress, 0: not) — Metric exposed by the etcd service itself Type: gauge | | | `etcd_server_leader_changes_seen_total` | Cluster Leader Changes — Metric exposed by the etcd service itself Type: counter | | | `grpc_server_handled_total` | gRPC Calls Total — Metric exposed by the etcd service itself Type: counter | | | `etcd_disk_wal_fsync_duration_seconds` | WAL Fsync Duration — Metric exposed by the etcd service itself Type: histogram | | | `etcd_server_proposals_failed_total` | Failed Proposals Total (a proposal is one request going through the Raft protocol) — Metric exposed by the etcd service itself Type: counter | | | `etcd_server_proposals_pending` | Pending Proposals (a proposal is one request going through the Raft protocol) — Metric exposed by the etcd service itself Type: gauge | | | `etcd_server_read_indexes_failed_total` | Read Index Failures Total (v3 indexes every key to speed up range operations) — Metric exposed by the etcd service itself Type: counter | | | `etcd_server_slow_read_indexes_total` | Slow Read Indexes Total (stale index reads or read timeouts) — Metric exposed by the etcd service itself Type: counter | | | `etcd_server_quota_backend_bytes` | Backend Storage Quota (DB size limit, set via quota-backend-bytes; default 2G, officially recommended not to exceed 8G) — Metric exposed by the etcd service itself Type: gauge | | | `etcd_mvcc_db_total_size_in_bytes` | Allocated DB Size (in-use size + free size) — Metric exposed by the etcd service itself Type: gauge | | | `etcd_mvcc_db_total_size_in_use_in_bytes` | DB Size In Use — Metric exposed by the etcd service itself Type: gauge | | | `etcd_mvcc_range_total` | Range Operations Total — Metric exposed by the etcd service itself Type: counter | | | `etcd_mvcc_put_total` | Put Operations Total — Metric exposed by the etcd service itself Type: counter | | | `etcd_mvcc_txn_total` | Transactions Total — Metric exposed by the etcd service itself Type: counter | | | `etcd_mvcc_delete_total` | Delete Operations Total — Metric exposed by the etcd service itself Type: counter | | | `process_cpu_seconds_total` | etcd CPU Usage — Metric exposed by the etcd service itself Type: counter | | | `process_resident_memory_bytes` | etcd Memory Usage — Metric exposed by the etcd service itself Type: gauge | | | `process_open_fds` | etcd Open File Descriptors — Metric exposed by the etcd service itself Type: gauge | | | `kube_job_info` | Job Information — Job metric from kube-state-metrics. | | | `kube_job_labels` | Kubernetes Labels Converted to Prometheus Labels — Job metric from kube-state-metrics. | | | `kube_job_owner` | Job Owner Information — Job metric from kube-state-metrics. | | | `kube_job_spec_parallelism` | Maximum Desired Pods Running at Any Given Time — Job metric from kube-state-metrics. | | | `kube_job_spec_completions` | Desired Successful Pod Completions — Job metric from kube-state-metrics. | | | `kube_job_spec_active_deadline_seconds` | Active Deadline Relative to StartTime Before Termination (Seconds) — Job metric from kube-state-metrics. | | | `kube_job_status_active` | Actively Running Pods — Job metric from kube-state-metrics. | | | `kube_job_status_succeeded` | Pods That Reached Succeeded Phase — Job metric from kube-state-metrics. | | | `kube_job_status_failed` | Pods That Reached Failed Phase — Job metric from kube-state-metrics. | | | `kube_job_status_start_time` | Job Start Time (Acknowledged by the Job Manager) — Job metric from kube-state-metrics. | | | `kube_job_status_completion_time` | Job Completion Time — Job metric from kube-state-metrics. | | | `kube_job_complete` | Job Completed — Job metric from kube-state-metrics. | | | `kube_job_failed` | Job Failed — Job metric from kube-state-metrics. | | | `kube_job_created` | Creation Unix Timestamp — Job metric from kube-state-metrics. | | | `node_netstat_Tcp_CurrEstab * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Current TCP Connections — Node indicators Type: \* | | | `node_filefd_allocated * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Number of file descriptors used — Node indicators Type: \* | | | `node_filefd_maximum * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | File Descriptor Maximum Limit — Node indicators Type: \* | | | `100 - (node_filesystem_files_free * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"} / node_filesystem_files * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"} * 100)` | File system inode usage — Node indicators Type:- | | | `100 - ((node_filesystem_avail_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"} * 100) / node_filesystem_size_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | File system usage — Node indicators Type:- | | | `sum(node_filesystem_device_error * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}) by (mountpoint)` | Number of file system errors — Node indicators Type: \* | | | `rate(node_disk_io_now[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Disk IO usage — Node indicators Type: \* | | | `rate(node_disk_writes_completed_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Disk Write IOPS — Node indicators Type: \* | | | `rate(node_disk_written_bytes_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Disk write rate — Node indicators Type: \* | | | `rate(node_disk_reads_completed_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Disk Read IOPS — Node indicators Type: \* | | | `rate(node_disk_read_bytes_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Disk read rate — Node indicators Type: \* | | | `rate(node_context_switches_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | System context switching rate — Node indicators Type: \* | | | `rate(node_intr_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | System interruption rate — Node indicators Type: \* | | | `sum(rate(node_network_transmit_drop_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Network transmission packet loss rate — Node indicators Type: \* | | | `sum(rate(node_network_transmit_bytes_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Network transmission bandwidth — Node indicators Type: \* | | | `sum(rate(node_network_transmit_errs_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Network transmission error rate — Node indicators Type: \* | | | `sum(rate(node_network_receive_drop_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Network reception packet loss rate — Node indicators Type: \* | | | `sum(rate(node_network_receive_bytes_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Network reception bandwidth — Node indicators Type: \* | | | `sum(rate(node_network_receive_errs_total[5m]) * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Network reception error rate — Node indicators Type: \* | | | `node_nf_conntrack_entries * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Number of network connection tracking entries — Node indicators Type: \* | | | `node_nf_conntrack_entries_limit * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Network connection tracking restrictions — Node indicators Type: \* | | | `sum by (instance) (rate(node_cpu_seconds_total{mode!~"idle\|iowait\|steal"}[5m])) * on(instance) group_left(nodename) node_uname_info{nodename=~"$node_name"} *100` | Node CPU usage — Node indicators Type: by | | | `node_load15 * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node 15-minute load — Node indicators Type: \* | | | `node_load1 * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node 1 minute load — Node indicators Type: \* | | | `node_load5 * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node 5-minute load — Node indicators Type: \* | | | `node_memory_SwapTotal_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"} - node_memory_SwapFree_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node Swap usage — Node indicators Type: \* | | | `node_memory_SwapTotal_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Total Node Swap — Node indicators Type: \* | | | `sum(kube_pod_info * on(node) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Number of Pods running on a node — Node indicators Type: \* | | | `sum(node_memory_MemTotal_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"} - node_memory_MemAvailable_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}) / sum(node_memory_MemTotal_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"})` | Node memory usage — Node indicators Type: \* | | | `node_memory_MemAvailable_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node memory details-Available — Node indicators Type: \* | | | `node_memory_MemTotal_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node Memory Details-Total — Node indicators Type: \* | | | `node_memory_MemFree_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node memory details-free — Node indicators Type: \* | | | `node_memory_Buffers_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node Memory Details-Buffer — Node indicators Type: \* | | | `node_memory_Cached_bytes * on(instance, cluster) group_left(nodename) node_uname_info{nodename=~"$node_name"}` | Node Memory Details-Cache — Node indicators Type: \* | | | `sum(container_fs_inodes_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)` | Number of Inodes — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(container_tasks_state{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*", state="uninterruptible"}) by (name)` | Number of uninterruptible tasks — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `(sum(container_memory_cache{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name))` | Container cache use — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `(sum(container_spec_cpu_quota{namespace="$namespace", pod=~"$pod_name"}/container_spec_cpu_period{namespace="$namespace", pod=~"$pod_name"}) by (name))` | Container CPU Limit — Pod's own indicators Type: pod = \~ "$pod \_ name"}/container \_ spec \_ cpu \_ period {namespace = "$namespace", | | | `sum(container_cpu_load_average_10s{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)` | Container CPU load 10 — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_cpu_usage_seconds_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}[1m])*100) by(name)` | Container CPU usage — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_cpu_usage_seconds_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}[1m])*100) by(name)/((sum(container_spec_cpu_quota{namespace="$namespace", pod=~"$pod_name"}/container_spec_cpu_period{namespace="$namespace", pod=~"$pod_name"}) by (name)))` | Container CPU usage after normalization — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(container_fs_io_current{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)` | Container I/O — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `(sum(container_memory_rss{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name))` | Container RSS memory usage — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(container_spec_memory_limit_bytes{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)` | Container Memory Limit — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `(sum(container_memory_usage_bytes{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name))` | Container memory usage — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `((sum(container_memory_usage_bytes{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)) /(sum(container_spec_memory_limit_bytes{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)))*100` | Container memory usage — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_cpu_system_seconds_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}[1m])*100) by(name)` | Container kernel mode CPU usage — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_cpu_cfs_throttled_periods_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}[1m]))by(name) *100` | The rate at which container CPU throttle occurs — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(container_oom_events_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)` | Number of OOM occurrences for container — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum((time()-container_start_time_seconds{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"})) by (name)` | Container startup time (hours) — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(container_fs_usage_bytes{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}) by (name)` | File system size used by the container — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_cpu_user_seconds_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}[1m])*100) by(name)` | Container user mode CPU usage — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_fs_writes_bytes_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}[1m])) by(name)` | File system write rate — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_fs_reads_bytes_total{namespace="$namespace", pod=~"$pod_name", image!~".*pause.*"}[1m])) by(name)` | File system read rate — Pod's own indicators Type: pod = \~ "$pod \_ name", | | | `sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | Number of packets lost by network transmission — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `sum(rate(container_network_transmit_packets_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | The network sends packets — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `sum(rate(container_network_transmit_bytes_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | Network transmission rate — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `sum(rate(container_network_transmit_errors_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | Number of network transmission errors — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | Number of packet losses received by network — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `sum(rate(container_network_receive_packets_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | Number of packets received by network — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `sum(rate(container_network_receive_bytes_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | Network reception rate — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `sum(rate(container_network_receive_errors_total{namespace="$namespace", pod=~"$pod_name"}[1m])) by(name, interface)` | Number of network reception errors — Pod's own indicators Type: pod = \~ "$pod \_ name"} \[1m])) | | | `go_gc_duration_seconds` | GC Duration — Metric of the kube-proxy service itself. Type: summary | | | `go_goroutines` | Goroutine Count — Metric of the kube-proxy service itself. Type: gauge | | | `go_threads` | Thread Count — Metric of the kube-proxy service itself. Type: gauge | | | `kubeproxy_network_programming_duration_seconds` | Time From Service/Pod Change to kube-proxy Rule Sync Completion (See kubernetes/community network\_programming\_latency.md) — Metric of the kube-proxy service itself. Type: histogram | | | `kubeproxy_sync_proxy_rules_duration_seconds` | Proxy Rule Sync Duration — Metric of the kube-proxy service itself. Type: histogram | | | `kubeproxy_sync_proxy_rules_endpoint_changes_pending` | Pending Rule Syncs From Endpoint Changes — Metric of the kube-proxy service itself. Type: gauge | | | `kubeproxy_sync_proxy_rules_endpoint_changes_total` | Total Rule Syncs From Endpoint Changes — Metric of the kube-proxy service itself. Type: counter | | | `kubeproxy_sync_proxy_rules_iptables_restore_failures_total` | Total iptables Restore Failures on Host — Metric of the kube-proxy service itself. Type: counter | | | `kubeproxy_sync_proxy_rules_last_queued_timestamp_seconds` | Last Rule Sync Request Timestamp (Sync Is Hung if Much Larger Than kubeproxy\_sync\_proxy\_rules\_last\_timestamp\_seconds) — Metric of the kube-proxy service itself. Type: gauge | | | `kubeproxy_sync_proxy_rules_last_timestamp_seconds` | Last Rule Sync Completion Timestamp — Metric of the kube-proxy service itself. Type: gauge | | | `kubeproxy_sync_proxy_rules_service_changes_pending` | Pending Rule Syncs From Service Changes — Metric of the kube-proxy service itself. Type: gauge | | | `kubeproxy_sync_proxy_rules_service_changes_total` | Total Rule Syncs From Service Changes — Metric of the kube-proxy service itself. Type: counter | | | `process_cpu_seconds_total` | Total CPU Seconds (Used to Compute CPU Utilization) — Metric of the kube-proxy service itself. Type: counter | | | `process_max_fds` | Max Open File Descriptors of Process — Metric of the kube-proxy service itself. Type: gauge | | | `process_open_fds` | Current Open File Descriptors of Process — Metric of the kube-proxy service itself. Type: gauge | | | `process_resident_memory_bytes` | Resident Memory Usage — Metric of the kube-proxy service itself. Type: gauge | | | `process_start_time_seconds` | Process Start Timestamp — Metric of the kube-proxy service itself. Type: gauge | | | `rest_client_request_duration_seconds` | API Server Request Latency (by URL and Verb) — Metric of the kube-proxy service itself. Type: histogram | | | `rest_client_requests_total` | Total API Server Requests (by Code, Method, Host) — Metric of the kube-proxy service itself. Type: counter | | | `go_gc_duration_seconds` | GC Duration (Summary Metric) — Metric exposed by the kubelet service itself Type: summary | | | `go_goroutines` | Goroutine Count — Metric exposed by the kubelet service itself Type: gauge | | | `go_threads` | Thread Count — Metric exposed by the kubelet service itself Type: gauge | | | `kubelet_cgroup_manager_duration_seconds` | Cgroup Operation Duration (by operation type) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_containers_per_pod_count` | Containers per Pod (count of spec.containers) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_docker_operations_duration_seconds` | Docker Operation Duration (by operation type) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_docker_operations_errors_total` | Docker Operation Errors Total (by operation type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_docker_operations_timeout_total` | Docker Operation Timeouts Total (by operation type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_docker_operations_total` | Docker Operations Total (by operation type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_eviction_stats_age_seconds` | Eviction Stats Age (by eviction signal) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_evictions` | Evictions Total (by eviction signal) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_http_inflight_requests` | Kubelet Inflight HTTP Requests (by method/path/server\_type, distinct from requests per second) — Metric exposed by the kubelet service itself Type: gauge | | | `kubelet_http_requests_duration_seconds` | Kubelet HTTP Request Duration (by method/path/server\_type) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_http_requests_total` | Kubelet HTTP Requests Total (by method/path/server\_type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_managed_ephemeral_containers` | Currently Managed Ephemeral Containers — Metric exposed by the kubelet service itself Type: gauge | | | `kubelet_network_plugin_operations_duration_seconds` | Network Plugin Operation Duration (by operation\_type; always 0 when --feature-gates=EphemeralContainers=false) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_network_plugin_operations_errors_total` | Network Plugin Operation Errors Total (by operation\_type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_network_plugin_operations_total` | Network Plugin Operations Total (by operation\_type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_node_name` | Node Name — Metric exposed by the kubelet service itself Type: gauge | | | `kubelet_pleg_discard_events` | PLEG (Pod Lifecycle Event Generator) Discarded Events — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_pleg_last_seen_seconds` | PLEG Last Seen Timestamp — Metric exposed by the kubelet service itself Type: gauge | | | `kubelet_pleg_relist_duration_seconds` | PLEG Relist Duration — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_pleg_relist_interval_seconds` | PLEG Relist Interval — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_pod_start_duration_seconds` | Pod Start Duration (from pending to running, i.e. from when kubelet sees the pod until all its containers are running; pod changes are watched from various source channels) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_pod_worker_duration_seconds` | Pod Worker Duration (by operation type: create/update/sync; a worker is kubelet's logical unit for handling a single pod) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_pod_worker_start_duration_seconds` | Pod Worker Start Duration (from kubelet seeing the pod to worker start) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_run_podsandbox_duration_seconds` | Pod Sandbox Start Duration — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_run_podsandbox_errors_total` | Pod Sandbox Start Errors Total — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_running_containers` | Running Containers (by container state: created/running/exited) — Metric exposed by the kubelet service itself Type: gauge | | | `kubelet_running_pods` | Currently Running Pods — Metric exposed by the kubelet service itself Type: gauge | | | `kubelet_runtime_operations_duration_seconds` | Container Runtime Operation Duration (container create/list/exec/remove/stop, etc.) — Metric exposed by the kubelet service itself Type: histogram | | | `kubelet_runtime_operations_errors_total` | Container Runtime Operation Errors Total (by operation type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_runtime_operations_total` | Container Runtime Operations Total (by operation type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_started_containers_errors_total` | Kubelet Started Container Errors Total (by code and container\_type) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_started_containers_total` | Kubelet Started Containers Total — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_started_pods_errors_total` | Kubelet Started Pod Errors Total (counted only when sandbox creation fails) — Metric exposed by the kubelet service itself Type: counter | | | `kubelet_started_pods_total` | Kubelet Started Pods Total — Metric exposed by the kubelet service itself Type: counter | | | `process_cpu_seconds_total` | CPU Usage — Metric exposed by the kubelet service itself Type: counter | | | `process_max_fds` | Max Allowed Open File Descriptors — Metric exposed by the kubelet service itself Type: gauge | | | `process_open_fds` | Currently Open File Descriptors — Metric exposed by the kubelet service itself Type: gauge | | | `process_resident_memory_bytes` | Process Resident Memory Size — Metric exposed by the kubelet service itself Type: gauge | | | `process_start_time_seconds` | Process Start Time — Metric exposed by the kubelet service itself Type: gauge | | | `rest_client_request_duration_seconds` | API Server Request Duration (by URL and request verb) — Metric exposed by the kubelet service itself Type: histogram | | | `rest_client_requests_total` | API Server Requests Total (by response code and request method) — Metric exposed by the kubelet service itself Type: counter | | | `storage_operation_duration_seconds` | Storage Operation Duration (by storage plugin such as configmap/emptydir/hostpath and operation\_name) — Metric exposed by the kubelet service itself Type: histogram | | | `volume_manager_total_volumes` | Volumes Mounted on Node (by plugin\_name and state; plugin\_name includes "host-path", "empty-dir", "configmap", "projected"; state is desired\_state\_of\_world or actual\_state\_of\_world) — Metric exposed by the kubelet service itself Type: gauge | | | `kube_namespace_created` | Creation Unix Timestamp — Namespace metric from kube-state-metrics | | | `kube_namespace_labels` | Kubernetes Labels Converted to Prometheus Labels — Namespace metric from kube-state-metrics | | | `kube_namespace_status_condition` | Namespace Condition — Namespace metric from kube-state-metrics | | | `kube_namespace_status_phase` | Namespace Status Phase — Namespace metric from kube-state-metrics | | | `kube_node_info` | Cluster Node Information — Node metric from kube-state-metrics | | | `kube_node_labels` | Kubernetes Labels Converted to Prometheus Labels — Node metric from kube-state-metrics | | | `kube_node_role` | Cluster Node Role — Node metric from kube-state-metrics | | | `kube_node_spec_unschedulable` | Whether Node Can Schedule New Pods — Node metric from kube-state-metrics | | | `kube_node_spec_taint` | Cluster Node Taints — Node metric from kube-state-metrics | | | `kube_node_status_capacity` | Node Resource Capacity — Node metric from kube-state-metrics | | | `kube_node_status_allocatable` | Node Allocatable Resources — Node metric from kube-state-metrics | | | `kube_node_status_condition` | Cluster Node Condition — Node metric from kube-state-metrics | | | `kube_node_created` | Creation Unix Timestamp — Node metric from kube-state-metrics | | | `rest_client_request_duration_seconds` | API Server Request Latency Distribution — Metric of the scheduler service itself. Type: histogram | | | `rest_client_requests_total` | Total API Server Requests (by Host, Code, Method) — Metric of the scheduler service itself. Type: counter | | | `leader_election_master_status` | Scheduler Leader Election Status (0: Backup, 1: Master) — Metric of the scheduler service itself. Type: gauge | | | `scheduler_queue_incoming_pods_total` | Pods Entering Scheduling Queue — Metric of the scheduler service itself. Type: counter | | | `scheduler_preemption_attempts_total` | Scheduler Preemption Attempts — Metric of the scheduler service itself. Type: counter | | | `scheduler_scheduler_cache_size` | Scheduler Cache Size (Nodes, Pods, Assumed Pods) — Metric of the scheduler service itself. Type: gauge | | | `scheduler_pending_pods` | Pending Pods (by Queue Type) — Metric of the scheduler service itself. Type: gauge | | | `scheduler_plugin_execution_duration_seconds` | Plugin Execution Duration per Extension Point (by extension\_point, plugin, status) — Metric of the scheduler service itself. Type: histogram | | | `scheduler_e2e_scheduling_duration_seconds` | E2E Scheduling Latency Distribution (Replaced by scheduling\_attempt\_duration\_seconds Since 1.23.0) — Metric of the scheduler service itself. Type: histogram | | | `scheduler_framework_extension_point_duration_seconds` | Framework Extension Point Latency Distribution (by extension\_point, profile, status) — Metric of the scheduler service itself. Type: histogram | | | `scheduler_pod_scheduling_attempts` | Scheduling Attempts Distribution Before Pod Scheduled Successfully — Metric of the scheduler service itself. Type: histogram | | | `scheduler_schedule_attempts_total` | Total Schedule Attempts by Result (unschedulable: Cannot Be Scheduled; error: Internal Scheduler Error) — Metric of the scheduler service itself. Type: counter | | | `scheduler_scheduler_goroutines` | Goroutine Count by Function (e.g. Binding, Filter) — Metric of the scheduler service itself. Type: gauge | | | `scheduler_scheduling_algorithm_duration_seconds` | Scheduling Algorithm Latency Distribution — Metric of the scheduler service itself. Type: histogram | | | `scheduler_scheduling_attempt_duration_seconds` | Scheduling Algorithm + Binding Latency Distribution — Metric of the scheduler service itself. Type: histogram | | | `scheduler_scheduler_goroutines` | Scheduler Goroutine Count — Metric of the scheduler service itself. Type: gauge | | | `kube_statefulset_status_replicas` | Replicas per StatefulSet — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_status_replicas_current` | Current Replicas per StatefulSet — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_status_replicas_ready` | Ready Replicas per StatefulSet — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_status_replicas_updated` | Updated Replicas per StatefulSet — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_status_observed_generation` | Generation Observed by StatefulSet Controller — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_replicas` | Desired Pod Count of StatefulSet — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_metadata_generation` | Desired State Generation Sequence Number — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_created` | Creation Unix Timestamp — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_labels` | Kubernetes Labels Converted to Prometheus Labels — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_status_current_revision` | StatefulSet Current Revision — StatefulSet metric from kube-state-metrics | | | `kube_statefulset_status_update_revision` | StatefulSet Update Revision — StatefulSet metric from kube-state-metrics | | ## Related[​](#related "Direct link to Related") * [Kubernetes template source](https://github.com/ccfos/nightingale/tree/main/integrations/Kubernetes): the raw JSON for dashboards, rules and collector config. Pull requests welcome. * [Import dashboards and alert rules](https://n9e.io/docs/data/integrations/import.md) * [Verify that metrics are stored](https://n9e.io/docs/data/integrations/verify.md) * [All integrations](https://n9e.io/integrations.md)