* [CHANGE] Use narrower selectors for StatefulSet informers in `Alertmanager` and `ThanosRuler` controllers. It is recommended to upgrade from `v0.85.0` (at least). #8246
* [CHANGE] Reject `EC2/Lightsail` SD for Prometheus >= 3.8.0 in `ScrapeConfig` CRD. #8175
* [FEATURE] Add `podManagementPolicy` field to `Prometheus`, `PrometheusAgent`, `Alertmanager` and `ThanosRuler` CRDs. #8119
* [FEATURE] Add `updateStrategy` field to `Prometheus`, `PrometheusAgent`, `Alertmanager` and `ThanosRuler` CRDs. #8202
* [FEATURE] Add `scrapeNativeHistograms` field to `Prometheus`, `PrometheusAgent`, `ServiceMonitor`, `PodMonitor`, `Probe` and `ScrapeConfig` CRDs. #8102
* [FEATURE] Add `scope` field to `AzureAD` remote write configuration. #8240
* [FEATURE] Add `workloadIdentity` field to `AzureAD` remote write configuration. #7998
* [FEATURE] Add support for PrometheusRule fields `groupLabels` and `queryOffset` in `ThanosRuler`. #8137
* [FEATURE] Add `slackAppToken` and `slackAppUrl` fields to Alertmanager global config for Slack App support. #8238
* [FEATURE] Add `incident.io` receiver support to `Alertmanager` config secret. #8190#8245#8228
* [FEATURE] Add `Mattermost` receiver support to `Alertmanager` config secret. #8188
* [FEATURE] Add `apiType` field to `Jira` receiver in `Alertmanager` config secret. #8218
* [FEATURE] Add `timeout` field to `PagerDuty` receiver in `AlertmanagerConfig` CRD. #8162
* [FEATURE] Add `timeout` field to `Slack` receiver in `AlertmanagerConfig` CRD. #8161
* [ENHANCEMENT] Use `minReadySeconds` to set `--dispatch.start-delay` in `Alertmanager`. #8177#8201
* [ENHANCEMENT] Expose native histograms in operator metrics. #8194
* [ENHANCEMENT] Add `NoSelectedResources` reason to status conditions. #8124
* [ENHANCEMENT] Add `enableHttp2` and `followRedirects` fields to HTTP configuration for `Probe` CRD. #8112
* [ENHANCEMENT] Add CEL validations for `DaemonSet` mode in `PrometheusAgent` CRD (requires the `PrometheusAgentDaemonSetFeature` featuregate). #7881
* [ENHANCEMENT] Improve validation for `Pushover`, `PagerDuty` and `VictorOps` receivers in `AlertmanagerConfig` CRD. #8239#8113#8220
* [ENHANCEMENT] Add `apiURL` validation for `WeChat`, `OpsGenie` and `Telegram` receivers in `AlertmanagerConfig` CRD. #8196#8206#8199
* [BUGFIX] Fix the generated Alertmanager configuration for `html` and `monospace` fields of `pushoverConfig` receiver in AlertmanagerConfig CRD. #8153
* [FEATURE] Add status subresource for `PrometheusRule` custom resources (requires the `StatusForConfigurationResources` feature gate). #8069#8086#8024#8005
* [ENHANCEMENT] Avoid statefulset's rollout due to changes in the number of rule configmaps for `Prometheus` and `ThanosRuler`. #8010
* [ENHANCEMENT] Support Azure system-assigned managed identities for remote-write configuration. #7815
* [ENHANCEMENT] Add `monospace` field to to `pushoverConfig` receiver in AlertmanagerConfig CRD. #8018
* [BUGFIX] Propagate Certificate Authority updates for HTTP configuration in Alertmanager's global configuration. #8089
> This release introduces the status subresource (behind the `StatusForConfigurationResources` feature gate) for `ServiceMonitor`, `PodMonitor`, `Probe` and `Scrapeconfig` custom resources. It is only supported for `Prometheus` resources.
> This release enables automatic UTF-8 character support in label names, metric names and PrometheusRule expressions for Prometheus/PrometheusAgent resources running with version >= 3.0.0.
>
> To preserve backward compatibility, the admission webhook service validates PrometheusRule resources against the legacy Prometheus scheme by default (but it can be changed with the `--name-validation-scheme` flag).
* [CHANGE] Remove automatic addition of the `metadata-wal-records` feature flag for Prometheus versions >= 3.4. #7893
* [CHANGE/FEATURE] Add support for UTF-8 characters to label names and metric names in `PrometheusRule` resources and relabel configurations. #7637#7985
* [FEATURE] Add the flag `--name-validation-scheme` to admission webhook to select between utf8 and legacy validations. #7985
* [FEATURE] Add status subresource for `ServiceMonitor` custom resources (requires the `StatusForConfigurationResources` feature gate). #7767#7836#7827#7795
* [FEATURE] Add status subresource for `PodMonitor` custom resources (requires the `StatusForConfigurationResources` feature gate). #7929#7914#7936
* [FEATURE] Add status subresource for `ScrapeConfig` custom resources (requires the `StatusForConfigurationResources` feature gate). #7958#7964#7969
* [CHANGE/BUGFIX] Add the `--watch-referenced-objects-in-all-namespaces` CLI argument. When enabled, the operator watches for secrets and configmaps in both workload and configuration resources. It ensures that reconciliation happens when a referenced secret/configmap is updated. #7615
* [ENHANCEMENT] Set `reason: ConfigurationUnmanaged` in the `Reconciled` condition when the operator doesn't managed the Prometheus configuration. #7661
* [ENHANCEMENT] AlertManager `matchType` defaults to `=` if neither `matchType` nor `regex` is set, and `=~` if `regex: true` is set without matchType. #7592
* [CHANGE] Remove the lower-case `hypervisor` and `instance` roles for the OpenStack Service Discovery from the ScrapeConfig CRD (users are required to use `Hypervisor` and `Instance` instead). #7370
* [FEATURE] Add `serviceName` field to the ThanosRuler and Alertmanager CRDs. #7325
* [FEATURE] Add `shardRetentionPolicy` field to the Prometheus CRD (it requires the `PrometheusShardRetentionPolicy` feature gate). #7274
* [FEATURE] Add support for the cluster mTLS configuration to the Alertmanager CRD. #7149
* [FEATURE] Add `LoadBalancer` role for the OpenStack Service Discovery to the ScrapeConfig CRD. #7356
* [ENHANCEMENT] Add `enableServiceLinks` field to the Alertmanager, Prometheus, PrometheusAgent and ThanosRuler CRDs. #7384
* [BUGFIX] Fix Alertmanager configuration with PushOver duration fields. #7249
> With this release, the operator will be deploying Prometheus `v3.0.1` unless a specific version is specified.
> Users who want to stay with Prometheus v2 need to set the `version` field under
> `Prometheus/PrometheusAgent` Spec.
* [CHANGE/FEATURE] Set Prometheus `v3.0.1` as default version installed by operator. #7158
* [CHANGE] Add more API validations to the ScrapeConfig CRD. #6738#7006
* [FEATURE] Set Alertmanager API version to v2 for Prometheus v3 and above. #7102
* [FEATURE] Add `enableOTLPReceiver` field in the Prometheus/PrometheusAgent CRDs. #7105
* [FEATURE] Add support for `TranslationStrategy` option for OTLP receiver. #7112
* [FEATURE] Add `nameValidationScheme` field in Prometheus global configuration. #7116
* [FEATURE] Add `scrapeFallbackProtocol` field in `Prometheus`, `PrometheusAgent`, `ServiceMonitor`, `PodMonitor`, `Probe`, `ScrapeConfig` and globally. #7131
* [FEATURE] Add support for labels in PrometheusRule CRD at rules level. #7140
* [FEATURE] Add support for ProxyConfig in Alertmanager Endpoints. #7191
* [FEATURE] Add support for configuring Alertmanager STS `persistentVolumeClaimRetentionPolicy`. #7183
* [FEATURE] Add `enableHTTP2` to ScrapeConfig CRD. #7024
* [FEATURE] Add `filter` field in `ConsulSDConfig` in ScrapeConfig CRD #7124
* [FEATURE] Add `messageThreadID` field to TelegramConfig in AlertmanagerConfig CRD. #6999
* [ENHANCEMENT] Allow users to control the sharding label(s). #7126
* [ENHANCEMENT] Add support for role selector field `selectorMechanism` on ServiceMonitor. #7086
* [ENHANCEMENT] Add support for role selector field `selectorMechanism` on PodMonitor. #7185
> Along with this release we also introduce a new command line tool [poctl](https://github.com/prometheus-operator/poctl). It is designed specifically for managing Prometheus Operator Custom Resources.
> This project is currently under active development and fully experimental, so expect breaking changes and rough edges. We encourage you to [try it out](https://github.com/prometheus-operator/poctl/releases/tag/v0.1.0) and provide the feedback.
> [!WARNING]
> Another notable change is that we have switched the logging framework from [go-kit/log](https://github.com/go-kit/log) to [slog](https://pkg.go.dev/log/slog) which may lead to some differences in the log output.
* [CHANGE] Add more API validation and changes to the `ScrapeConfig` CRD. #6921#6855#6744
* [CHANGE] Update the validating webhook to reject `PrometheusRule` objects that would generate configuration greater than the max ConfigMap size (512KiB). #6606
* [FEATURE] Add experimental Prometheus 3 support. It allows to try the new Prometheus `v3.0.0-beta.0`. #6940
* Feature requests in this version won't be supported until Prometheus 3 reaches a stable release. However, users are encouraged to try the beta version with the Operator and report any issues they encounter.
* [FEATURE] Add support for `PodMonitor` to the `PrometheusAgent` CRD in DaemonSet mode. #6879
* [FEATURE] Add `-kubelet-endpointslice` argument to support `EndpointSlice` for the kubelet controller. #6882
* [FEATURE] Add `outOfOrderTimeWindow` field to the `PrometheusAgent` CRD. #6851
* [FEATURE] Add `matchFirstNetwork` field to the `ScrapeConfig` CRD for the Docker Service Discovery. #6849#6894
* [FEATURE] Add the `otlp` field to the `Prometheus` and `PrometheusAgent` CRDs. #6780
* [FEATURE] Add support for certificate, key and client CA file paths to the web TLS configuration. #6358
* [FEATURE] Add `noProxy`, `proxyFromEnvironment` and `proxyConnectHeader` fields to the AlertmanagerConfig CRD for OAuth2 clients. #6922
* [FEATURE] Add `noProxy`, `proxyFromEnvironment` and `proxyConnectHeader` fields to the AlertmanagerConfig CRD for HTTP clients. #6864
* [FEATURE] Add Ionos Service Discovery support to `ScrapeConfig` CRD. (#6732)
* [BUGFIX] Fix panic when processing an invalid `AlertmanagerConfig` object used for global configuration. #6931
* [BUGFIX] Allow configuring proxy URL for all Prometheus versions. #6845
* [CHANGE] Enhanced secret management in Prometheus and PrometheusAgent CRDs by switching the secrets field from atomic to listType: set, allowing independent handling of entries by different managers to prevent conflicts and improve deployment stability. #6762
* [CHANGE] Add API-level validations to Kubernetes SD in the ScrapeConfig CRD. #6678
* [FEATURE] Add TLS and Proxy settings to OAuth2 configuration for Prometheus and PrometheusAgent CRDs. #6735
* [FEATURE] Add support for OAuth2 in the ScrapeConfig CRD. #6814
* [FEATURE] Add Scaleway service discovery to the ScrapeConfig CRD. #6711
* [FEATURE] Add `serviceDiscoveryRole` field to the Prometheus and PrometheusAgent CRDs to select between Endpoints (default) and EndpointSlice for discovering scrape and alerting targets. #6672
* [ENHANCEMENT] Make the `namespace` field optional in the Alertmanager endpoints configuration of the Prometheus CRD, if not defined it will use the `default` namespace. #6338
* [ENHANCEMENT] Add support to configure the TLS version for Prometheus, PrometheusAgent and Alertmanager CRDs. #6736
* [ENHANCEMENT] Add `-secret-label-selector` argument to the operator to filter the Secrets being watched. #6731
* [ENHANCEMENT] Add `attachMetadata` field to ScrapeClasses. #6756
* [BUGFIX] Add support for all proxy settings in the Alertmanager configuration. #6818
* [CHANGE/BUGFIX] Change the `proxyConnectHeader` field of the proxy configuration in the `ScrapeConfig` CRD from a map of strings to a map of string slices. #6541
* [CHANGE] Automatically set the GOMAXPROCS variable according to the CPU limits of the process cgroup. #6576
* [FEATURE] Add support for the Nomad Service Discovery to the `ScrapeConfig` CRD. #6485
* [FEATURE] Add `relabelings` field for Alertmanager's endpoints in the `Prometheus` CRD. #6467
* [FEATURE] Add `alertRelabelings` field for the Alertmanager's endpoints in the `Prometheus` CRD. #6450
* [FEATURE] Add support for the Azure SDK authentication to the Azure Service Discovery in the `ScrapeConfig` CRD. #6595
* [FEATURE] Add support for the Azure SDK authentication to the remote-write configuration in the `Prometheus` and `PrometheusAgent` CRDs. #6596
* [ENHANCEMENT] Add Go runtime scheduler metrics to the operator. #6563
* [ENHANCEMENT] Add `go_sync_mutex_wait_total_seconds_total` metric to the operator. #6581
* [BUGFIX] Allow templated strings for the responder's `type` in OpsGenie configuration. #6378
* [BUGFIX] Fix nil pointer panic in the `AlertmanagerConfig` Webex receiver. #6582
* [BUGFIX] Apply TLS scrape class settings to all `PodMonitor`, `Probe` and `ScrapeConfig` objects. #6573
* [CHANGE/BUGFIX] Add `proxyURL` validation for ServiceMonitor, PodMonitor and Probe objects. **It could be a breaking change for those who rely on the configuration reloader sidecar to expand environment variables in the generated configuration file.**#6464
In this release we want to highlight the addition of ScrapeClasses.
ScrapeClasses is an experimental feature that allows declaring multiple scrape configurations in the Prometheus/PrometheusAgent objects that can be consumed by selected scrape-related resources, like ScrapeConfig, ServiceMonitor, PodMonitor and Probes. Initial support only allows the declaration of TLS configuration, but we plan to extend ScrapeClasses with more options in the future, e.g. Authorization, relabelings and scrape protocols.
* [FEATURE] Add `ScrapeClasses` to Prometheus, PrometheusAgent, ScrapeConfig, ServiceMonitor, PodMonitor and Probe CRDs. #6199
* [FEATURE] Add the `enableCompression` field to the ScrapeConfig CRD. #6236
* [FEATURE] Add DigitalOcean service discovery to the ScrapeConfig CRD. #6220
* [FEATURE] Add support for `scrapeProtocols` to the ScrapeConfig, ServiceMonitor and PodMonitor CRDs. #6235#6268
* [FEATURE] Add support for NS records to the DNS service discovery in the ScrapeConfig CRD. #6235
* [FEATURE] Add support for `keepFiringFor` in the ThanosRuler CRD. #6283
* [ENHANCEMENT] Enable usage of sharding with ScrapeConfig. #6135
* [BUGFIX] Correctly mount file subpaths in ThanosRuler CRD. #6243
* [BUGFIX] Fix conversion of the `continue` field in AlertmanagerConfig between v1alpha1 and v1beta1 versions. #6282
* [BUGFIX] Fix failure of metadata informers when processing deleted objects. #6298
* [CHANGE] Use camelCase on scrapeConfig in prometheus job names. #6124
* [CHANGE] Update field names for ConsulSD configs. #6094
* [FEATURE] Add `trackTimestampsStaleness` field to `ServiceMonitor`, `PodMonitor` and `ScrapeConfig` CRDs. #6105
* [FEATURE] Add `persistentVolumeClaimRetentionPolicy` field to the `Prometheus` and `PrometheusAgent` CRDs. #6038
* [FEATURE] Support service,pod,endpoints,endpointslice,ingress roles to `KubernetesSDConfigs` in `ScrapeConfig` CRD. #6089
* [FEATURE] Add support for Azure SD and GCE SD in `ScrapeConfig` CRD. #6078#6009
* [FEATURE] Add azure AD oauth support to remote write. #6037
* [FEATURE] Add `.spec.reloadStrategy` to the Prometheus and PrometheusAgent CRDs. The value can be `HTTP` (default if not specified) or `ProcessSignal`. #5690
* [ENHANCEMENT] Expose flag to enable the reloader probes in the jsonnet configuration. #6075
* [ENHANCEMENT] Reduce memory usage for clusters with large number of secrets and/or configmaps. #5993
* [ENHANCEMENT] Config reloader now sends all logs to `os.Stdout`. #6085
* [BUGFIX] Remove verbose logging in admission-webhook pods. #6095
* [BUGFIX] Fixed HTTP scheme value in the Consul SD configurations. #6098
* [BUGFIX] Skip adding shard relabel rules if they were already set in `additionalScrapeConfigs`. #6099
* [BUGFIX] Fix nil pointer exception when title and text are not set in MS Teams config. #6109
* [CHANGE] Consider secret references without `optional` value to be mandatory instead of optional for `.spec.additionalScrapeConfigs` configs. #5985
* [CHANGE] Remove `prometheus_operator_rule_validation_triggered_total`, `prometheus_operator_rule_validation_errors_total`, `prometheus_operator_alertmanager_config_validation_triggered_total` and `prometheus_operator_alertmanager_config_validation_errors_total` metrics which have been deprecated since v0.55.0. The same information is available from the `apiserver_admission_webhook_rejection_count` metric available since Kubernetes v1.16.0. #6024
* [CHANGE/BUGFIX] Disable HTTP2 connections by default to mitigate CVE-2023-44487. #6028
* [FEATURE] Add support for EC2 service discovery to the ScrapeConfig CRD. #5902#6012
* [FEATURE] Support MSTeams receiver in the AlertmanagerConfig CRD. #6002
* [FEATURE] Add the `sigv4` field to the Alertmanager endpoints for the Prometheus CRD. #6036
* [FEATURE] Support AzureAD authentication for Prometheus remote write. #5852
* [FEATURE] Add the `userKeyFile` and `tokenFile` fields for Pushover to the AlertmanagerConfig CRD. #5886
* [FEATURE] Add `--as` to the operator's CLI arguments for user impersonation. #5906
* [FEATURE] Add the `selectors` field for the Kubernetes service discovery to the ScrapeConfig CRD. #6053
* [ENHANCEMENT] Support `url_file` for Webhook receiver in the Alertmanager configuration. #5876
* [ENHANCEMENT] Support `user_key_file` and `token_file` for Pushover receiver in the Alertmanager configuration. #5876
* [ENHANCEMENT] Use server-side apply instead of update when reconciling the resource's status. #5883#5913#5912
* [ENHANCEMENT] Detect when an invalid storage class is defined. #5792
* [ENHANCEMENT] Add OCI labels to container images. #5946
* [ENHANCEMENT] Add the `operator.prometheus.io/version` annotation to the CRD manifests. #6050
* [BUGFIX] Detect namespace changes without list/watch permissions on the namespace resources. #5934#5898
* [BUGFIX] Avoid operator panic when using ScrapeConfig with the Consul service discovery. #5937
* [BUGFIX] Don't enable Prometheus unmanaged mode when only `.spec.scrapeConfigSelector` is defined. #5941
* [BUGFIX] Prevent Alertmanager pods from joining random clusters. #5945
* [BUGFIX] Fix race condition when deleting Alertmanager, Prometheus, PrometheusAgent and ThanosRuler instances. #5954
* [BUGFIX] Enable klog verbose level only when the log level is debug.. #5981
* [BUGFIX] Reduce memory usage by using secret metadata informer in the PrometheusAgent controller. #5982
* [BUGFIX] Do not strip mandatory fields in the `stripped-down-crds.yaml` manifest file. #6008
* [BUGFIX] Update the resource requests and limits of the config reloader sidecar when updated. #5971
After research, we noticed how the default CPU requests of the config-reloader container were too high compared to the effective usage. We decided to decrease the default CPU requests from `100m` to `10m`. For most environments, this won't make a difference but if you need to increase the values, you can use the `-config-reloader-cpu-request` and `-config-reloader-cpu-limit` CLI arguments of the Prometheus operator.
* [CHANGE] Decrease the default CPU requests for the config-reloader container from `100m` to `10m`. #5539
* [FEATURE] Add `spec.alertmanagerConfiguration.global.PagerdutyURL` to the `Alertmanager` CRD. #5469
2. Update the Prometheus operator's RBAC permissions to manage `ScrapeConfig` resources
(see `example/rbac/prometheus-operator/prometheus-operator-cluster-role.yaml`).
**NOTE**: if these conditions aren't met, the operator will start but it won't
be able to reconcile the `ScrapeConfig` resources.
* [FEATURE] Add the `status` subresource for the `ThanosRuler` CRD. #5520
* [FEATURE] Add `spec.web.timeout` and `spec.web.getConcurrency` to the `Alertmanager` CRD. #5478
* [FEATURE] Add `spec.groups[].limit` to the `Prometheus` CRD. #4999
* [FEATURE] Add ScrapeConfig CRD. #5335
* [ENHANCEMENT] Set a default for `seccompProfile` on the operator and webhook Deployments to `RuntimeDefault`. #5477
* [ENHANCEMENT] Add optional liveness and readiness probes to `prometheus-config-reloader`. This can be enabled via the `--enable-config-reloader-probes` CLI flag. #5449
* [BUGFIX] Don't start the `PrometheusAgent` controller if the CRD isn't present or the operator lacks permissions. #5476
* [BUGFIX] Declare `spec.rules` optional in `PrometheusRule` CRD. #5481
* [BUGFIX] Fix incorrect metric counter value for failed sync status. #5533
2. Update the Prometheus operator's RBAC permissions to manage PrometheusAgents resources
(see `example/rbac/prometheus-operator/prometheus-operator-cluster-role.yaml`).
**NOTE**: if these conditions aren't met, the operator will start but it won't
be able to reconcile the PrometheusAgent resources.
For the first time, the container images associated to this release are signed
using [sigstore](https://www.sigstore.dev/).
* [CHANGE] Remove the `/apis` endpoints from the operator's web server. #5396
* [CHANGE] Set default default value of `spec.portName` to `web`. #5350
* [FEATURE] Add v1alpha1 `PrometheusAgent` CRD to run Prometheus in agent mode. #5385
* [FEATURE] Add `--reload-timeout` argument to the config-reloader binary which controls how long the program will wait for the reload operation to complete (default: 30s). #5349
* [ENHANCEMENT] Set web server's `ReadTimeout` and `ReadHeaderTimeout` to 30s for Prometheus operator and config-reloader to avoid potential slowloris attacks. #5340
* [ENHANCEMENT] Add support for `DropEqual` and `KeepEqual` relabeling actions. #5368
* [ENHANCEMENT] Drop invalid `PrometheusRule` objects instead of failing the reconciliation of Prometheus and ThanosRuler objects. #5221
* [ENHANCEMENT] Add `spec.thanos.blockSize` field to the `Prometheus` CRD. #5360
* [ENHANCEMENT] Add `spec.thanos.configTimeout` and `spec.thanos.configInterval` to the Prometheus CRD. #5399
* [ENHANCEMENT] Add `spec.alertmanagerConfiguration.global.slackApiUrl` field to the `Alertmanager` CRD. #5383
* [ENHANCEMENT] Add `spec.alertmanagerConfiguration.global.opsGenieApiUrl` and `spec.alertmanagerConfiguration.global.opsGenieApiKey` fields to the `Alertmanager` CRD. #5422
* [ENHANCEMENT] Reduce the operator's memory usage by using metadata informers for Kubernetes secrets and configmaps. #5424#5448
* [BUGFIX] Add `init-config-reloader` init container to avoid a restart of the Alertmanager's `config-reloader` container when the pod starts. #5358
* [ENHANCEMENT] Added `enableHttp2` field to `AlertingEndpoints`#5152
* [ENHANCEMENT] Updated `ThanosRuler` arguments (`QueryConfig`, `AlertManagerConfig`, `ObjectStorageConfig` and `TracingConfig`) to be directly read from secrets instead of using ENV vars. #5122
* [CHANGE] Added `filterRunning` field to the PodMonitor CRD. By default, non-running pods are dropped by the Prometheus service discovery. To preserve the old behavior and keep pods which aren't running, set `filterRunning: false`. #5049
* [FEATURE] Added `grpcListenLocal` and `httpListenLocal` fields to the Thanos sidecar configuration of the Prometheus CRD. #5045
* [FEATURE] Added `hostNetwork` field to the Prometheus CRD. #5010
* [FEATURE] Added `spec.tsdb.outOfOrderTimeWindow` field to the Prometheus CRD to allow out-of-order samples in TSDB. #5071
* [ENHANCEMENT] Added columns showing the Prometheus conditions to the output of `kubectl get prometheus`. #5055
* [ENHANCEMENT] Added `observedGeneration` field to the Prometheus status conditions. #5005
* [ENHANCEMENT] Added configuration option in the jsonnet mixins to specify the aggregation labels. #4814
* [ENHANCEMENT] Added `attachMetadata` field to the PodMonitor CRD. #4792
* [BUGFIX] Fixed the curl command for exec probes when `listenLocal` is set to true in the Prometheus object. It avoids temporary service outage due to long WAL replays. #4804
* [CHANGE] Added validation at the API level for size-based fields of the Prometheus CRD. #4661
* [CHANGE] Added validation at the API level for log level and format fields of the Alertmanager, Prometheus and ThanosRuler CRDs. #4638
* [CHANGE] Added validation at the API level for duration and time-based fields of the Prometheus CRD. #4684
* [CHANGE] Added shortnames for custom resources (`amcfg` for AlertmanagerConfig, `am` for Alertmanager, `pmon` for PodMonitor, `prb` for Probe, `prom` for Prometheus, `smon` for ServiceMonitor, `ruler` for Thanos Ruler). #4680
* [FEATURE] Added `status` subresource to the Prometheus CRD. #4580
* [ENHANCEMENT] Added `excludedFromEnforce` field to the Prometheus CRD. It allows to define PodMonitor, ServiceMonitor, Probe or PrometheusRule objects for which the namespace label enforcement (if enabled) should not be applied. This deprecates `prometheusRulesExcludedFromEnforce` which is still supported but users are encouraged to migrate to the new field. #4397
* [ENHANCEMENT] Added `enableRemoteWriteReceiver` field to the Prometheus CRD. #4633
* [ENHANCEMENT] Added `entity` and `actions` fields for the OpsGenie receiver to the AlertmanagerConfig CRD. #4697
* [CHANGE] Enabled read-only root filesystem for containers generated from the Prometheus, Alertmanager and ThanosRuler objects. #4552
* [CHANGE] Disabled privilege escalation for the containers generated from Prometheus, Alertmanager and ThanosRuler objects. #4552
* [CHANGE] Dropped all capabilities for the containers generated from Prometheus, Alertmanager and ThanosRuler objects. #4552
* [CHANGE] Added `emptyDir` volume to the Prometheus statefulset when `spec.queryLogFile` is only a base filename (e.g. `query.log` as opposed to `/tmp/query.log`). When the path contains a full path, a volume + volume mount should be explicitly given in the Prometheus spec since the root file system is now read-only. #4566
* [CHANGE/BUGFIX] Added skip TLS verify for the config-reloader HTTP client when informing Prometheus/Alertmanager on a config reload (partial fix for #4273). #4592
* [CHANGE] Switched using the `endpointslice` role for Prometheus by default if it is supported by the Kubernetes API. #4535
* [CHANGE] Remove deprecated flags `--config-reloader-memory` and `--config-reloader-cpu` in favor of `--config-reloader-memory-limit`, `--config-reloader-memory-request`, `--config-reloader-cpu-limit`, and `--config-reloader-cpu-request`. #3884
* [CHANGE] Remove use of Kubernetes API versions being removed in v1.22. #4171
* [FEATURE] Added support for OAuth2 authentication in remote read and remote write configuration. #4113
* [FEATURE] Added OAuth2 configuration for ServiceMonitor, PodMonitor and Probe. #4170
* [FEATURE] Added `prometheus_operator_spec_shards` metric for exposing the number of shards set on prometheus operator spec. #4173
* [FEATURE] Support for `Authorization` section in various prometheus sections. #4180
* [FEATURE] Support `label_limit`, `label_name_length_limit` and `label_value_length_limit` configuration fields at the Prometheus CRD level as well as support individual limits per ServiceMonitor, PodMonitor and Probe resources. #4195
* [FEATURE] Added sample and target limits to Probe. #4207
* [FEATURE] Added `send_exemplars` field to the `remote_write` configuration in Prometheus. #4215#4160
* [ENHANCEMENT] Support loading ClusterConfig from concatenated KUBECONFIG env. #4154
* [ENHANCEMENT] Include PrometheusRule in prometheus-operator CRD category. #4213
* [ENHANCEMENT] Preserve annotations set by kubectl. #4185
* [BUGFIX] Thanos: listen to all available IP addresses instead of `POD_IP`, simplifies istio management. #4038
* [BUGFIX] Add port name mapping to ConfigReloader to avoid reloader-web probe failure. #4187
* [BUGFIX] Handle Thanos rules `partial_response_strategy` field in validating admission webhook. #4217
will follow in future releases of the operator. The current version of the CRD
is `v1alpha1` meaning that testing/feedback is encouraged and welcome but the
feature is not yet considered stable and the API is subject to change in the
future.
* [CHANGE] Use a single reloader sidecar (instead of 2) for Prometheus. The `--config-reloader-image` flag is deprecated and will be removed in a future release (not before v0.45.0). *Make sure to start the operator with a version of `--prometheus-config-reloader` that is at least `v0.43.0` otherwise the Prometheus pods will fail to start.*#3457
* [FEATURE] Add `targetLimit` and `enforcedTargetLimit` to the Prometheus CRD. #3571
* [FEATURE] Add initial support for `AlertmanagerConfig` CRD. #3451
* [FEATURE] Add support for Pod Topology Spread Constraints to Prometheus, Alertmanager, and ThanosRuler CRDs. #3598
* [ENHANCEMENT] Allow customization of the Prometheus web page title. #3525
* [ENHANCEMENT] Add metrics for selected/rejected resources and synchronization status. #3421
* [ENHANCEMENT] Configure Thanos sidecar for uploads only when needed. #3485
* [ENHANCEMENT] Add `--version` flag to all binaries + `prometheus_operator_build_info` metric. #359
* [CHANGE] Remove gossip configuration from Thanos sidecar. This means only non-gossip configurations can be used going forward. (#2623, #2629)
* [FEATURE] Add PodMonitor, allowing monitoring pods directly without the necessity to go through a Endpoints of a Service, this is an experimental feature, it may break at any time without notice. (#2566)
* [FEATURE] Add admission webhook to validate `PrometheusRule` objects with Prometheus' promtool linting. (#2551)
* [FEATURE] Add ability to select subset of Prometheus objects to reconcile against, configurable with `--prometheus-instance-selector` flag. (#2615)
* [FEATURE] Add ability to configure size based retention on Prometheus. (#2608)
* [FEATURE] Add ability to use StatefulSet ordinal in external labels. (#2591)
* [ENHANCEMENT] Use /-/healthy and /-/ready for probes in Alertmanager. (#2600)
Note: Both kube-prometheus (#2554) and the Helm Chart (#2416) have been removed from this repository.
kube-prometheus is not hosted as github.com/coroes/kube-prometheus and the helm chart is available at https://github.com/helm/charts/tree/master/stable/prometheus-operator
* [FEATURE] Introduce new flag to control logging format (#1475)
* [FEATURE] Ensure Prometheus Operator container runs as `nobody` user by default (#1393)
* [BUGFIX] Fix reconciliation of Prometheus StatefulSets due to ServiceMonitors and PrometheusRules changes when a single namespace is being watched (#1749)
From this release onwards only Kubernetes versions v1.8 and higher are supported. If you have an older version of Kubernetes and the Prometheus Operator running, we recommend upgrading Kubernetes first and then the Prometheus Operator.
While multiple validation issues have been fixed, it will remain a beta feature in this release. If you want to update validations, you need to either apply the CustomResourceDefinitions located in `example/prometheus-operator-crd` or delete all CRDs and restart the Prometheus Operator.
Some changes cause Prometheus and Alertmanager clusters to be redeployed. If you do not have persistent storage backing your data, this means you will loose the amount of data equal to your retention time.
* [CHANGE] Use canonical `/prometheus` and `/alertmanager` as data dirs in containers.
* [FEATURE] Allow configuring Prometheus and Alertmanager servers to listen on loopback interface, allowing proxies to be the ingress point of those Pods.
* [FEATURE] Allow configuring additional containers in Prometheus and Alertmanager Pods.
* [FEATURE] Add ability to whitelist Kubernetes labels to become Prometheus labels.
* [FEATURE] Add metrics for reconciliation errors: `prometheus_operator_alertmanager_reconcile_errors_total` and `prometheus_operator_prometheus_reconcile_errors_total`.
* [FEATURE] Support `read_recent` and `required_matchers` fields for remote read configurations.
* [FEATURE] Allow disabling any defaults of `SecurityContext` fields of Pods.
This release adds validations as a beta feature. It will only be installed on new clusters, existing CRD definitions will not be updated, this will be done in a future release. Please try out this feature and give us feedback!
* [CHANGE] Default Prometheus version v2.2.0-rc.0.
* [CHANGE] Default Alertmanager version v0.14.0.
* [FEATURE] Generate and add CRD validations.
* [FEATURE] Add ability to set `serviceAccountName` for Alertmanager Pods.
* [FEATURE] Add ability to specify custom `securityContext` for Alertmanager Pods.
* [ENHANCEMENT] Default to non-root security context for Alertmanager Pods.
Starting with this release only Kubernetes `v1.7.x` and up is supported as CustomResourceDefinitions are a requirement for the Prometheus Operator and are only available from those versions and up.
Additionally all objects have been promoted from `v1alpha1` to `v1`. On start up of this version of the Prometheus Operator the previously used `ThirdPartyResource`s and the associated `v1alpha1` objects will be automatically migrated to their `v1` equivalent `CustomResourceDefinition`.
* [CHANGE] All manifests created and used by the Prometheus Operator have been promoted from `v1alpha1` to `v1`.
* [CHANGE] Use Kubernetes `CustomResourceDefinition`s instead of `ThirdPartyResource`s.
* [FEATURE] Add ability to set scrape timeout to `ServiceMonitor`.
* [ENHANCEMENT] Use `StatefulSet` rolling deployments.
* [ENHANCEMENT] Properly set `SecurityContext` for Prometheus 2.0 deployments.
* [ENHANCEMENT] Enable web lifecycle APIs for Prometheus 2.0 deployments.
Warning: This release deprecates the previously used storage definition in favor of upstream PersistentVolumeClaim templates. While this should not have an immediate effect on a running cluster, Prometheus object definitions that have storage configured need to be adapted. The previously existing fields are still there, but have no effect anymore.
* [FEATURE] Add Prometheus 2.0 alpha3 support.
* [FEATURE] Use PVC templates instead of custom storage definition.
* [FEATURE] Add cAdvisor port to kubelet sync.
* [FEATURE] Allow default base images to be configurable.
* [FEATURE] Configure Prometheus to only use necessary namespaces.
* [ENHANCEMENT] Improve rollout detection for Alertmanager clusters.
