Skip to main content

Release Notes v1.86

YAKE/23KE release notes and upgrade guide​

danger

This update renames 23ke to YAKE. You need to rename the config secret and GitRepository resource as described.

danger

This update definitely needs backups to be configured. If you are running a 23KE instance without backups, enable backups before performing this update.

Prerequisites​

etcd downgrade​

In order to align the versions of etcd and etcd-backup-restore with gardener/etcd-druid, we perform a downgrade to etcd-3.4.26 and an upgrade to etcd-backup-restore-0.24.7. This is also expected to improve the stability of the backup process. For the upgrade, you need to

  • Make sure you have an up-to-date backup of etcd-0. To perform a full backup you can use the following request:

    kubectl -n garden exec -it etcd-0 curl localhost:8080/snapshot/full

  • Delete the statefulset etcd and etcd-events in the garden namespace

    kubectl delete statefulset -n garden etcd
    kubectl delete statefulset -n garden etcd-events

During the upgrade helm will create new persistentVolumes for the virtual garden's etcds. These volumes are prefixed by virtual-garden-.

Temporarily remove gardener-metrics-exporter​

To work around an issue with how YAKE uses gardener-metrics-exporter's chart, delete its deployment

kubectl delete -n garden deployment gardener-metrics-exporter

Upgrade​

Perform the migration from 23ke to YAKE execute the following steps.​

  • Create copy of Secret 23ke-config named yake-config

    kubectl get secret -n flux-system 23ke-config -o yaml | kubectl-neat | yq '.metadata.name="yake-config"' | kubectl apply -f -

  • Create new GitRepository source named yake.

    cat <<EOF | kubectl apply -f -
    apiVersion: source.toolkit.fluxcd.io/v1
    kind: GitRepository
    metadata:
      name: yake
      namespace: flux-system
    spec:
      interval: 1m
      ref:
        tag: v1.86.0-0
      timeout: 60s
      url: https://github.com/yakecloud/yake
    EOF

  • Suspend 23ke Kustomization. This can be named differently in your setup.

    flux suspend ks 23ke

  • Relabel Kustomizations created by the main 23ke Kustomization

    kubectl label ks -n flux-system -l kustomize.toolkit.fluxcd.io/name=23ke kustomize.toolkit.fluxcd.io/name=yake --overwrite

  • Recreate the main Kustomization with name yake

    kubectl get ks -n flux-system 23ke -o yaml | kubectl-neat | yq '.metadata.name="yake" | .spec.sourceRef.name="yake"' | .spec.sourceRef.kind="GitRepository"' | kubectl apply -f -

  • Resume the yake Kustomization

    flux resume ks yake

Cleanup obsolete resources​

Once you confirmed everything's working correctly you can remove obsolete resources.

warning

If your base kustomization is itself reconciled by flux, make sure to remove it via git rather than manually. Otherwise, it will be restored in an active state and may break your setup!

  • Delete the old 23ke Kustomization

    kubectl delete ks -n flux-system 23ke

  • Delete the old GitRepository resource

    kubectl delete gitrepo -n flux-system 23ke

  • Delete Secret 23ke-config

    kubectl delete secret -n flux-system 23ke-config

  • (Optional) Delete the old persistentVolumeClaims and their persistentVolumes belonging to the already deleted statefulsets of etcd and etcd-events.

    kubectl get pvc -n garden | grep '^etcd'
    kubectl get pv | grep garden/etcd

Related upstream release notes / changelogs​

Update shoot-networking-filter to 0.16.0

[gardener/gardener-extension-shoot-networking-filter]

✨ New Features​

  • [USER] Update image of egress-filter to 0.14.0 by @axel7born [#107]
  • [USER] Mount /run/xtables.lock to prevent concurrent modifications of iptables rules. by @axel7born [#106]

🏃 Others​

  • [OPERATOR] Bump github.com/gardener/gardener from 1.84.0 to 1.84.1. by @dependabot[bot] [#102]
  • [OPERATOR] Bumps golang from 1.21.4 to 1.21.5. by @dependabot[bot] [#105]
  • [OPERATOR] Bump github.com/gardener/gardener from 1.84.1 to 1.85.0. by @dependabot[bot] [#104]

Docker Images​

  • gardener-extension-shoot-networking-filter: eu.gcr.io/gardener-project/gardener/extensions/shoot-networking-filter:v0.16.0
Update provider-aws to 1.51.0

[gardener/gardener-extension-provider-aws]

🏃 Others​

  • [OPERATOR] The following golang dependencies have been upgraded :
    • gardener/gardener: v1.81.6->v1.83.2 by @shafeeqes [#828]
  • [OPERATOR] Add documentation for the "flow" infrastructure reconciler. by @kon-angelo [#827]
  • [DEVELOPER] Add new unit tests. by @axel7born [#829]

Docker Images​

  • gardener-extension-admission-aws: eu.gcr.io/gardener-project/gardener/extensions/admission-aws:v1.51.0
  • gardener-extension-provider-aws: eu.gcr.io/gardener-project/gardener/extensions/provider-aws:v1.51.0
Update shoot-rsyslog-relp to 0.3.0

[gardener/gardener-extension-shoot-rsyslog-relp]

⚠️ Breaking Changes​

  • [OPERATOR] The security.gardener.cloud/pod-security-enforce annotation in the ControllerRegistration is set to baseline. With this, the pods running in the extension namespace should comply with baseline pod-security standard. by @AleksandarSavchev [#17]

✨ New Features​

  • [USER] The shoot-rsyslog-relp configuration now allows users to specify which tls library should be used by librerlp when tls communication is enabled via the tls.tlsLib optional field. The possible options are gnutls and openssl. When the field is omitted, librelp uses its default tls library which in most cases is gnutls. More information can be found here: https://www.rsyslog.com/doc/v8-stable/configuration/modules/imrelp.html#tls-tlslib by @plkokanov [#27]
  • [USER] shoot-rsyslog-relp extension now supports Shoot Force Deletion. by @acumino [#24]

🏃 Others​

  • [OPERATOR] Metrics for the rsyslog service running on the shoot nodes are now exposed and collected according to the following:
    • The metrics are available on the node-exporter's /metrics endpoint.
    • The names of the new metrics match the rsyslog_pstat_.+ regex.
    • The metrics are scraped and collected in the shoot's prometheus instance.
    • A dedicated plutono dashboard is added which displays the rsyslog metrics. by @plkokanov [#32]
  • [OPERATOR] Fixed an issue where the rsyslog systemd unit could become stuck in a failed state immediately after it is installed on the shoot's nodes, if the shoot-rsyslog-relp extension was enabled on the shoot before that. The configure-rsyslog.sh script which is responsible for configuring and restarting the rsyslog systemd unit will now wait for the syslog.service symlink to be created before attempting to configure and restart the rsyslog systemd unit. by @plkokanov [#34]
  • [OPERATOR] The shoot-rsyslog-relp extension is now aligned with Gardener's component checklist:
    • RBAC for the shoot-rsyslog-relp extension controller have been drastically reduced to only the required ones.
    • The deployment for the shoot-rsyslog-relp extension controller now contains the proper label for HA - high-availability-config.resources.gardener.cloud/type: controller
    • The shoot-rsyslog-relp admission pod no longer has a SecurityContext. This will be automatically added by the seccomp-profile webhook of the gardener-resource-manager
    • The rsyslog-relp-configurator and rsyslog-relp-configuration-cleaner pods now use the RuntimeDefault seccomp profile.
    • The init containers of the rsyslog-relp-configurator and rsyslog-relp-configuration-cleaner pods no longer run in privileged mode.
    • The rsyslog-relp-configurator and rsyslog-relp-configuration-cleaner now specify resource requests and limits.
    • PodSecurityPolicys for the rsyslog-relp-configurator and rsyslog-relp-configuration-cleaner are now deployed in the shoot cluster, if its kubernetes version is 1.24.x. by @plkokanov [#29]
  • [OPERATOR] The healthcheck controller is now removed. Starting v1.65.0, gardenlet perform health checks for all ManagedResources in the Shoot control plane in the Seed. There is no longer need of the custom healthcheck controller in the shoot-rsyslog-relp extension as it was doing the same job. It was performing health check for the ManagedResource it deploys. by @plkokanov [#28]
  • [OPERATOR] The rsyslog-relp-configuration-cleaner is no longer deployed on Shoot deletion with shoot-rsyslog-relp extension enabled. The Extension deletion occurs after the Worker deletion. There are no Nodes, hence there is no need to clean up registry configuration. by @plkokanov [#30]

Docker Images​

  • gardener-extension-shoot-rsyslog-relp-admission: eu.gcr.io/gardener-project/gardener/extensions/shoot-rsyslog-relp-admission:v0.3.0
  • gardener-extension-shoot-rsyslog-relp: eu.gcr.io/gardener-project/gardener/extensions/shoot-rsyslog-relp:v0.3.0
Update etcd to 6.0.0

What's Changed​

New Contributors​

Full Changelog: https://github.com/gardener-community/etcd/compare/5.3.2...6.0.0

Update etcd to 6.0.0

What's Changed​

New Contributors​

Full Changelog: https://github.com/gardener-community/etcd/compare/5.3.2...6.0.0

Update gardener-metrics-exporter to 0.28.0

[gardener/gardener-metrics-exporter]

🏃 Others​

  • [OPERATOR] Metrics are exported for pending shoots as well. by @timebertt [#91]
  • [OPERATOR] Minimum and maximum shoot node counts are no longer reversed. by @Sinscerly [#90]

Docker Images​

  • metrics-exporter: eu.gcr.io/gardener-project/gardener/metrics-exporter:0.28.0
Update gardener-metrics-exporter to 0.28.0

[gardener/gardener-metrics-exporter]

🏃 Others​

  • [OPERATOR] Metrics are exported for pending shoots as well. by @timebertt [#91]
  • [OPERATOR] Minimum and maximum shoot node counts are no longer reversed. by @Sinscerly [#90]

Docker Images​

  • metrics-exporter: eu.gcr.io/gardener-project/gardener/metrics-exporter:0.28.0
Update shoot-networking-filter to 0.16.1

no release notes available

Docker Images​

  • gardener-extension-shoot-networking-filter: eu.gcr.io/gardener-project/gardener/extensions/shoot-networking-filter:v0.16.1
Update provider-alicloud to 1.50.1

no release notes available

Docker Images​

  • gardener-extension-admission-alicloud: eu.gcr.io/gardener-project/gardener/extensions/admission-alicloud:v1.50.1
  • gardener-extension-provider-alicloud: eu.gcr.io/gardener-project/gardener/extensions/provider-alicloud:v1.50.1
Update provider-aws to 1.51.1

[gardener/gardener-extension-provider-aws]

🏃 Others​

Docker Images​

  • gardener-extension-admission-aws: eu.gcr.io/gardener-project/gardener/extensions/admission-aws:v1.51.1
  • gardener-extension-provider-aws: eu.gcr.io/gardener-project/gardener/extensions/provider-aws:v1.51.1
Update shoot-dns-service to 1.41.0

[gardener/gardener-extension-shoot-dns-service]

⚠️ Breaking Changes​

  • [OPERATOR] CA and server certificates for the admission component are managed automatically. Passing custom certificates via Helm values is not supported anymore. by @timuthy [#266]

🐛 Bug Fixes​

  • [OPERATOR] An issue was fixed that caused shoot clusters with a shoot-dns-service extension configuration not to be validated during creation. Potential validation errors only happened later and remained unnoticed, e.g. when update requests from Gardenlet were denied and shoot reconciliation got stuck. by @timuthy [#270]

🏃 Others​

  • [OPERATOR] Bumps golang from 1.21.4 to 1.21.5. by @dependabot[bot] [#267]
  • [OPERATOR] Bump github.com/gardener/gardener from 1.83.0 to 1.84.0. by @dependabot[bot] [#261]
  • [OPERATOR] Bump github.com/gardener/gardener from 1.84.1 to 1.85.0. by @timuthy [#266]
  • [OPERATOR] Bump github.com/gardener/gardener from 1.84.0 to 1.84.1. by @dependabot[bot] [#262]

[gardener/external-dns-management]

⚠️ Breaking Changes​

  • [USER] NS records are not retrieved anymore for all accessible hosted zones to avoid reading all DNS record sets of all hosted zones periodically independently if they are used. Only hosted zones with active DNSProviders are synched, but without caring about consequences of NS records for subdomains. If there are many large hosted zones accessible for given credentials and there are only DNSProviders using a few of these zones (either by domain or zone include), the period synchronisation of the zone state for all other hosted zones is avoided. This can result in a significant reduction of requests to the provider backend. As a downside of this change, applying a DNSEntry for a forwarded subdomain now results in a DNS record set in the parent hosted zone, if the real hosted zone is unknown to the controller. Formerly, applying such a DNSEnty resulted in an error state. No action is necessary from the users, this is only a "heads up" for the changed behaviour if NS records are used for subdomains. by @MartinWeindel [gardener/external-dns-management#336]

🏃 Others​

  • [USER] Validate provider domain includes and excludes for forbidden wildcard domains. by @MartinWeindel [gardener/external-dns-management#335]
  • [OPERATOR] Bumps golang from 1.21.3 to 1.21.4. by @dependabot[bot] [gardener/external-dns-management#333]

Docker Images​

  • gardener-extension-admission-shoot-dns-service: eu.gcr.io/gardener-project/gardener/extensions/admission-shoot-dns-service:v1.41.0
  • gardener-extension-shoot-dns-service: eu.gcr.io/gardener-project/gardener/extensions/shoot-dns-service:v1.41.0
Update gardener-controlplane to 1.86.0

[gardener/gardener]

⚠️ Breaking Changes​

  • [OPERATOR] All virtual garden access Secrets have to be labeled with with resources.gardener.cloud/class=shoot. Otherwise the virtual-GRM won't consider the Secrets and won't renew them. by @rfranzke [#8883]
  • [OPERATOR] The ContainerdRegistryHostsDir feature gate has been promoted to beta and is now turned on by default. by @ialidzhikov [#8873]
  • [DEVELOPER] Support for the deprecated NetworkPolicy annotations networking.resources.gardener.cloud/from-policy-allowed-ports and networking.resources.gardener.cloud/from-policy-pod-label-selector has been removed. Use networking.resources.gardener.cloud/from-<some-alias>-allowed-ports instead (documentation). by @rfranzke [#8883]

📰 Noteworthy​

  • [DEVELOPER] The local Gardener environments for e2e tests running in Prow are now backed by the registry-cache extensions enabled in the Prow cluster. This should have a positive impact on the network I/O for image pulls and resulting costs. by @oliver-goetz [#8880]
  • [OPERATOR] The WorkerlessShoots has been promoted to GA and is now locked to "enabled by default". by @acumino [#8906]

✨ New Features​

  • [USER] It is now possible to configure the resources encrypted in the ETCD for shoot clusters, see this document for more details. by @shafeeqes [#8842]
  • [USER] The shoots/viewerkubeconfig subresource now also restricts viewer access to resources which are specified in the spec.kubernetes.kubeAPIServer.encryptionConfig in the Shoot in addition to Secrets. by @shafeeqes [#8966]
  • [USER] It is now possible to request a kubeconfig with read-only access (all APIs except core/v1.Secret) for shoot clusters by using the new shoots/viewerkubeconfig subresource. Read all about it here. by @rfranzke [#8870]
  • [OPERATOR] The vpn-seed-server component now supports IPv4 seed clusters hosting IPv6 shoot clusters. by @DockToFuture [#8830]
  • [OPERATOR] It is now possible to configure the resources encrypted in the ETCD for the virtual garden cluster, see this document for more details. by @shafeeqes [#8842]

🐛 Bug Fixes​

  • [DEPENDENCY] extension library: An issue causing the Worker restore operation to fail for hibernated Shoots is now fixed. by @ialidzhikov [#8943]
  • [OPERATOR] A bug causing the Shoot to use the wrong istio load balancer if the ExposureClass name and the exposureclass handler name are not the same is now fixed. by @shafeeqes [#8926]
  • [OPERATOR] Fixed a bug where a Shoot with an expired machine image or Kubernetes version could be created. For machine images: only allow updating to a higher expired machine image version for an existing worker pool For Kubernetes versions: do not allow creation of a worker pool with an expired K8s version, but still allow updating an existing worker pool to a higher expired version. by @danielfoehrKn [#8854]
  • [OPERATOR] gardener-node-agent's OperatingSystemConfig controller now respects the reconciliation timeout and aborts the reconciliation if it takes too long. by @rfranzke [#8907]
  • [OPERATOR] gardener-node-agent now creates temporary directories and files under /var/lib/gardener-node-agent/tmp instead of /tmp. This fixes issues during OperatingSystemConfig reconciliation which occur when /var and /tmp are backed by different file systems or devices. by @rfranzke [#8894]
  • [OPERATOR] gardener-node-agent now skips disablement and stop attempts of deleted units in case their unit files have already been cleaned up by third parties. by @rfranzke [#8898]
  • [OPERATOR] gardener-node-agent now converts the hostname to lower case to match kubelet behaviour when it maintains the kubernetes.io/hostname label on Nodes. by @rfranzke [#8902]

🏃 Others​

  • [OPERATOR] gardener-node-agent now stops waiting for systemd command results if they don't respond back after 10s. by @rfranzke [#8919]
  • [OPERATOR] Add unhealthy nodes dashboard. by @adenitiu [#8869]
  • [OPERATOR] Add egressCIDRs field to the infrastructureStatus resource. This allows provider-extensions to specify a list of stable CIDRs used as source IP for traffic generated by the shoot's worker nodes. by @kon-angelo [#8888]
  • [DEVELOPER] Add support for optional SCRIPT_ROOT environment var in vgopath enabled hack scripts by @afritzler [#8935]

[gardener/vpn2]

⚠️ Breaking Changes​

  • [OPERATOR] Change OCI Image Registry from GCR (eu.gcr.io/gardener-project) to Artifact-Registry (europe-docker.pkg.dev/gardener-project/releases). Users should update their references. by @ccwienk [gardener/vpn2#62]

📰 Noteworthy​

  • [OPERATOR] added ipv6 single-stack support by @nschad [gardener/vpn2#45]
  • [OPERATOR] Add iptables backend detection to firewall script. by @axel7born [gardener/vpn2#64]

[gardener/apiserver-proxy]

📰 Noteworthy​

  • [OPERATOR] Remove the optional creation of iptables rules and the flag--setup-iptables. by @axel7born [gardener/apiserver-proxy#70]

[gardener/gardener-metrics-exporter]

🏃 Others​

  • [OPERATOR] Metrics are exported for pending shoots as well. by @timebertt [gardener/gardener-metrics-exporter#91]

Docker Images​

  • admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.86.0
  • apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.86.0
  • controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.86.0
  • gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.86.0
  • node-agent: eu.gcr.io/gardener-project/gardener/node-agent:v1.86.0
  • operator: eu.gcr.io/gardener-project/gardener/operator:v1.86.0
  • resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.86.0
  • scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.86.0
Update gardener-controlplane to 1.86.0

[gardener/gardener]

⚠️ Breaking Changes​

  • [OPERATOR] All virtual garden access Secrets have to be labeled with with resources.gardener.cloud/class=shoot. Otherwise the virtual-GRM won't consider the Secrets and won't renew them. by @rfranzke [#8883]
  • [OPERATOR] The ContainerdRegistryHostsDir feature gate has been promoted to beta and is now turned on by default. by @ialidzhikov [#8873]
  • [DEVELOPER] Support for the deprecated NetworkPolicy annotations networking.resources.gardener.cloud/from-policy-allowed-ports and networking.resources.gardener.cloud/from-policy-pod-label-selector has been removed. Use networking.resources.gardener.cloud/from-<some-alias>-allowed-ports instead (documentation). by @rfranzke [#8883]

📰 Noteworthy​

  • [DEVELOPER] The local Gardener environments for e2e tests running in Prow are now backed by the registry-cache extensions enabled in the Prow cluster. This should have a positive impact on the network I/O for image pulls and resulting costs. by @oliver-goetz [#8880]
  • [OPERATOR] The WorkerlessShoots has been promoted to GA and is now locked to "enabled by default". by @acumino [#8906]

✨ New Features​

  • [USER] It is now possible to configure the resources encrypted in the ETCD for shoot clusters, see this document for more details. by @shafeeqes [#8842]
  • [USER] The shoots/viewerkubeconfig subresource now also restricts viewer access to resources which are specified in the spec.kubernetes.kubeAPIServer.encryptionConfig in the Shoot in addition to Secrets. by @shafeeqes [#8966]
  • [USER] It is now possible to request a kubeconfig with read-only access (all APIs except core/v1.Secret) for shoot clusters by using the new shoots/viewerkubeconfig subresource. Read all about it here. by @rfranzke [#8870]
  • [OPERATOR] The vpn-seed-server component now supports IPv4 seed clusters hosting IPv6 shoot clusters. by @DockToFuture [#8830]
  • [OPERATOR] It is now possible to configure the resources encrypted in the ETCD for the virtual garden cluster, see this document for more details. by @shafeeqes [#8842]

🐛 Bug Fixes​

  • [DEPENDENCY] extension library: An issue causing the Worker restore operation to fail for hibernated Shoots is now fixed. by @ialidzhikov [#8943]
  • [OPERATOR] A bug causing the Shoot to use the wrong istio load balancer if the ExposureClass name and the exposureclass handler name are not the same is now fixed. by @shafeeqes [#8926]
  • [OPERATOR] Fixed a bug where a Shoot with an expired machine image or Kubernetes version could be created. For machine images: only allow updating to a higher expired machine image version for an existing worker pool For Kubernetes versions: do not allow creation of a worker pool with an expired K8s version, but still allow updating an existing worker pool to a higher expired version. by @danielfoehrKn [#8854]
  • [OPERATOR] gardener-node-agent's OperatingSystemConfig controller now respects the reconciliation timeout and aborts the reconciliation if it takes too long. by @rfranzke [#8907]
  • [OPERATOR] gardener-node-agent now creates temporary directories and files under /var/lib/gardener-node-agent/tmp instead of /tmp. This fixes issues during OperatingSystemConfig reconciliation which occur when /var and /tmp are backed by different file systems or devices. by @rfranzke [#8894]
  • [OPERATOR] gardener-node-agent now skips disablement and stop attempts of deleted units in case their unit files have already been cleaned up by third parties. by @rfranzke [#8898]
  • [OPERATOR] gardener-node-agent now converts the hostname to lower case to match kubelet behaviour when it maintains the kubernetes.io/hostname label on Nodes. by @rfranzke [#8902]

🏃 Others​

  • [OPERATOR] gardener-node-agent now stops waiting for systemd command results if they don't respond back after 10s. by @rfranzke [#8919]
  • [OPERATOR] Add unhealthy nodes dashboard. by @adenitiu [#8869]
  • [OPERATOR] Add egressCIDRs field to the infrastructureStatus resource. This allows provider-extensions to specify a list of stable CIDRs used as source IP for traffic generated by the shoot's worker nodes. by @kon-angelo [#8888]
  • [DEVELOPER] Add support for optional SCRIPT_ROOT environment var in vgopath enabled hack scripts by @afritzler [#8935]

[gardener/vpn2]

⚠️ Breaking Changes​

  • [OPERATOR] Change OCI Image Registry from GCR (eu.gcr.io/gardener-project) to Artifact-Registry (europe-docker.pkg.dev/gardener-project/releases). Users should update their references. by @ccwienk [gardener/vpn2#62]

📰 Noteworthy​

  • [OPERATOR] added ipv6 single-stack support by @nschad [gardener/vpn2#45]
  • [OPERATOR] Add iptables backend detection to firewall script. by @axel7born [gardener/vpn2#64]

[gardener/apiserver-proxy]

📰 Noteworthy​

  • [OPERATOR] Remove the optional creation of iptables rules and the flag--setup-iptables. by @axel7born [gardener/apiserver-proxy#70]

[gardener/gardener-metrics-exporter]

🏃 Others​

  • [OPERATOR] Metrics are exported for pending shoots as well. by @timebertt [gardener/gardener-metrics-exporter#91]

Docker Images​

  • admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.86.0
  • apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.86.0
  • controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.86.0
  • gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.86.0
  • node-agent: eu.gcr.io/gardener-project/gardener/node-agent:v1.86.0
  • operator: eu.gcr.io/gardener-project/gardener/operator:v1.86.0
  • resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.86.0
  • scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.86.0
Update gardenlet to 1.86.0

[gardener/gardener]

⚠️ Breaking Changes​

  • [OPERATOR] All virtual garden access Secrets have to be labeled with with resources.gardener.cloud/class=shoot. Otherwise the virtual-GRM won't consider the Secrets and won't renew them. by @rfranzke [#8883]
  • [OPERATOR] The ContainerdRegistryHostsDir feature gate has been promoted to beta and is now turned on by default. by @ialidzhikov [#8873]
  • [DEVELOPER] Support for the deprecated NetworkPolicy annotations networking.resources.gardener.cloud/from-policy-allowed-ports and networking.resources.gardener.cloud/from-policy-pod-label-selector has been removed. Use networking.resources.gardener.cloud/from-<some-alias>-allowed-ports instead (documentation). by @rfranzke [#8883]

📰 Noteworthy​

  • [DEVELOPER] The local Gardener environments for e2e tests running in Prow are now backed by the registry-cache extensions enabled in the Prow cluster. This should have a positive impact on the network I/O for image pulls and resulting costs. by @oliver-goetz [#8880]
  • [OPERATOR] The WorkerlessShoots has been promoted to GA and is now locked to "enabled by default". by @acumino [#8906]

✨ New Features​

  • [USER] It is now possible to configure the resources encrypted in the ETCD for shoot clusters, see this document for more details. by @shafeeqes [#8842]
  • [USER] The shoots/viewerkubeconfig subresource now also restricts viewer access to resources which are specified in the spec.kubernetes.kubeAPIServer.encryptionConfig in the Shoot in addition to Secrets. by @shafeeqes [#8966]
  • [USER] It is now possible to request a kubeconfig with read-only access (all APIs except core/v1.Secret) for shoot clusters by using the new shoots/viewerkubeconfig subresource. Read all about it here. by @rfranzke [#8870]
  • [OPERATOR] The vpn-seed-server component now supports IPv4 seed clusters hosting IPv6 shoot clusters. by @DockToFuture [#8830]
  • [OPERATOR] It is now possible to configure the resources encrypted in the ETCD for the virtual garden cluster, see this document for more details. by @shafeeqes [#8842]

🐛 Bug Fixes​

  • [DEPENDENCY] extension library: An issue causing the Worker restore operation to fail for hibernated Shoots is now fixed. by @ialidzhikov [#8943]
  • [OPERATOR] A bug causing the Shoot to use the wrong istio load balancer if the ExposureClass name and the exposureclass handler name are not the same is now fixed. by @shafeeqes [#8926]
  • [OPERATOR] Fixed a bug where a Shoot with an expired machine image or Kubernetes version could be created. For machine images: only allow updating to a higher expired machine image version for an existing worker pool For Kubernetes versions: do not allow creation of a worker pool with an expired K8s version, but still allow updating an existing worker pool to a higher expired version. by @danielfoehrKn [#8854]
  • [OPERATOR] gardener-node-agent's OperatingSystemConfig controller now respects the reconciliation timeout and aborts the reconciliation if it takes too long. by @rfranzke [#8907]
  • [OPERATOR] gardener-node-agent now creates temporary directories and files under /var/lib/gardener-node-agent/tmp instead of /tmp. This fixes issues during OperatingSystemConfig reconciliation which occur when /var and /tmp are backed by different file systems or devices. by @rfranzke [#8894]
  • [OPERATOR] gardener-node-agent now skips disablement and stop attempts of deleted units in case their unit files have already been cleaned up by third parties. by @rfranzke [#8898]
  • [OPERATOR] gardener-node-agent now converts the hostname to lower case to match kubelet behaviour when it maintains the kubernetes.io/hostname label on Nodes. by @rfranzke [#8902]

🏃 Others​

  • [OPERATOR] gardener-node-agent now stops waiting for systemd command results if they don't respond back after 10s. by @rfranzke [#8919]
  • [OPERATOR] Add unhealthy nodes dashboard. by @adenitiu [#8869]
  • [OPERATOR] Add egressCIDRs field to the infrastructureStatus resource. This allows provider-extensions to specify a list of stable CIDRs used as source IP for traffic generated by the shoot's worker nodes. by @kon-angelo [#8888]
  • [DEVELOPER] Add support for optional SCRIPT_ROOT environment var in vgopath enabled hack scripts by @afritzler [#8935]

[gardener/vpn2]

⚠️ Breaking Changes​

  • [OPERATOR] Change OCI Image Registry from GCR (eu.gcr.io/gardener-project) to Artifact-Registry (europe-docker.pkg.dev/gardener-project/releases). Users should update their references. by @ccwienk [gardener/vpn2#62]

📰 Noteworthy​

  • [OPERATOR] added ipv6 single-stack support by @nschad [gardener/vpn2#45]
  • [OPERATOR] Add iptables backend detection to firewall script. by @axel7born [gardener/vpn2#64]

[gardener/apiserver-proxy]

📰 Noteworthy​

  • [OPERATOR] Remove the optional creation of iptables rules and the flag--setup-iptables. by @axel7born [gardener/apiserver-proxy#70]

[gardener/gardener-metrics-exporter]

🏃 Others​

  • [OPERATOR] Metrics are exported for pending shoots as well. by @timebertt [gardener/gardener-metrics-exporter#91]

Docker Images​

  • admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.86.0
  • apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.86.0
  • controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.86.0
  • gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.86.0
  • node-agent: eu.gcr.io/gardener-project/gardener/node-agent:v1.86.0
  • operator: eu.gcr.io/gardener-project/gardener/operator:v1.86.0
  • resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.86.0
  • scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.86.0
Update cert-management to 0.11.4

[gardener/cert-management]

⚠️ Breaking Changes​

  • [OPERATOR] Change OCI Image Registry from GCR (eu.gcr.io/gardener-project) to Artifact-Registry (europe-docker.pkg.dev/gardener-project/releases). Users should update their references. by @ccwienk [#156]

🏃 Others​

  • [OPERATOR] Update expirationDate if secret is shared between certs by @MartinWeindel [#152]
  • [OPERATOR] Updated dependencies controller-manager-library and kubernetes from v1.28.2 to v1.28.3 by @MartinWeindel [#149]
  • [OPERATOR] Bumps golang from 1.21.3 to 1.21.4. by @dependabot[bot] [#148]
  • [OPERATOR] Bumps golang from 1.21.4 to 1.21.5. by @dependabot[bot] [#154]
  • [USER] Improve message if DNS entry is not getting ready. by @MartinWeindel [#151]
  • [DEVELOPER] Remove vendoring by @MartinWeindel [#157]

Docker Images​

  • cert-management: europe-docker.pkg.dev/gardener-project/releases/cert-controller-manager:v0.11.4
Update dashboard to 1.71.0

[gardener/dashboard]

⚠️ Breaking Changes​

  • [OPERATOR] Change OCI Image Registry from GCR (eu.gcr.io/gardener-project) to Artifact-Registry (europe-docker.pkg.dev/gardener-project/releases) by @ccwienk [#1645]

✨ New Features​

  • [USER] Enhanced Client-Side Validation: Previously, if there was a validation error, the submit buttons on Dialogs and the Create Cluster page would be greyed out. Now, these buttons remain active even if there is a validation error. Clicking on the submit button in the presence of a validation error will not execute the submit logic. Instead, it will trigger the validation process to ensure that all validation error messages are clearly visible at the respective input fields. In addition, we've made it easier to view all error messages. They are now collected and displayed in an alert, similar to how server-side error messages are shown. This enhancement ensures that users are fully aware of any errors that prevent to proceed with the action. by @grolu [#1633]
  • [USER] An improvement in performance and memory usage on the shoot list has been achieved when a large number of clusters are present. In the past, under heavy load, there were repeated instances where the dashboard became unresponsive due to very high memory consumption. This has been achieved by implementing the following two changes:
    • Throttling of shoot events in the frontend. Now, only the uid of the modified object is sent to the client, coupled with periodic synchronization of associated shoots.
    • Removal of the key property in the g-shoot-list-row component
    • Improved performance of sorting and filtering implementation
    • Faster response times for list shoot request (experimental: must be enabled by an operator)
    • Reduced network traffic for invisible browser tabs by @holgerkoser [#1637]
  • [OPERATOR] Added a configuration parameter Values.global.dashboard.maxRequestBodySize that specifies the maximum size of the request body. It's value defaults to 100kb. by @holgerkoser [#1656]
  • [OPERATOR] Experimental Features:
    • Enhanced Watch Cache Control for List Shoots Requests. We've introduced a new feature to fine-tune caching behavior for list shoots requests. A new configuration option, Values.global.dashboard.experimentalUseWatchCacheForListShoots, has been added to the gardener-dashboard Helm chart. This allows for more precise control over caching with four settings: never, no, yes, and always. By default, this is set to never. As an experimental feature, we welcome feedback and suggest caution in production environments.
    • Fine-tune the throttle delay per cluster. This option, found under Values.global.dashboard.frontendConfig.experimental.throttleDelayPerCluster, allows administrators to set the base number of milliseconds delay per cluster. This delay dynamically adjusts the synchronization throttle based on the number of active clusters, optimizing performance and resource utilization in environments with a varying number of clusters. by @holgerkoser [#1637]

🐛 Bug Fixes​

  • [OPERATOR] Fixed garden cluster terminals when the host cluster is a managed Seed by @petersutter [#1657]
  • [OPERATOR] Dashboard pods were not recreated after assets have been changed by @petersutter [#1627]
  • [USER] Fixed issues with the machine image input field:
    • image description was not shown
    • certain mache image hints were not shown when editing the worker for an existing shoot by @petersutter [#1635]
  • [USER] Fxed issue where the kubernetes dashboard was not shown on cluster details page in case the addon was enabled. The issue occurs when static token kubeconfig is disabled. by @petersutter [#1658]
  • [USER] Fixed the owner selection via mouse click on the project administration page. by @holgerkoser [#1632]
  • [USER] Fixed issue on create cluster page: On error, alert should scroll into view by @petersutter [#1629]
  • [USER] Users will now be redirected back to the last visited URL after automatic logout due to session timeout or authentication errors. by @holgerkoser [#1624]
  • [USER] Fixed several bugs on the project filter menu. More details in this PR by @petersutter [#1623]
  • [USER] Fixed vSphere Load Balancer Class selection by @grolu [#1625]

Docker Images​

  • dashboard: europe-docker.pkg.dev/gardener-project/releases/gardener/dashboard:1.71.0
Update dashboard to 1.71.0

[gardener/dashboard]

⚠️ Breaking Changes​

  • [OPERATOR] Change OCI Image Registry from GCR (eu.gcr.io/gardener-project) to Artifact-Registry (europe-docker.pkg.dev/gardener-project/releases) by @ccwienk [#1645]

✨ New Features​

  • [USER] Enhanced Client-Side Validation: Previously, if there was a validation error, the submit buttons on Dialogs and the Create Cluster page would be greyed out. Now, these buttons remain active even if there is a validation error. Clicking on the submit button in the presence of a validation error will not execute the submit logic. Instead, it will trigger the validation process to ensure that all validation error messages are clearly visible at the respective input fields. In addition, we've made it easier to view all error messages. They are now collected and displayed in an alert, similar to how server-side error messages are shown. This enhancement ensures that users are fully aware of any errors that prevent to proceed with the action. by @grolu [#1633]
  • [USER] An improvement in performance and memory usage on the shoot list has been achieved when a large number of clusters are present. In the past, under heavy load, there were repeated instances where the dashboard became unresponsive due to very high memory consumption. This has been achieved by implementing the following two changes:
    • Throttling of shoot events in the frontend. Now, only the uid of the modified object is sent to the client, coupled with periodic synchronization of associated shoots.
    • Removal of the key property in the g-shoot-list-row component
    • Improved performance of sorting and filtering implementation
    • Faster response times for list shoot request (experimental: must be enabled by an operator)
    • Reduced network traffic for invisible browser tabs by @holgerkoser [#1637]
  • [OPERATOR] Added a configuration parameter Values.global.dashboard.maxRequestBodySize that specifies the maximum size of the request body. It's value defaults to 100kb. by @holgerkoser [#1656]
  • [OPERATOR] Experimental Features:
    • Enhanced Watch Cache Control for List Shoots Requests. We've introduced a new feature to fine-tune caching behavior for list shoots requests. A new configuration option, Values.global.dashboard.experimentalUseWatchCacheForListShoots, has been added to the gardener-dashboard Helm chart. This allows for more precise control over caching with four settings: never, no, yes, and always. By default, this is set to never. As an experimental feature, we welcome feedback and suggest caution in production environments.
    • Fine-tune the throttle delay per cluster. This option, found under Values.global.dashboard.frontendConfig.experimental.throttleDelayPerCluster, allows administrators to set the base number of milliseconds delay per cluster. This delay dynamically adjusts the synchronization throttle based on the number of active clusters, optimizing performance and resource utilization in environments with a varying number of clusters. by @holgerkoser [#1637]

🐛 Bug Fixes​

  • [OPERATOR] Fixed garden cluster terminals when the host cluster is a managed Seed by @petersutter [#1657]
  • [OPERATOR] Dashboard pods were not recreated after assets have been changed by @petersutter [#1627]
  • [USER] Fixed issues with the machine image input field:
    • image description was not shown
    • certain mache image hints were not shown when editing the worker for an existing shoot by @petersutter [#1635]
  • [USER] Fxed issue where the kubernetes dashboard was not shown on cluster details page in case the addon was enabled. The issue occurs when static token kubeconfig is disabled. by @petersutter [#1658]
  • [USER] Fixed the owner selection via mouse click on the project administration page. by @holgerkoser [#1632]
  • [USER] Fixed issue on create cluster page: On error, alert should scroll into view by @petersutter [#1629]
  • [USER] Users will now be redirected back to the last visited URL after automatic logout due to session timeout or authentication errors. by @holgerkoser [#1624]
  • [USER] Fixed several bugs on the project filter menu. More details in this PR by @petersutter [#1623]
  • [USER] Fixed vSphere Load Balancer Class selection by @grolu [#1625]

Docker Images​

  • dashboard: europe-docker.pkg.dev/gardener-project/releases/gardener/dashboard:1.71.0
Update shoot-rsyslog-relp to 0.3.1

[gardener/gardener-extension-shoot-rsyslog-relp]

🏃 Others​

  • [OPERATOR] Incresed memory limits for the init containers of the rsyslog-relp-configurator and rsyslog-relp-configuration-cleaner daemonsets from 16Mi to 32Mi by @plkokanov [#51]

Docker Images​

  • gardener-extension-shoot-rsyslog-relp-admission: eu.gcr.io/gardener-project/gardener/extensions/shoot-rsyslog-relp-admission:v0.3.1
  • gardener-extension-shoot-rsyslog-relp: eu.gcr.io/gardener-project/gardener/extensions/shoot-rsyslog-relp:v0.3.1