KubeSphere Enterprise v4.1.2 Release Notes

Release Notes

KubeSphere Core v4.1.2
WhizardTelemetry Observability Platform v1.1.0
Platform Management
Database and Middleware
- RadonDB DMP v2.1.3
DevOps
- DevOps v1.1.1
Networking
Security
AI
- NVIDIA GPU Operator v23.9.2

KubeSphere Core v4.1.2

Features

Support API rate limiting.
Support configuring initialization tasks for extensions.
Support creating persistent volumes on the page.

Enhancements

Reset recommended configurations during installation, update, and configuration of extensions.
Support automatic cleanup of installation pods for extensions.
Support adjusting ks-agent parameters for member clusters.
Pagination support for the "licenses information" and "licenses management" pages.
Adjust sorting and filtering fields on the "licenses information" and "licenses management" pages.
Display specific reasons for license errors.
Support triggering cluster resource synchronization proactively.
User experience improvements on the workload pages under clusters.
User experience improvements on the application list page.
User experience improvements on the persistent volume claim and storage class list pages.
Optimize the display of excessively long resource names.
Support modifying ports for redis containers and services.
Automatically remove discontinued extensions in the Extensions Center.
Support globally enabling fieldValidation.
Increase default installation timeout for extensions.
Support horizontal movement on the cluster nodes list page.

Bug Fixes

Fix inconsistency between selected and actual roles when modifying project members' roles.
Fix the blank display issue on container details page.
Fix the default skip certificate verification issue when selecting HTTPS for image registry addresses when creating secrets.
Fix inability to edit project roles for service accounts of member clusters.
Fix inability to edit settings for configmaps without key-value pairs.
Fix inability to edit or delete key-value pair data for configmaps of member clusters.
Fix the display issue with pop-up dialog when removing unready clusters on the cluster management page.
Fix the progress bar display issue when removing clusters on cluster management page.
Fix disappearance of the selection status of previously selected clusters after searching for clusters in pop-up dialog for "Add tags to clusters".
Fix pagination issue for pods on workload details page.
Fix display of HTML comments in changelogs on extension details page.
Fix abnormal cluster selection during installation, update, and configuration of extensions.
Fix incomplete display of floating elements on list pages under certain circumstances.
Fix abnormal display of error messages in the top right corner.
Fix display issue with pop-up dialog for creating workspaces.
Fix inability to search for Harbor (version 2.8.0 and later) images.
Fix slow loading of console under HTTPS protocol.
Specify cluster creator as cluster administrator by default.
Remove authorization to workspaces simultaneously after removing clusters.
Fix the issue where built-in roles are not generated after creating projects.
Fix the issue where built-in roles are not generated after creating DevOps projects.
Fix the issue where jobs cannot be rerun.
Fix exception when deleting labels from nodes.
Fix the issue where page does not refresh in real time when adding member clusters.
Add prompt message when uploading files in containers.
Fix the issue where clusters are not filtered based on user permissions when selecting clusters.
Fix potential permission escalation and authorization risks in helm application deployment.
Fix the issue where file upload freezes when creating application templates.
Fix the issue where applications created in one project are visible in other projects.
Fix the issue where bitnami source in application repository cannot be synchronized.
Fix the issue where application template displays no data.
Fix the issue where unauthorized users encounter a blank page when deploying applications from the app store.
Fix incorrect display of types of secrets.
Fix display issue with workspace list.
Fix incorrect status in persistent volume list.
Fix failure to create PVC based on snapshots.
Remove unnecessary prompt for persistent volume expansion.
Fix incorrect display of type dropdown when creating secrets.
Fix data filling error when creating secrets with the "Image registry information" type.
Fix the issue where workload list cannot retrieve all projects.
Fix abnormal display of prompt information for pods in workloads.
Fix the issue where versions displayed in CRDs page is not the latest.
Fix display issue when searching for clusters in cluster list.
Fix the issue where after OAuth login, it fails to redirect back to application page.
Fix the issue where web kubectl terminal is unusable in EKS environment.
Fix abnormal installation status for extensions.
Fix potential sensitive information leakage in configz API.
Fix potential sensitive information leakage in oauth config API.

WhizardTelemetry Observability Platform v1.1.0

The WhizardTelemetry Observability Platform has been continuously evolving since the release of its first version, v1.0.0, in KubeSphere Enterprise v4.1.0. The second version, v1.1.0, of the WhizardTelemetry Observability Platform is released in KubeSphere Enterprise v4.1.2, with the following key updates:

Introduce the new extension Grafana for WhizardTelemetry, with built-in customized Dashboards for K8s, KubeSphere, AI Infra, Loki, and more.
Introduce the new extension Grafana Loki for WhizardTelemetry, supporting the storage of KubeSphere's log, audit, event, and notification history data through Loki in more cost-effective object storage, and enabling querying from object storage.
WhizardTelemetry Events Alerting now supports log alerting functionality, enabling keyword alerts and sliding window alerts for streaming log data.
WhizardTelemetry Notifications now supports configuring WeChat group robots for sending notifications to WeChat groups.
Whizard Observability Center supports cross-cluster resource queries, currently supports cross-cluster Pod queries. Pods can be queried across clusters based on cluster, node, namespace, Pod name, Pod status, and Pod IP.
WhizardTelemetry Monitoring adapts for EdgeWize 3.0, providing better support for monitoring and alerts on edge nodes and applications.
The extensions of the WhizardTelemetry Logging series support for customizing index formats of OpenSearch and filtering logs by namespace and pod.
WhizardTelemetry Logging now supports collection of Calico CNI logs, allowing querying of information related to Calico's allocation and release of IP addresses for Pods.

WhizardTelemetry Platform Service v1.2.2

Features

Support customizing the index format of OpenSearch when using it as the backend storage.

Enhancements

Support nanosecond-level sorting queries when using OpenSearch as the backend storage.

WhizardTelemetry Data Pipeline v1.0.4

Features

Support exporting metrics from Vector to Prometheus.

Misc

Upgrade Vector to version 0.39.0.

WhizardTelemetry Alerting v1.0.3

Enhancements

The namespace and pod labels are removed from the node alerts because they are irrelevant to nodes.

Bug Fixes

Fix the issue where clusters without the alerting extension installed still display its entry.

Misc

Upgrade thanos to v0.36.1

WhizardTelemetry Auditing v1.2.0

Features

Support using Loki as the backend storage for audits.
Support customizing the index format of OpenSearch when using it as the backend storage.

Enhancements

Support nanosecond-level sorting queries when using OpenSearch as the backend storage.

WhizardTelemetry Events v1.2.0

Features

Support using Loki as the backend storage for events.
Support customizing the index format of OpenSearch when using it as the backend storage.

Enhancements

Support nanosecond-level sorting queries when using OpenSearch as the backend storage.

WhizardTelemetry Logging v1.2.2

Features

Support filtering logs by namespace and pod.
Support using Loki as the backend storage for logs.
Support customizing the index format of OpenSearch when using it as the backend storage.
Introduce calico CNI log collection, allowing querying of information such as IP allocation and release for pods by calico.

Enhancements

Support nanosecond-level sorting queries when using OpenSearch as the backend storage.

Bug Fixes

Fix the issue where the Log Receiver page is blank.
Fix wrong display of time range for topology graphs on the Container Log Search page.
Fix the issue where the page crashes when empty logs are present during container log keyword queries.

WhizardTelemetry Monitoring v1.1.0

Features

Whizard Observability Center supports cross-cluster resource queries. Currently supports cross-cluster Pod queries, enabling Pod searches by cluster, node, namespace, Pod name, Pod status, and Pod IP.
Support for edge resource and application monitoring for EdgeWize.
Support for separately deploying cAdvisor.
Support for retaining InfiniBand network metrics for nodes.

Enhancements

Improve user experience in Whizard Observability Center.
Enhance stability by optimizing recording rules configuration in Kube-Prometheus-Stack.
Optimize default maximum time interval for monitoring query charts.
Add average disk usage to the cluster node list.
When Whizard Observability Center is enabled, Prometheus rules in all namespaces are selected for computation by default to fix the issue of gateway having no monitoring data.

Bug Fixes

Fix the issue with excessive legend information in monitoring query charts causing abnormal display.
Fix missing metrics in the Monitoring tab of container details page.
Fix inconsistency between actual time range and selected time range of real-time monitoring for project quantity on the Application Resources page.
Fix inability to add again after upgrading configurations of components in Whizard Observability Center.
Fix the issue where inbound/outbound traffic is displayed reversed in Pod network traffic monitoring.
Fix the issue where default configuration is not populated in the component configuration in Whizard Observability Center.

Misc

Upgrade Prometheus Operator to v0.75.1
Upgrade Prometheus to v2.51.2
Upgrade kube-state-metrics to v2.12.0
Upgrade node-exporter to v1.8.1
Upgrade thanos to v0.36.1

WhizardTelemetry Notification v2.5.9

Features

Add support for configuring WeChat Work group robots to send notifications to WeChat Work groups.

Enhancements

Improve user experience on the notification subscription page.
Add time information to notification messages.
Support repeated notifications triggered by the same alerts.

Bug Fixes

Fix the issue where the alert type is missing in the notification history after a logging alert is triggered.
Fix the issue in multi-cluster environments where tenant information for other clusters does not update when there is an unready cluster.
Fix an installation compatibility issue on K8s clusters (below v1.22).
Fix the issue where the alertmanager could not start properly in certain network environments.

Misc

Upgrade alertmanager to v0.27.0
Upgrade alertmanager-proxy to v0.2.0

WhizardTelemetry Events Alerting v1.2.0

Features

Support logging alerts based on keyword matching.
Support logging alerts based on the number of keyword occurrences within a sliding window.

Bug Fixes

Fix the issue where the label and annotation values are incorrect when sending auditing alerts and event alerts.

Grafana for WhizardTelemetry v10.4.1

This is the first version of the Grafana for WhizardTelemetry extension, featuring numerous built-in dashboards to enhance the visualization capabilities of WhizardTelemetry.

Features

Convert Grafana into an extension that adapts to the pluggable architecture of KubeSphere LuBan.
Built-in kube-prometheus monitoring dashboard, providing comprehensive monitoring of a single Kubernetes cluster, including usage of resources such as nodes, pods, and containers.
Add the Whizard Monitoring for KubeSphere multi-cluster dashboard, allowing easy querying of monitoring data from multiple KubeSphere clusters.
Introduce multiple Loki query panels to query KubeSphere logs, audits, events, notification history, etc., stored in Loki, and display metric charts generated based them.
Enhance support for monitoring AI infrastructure, adding the following AI infrastructure monitoring panels:
1. GPFS
2. Huawei Ascend NPU and RoCE network
3. Infiniband network
4. Hexaflake GPU
5. NVIDIA GPU

Grafana Loki for WhizardTelemetry v1.0.0

Logs typically have a huge volume of data, and storing logs on disk or other block storage devices can be costly. Grafana Loki offers a solution by storing log data in object storage and supporting querying from object storage. Additionally, Grafana Loki has innovatively introduced an indexing mechanism similar to Prometheus, which indexes only the metadata of log streams rather than full-text indexing, along with mechanisms like log stream compression. This approach significantly reduces log storage costs while maintaining fast log querying speeds.

Features

This is the first version of Grafana Loki released as part of the WhizardTelemetry Observability Platform with the KubeSphere 4.x LuBan pluggable architecture. In addition to deploying Grafana Loki as an extension, it can also be combined with the built-in Loki Dashboards in the Grafana for WhizardTelemetry extension to effectively query KubeSphere logs, audits, events, and notification histories stored in object storage.

Platform Management

App Store Management v2.0.1

Bug Fixes

Fix the issue where application categories are not displayed when exceeding 10 entries.
Fix potential issue with excessively long application names.
Fix potential issue with repo synchronization not being executed.
Fix the issue where clusters connected by tower cannot install applications.

Service Mesh v1.0.0

Bug Fixes

Fix the issue where the ingress does not have a gateway address after deploying the sample application.
Fix the format error of routing rules when deploying the sample application.

Spring Cloud v1.0.1

Enhancements

Enable persistent storage for Nacos by default.

Application Management for Cluster Federation v1.0.0

Bug Fixes

Fix the issue where composed applications with application governance enabled still display "Application governance disabled" on their details page.
Fix the error when saving changes to ingresses in federated projects.
Fix the issue where no data is displayed on the "Traffic Management" page of composed applications.

Database and Middleware

RadonDB DMP v2.1.3

Bug Fixes

Fix failure of obtaining licenses for DMP.
Fix the inaccurate monitoring data on the Kafka details page.
Fix the issue of data disappearing after refreshing the Monitoring page on the cluster instance details page.
Fix translation error on the Two-Factor Authentication page caused by this extension.
Fix the error when deleting a MySQL instance without installing the WhizardTelemetry Alerting extension.
Fix the incorrect image version when creating a MySQL instance.

DevOps

DevOps v1.1.1

Features

Support for creating credentials with the kubeconfig type.

Bug Fixes

Fix the issue where the devops-jenkins-rules rule group is missing.
Fix the issue where pipeline running records are not shown in specific cases.
Fix cluster selection logic on DevOps list page under workspaces.
Fix the issue where login information of project members was not updated in DevOps project member lists.
Fix devops-apiserver exception caused by updating member cluster agent configuration.
Fix failure of email sending steps in pipelines.

Networking

Gateway v1.0.2

Enhancements

Improve user experience on the Gateway Settings page.

Bug Fixes

Fix the issue where all gateways of the cluster are displayed in the kubesphere-system namespace.
Fix the issue where all gateways use the same lease, resulting in only one gateway running.
Fix the issue where ingress-nginx cannot use global image configuration.
Fix the issue where multiple project gateways selecting the master lead to the inability to update the ingress loadBalancer IP.

Network v1.1.0

Features

Support associating multiple ippools with workloads.

Bug Fixes

Fix the issue where the IP pool blocksize is not automatically populated when creating an ippool.
Correct the wrong behavior of the "Refresh" and "Clear" operations in the IP pool selection list.
Fix permission issues when creating a new IP pool in workloads.

Ingress Utils v1.0.0

Features

Refactor based on the new microkernel architecture KubeSphere LuBan.
Shield underlying configuration details, supporting one-click activation of application routing tools.
Support reuse validation for already added domain names.

Security

Gatekeeper v1.0.1

Enhancements

Adapt for KSE v4.1.2

OAuth2-Proxy v7.6.2

This is the first version of the OAuth2-Proxy extension, integrating OAuth2-Proxy into the OpenResty proxy to achieve powerful authentication and access control capabilities, and enhancing application security and user management convenience.

Features

Optimize the deployment process of OAuth2-Proxy, making application integration easier.
Add support for NodePort and Ingress, offering more deployment options for increased flexibility.
Introduce email whitelist authentication feature, enabling authorization control through an email whitelist to ensure that only authorized users can access the service.

Cert Manager v1.0.0

Using cert-manager to sign and manage certificates for extensions or workloads.

cert-manager is a powerful and extensible X.509 certificate controller for Kubernetes and OpenShift workloads. It will obtain certificates from a variety of Issuers, both popular public Issuers as well as private Issuers, and ensure the certificates are valid and up-to-date, and will attempt to renew certificates at a configured time before expiry.

Features

Specify a default issuer.
Create signed-off certificates for Ingresses created by KubeSphere Gateway.

AI

NVIDIA GPU Operator v23.9.2

This is the first official version of NVIDIA GPU Operator for KSE v4.x, Only [gpu-operator] (https://github.com/NVIDIA/gpu-operator) is converted to an extension that meets KubeSphere LuBan pluggable architecture.

Features

Convert gpu-operator into an extension that complies with the pluggable architecture of KubeSphere LuBan.

Release Notes