Kyverno 1.12 Released

The Kyverno team is thrilled to announce the launch of version 1.12 hot on the heels of KubeCon EU 2024. Our interactions with Kyverno users during the conference have been incredibly insightful, with many expressing eager anticipation for the features introduced in this latest release. With over 330 pull requests merged, and 32 new contributors onboarded, we extend our heartfelt thanks to the Kyverno community for its invaluable contributions. Join us as we delve into the exciting new features of the 1.12 release.

Key New Features in Kyverno 1.12

This release introduces support for alternative storage for policy reports, brings Kyverno JSON to the CLI and includes various other enhancements in policy exception handling, ValidatingAdmissionPolicy and pod security sub-rule. Significant performance improvements have been made, ensuring resilience for large-scale systems.

Reports Server - Alternative Reports Storage

Policy reports are vital for tools like Kyverno, yet in large clusters, etcd’s storage limitations can hinder efficient policy management. Reports Server is designed to tackle these challenges head-on and streamline policy reporting within Kubernetes environments. Reports server provides a scalable solution for storing policy reports and cluster policy reports. It moves reports out of etcd and stores them in a PostgreSQL database instance.

Reports Server offers a straightforward solution. It leverages the Extension API server with a database to seamlessly integrate with Kubernetes, providing a dedicated space for storing and retrieving policy reports.

Advantages of Reports Server:

• Scalability: Overcomes etcd’s storage constraints, allowing for seamless scalability within Kubernetes clusters.
• Performance: By storing policy reports in a dedicated database, Reports Server improves cluster performance, particularly during heavy operations.
• Simplified Querying: Offers enhanced querying capabilities compared to etcd, simplifying access to policy reports.

Explore Reports Server and contribute to its development on GitHub: https://github.com/kyverno/reports-server.

Global Context Entry

Kyverno’s Global Context feature enhances policy enforcement within Kubernetes environments by enabling the caching of resources or external API calls for efficient reference across policies. Let’s delve into how this functionality enhances flexibility and performance in policy management.

The cornerstone of Global Context is the Global Context Entry, a custom resource type introduced by Kyverno. With Global Context Entry, users can declare resource or API call data globally, independent of any specific policies. These entries can then be effortlessly referenced within policies as needed.

Types of Global Context Entries

Kubernetes Resource

Users can reference any Kubernetes resource, optionally within a specific namespace, to populate the cache with its latest state. This is particularly useful for scenarios where the latest state of resources needs to be accessed across multiple policies.

Example:

 1apiVersion: kyverno.io/v2alpha1
 2kind: GlobalContextEntry
 3metadata:
 4  name: deployments
 5spec:
 6  kubernetesResource:
 7    group: apps
 8    version: v1
 9    resource: deployments
10    namespace: test-globalcontext

Notes: When dealing with cluster-scoped resources, leave the namespace field empty.

API Call

Users can define an external API call, specifying the URL, CA bundle for certificate verification, and refresh interval to ensure the cached data remains up-to-date. This empowers policies to dynamically fetch data from external sources and utilize it for enforcement.

Example:

1apiVersion: kyverno.io/v2alpha1
2kind: GlobalContextEntry
3metadata:
4  name: deployments
5spec:
6  apiCall:
7    urlPath: "/apis/apps/v1/namespaces/test-globalcontext/deployments"
8    refreshInterval: 10s

Notes: It’s important to be mindful of potential data staleness with API calls, as the cache is updated periodically based on the defined refresh interval.

Integration with Policies

Global Context Entries seamlessly integrate into policies, allowing users to harness the cached data within their enforcement logic. By referencing Global Context Entries within policy contexts, users can leverage pre-fetched data without redundancy.

Example:

1context:
2  - name: deploymentCount
3     globalReference:
4       name: deployments

Note: Depending on whether the data is sourced from Kubernetes resources or API calls, the JMESPath expression used to manipulate the data may vary. For Kubernetes resources, utilize length(@) to obtain the length of the array, while for API calls returning a list, use items | length(@).

In conclusion, Global Context in Kyverno empowers users to streamline policy management by centralizing and reusing data across policies, thereby enhancing efficiency and maintainability in Kubernetes environments.

Kyverno JSON In CLI

Kyverno JSON was introduced around the same time as Kyverno 1.11.0, expanding Kyverno’s capabilities beyond Kubernetes. With the latest 1.12 release, Kyverno JSON is now supported by the Kyverno CLI for validating any JSON payload.

The following illustrates how to use the new checks stanza in a Kyverno CLI test:

 1# ...
 2checks:
 3- match:
 4    resource:
 5      kind: Namespace
 6      metadata:
 7        name: hello-world-namespace
 8    policy:
 9      kind: ClusterPolicy
10      metadata:
11        name: sync-secret
12    rule:
13      name: sync-my-secret
14  assert:
15    status: pass

Policy Exception Enhancements

PolicyExceptions now support a conditions{} block (optional) that uses common expressions similar to those found in preconditions and deny rules to query the contents of the selected resources in order to refine the selection process.

The following is a PolicyException that checks if the Pod/Deployment has a specific label:

 1apiVersion: kyverno.io/v2beta1
 2kind: PolicyException
 3metadata:
 4  name: container-exception
 5spec:
 6  exceptions:
 7  - policyName: max-containers
 8     ruleNames:
 9     - max-two-containers
10     - autogen-max-two-containers
11  match:
12    any:
13    - resources:
14        kinds:
15        - Pod
16        - Deployment
17  conditions:
18    any:
19    - key: "{{ request.object.metadata.labels.color || '' }}"
20       operator: Equals
21       value: blue

In addition, PolicyException can be used to define an exemption for the Pod through the podSecurity{} block. It can be used to define controls that are exempted from the policy.

The following PolicyException exempts the containers running either the nginx or redis image from following the Capabilities control.

 1apiVersion: kyverno.io/v2beta1
 2kind: PolicyException
 3metadata:
 4  name: pod-security-exception
 5  namespace: policy-exception-ns
 6spec:
 7  exceptions:
 8  - policyName: psa
 9    ruleNames:
10    - restricted
11  match:
12    any:
13    - resources:
14        namespaces:
15        - delta
16  podSecurity:
17    - controlName: Capabilities
18      images:
19          - nginx*
20          - redis*

Furthermore, Kyverno CLI supports applying PolicyExceptions alongside policies to resources by using the -e or --exceptions flag to pass the Policy Exception manifest.

1kyverno apply /path/to/policy.yaml --resource /path/to/resource.yaml --exception /path/to/exception.yaml

Resources that match the given exception will get a skip as a result.

ValidatingAdmissionPolicy in CLI and Reports

In 1.12, Kyverno CLI now supports applying the ValidatingAdmissionPolicyBinding along with the policy definition to the resources. Furthermore, the reports controller generates reports as a result of applying the ValidatingAdmissionPolicy along with its binding to resources.

Here is a report snippet that is generated for ValidatingAdmissionPolicy with its binding:

 1 apiVersion: wgpolicyk8s.io/v1alpha2
 2  kind: PolicyReport
 3  metadata:
 4    labels:
 5      app.kubernetes.io/managed-by: kyverno
 6    namespace: staging-ns
 7    ownerReferences:
 8    - apiVersion: apps/v1
 9      kind: Deployment
10      name: deployment-3
11  results:
12  - message: 'failed expression: object.spec.replicas <= 5'
13    policy: check-deployment-replicas
14    properties:
15      binding: check-deployment-replicas-binding
16    result: fail
17    source: ValidatingAdmissionPolicy
18  scope:
19    apiVersion: apps/v1
20    kind: Deployment
21    name: deployment-3
22    namespace: staging-ns
23  summary:
24    error: 0
25    fail: 1
26    pass: 0
27    skip: 0
28    warn: 0

Fine-grained PodSecurity Validations

The validate.podSecurity sub-rule now has the ability to further classify exclusions by excluding specific fields as well as their values. This allows you to both use the easy style of policy language but get down to the lowest level of the funnel. For example, this is how you’ll be able to enforce the entire baseline profile of the Pod Security Standards but only exclude Istio’s and Linkerd’s images from specifically the initContainers list.

 1apiVersion: kyverno.io/v1
 2kind: ClusterPolicy
 3metadata:
 4  name: pod-security-standards
 5spec:
 6  background: true
 7  validationFailureAction: Enforce
 8  rules:
 9    - name: baseline-service-mesh
10      match:
11        any:
12          - resources:
13              kinds:
14                - Pod
15      validate:
16        podSecurity:
17          level: baseline
18          version: latest
19          exclude:
20            - controlName: Capabilities
21              images:
22                - "*/istio/proxyv2*"
23                - "*/linkerd/proxy-init*"
24              restrictedField: spec.initContainers[*].securityContext.capabilities.add
25              values:
26                - NET_ADMIN
27                - NET_RAW

Increased Performance

Kyverno 1.12 has undergone significant performance enhancements, making it even more suitable for large-scale deployments. Let’s dive into the challenges faced, the optimization journey, and the impressive results achieved.

Facing the Performance Bottlenecks

While Kyverno functioned well under normal conditions, its performance degraded under high loads, impacting large clusters. To address this, the Kyverno team, in collaboration with customers and the community, embarked on a mission to optimize performance.

Unveiling the Culprits

Load testing with K6 and profiling with Golang’s pprof tool revealed the primary culprits: inefficient context storage, repetitive JSON operations, and excessive goroutine creation during event processing.

Strategic Optimizations

Armed with these insights, the team implemented targeted optimizations:

• Context Storage: Replacing JSON structures with in-memory Golang maps significantly reduced overhead.
• JSON Handling: Switching to the lightweight jsoniter library boosted JSON processing efficiency.
• Event Processing: A custom Watcher function optimized event generation and minimized goroutine overhead.
• Policy Matching: Restructuring the matching logic prioritized simpler comparisons, leading to faster processing.
• Webhook Configuration: Fine-grained configuration minimized unnecessary Kyverno invocations.

The Performance Leap

These optimizations yielded remarkable results:

• 8x improvement in average and p95 latency.
• Stable performance even with high iteration counts.

Continuous Improvement through Automation

To ensure the longevity of these improvements, automated K6 load testing is now integrated into the CI/CD pipeline. This allows for early detection of potential regressions and facilitates continuous performance optimization.

Kyverno: Ready for Large-Scale Deployments

These advancements solidify Kyverno’s position as a robust and scalable policy engine for Kubernetes, capable of handling demanding workloads with ease. The collaborative effort behind these improvements exemplifies the project’s commitment to delivering a secure and efficient policy management solution for the Kubernetes community.

Policy Reporter UI V2

Policy Reporter UI v2 started its alpha phase with an updated techstack and complete restructuring of dashboards, details pages and the plugin system.

Content Restructuring

Policy Reporter UI v2 offers two types of views, resource-based and policy-based. The resource-based dashboards replace the old report views in Policy Reporter UI v1. The new focus of resource-based dashboards makes them much more scalable in larger environments and improves overall performance. Policy-based dashboards were a Kyverno-only feature in the old UI and only available with an additional Kyverno plugin. The revised views are available without additional requirements and are available for all provided sources.

New Plugin System

The plugin system was introduced to provide additional engine-specific functionality, but the old implementation required direct changes to the UI code, which made adding custom plugins very difficult.

The new plugin system requires an external service with a predefined (sub)set of REST APIs, where each plugin points to a source and provides additional information about this source. This additional information is automatically embedded in the existing views. These workflows make it possible to develop and use custom plugins without having to work on the actual user interface.

Custom Boards

One disadvantage of predefined views is the lack of customization options. Although they provide an overview of the entire cluster, it is often the case that teams within a company are only interested in a subset of namespaces and engine reports.

Custom Boards are a way of extending the UI with customized, application-specific views that provide only the required subset of information. The various configuration options enable the selection of namespaces based on a list or selector, the selection of displayed sources and the activation or deactivation of cluster-specific resource results.

Additional Features

• OpenIDConnect and OAuth2 authentication support
• Updated frontend tech stack: Bun, Vue 3.x, Nuxt 3.x, Vuetify 3.x, Vite 5.x
• Color Blind Themes
• General performance improvements

Kyverno Playground

The Kyverno playground with the support of ValidatingAdmissionPolicyBinding is now available for the new 1.12 release. You can check it out today via https://playground.kyverno.io/next/#/.

Closing

We have achieved another significant milestone, thanks to the support and contributions from our incredible community. Alongside the highlighted new features, this release includes critical fixes and other important changes. For a comprehensive overview, please refer to the full release notes on GitHub.

You can find us in the Kyverno channel on Kubernetes Slack, come and join one of our community meetings to discuss your contribution issues, or just catch us on Twitter. And if you’re already a Kyverno adopter, sign up to be an official adopter by updating the Adopters form here.

• ←Previous
• Next→