<!-- **Note:** When your KEP is complete, all of these comment blocks should be removed. To get started with this template: - [ ] **Pick a hosting SIG.** Make sure that the problem space is something the SIG is interested in taking up. KEPs should not be checked in without a sponsoring SIG. - [ ] **Create an issue in kubernetes/enhancements** When filing an enhancement tracking issue, please make sure to complete all fields in that template. One of the fields asks for a link to the KEP. You can leave that blank until this KEP is filed, and then go back to the enhancement and add the link. - [ ] **Make a copy of this template directory.** Copy this template into the owning SIG's directory and name it `NNNN-short-descriptive-title`, where `NNNN` is the issue number (with no leading-zero padding) assigned to your enhancement above. - [ ] **Fill out as much of the kep.yaml file as you can.** At minimum, you should fill in the "Title", "Authors", "Owning-sig", "Status", and date-related fields. - [ ] **Fill out this file as best you can.** At minimum, you should fill in the "Summary" and "Motivation" sections. These should be easy if you've preflighted the idea of the KEP with the appropriate SIG(s). - [ ] **Create a PR for this KEP.** Assign it to people in the SIG who are sponsoring this process. - [ ] **Merge early and iterate.** Avoid getting hung up on specific details and instead aim to get the goals of the KEP clarified and merged quickly. The best way to do this is to just start with the high-level sections and fill out details incrementally in subsequent PRs. Just because a KEP is merged does not mean it is complete or approved. Any KEP marked as `provisional` is a working document and subject to change. You can denote sections that are under active debate as follows: ``` <<[UNRESOLVED optional short context or usernames ]>> Stuff that is being argued. <<[/UNRESOLVED]>> ``` When editing KEPS, aim for tightly-scoped, single-topic PRs to keep discussions focused. If you disagree with what is already in a document, open a new PR with suggested changes. One KEP corresponds to one "feature" or "enhancement" for its whole lifecycle. You do not need a new KEP to move from beta to GA, for example. If new details emerge that belong in the KEP, edit the KEP. Once a feature has become "implemented", major changes should get new KEPs. The canonical place for the latest set of instructions (and the likely source of this file) is [here](/keps/NNNN-kep-template/README.md). **Note:** Any PRs to move a KEP to `implementable`, or significant changes once it is marked `implementable`, must be approved by each of the KEP approvers. If none of those approvers are still appropriate, then changes to that list should be approved by the remaining approvers and/or the owning SIG (or SIG Architecture for cross-cutting KEPs). -->

KEP-3503: Host Network Support for Windows Pods

<!-- A table of contents is helpful for quickly jumping to sections of a KEP and for highlighting any additional information provided beyond the standard KEP template. Ensure the TOC is wrapped with <code>&lt;!-- toc --&rt;&lt;!-- /toc --&rt;</code> tags, and then generate with `hack/update-toc.sh`. --> <!-- toc -->

• Release Signoff Checklist
• Summary
• Motivation
  • Goals
  • Non-Goals
• Proposal
  • User Stories (Optional)
    • Story 1
    • Story 2
    • Story 3
  • Notes/Constraints/Caveats (Optional)
  • Risks and Mitigations
• Design Details
  • CRI / Kubelet Updates
  • Container Runtime Support
  • Test Plan
    • Prerequisite testing updates
    • Unit tests
    • Integration tests
    • e2e tests
  • Graduation Criteria
    • Alpha
    • Beta
    • GA
  • Upgrade / Downgrade Strategy
  • Version Skew Strategy
• Production Readiness Review Questionnaire
  • Feature Enablement and Rollback
  • Rollout, Upgrade and Rollback Planning
  • Monitoring Requirements
  • Dependencies
  • Scalability
  • Troubleshooting
• Implementation History
• Drawbacks
• Alternatives
• Infrastructure Needed (Optional)

<!-- /toc -->

§ Release Signoff Checklist

<!-- **ACTION REQUIRED:** In order to merge code into a release, there must be an issue in [kubernetes/enhancements] referencing this KEP and targeting a release milestone **before the [Enhancement Freeze](https://git.k8s.io/sig-release/releases) of the targeted release**. For enhancements that make changes to code or processes/procedures in core Kubernetes—i.e., [kubernetes/kubernetes], we require the following Release Signoff checklist to be completed. Check these off as they are completed for the Release Team to track. These checklist items _must_ be updated for the enhancement to be released. -->

Items marked with (R) are required prior to targeting to a milestone / release.

• (R) Enhancement issue in release milestone, which links to KEP dir in kubernetes/enhancements (not the initial KEP PR)
• (R) KEP approvers have approved the KEP status as implementable
• (R) Design details are appropriately documented
• (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
  • e2e Tests for all Beta API Operations (endpoints)
  • (R) Ensure GA e2e tests meet requirements for Conformance Tests
  • (R) Minimum Two Week Window for GA e2e tests to prove flake free
• (R) Graduation criteria is in place
  • (R) all GA Endpoints must be hit by Conformance Tests
• (R) Production readiness review completed
• (R) Production readiness review approved
• "Implementation History" section is up-to-date for milestone
• User-facing documentation has been created in kubernetes/website, for publication to kubernetes.io
• Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes

<!-- **Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone. -->

§ Summary

Windows has all of the support needed to enable having containers use the node's networking namespace. This enhancement details the work needed to enable this functionality in Kubernetes.

<!-- This section is incredibly important for producing high-quality, user-focused documentation such as release notes or a development roadmap. It should be possible to collect this information before implementation begins, in order to avoid requiring implementors to split their attention between writing release notes and implementing the feature itself. KEP editors and SIG Docs should help to ensure that the tone and content of the `Summary` section is useful for a wide audience. A good summary is probably at least a paragraph in length. Both in this section and below, follow the guidelines of the [documentation style guide]. In particular, wrap lines to a reasonable length, to make it easier for reviewers to cite specific portions, and to minimize diff churn on updates. [documentation style guide]: https://github.com/kubernetes/community/blob/master/contributors/guide/style-guide.md -->

§ Motivation

The two main motivating factors are 1) parity with Linux and 2) improved cluster density.

Today it is possible to set hostNetwork=true for Windows pods but it doesn't change anything (unless the pod contains hostProcess containers). This can be confusing for users.

In clusters with large amounts of services Windows nodes can experience port exhaustion. One such situation is where a small amount of pods need to expose many ports it can then be desirable to use use host networking instead of using nodePorts. Another situation is using hostNetwork=true as alternative to relying on 'hostPort' CNI feature for exposing many ports in many pods.

<!-- This section is for explicitly listing the motivation, goals, and non-goals of this KEP. Describe why the change is important and the benefits to users. The motivation section can optionally provide links to [experience reports] to demonstrate the interest in a KEP within the wider Kubernetes community. [experience reports]: https://github.com/golang/go/wiki/ExperienceReports -->

Goals

• Enable Windows containers to use node's networking namespace for non-hostprocess containers.

Non-Goals

• Discuss implementation details on how container runtimes configure Pod sandboxes for pods that should be joined to the node's network namespace.

§ Proposal

<!-- This is where we get down to the specifics of what the proposal actually is. This should have enough detail that reviewers can understand exactly what you're proposing, but should not include things like API designs or implementation. What is the desired outcome and how do we measure success?. The "Design Details" section below is for the real nitty-gritty. -->

Host Network support is already supported for Linux Pods and this enhancement will bring feature parity to Windows Pods.

Changes to enable this for Windows entail updating the Kubelet to populate the necessary CRI-API fields when running on Windows to instruct the container runtime (containerd) to use the node's network namespace when configuring the sandbox for Pods that specify hostNetwork=true.

Container runtimes will also need to be updated but that work is out-of-scope for this proposal.

User Stories (Optional)

<!-- Detail the things that people will be able to do if this KEP is implemented. Include as much detail as possible so that people can understand the "how" of the system. The goal here is to make this feel real for users without getting bogged down. -->

Story 1

As a user of a legacy application I want to bind many arbitrary ports to a host network namespace on a single node, as opposed to taking all the node ports of a cluster.

Story 2

As a DaemonSet which runs before CNI providers are installed, for example for security, application bootstrapping, cni bootstrapping, and so on - I want to be able to run a container that isn't fully privileged (i.e. that isn't a HostProcessContainer) but which is on the host's network

Story 3

As a user creating a windows pod with hostNetwork=true, I want correct behaviour (i.e. I don't want to silently ignore the hostNetwork=true setting in Pod specs).

Notes/Constraints/Caveats (Optional)

<!-- What are the caveats to the proposal? What are some important details that didn't come across above? Go in to as much detail as necessary here. This might be a good place to talk about core concepts and how they relate. -->

Risks and Mitigations

Very low risk.

§ Design Details

<!-- This section should contain enough information that the specifics of your change are understandable. This may include API specs (though not always required) or even code snippets. If there's any ambiguity about HOW your proposal will be implemented, this is the place to discuss them. -->

CRI / Kubelet Updates

Add a new WindowsNamespaceOption struct to CRI-API that mirrors the Linux-specific NameSpaceOption struct and contains only options on Windows.

// WindowsNamepaceOption provides options for Windows namespaces.
message WindowsNamespaceOption {
  // Network namespace for this container/sandbox.
  // Note: There is currently no way to set CONTAINER scoped network in the Kubernetes API.
  // Namespaces currently set by the kubelet: POD, NODE
  NamespaceMode network = 1;
}

Update WindowsSandboxSecurityContext to include WindowsNamespaceOption

// WindowsSandboxSecurityContext holds platform-specific configurations that will be
// applied to a sandbox.
// These settings will only apply to the sandbox container.
message WindowsSandboxSecurityContext {
  ...
  // Configurations for the sandbox's namespaces.
  WindowsNamespaceOption namespace_options = 4;
}

Update Kubelet to set new CRI-API fields based on contents of incoming Pod specs.

Container Runtime Support

Update Containerd to check for new CRI-API fields in RunPodSandbox and configure networking appropriately.

Test Plan

<!-- **Note:** *Not required until targeted at a release.* The goal is to ensure that we don't accept enhancements with inadequate testing. All code is expected to have adequate tests (eventually with coverage expectations). Please adhere to the [Kubernetes testing guidelines][testing-guidelines] when drafting this test plan. [testing-guidelines]: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md -->

[x] I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement.

Prerequisite testing updates

<!-- Based on reviewers feedback describe what additional tests need to be added prior implementing this enhancement to ensure the enhancements have also solid foundations. -->

Unit tests

<!-- In principle every added code should have complete unit test coverage, so providing the exact set of tests will not bring additional value. However, if complete unit test coverage is not possible, explain the reason of it together with explanation why this is acceptable. --> <!-- Additionally, for Alpha try to enumerate the core package you will be touching to implement this enhancement and provide the current unit coverage for those in the form of: - <package>: <date> - <current test coverage> The data can be easily read from: https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-coverage-unit This can inform certain test coverage improvements that we want to do before extending the production code to implement this enhancement. -->

• k8s.io/kubernetes/pkg/kubelet/kuberuntime: 2022-09-12 - 67%

Integration tests

<!-- This question should be filled when targeting a release. For Alpha, describe what tests will be added to ensure proper quality of the enhancement. For Beta and GA, add links to added tests together with links to k8s-triage for those tests: https://storage.googleapis.com/k8s-triage/index.html -->

N/A - There is currently no way to run integration tests that target Windows specific functionality.

e2e tests

<!-- This question should be filled when targeting a release. For Alpha, describe what tests will be added to ensure proper quality of the enhancement. For Beta and GA, add links to added tests together with links to k8s-triage for those tests: https://storage.googleapis.com/k8s-triage/index.html We expect no non-infra related flakes in the last month as a GA graduation criteria. -->

• Existing SIG-Network e2e tests for hostNetwork containers will be updated to run for Windows.

<!-- - <test>: <link to test coverage> -->

Graduation Criteria

<!-- **Note:** *Not required until targeted at a release.* Define graduation milestones. These may be defined in terms of API maturity, [feature gate] graduations, or as something else. The KEP should keep this high-level with a focus on what signals will be looked at to determine graduation. Consider the following in developing the graduation criteria for this enhancement: - [Maturity levels (`alpha`, `beta`, `stable`)][maturity-levels] - [Feature gate][feature gate] lifecycle - [Deprecation policy][deprecation-policy] Clearly define what graduation means by either linking to the [API doc definition](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning) or by redefining what graduation means. In general we try to use the same stages (alpha, beta, GA), regardless of how the functionality is accessed. [feature gate]: https://git.k8s.io/community/contributors/devel/sig-architecture/feature-gates.md [maturity-levels]: https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions [deprecation-policy]: https://kubernetes.io/docs/reference/using-api/deprecation-policy/ Below are some examples to consider, in addition to the aforementioned [maturity levels][maturity-levels]. #### Alpha - Feature implemented behind a feature flag - Initial e2e tests completed and enabled #### Beta - Gather feedback from developers and surveys - Complete features A, B, C - Additional tests are in Testgrid and linked in KEP #### GA - N examples of real-world usage - N installs - More rigorous forms of testing—e.g., downgrade tests and scalability tests - Allowing time for feedback **Note:** Generally we also wait at least two releases between beta and GA/stable, because there's no opportunity for user feedback, or even bug reports, in back-to-back releases. **For non-optional features moving to GA, the graduation criteria must include [conformance tests].** [conformance tests]: https://git.k8s.io/community/contributors/devel/sig-architecture/conformance-tests.md #### Deprecation - Announce deprecation and support policy of the existing flag - Two versions passed since introducing the functionality that deprecates the flag (to address version skew) - Address feedback on usage/changed behavior, provided on GitHub issues - Deprecate the flag -->

Alpha

• CRI-API updates added to codebase
• Unit tests added to validate correct CRI-API fields are set depending if hostNetwork=true is set on Windows
• Feature implemented behind a feature flag
• Initial e2e tests completed and enabled

Beta

• A version of containerd w/ support for configuing pod's to use the node's network namespace is released (target v1.8)
• Functionality is validated as part of Windows Operational Readiness validation.

GA

• All feedback from alpha/beta usage is addressed

Upgrade / Downgrade Strategy

<!-- If applicable, how will the component be upgraded and downgraded? Make sure this is in the test plan. Consider the following in developing an upgrade/downgrade strategy for this enhancement: - What changes (in invocations, configurations, API use, etc.) is an existing cluster required to make on upgrade, in order to maintain previous behavior? - What changes (in invocations, configurations, API use, etc.) is an existing cluster required to