• 约 13 分钟

Windows 上 Docker Desktop 内置 K8s 启动失败:端口保留与 portproxy 的完整排查和自动化修复

使用 Windows 版 Docker Desktop 的内置 Kubernetes 时,可能会遇到一个很恼人的问题:重启 Docker Desktop 后,K8s cluster 起不来。报错常常是那句含糊的 “Unable to start a cluster, try again”。

在网上找一圈,方案大多是重置 cluster,代价太大,数据也没了。折腾了好几次之后,我总结出两个真正的罪魁祸首,以及一套可以「一劳永逸」的自动化方案。整个过程不需要重置 cluster,也不会丢数据。

两个罪魁祸首

一、portproxy 抢占了 6443

如果你之前为了从外部访问 K8s API server,用 netsh 建过端口转发,那这个转发很可能就是元凶之一。

先看看有没有:

netsh interface portproxy show all

如果输出里包含 K8s API server 监听的端口(默认 6443),基本就可以确定问题了:

  • 0.0.0.0:6443 这个通配监听,会挡住 Docker Desktop 把 127.0.0.1:6443 绑给自己;
  • 于是轮不到 apiserver 起来,Docker 就报 cluster 启动失败。

临时处理:K8s 停止时删掉转发,等 cluster 起来后再加回来。

# K8s 未启动时删掉
netsh interface portproxy delete v4tov4 listenport=6443 listenaddress=0.0.0.0

# cluster 启动成功后再加回来
netsh interface portproxy add v4tov4 listenport=6443 listenaddress=0.0.0.0 connectport=6443 connectaddress=127.0.0.1

二、Windows 保留端口段(Hyper-V / winnat)

这才是最隐蔽、也最常见的原因。Windows 的 Hyper-V/WSL 会在启动时预留一大批动态端口,如果 6443 恰好落在被保留的区间里,Docker 绑定它时会被系统直接拒绝。

排查:

netsh int ipv4 show excludedportrange protocol=tcp

例如下面这样,6443 落在 6347-6446 里:

协议 tcp 端口排除范围

开始端口    结束端口
      6247        6346
      6347        6446   <-- 6443 在里面
      6447        6546
      ...

再看 Docker 的后端日志(%LOCALAPPDATA%\Docker\log\host\com.docker.backend.exe.log),会看到非常明确的报错:

kubernetes failed to start: {"progressMessage":"exposing apiserver port", ...}
POST /forwards/expose: listen tcp 127.0.0.1:6443: bind: An attempt was made to
access a socket in a way forbidden by its access permissions.

你也可以直接测端口能不能绑定:

Test-NetConnection 127.0.0.1 -Port 6443

手动修复

1. 修复保留端口段:先把 winnat 停掉(会瞬断 Hyper-V/WSL 网络 1–2 秒),把 6443 注册成持久保留,再启动 winnat。这样以后 Hyper-V 就不会再抢它。

net stop winnat
netsh int ipv4 add excludedportrange protocol=tcp startport=6443 numberofports=1 store=persistent
net start winnat

如果直接 add 报「另一个程序正在使用此文件」,说明 6443 已被 Hyper-V 的范围占着,必须先停 winnat再 add(顺序很重要)。

2. 修复 portproxy:按上面的方式,K8s 停止时删掉、启动后再加回来。

3. 重启 Docker Desktop 的 Kubernetes,应该就能正常起来了。

自动化:一次装好,开机自愈

手动做一次两次还行,每次重启都要弄就太烦了。于是写了两个 PowerShell 脚本 + 两个任务计划程序:

  • Reserve-ApiserverPort.ps1:开机时把 6443 注册为持久保留(治本);
  • Watch-K8sPortProxy.ps1:常驻看护,探测到 apiserver 起来就自动加 portproxy,停止就自动删。

把两个脚本放到 C:\ProgramData\DockerK8s\ 即可。

脚本一:Reserve-ApiserverPort.ps1

<#
.SYNOPSIS
    Ensures the Kubernetes apiserver port (default 6443) is persistently reserved
    so that Hyper-V/winnat does not grab it and Docker Desktop Kubernetes can bind it.

.DESCRIPTION
    Idempotent. On each run:
      1. Verifies administrator privileges.
      2. Tries to add a persistent excluded-port-range reservation for <Port>.
      3. If 127.0.0.1:<Port> is still not bindable (range owned by Hyper-V/winnat),
         recycles the winnat service (stop -> add reservation -> start) and retries.
      4. Verifies the port is bindable again.

    This script is safe to run repeatedly and safe to schedule at startup: when the
    reservation is already in place it only tries to (re)add it, which is harmless.

    It can also be dot-sourced (to load Invoke-ApiserverReservation into the caller),
    which is how Install.ps1 invokes it in the already-elevated session.

.PARAMETER Port
    Apiserver port. Default 6443.

.PARAMETER LogPath
    Log file path. Defaults to reserve.log next to this script.

.NOTES
    Compatible with Windows PowerShell 5.1 and PowerShell 7+.
#>

[CmdletBinding()]
param(
    [int]$Port = 6443,
    [string]$LogPath
)

function Write-ReserveLog {
    param(
        [Parameter(Mandatory)][string]$Message,
        [Parameter(Mandatory)][string]$Path
    )
    $line = '{0} [reserve] {1}' -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $Message
    try {
        # Always UTF-8 with BOM so any reader/editor detects the encoding.
        $utf8Bom = New-Object System.Text.UTF8Encoding($true)
        [System.IO.File]::AppendAllText($Path, $line + [Environment]::NewLine, $utf8Bom)
    }
    catch { }
    Write-Verbose $line
}

function Test-IsAdmin {
    return ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
}

function Test-PortBindable {
    param([Parameter(Mandatory)][int]$PortNumber)
    $listener = $null
    try {
        $listener = New-Object System.Net.Sockets.TcpListener([System.Net.IPAddress]::Loopback, $PortNumber)
        $listener.Start()
        return $true
    }
    catch {
        return $false
    }
    finally {
        if ($null -ne $listener) {
            try { $listener.Stop() } catch { }
            $listener = $null
        }
    }
}

function Test-AnyListenerOnPort {
    param([Parameter(Mandatory)][int]$PortNumber)
    $listeners = [System.Net.NetworkInformation.IPGlobalProperties]::GetIPGlobalProperties().GetActiveTcpListeners()
    foreach ($ep in $listeners) {
        if ($ep.Port -eq $PortNumber) { return $true }
    }
    return $false
}

function Add-PersistentReservation {
    param(
        [Parameter(Mandatory)][int]$PortNumber,
        [Parameter(Mandatory)][string]$LogPath
    )
    $output = & netsh.exe int ipv4 add excludedportrange protocol=tcp startport=$PortNumber numberofports=1 store=persistent 2>&1
    $code = $LASTEXITCODE
    $text = ($output | Out-String).Trim()
    Write-ReserveLog -Message ("netsh add excludedportrange exit=$code out='$text'") -Path $LogPath
    if ($code -eq 0) { return $true }
    if ($text -match 'already exists|已存在|对象已存在|attribute is already') { return $true }
    return $false
}

function Invoke-ApiserverReservation {
    [CmdletBinding()]
    param(
        [int]$Port = 6443,
        [string]$LogPath
    )

    if (-not $LogPath) {
        $base = if ($PSScriptRoot) { $PSScriptRoot } else { $PWD.Path }
        $LogPath = Join-Path $base 'reserve.log'
    }

    Write-ReserveLog -Message ("start port=$Port log=$LogPath elevated=$(Test-IsAdmin)") -Path $LogPath

    if (-not (Test-IsAdmin)) {
        Write-ReserveLog -Message 'not running elevated; cannot reserve port' -Path $LogPath
        return $false
    }

    $reserved = Add-PersistentReservation -PortNumber $Port -LogPath $LogPath

    # Always verify by actually binding. If the port is already in use by a live
    # listener (e.g. the running apiserver), do NOT recycle winnat.
    if (-not (Test-PortBindable -PortNumber $Port)) {
        if (Test-AnyListenerOnPort -PortNumber $Port) {
            Write-ReserveLog -Message ("port $Port is in use by a listener; skipping winnat recycle") -Path $LogPath
            return $true
        }
        Write-ReserveLog -Message 'port not bindable; recycling winnat' -Path $LogPath
        & net.exe stop winnat 2>&1 | Out-Null
        Start-Sleep -Seconds 2
        $reserved = Add-PersistentReservation -PortNumber $Port -LogPath $LogPath
        & net.exe start winnat 2>&1 | Out-Null
        Start-Sleep -Seconds 2
    }

    $bindable = Test-PortBindable -PortNumber $Port
    Write-ReserveLog -Message ("result bindable=$bindable reserved=$reserved") -Path $LogPath
    return $bindable
}

# Run only when executed as a standalone script (skipped when dot-sourced).
if ($MyInvocation.InvocationName -ne '.') {
    $ok = Invoke-ApiserverReservation -Port $Port -LogPath $LogPath
    if ($ok) { exit 0 } else { exit 2 }
}

脚本二:Watch-K8sPortProxy.ps1

<#
.SYNOPSIS
    Resident watchdog that publishes the Docker Desktop Kubernetes apiserver
    (127.0.0.1:<Port>) on all host IPv4/IPv6 interfaces via netsh portproxy,
    but only while the apiserver is actually listening.

.DESCRIPTION
    Loop (default every 10s):
      * Checks whether the REAL apiserver owns a 127.0.0.1:<Port> listener. It must
        not be a plain TCP connect: our own wildcard portproxy (0.0.0.0/::) would
        satisfy that and cause a false "up".
      * If UP   -> idempotently adds v4tov4 (0.0.0.0 -> 127.0.0.1) and
                   v6tov4 (:: -> 127.0.0.1) portproxy rules.
      * If DOWN for FailThreshold consecutive checks -> removes any portproxy
                   rules for <Port>. This prevents a wildcard 0.0.0.0 listener
                   from blocking Docker's own 127.0.0.1 bind on the next start.
      * On startup, removes any stale portproxy left over from a previous boot
                   (portproxy rules persist across reboots and would otherwise
                   sit on 0.0.0.0:<Port> before Docker starts).

    Design notes:
      * Single-threaded, no queues/locks -> no deadlocks.
      * Listener enumeration only (no TIME_WAIT churn); no state grows unbounded
        -> no handle/memory leaks.
      * Log file is size-capped and rotated -> no unbounded disk growth.
      * Compatible with Windows PowerShell 5.1 and PowerShell 7+.

    Requires administrator privileges (netsh portproxy). Intended to run as
    SYSTEM via a scheduled task.

.PARAMETER Port
    Apiserver port. Default 6443.

.PARAMETER IntervalSeconds
    Seconds between probes. Default 10.

.PARAMETER FailThreshold
    Consecutive failed probes before removing the portproxy. Default 3.

.PARAMETER ConnectAddress
    Backend address the portproxy forwards to. Default 127.0.0.1.

.PARAMETER LogPath
    Log file path. Defaults to watch.log next to this script.

.PARAMETER MaxLogBytes
    Rotate the log once it exceeds this size. Default 1 MiB.
#>

[CmdletBinding()]
param(
    [int]$Port = 6443,
    [int]$IntervalSeconds = 10,
    [int]$FailThreshold = 3,
    [string]$ConnectAddress = '127.0.0.1',
    [string]$LogPath,
    [int]$MaxLogBytes = 1048576
)

$ErrorActionPreference = 'Continue'
if (-not $LogPath) {
    $base = if ($PSScriptRoot) { $PSScriptRoot } else { $PWD.Path }
    $LogPath = Join-Path $base 'watch.log'
}

function Write-Log {
    param([Parameter(Mandatory)][string]$Message)
    $line = '{0} [watch] {1}' -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $Message
    try {
        if ((Test-Path -LiteralPath $LogPath) -and `
            ((Get-Item -LiteralPath $LogPath).Length -gt $MaxLogBytes)) {
            $bak = "$LogPath.1"
            Move-Item -LiteralPath $LogPath -Destination $bak -Force -ErrorAction SilentlyContinue
        }
        $utf8Bom = New-Object System.Text.UTF8Encoding($true)
        [System.IO.File]::AppendAllText($LogPath, $line + [Environment]::NewLine, $utf8Bom)
    }
    catch { }
    Write-Verbose $line
}

function Test-ApiserverListening {
    # True only when the real apiserver holds 127.0.0.1:<Port> (Docker exposes it
    # with in_ip=127.0.0.1). Our own portproxy listens on 0.0.0.0/::, so it can
    # never be mistaken for the apiserver. GetActiveTcpListeners enumerates only
    # listeners (not TIME_WAIT), so it is fast and does not churn connections.
    param([Parameter(Mandatory)][int]$PortNumber)
    $loopbacks = @(
        [System.Net.IPAddress]::Loopback
        [System.Net.IPAddress]::IPv6Loopback
    )
    $listeners = [System.Net.NetworkInformation.IPGlobalProperties]::GetIPGlobalProperties().GetActiveTcpListeners()
    foreach ($ep in $listeners) {
        if ($ep.Port -ne $PortNumber) { continue }
        foreach ($addr in $loopbacks) {
            if ($ep.Address.Equals($addr)) { return $true }
        }
    }
    return $false
}

function Test-V4ProxyExists {
    param([Parameter(Mandatory)][int]$PortNumber)
    $pattern = '^\s*0\.0\.0\.0\s+' + [regex]::Escape([string]$PortNumber) + '\s+'
    foreach ($line in (& netsh.exe interface portproxy show v4tov4 2>$null)) {
        if ($line -match $pattern) { return $true }
    }
    return $false
}

function Test-V6ProxyExists {
    param([Parameter(Mandatory)][int]$PortNumber)
    $pattern = '^\s*(::|\*)\s+' + [regex]::Escape([string]$PortNumber) + '\s+'
    foreach ($line in (& netsh.exe interface portproxy show v6tov4 2>$null)) {
        if ($line -match $pattern) { return $true }
    }
    return $false
}

function Add-Proxy {
    param([Parameter(Mandatory)][int]$PortNumber)
    if (-not (Test-V4ProxyExists -PortNumber $PortNumber)) {
        & netsh.exe interface portproxy add v4tov4 listenport=$PortNumber listenaddress=0.0.0.0 connectport=$PortNumber connectaddress=$ConnectAddress 2>&1 | Out-Null
        Write-Log "added v4tov4 0.0.0.0:$PortNumber -> ${ConnectAddress}:$PortNumber (exit=$LASTEXITCODE)"
    }
    if (-not (Test-V6ProxyExists -PortNumber $PortNumber)) {
        & netsh.exe interface portproxy add v6tov4 listenport=$PortNumber listenaddress=:: connectport=$PortNumber connectaddress=$ConnectAddress 2>&1 | Out-Null
        Write-Log "added v6tov4 [::]:$PortNumber -> ${ConnectAddress}:$PortNumber (exit=$LASTEXITCODE)"
    }
}

function Remove-Proxy {
    param([Parameter(Mandatory)][int]$PortNumber)
    if (Test-V4ProxyExists -PortNumber $PortNumber) {
        & netsh.exe interface portproxy delete v4tov4 listenport=$PortNumber listenaddress=0.0.0.0 2>&1 | Out-Null
        Write-Log "removed v4tov4 for port $PortNumber (exit=$LASTEXITCODE)"
    }
    if (Test-V6ProxyExists -PortNumber $PortNumber) {
        & netsh.exe interface portproxy delete v6tov4 listenport=$PortNumber listenaddress=:: 2>&1 | Out-Null
        Write-Log "removed v6tov4 for port $PortNumber (exit=$LASTEXITCODE)"
    }
}

Write-Log "start pid=$PID port=$Port interval=${IntervalSeconds}s failThreshold=$FailThreshold"

# A portproxy rule persists across reboots and would occupy 0.0.0.0/<Port> before
# Docker starts, blocking its 127.0.0.1 bind. Remove any stale rule on startup
# unless the real apiserver is already listening.
if (-not (Test-ApiserverListening -PortNumber $Port)) {
    Remove-Proxy -PortNumber $Port
}

$failCount = 0
while ($true) {
    try {
        if (Test-ApiserverListening -PortNumber $Port) {
            $failCount = 0
            if (-not (Test-V4ProxyExists -PortNumber $Port)) {
                Write-Log 'apiserver is up; ensuring portproxy'
                Add-Proxy -PortNumber $Port
            }
        }
        else {
            if ($failCount -lt $FailThreshold) { $failCount++ }
            if ($failCount -ge $FailThreshold -and (Test-V4ProxyExists -PortNumber $Port)) {
                Write-Log 'apiserver is down; removing portproxy'
                Remove-Proxy -PortNumber $Port
            }
        }
    }
    catch {
        Write-Log ('loop error: ' + $_.Exception.Message)
    }

    Start-Sleep -Seconds $IntervalSeconds
}

注册计划任务

以管理员身份运行一次(端口保留需要管理员权限):

# 1. 先跑一次端口保留
& "C:\ProgramData\DockerK8s\Reserve-ApiserverPort.ps1"

# 2. 注册两个计划任务
$ps  = "$env:SystemRoot\System32\WindowsPowerShell\v1.0\powershell.exe"
$dir = "C:\ProgramData\DockerK8s"

$principal = New-ScheduledTaskPrincipal -UserId SYSTEM -LogonType ServiceAccount -RunLevel Highest
$settings  = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries `
    -StartWhenAvailable -ExecutionTimeLimit ([TimeSpan]::Zero) `
    -RestartCount 3 -RestartInterval (New-TimeSpan -Minutes 1) -MultipleInstances IgnoreNew

Register-ScheduledTask -TaskName DockerK8s-ApiserverReserve -Force `
    -Action (New-ScheduledTaskAction -Execute $ps -Argument "-NoProfile -NonInteractive -ExecutionPolicy Bypass -WindowStyle Hidden -File `"$dir\Reserve-ApiserverPort.ps1`"") `
    -Trigger (New-ScheduledTaskTrigger -AtStartup) `
    -Principal $principal -Settings $settings

Register-ScheduledTask -TaskName DockerK8s-PortProxyGate -Force `
    -Action (New-ScheduledTaskAction -Execute $ps -Argument "-NoProfile -NonInteractive -ExecutionPolicy Bypass -WindowStyle Hidden -File `"$dir\Watch-K8sPortProxy.ps1`"") `
    -Trigger @((New-ScheduledTaskTrigger -AtStartup), (New-ScheduledTaskTrigger -AtLogOn)) `
    -Principal $principal -Settings $settings

Start-ScheduledTask -TaskName DockerK8s-PortProxyGate

完成后,验证:

kubectl get nodes
netsh interface portproxy show all
Get-Content "C:\ProgramData\DockerK8s\watch.log" -Encoding UTF8 -Tail 20

K8s 起来后大约 10 秒内,看护脚本会自动把 0.0.0.0:6443(以及 IPv6 的 [::]:6443)转发到 127.0.0.1:6443;K8s 停止时再自动移除。

踩坑记录

一、提权会话里 Start-Process 的子进程被降权

安装脚本里如果这样启动子脚本:

Start-Process powershell -Wait -ArgumentList '-File', $reserveScript

在某些 UAC 配置下,子进程会丢掉提权令牌,于是被脚本开头的 #requires -RunAsAdministrator 直接拦下,表现为:退出码 1、日志文件根本没生成,查半天都查不出原因。

解决办法:不要起子进程,改为在当前已提权的同一会话里 dot-source 执行:

. $reserveScript
Invoke-ApiserverReservation -Port 6443

脚本里用 $MyInvocation.InvocationName -ne '.' 判断是被 dot-source 还是独立运行,只有独立运行时才 exit,这样既能被别的脚本复用,也能被计划任务单独调用。

二、portproxy 规则会跨重启保留(最坑的一个)

netsh interface portproxy add 添加的规则是持久化的,重启后依然存在。如果它在 Docker 启动之前就占了 0.0.0.0:6443,就会再次触发本文开头那个 “exposing apiserver port / bind forbidden”。

更坑的是:如果看护脚本用「裸 TCP 连接 127.0.0.1:6443」来判断 apiserver 是否存活,这个连接会被脚本自己刚加的 0.0.0.0 通配监听接住,于是误判为“已启动”,那条规则就永远不被清理——每次重启都复发。

正确做法有两点:

  1. 启动时先清理本端口的 portproxy,避免上次重启残留的规则挡路;
  2. 用监听地址区分真假 apiserver:Docker 用 in_ip=127.0.0.1 暴露,所以真 apiserver 会绑定 127.0.0.1:6443;而 portproxy 只监听 0.0.0.0/::。判断是否存在 127.0.0.1:6443 监听器即可,且该 API 不会枚举 TIME_WAIT,没有连接churn:
[System.Net.NetworkInformation.IPGlobalProperties]::GetIPGlobalProperties().GetActiveTcpListeners() |
    Where-Object { $_.Port -eq 6443 -and $_.Address.Equals([System.Net.IPAddress]::Loopback) }

三、日志乱码:鍙︿竴涓…

netsh 这类系统工具的输出是系统本地码页(中文 Windows 是 GBK/936),而日志如果写成了 UTF-8,读的时候又用 GBK 去解,就会看到 鍙︿竴涓… 这种乱码。

处理办法:

  • 日志统一用 UTF-8 带 BOM 写入(UTF8Encoding($true) + AppendAllText),任何编辑器都能正确识别;
  • 控制台只打印 ASCII,不要直接把本地化中文往终端里塞;
  • 读取日志时显式指定编码:Get-Content xxx.log -Encoding UTF8。

四、计划任务以 SYSTEM 注册后,非提权看不到

以 SYSTEM / 最高权限注册的任务,默认的安全描述符只允许管理员读取。用普通权限打开任务计划程序根本看不到这两个任务,甚至查询会返回「拒绝访问」。

用管理员身份打开任务计划程序,在左侧 「任务计划程序库」根目录(不是子文件夹)就能找到:

  • DockerK8s-ApiserverReserve
  • DockerK8s-PortProxyGate

五、常驻脚本要防泄漏/死锁

看护脚本是常驻的,写的时候特意注意了这些:

  • 单线程、无锁、无队列,不会死锁;
  • 每次探测的 TcpClient 和 WaitHandle 都在 finally 里关闭,不泄漏句柄;
  • 不累积集合、失败计数器封顶,内存稳定;
  • 日志超过 1 MiB 自动轮转,磁盘不涨;
  • 计划任务设置 -MultipleInstances IgnoreNew,不会跑出多个实例。

卸载 / 回滚

Stop-ScheduledTask -TaskName DockerK8s-PortProxyGate -ErrorAction SilentlyContinue
Unregister-ScheduledTask -TaskName DockerK8s-PortProxyGate -Confirm:$false
Unregister-ScheduledTask -TaskName DockerK8s-ApiserverReserve -Confirm:$false

netsh interface portproxy delete v4tov4 listenport=6443 listenaddress=0.0.0.0
netsh interface portproxy delete v6tov4 listenport=6443 listenaddress=::

Remove-Item "C:\ProgramData\DockerK8s" -Recurse -Force

# 可选:撤销端口保留
netsh int ipv4 delete excludedportrange protocol=tcp startport=6443 numberofports=1 store=persistent
林威
林威 咖味十足的软件工程师