forked from mirror/coolify-cli
refactor(init): make podman/coold/firewall unconditional, add --skip-default-deny
Remove --podman, --default-deny, --install-coold flags. Podman stack, coold/corrosion agents, and default-deny iptables scaffold now always install. --skip-default-deny opts out of firewall scaffold for testing.
This commit is contained in:
Andras Bacsai
@@ -183,8 +183,9 @@ type Resource struct {
|
||||
- Establishes a full-mesh WireGuard overlay across N hosts.
|
||||
- Each host gets a mgmt IP `/32` from `--wg-mgmt-pool` (default `100.64.0.0/16`, RFC 6598 CGNAT) on `wg0`.
|
||||
- Each host gets a container subnet `/<container-prefix>` from `--container-pool` (default `10.210.0.0/16`, default prefix `/24`) owned by a Podman bridge named `coolify-mesh`.
|
||||
- Optionally installs Podman + enables `podman.socket` + creates the bridge + installs `coolify-mesh-fw.service` (`--podman` flag).
|
||||
- Optionally installs default-deny firewall scaffold (`--default-deny` flag) — adds `COOLIFY-INTRA` and empty `COOLIFY-ALLOW` chains.
|
||||
- Installs Podman + enables `podman.socket` + creates the bridge + installs `coolify-mesh-fw.service` (always; required for v5 runtime).
|
||||
- Installs coold + corrosion (v5 control-plane agents; always) from `--coold-binary` / `--corrosion-binary`.
|
||||
- Installs default-deny firewall scaffold by default — `COOLIFY-INTRA` + empty `COOLIFY-ALLOW` chains. Use `--skip-default-deny` to fall back to blanket-allow (mode A) for testing.
|
||||
|
||||
### Architecture (why this layout)
|
||||
|
||||
@@ -207,10 +208,10 @@ Critical: `AllowedIPs` lists the peer's full `/24` so kernel routes `10.210.<pee
|
||||
|
||||
Podman network `coolify-mesh` is created with `--disable-dns` — bridge gateway `10.210.X.1:53` is reserved for coold's embedded cluster DNS (see CONTROL_PLANE.md §5). Pre-alpha networks with `dns_enabled=true` are detected on re-run and recreated.
|
||||
|
||||
Firewall service (`coolify-mesh-fw.service`) installed by `--podman`:
|
||||
Firewall service (`coolify-mesh-fw.service`) installed unconditionally:
|
||||
- POSTROUTING `RETURN` rule prevents Podman MASQUERADE from rewriting container egress source on `wg0` (would break reverse routing because wg0 has no IP in the container subnet).
|
||||
- Mode A (no `--default-deny`): blanket FORWARD ACCEPT for container subnet.
|
||||
- Mode B (`--default-deny`): COOLIFY-INTRA chain (ESTABLISHED accept → COOLIFY-ALLOW → DROP), FORWARD jumps for `-s/-d <container-subnet>`. v5 control plane fills `COOLIFY-ALLOW`.
|
||||
- Mode A (`--skip-default-deny`): blanket FORWARD ACCEPT for container subnet.
|
||||
- Mode B (default): COOLIFY-INTRA chain (ESTABLISHED accept → COOLIFY-ALLOW → DROP), FORWARD jumps for `-s/-d <container-subnet>`. v5 control plane fills `COOLIFY-ALLOW`.
|
||||
|
||||
### Cross-host vs intra-host firewall
|
||||
|
||||
@@ -220,8 +221,8 @@ Firewall service (`coolify-mesh-fw.service`) installed by `--podman`:
|
||||
### Subcommands
|
||||
|
||||
```bash
|
||||
coolify init plan --servers IP1,IP2 --ssh-key ~/.ssh/id_ed25519 [--podman --default-deny]
|
||||
coolify init apply --servers IP1,IP2 --ssh-key ~/.ssh/id_ed25519 [--podman --default-deny] [--yes]
|
||||
coolify init plan --servers IP1,IP2 --ssh-key ~/.ssh/id_ed25519 [--skip-default-deny]
|
||||
coolify init apply --servers IP1,IP2 --ssh-key ~/.ssh/id_ed25519 [--skip-default-deny] [--yes]
|
||||
```
|
||||
|
||||
- `plan` is read-only: SSH-probes each host, reconstructs current state, shows what `apply` would do. Idempotent.
|
||||
@@ -241,9 +242,12 @@ coolify init apply --servers IP1,IP2 --ssh-key ~/.ssh/id_ed25519 [--podman --de
|
||||
| `--container-prefix` | `24` | per-host container subnet prefix |
|
||||
| `--wg-interface` | `wg0` | WG iface name on remote |
|
||||
| `--wg-listen-port` | `51820` | WG UDP port |
|
||||
| `--podman` | false | install podman + bridge + firewall service |
|
||||
| `--podman-network` | `coolify-mesh` | bridge network name |
|
||||
| `--default-deny` | false | requires `--podman`. Install COOLIFY-INTRA + empty COOLIFY-ALLOW chains for cross-host deny |
|
||||
| `--skip-default-deny` | false | skip the default-deny firewall scaffold. Default installs COOLIFY-INTRA + empty COOLIFY-ALLOW chains for cross-host deny |
|
||||
| `--coold-binary` | `$HOME/devel/coold/target/release/coold` | local path to the coold Linux/arm64 binary (required — uploaded to every host) |
|
||||
| `--corrosion-binary` | `$HOME/devel/corrosion/target/release/corrosion` | local path to the corrosion Linux/arm64 binary (required — uploaded to every host) |
|
||||
| `--corrosion-gossip-port` | `8787` | corrosion SWIM gossip port (bound to wg0 mgmt IP) |
|
||||
| `--corrosion-api-port` | `8080` | corrosion HTTP API port (bound to 127.0.0.1) |
|
||||
| `--concurrency` | `10` | parallel SSH connections |
|
||||
| `--ssh-timeout` | `30s` | SSH connect timeout |
|
||||
| `--yes`, `-y` | false | skip alpha confirmation prompt |
|
||||
@@ -269,7 +273,7 @@ coolify init apply --servers IP1,IP2 --ssh-key ~/.ssh/id_ed25519 [--podman --de
|
||||
### Key invariants
|
||||
|
||||
- **Reconstructed-only state**: no local state file. Every `plan`/`apply` re-probes via SSH. State lives on the hosts.
|
||||
- **Idempotent**: re-running with no changes produces empty plan. State drift triggers re-converge (e.g. flipping `--default-deny` reinstalls the firewall service).
|
||||
- **Idempotent**: re-running with no changes produces empty plan. State drift triggers re-converge (e.g. flipping `--skip-default-deny` reinstalls the firewall service).
|
||||
- **Private key never leaves host**: WG private key generated on remote via `wg genkey`; config written using `$PRIVKEY=$(cat /etc/wireguard/privatekey)` shell expansion.
|
||||
- **Atomic config writes**: write to `.conf.tmp`, `mv` to `.conf`.
|
||||
- **Stable subnet assignment**: existing valid assignments are preserved across re-runs; only invalid (out-of-pool, wrong prefix, duplicate, network/broadcast IP) trigger reassignment with warning.
|
||||
@@ -297,7 +301,7 @@ Use the SSH `Runner` interface for mocking — never open real SSH connections i
|
||||
|
||||
## `coolify firewall` — cross-host allow-rule client (alpha, v5)
|
||||
|
||||
**This subcommand is the second outlier** (alongside `coolify init`): it does NOT talk to the Coolify API. It is a thin REST client of the **coold** per-host agent installed by `coolify init --install-coold`. `allow` / `revoke` / `list` all go through coold's REST API (`/api/v1/firewall/allow`). `containers` stays SSH+podman because coold has no container surface yet. Transport is **SSH-bounce**: the laptop running the CLI is not a mesh peer, so it SSHes into the target host and the shell there runs `curl "http://$(wg0-mgmt-ip):8443/api/v1/firewall/..."` against coold on localhost.
|
||||
**This subcommand is the second outlier** (alongside `coolify init`): it does NOT talk to the Coolify API. It is a thin REST client of the **coold** per-host agent installed by `coolify init` (coold install is unconditional as of v1.6.3). `allow` / `revoke` / `list` all go through coold's REST API (`/api/v1/firewall/allow`). `containers` stays SSH+podman because coold has no container surface yet. Transport is **SSH-bounce**: the laptop running the CLI is not a mesh peer, so it SSHes into the target host and the shell there runs `curl "http://$(wg0-mgmt-ip):8443/api/v1/firewall/..."` against coold on localhost.
|
||||
|
||||
coold owns all kernel-rule + persistence logic (iptables/nft backend detection, `/etc/coolify/allow.rules` snapshot, `coolify-mesh-allow.service`). The CLI never writes iptables or systemd units directly.
|
||||
|
||||
@@ -422,7 +426,7 @@ Uses `fakeCooldRunner` / `cmdFakeRunner` pattern (substring → canned stdout ma
|
||||
|
||||
### End-to-end flow (verified on real hosts)
|
||||
|
||||
After `coolify init apply --podman --default-deny --install-coold --servers A,B ...` ran (coold must be up):
|
||||
After `coolify init apply --servers A,B ...` ran (coold must be up):
|
||||
|
||||
1. Baseline cross-host traffic DROPped by `COOLIFY-INTRA`.
|
||||
2. `coolify firewall containers --servers A,B --ssh-key KEY` → discovery table.
|
||||
|
||||
+22
-27
@@ -42,32 +42,27 @@ func runApply(ctx context.Context, cmd *cobra.Command, flags *InitFlags) error {
|
||||
}
|
||||
|
||||
var corrosionSha, cooldSha string
|
||||
if flags.InstallCoold {
|
||||
if !flags.InstallPodman {
|
||||
return fmt.Errorf("--install-coold requires --podman")
|
||||
for _, bp := range []struct {
|
||||
label, path string
|
||||
out *string
|
||||
}{
|
||||
{"corrosion", flags.CorrosionBinaryPath, &corrosionSha},
|
||||
{"coold", flags.CooldBinaryPath, &cooldSha},
|
||||
} {
|
||||
if bp.path == "" {
|
||||
return fmt.Errorf("--%s-binary is required", bp.label)
|
||||
}
|
||||
for _, bp := range []struct {
|
||||
label, path string
|
||||
out *string
|
||||
}{
|
||||
{"corrosion", flags.CorrosionBinaryPath, &corrosionSha},
|
||||
{"coold", flags.CooldBinaryPath, &cooldSha},
|
||||
} {
|
||||
if bp.path == "" {
|
||||
return fmt.Errorf("--%s-binary is required with --install-coold", bp.label)
|
||||
}
|
||||
if _, err := os.Stat(bp.path); err != nil {
|
||||
return fmt.Errorf("%s binary %q: %w", bp.label, bp.path, err)
|
||||
}
|
||||
if err := services.VerifyLinuxARM64(bp.path); err != nil {
|
||||
return fmt.Errorf("%s binary: %w", bp.label, err)
|
||||
}
|
||||
sum, err := wireguard.FileSha256(bp.path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("hash %s binary: %w", bp.label, err)
|
||||
}
|
||||
*bp.out = sum
|
||||
if _, err := os.Stat(bp.path); err != nil {
|
||||
return fmt.Errorf("%s binary %q: %w", bp.label, bp.path, err)
|
||||
}
|
||||
if err := services.VerifyLinuxARM64(bp.path); err != nil {
|
||||
return fmt.Errorf("%s binary: %w", bp.label, err)
|
||||
}
|
||||
sum, err := wireguard.FileSha256(bp.path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("hash %s binary: %w", bp.label, err)
|
||||
}
|
||||
*bp.out = sum
|
||||
}
|
||||
|
||||
// Alpha gate: block unless bypassed.
|
||||
@@ -96,10 +91,10 @@ func runApply(ctx context.Context, cmd *cobra.Command, flags *InitFlags) error {
|
||||
ContainerPool: contPool,
|
||||
ContainerPrefix: flags.ContainerPrefix,
|
||||
ListenPort: flags.WGListenPort,
|
||||
InstallPodman: flags.InstallPodman,
|
||||
InstallPodman: true,
|
||||
PodmanNetworkName: flags.PodmanNetworkName,
|
||||
DefaultDenyContainers: flags.DefaultDenyContainers,
|
||||
InstallCoold: flags.InstallCoold,
|
||||
DefaultDenyContainers: !flags.SkipDefaultDeny,
|
||||
InstallCoold: true,
|
||||
CooldBinaryPath: flags.CooldBinaryPath,
|
||||
CorrosionBinaryPath: flags.CorrosionBinaryPath,
|
||||
CorrosionGossipPort: flags.CorrosionGossipPort,
|
||||
|
||||
@@ -37,9 +37,8 @@ func TestNewInitCommand_PersistentFlags(t *testing.T) {
|
||||
assert.NotNil(t, pf.Lookup("container-prefix"))
|
||||
assert.NotNil(t, pf.Lookup("wg-interface"))
|
||||
assert.NotNil(t, pf.Lookup("wg-listen-port"))
|
||||
assert.NotNil(t, pf.Lookup("podman"))
|
||||
assert.NotNil(t, pf.Lookup("podman-network"))
|
||||
assert.NotNil(t, pf.Lookup("default-deny"))
|
||||
assert.NotNil(t, pf.Lookup("skip-default-deny"))
|
||||
assert.NotNil(t, pf.Lookup("concurrency"))
|
||||
assert.NotNil(t, pf.Lookup("ssh-timeout"))
|
||||
assert.NotNil(t, pf.Lookup("yes"))
|
||||
@@ -47,6 +46,9 @@ func TestNewInitCommand_PersistentFlags(t *testing.T) {
|
||||
assert.Nil(t, pf.Lookup("wg-pool"))
|
||||
assert.Nil(t, pf.Lookup("wg-host-prefix"))
|
||||
assert.Nil(t, pf.Lookup("wg-subnet"))
|
||||
assert.Nil(t, pf.Lookup("podman"))
|
||||
assert.Nil(t, pf.Lookup("default-deny"))
|
||||
assert.Nil(t, pf.Lookup("install-coold"))
|
||||
}
|
||||
|
||||
// TestNewInitCommand_FlagDefaults verifies default values.
|
||||
@@ -82,17 +84,13 @@ func TestNewInitCommand_FlagDefaults(t *testing.T) {
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 51820, listenPort)
|
||||
|
||||
podman, err := pf.GetBool("podman")
|
||||
require.NoError(t, err)
|
||||
assert.False(t, podman)
|
||||
|
||||
podmanNet, err := pf.GetString("podman-network")
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "coolify-mesh", podmanNet)
|
||||
|
||||
defaultDeny, err := pf.GetBool("default-deny")
|
||||
skipDefaultDeny, err := pf.GetBool("skip-default-deny")
|
||||
require.NoError(t, err)
|
||||
assert.False(t, defaultDeny)
|
||||
assert.False(t, skipDefaultDeny)
|
||||
|
||||
concurrency, err := pf.GetInt("concurrency")
|
||||
require.NoError(t, err)
|
||||
|
||||
+17
-23
@@ -14,20 +14,18 @@ import (
|
||||
type InitFlags struct {
|
||||
common.SSHMeshFlags
|
||||
|
||||
WGMgmtPool string
|
||||
ContainerPool string
|
||||
ContainerPrefix int
|
||||
WGInterface string
|
||||
WGListenPort int
|
||||
InstallPodman bool
|
||||
PodmanNetworkName string
|
||||
DefaultDenyContainers bool
|
||||
InstallCoold bool
|
||||
CooldBinaryPath string
|
||||
CorrosionBinaryPath string
|
||||
CorrosionGossipPort int
|
||||
CorrosionAPIPort int
|
||||
Yes bool
|
||||
WGMgmtPool string
|
||||
ContainerPool string
|
||||
ContainerPrefix int
|
||||
WGInterface string
|
||||
WGListenPort int
|
||||
PodmanNetworkName string
|
||||
SkipDefaultDeny bool
|
||||
CooldBinaryPath string
|
||||
CorrosionBinaryPath string
|
||||
CorrosionGossipPort int
|
||||
CorrosionAPIPort int
|
||||
Yes bool
|
||||
}
|
||||
|
||||
// bindInitFlags registers all shared flags as PersistentFlags on cmd.
|
||||
@@ -46,20 +44,16 @@ func bindInitFlags(cmd *cobra.Command, f *InitFlags) {
|
||||
"WireGuard interface name on the remote hosts")
|
||||
pf.IntVar(&f.WGListenPort, "wg-listen-port", 51820,
|
||||
"WireGuard UDP listen port")
|
||||
pf.BoolVar(&f.InstallPodman, "podman", false,
|
||||
"Install Podman, enable its socket, create a per-host bridge network, install firewall rules, and enable IP forwarding")
|
||||
pf.StringVar(&f.PodmanNetworkName, "podman-network", "coolify-mesh",
|
||||
"Name of the Podman bridge network created on each host (requires --podman)")
|
||||
pf.BoolVar(&f.DefaultDenyContainers, "default-deny", false,
|
||||
"With --podman: install default-deny iptables rules for CROSS-HOST container traffic (between hosts via wg0). Intra-host (same bridge) traffic is NOT enforced — defer to per-app podman networks. The v5 control plane manages allows in the COOLIFY-ALLOW chain on the host that owns each destination IP")
|
||||
pf.BoolVar(&f.InstallCoold, "install-coold", false,
|
||||
"Install the Coolify v5 control-plane agents (corrosion + coold). Requires --podman. Uploads the binaries from --corrosion-binary / --coold-binary")
|
||||
"Name of the Podman bridge network created on each host")
|
||||
pf.BoolVar(&f.SkipDefaultDeny, "skip-default-deny", false,
|
||||
"Skip installing the default-deny iptables scaffold (COOLIFY-INTRA / COOLIFY-ALLOW). By default, cross-host container traffic is blocked except where coold installs allow rules. Intra-host (same bridge) traffic is NOT enforced — defer to per-app podman networks")
|
||||
pf.StringVar(&f.CooldBinaryPath, "coold-binary",
|
||||
os.ExpandEnv("$HOME/devel/coold/target/release/coold"),
|
||||
"Local path to the coold Linux/arm64 binary (used with --install-coold)")
|
||||
"Local path to the coold Linux/arm64 binary")
|
||||
pf.StringVar(&f.CorrosionBinaryPath, "corrosion-binary",
|
||||
os.ExpandEnv("$HOME/devel/corrosion/target/release/corrosion"),
|
||||
"Local path to the corrosion Linux/arm64 binary (used with --install-coold)")
|
||||
"Local path to the corrosion Linux/arm64 binary")
|
||||
pf.IntVar(&f.CorrosionGossipPort, "corrosion-gossip-port", 8787,
|
||||
"Corrosion SWIM gossip port (bound to the wg0 mgmt IP)")
|
||||
pf.IntVar(&f.CorrosionAPIPort, "corrosion-api-port", 8080,
|
||||
|
||||
+7
-2
@@ -21,8 +21,13 @@ func NewInitCommand() *cobra.Command {
|
||||
cmd := &cobra.Command{
|
||||
Use: "init",
|
||||
Short: "[ALPHA] Initialize WireGuard mesh for Coolify v5",
|
||||
Long: `[ALPHA] Bootstrap a WireGuard full-mesh overlay between servers so
|
||||
the Coolify v5 control plane can reach them over a private network.
|
||||
Long: `[ALPHA] Bootstrap a WireGuard full-mesh overlay between servers and
|
||||
provision each host with the Coolify v5 runtime stack: Podman + bridge
|
||||
network, default-deny iptables scaffold, and the coold/corrosion
|
||||
control-plane agents.
|
||||
|
||||
All of the above install by default. Use --skip-default-deny to leave
|
||||
the firewall in blanket-allow mode for testing.
|
||||
|
||||
Subcommands:
|
||||
plan Show what would change without touching anything.
|
||||
|
||||
+12
-14
@@ -43,19 +43,17 @@ func runPlan(ctx context.Context, cmd *cobra.Command, flags *InitFlags) error {
|
||||
}
|
||||
|
||||
var corrosionSha, cooldSha string
|
||||
if flags.InstallCoold {
|
||||
if flags.CorrosionBinaryPath != "" {
|
||||
if _, err := os.Stat(flags.CorrosionBinaryPath); err == nil {
|
||||
if s, herr := wireguard.FileSha256(flags.CorrosionBinaryPath); herr == nil {
|
||||
corrosionSha = s
|
||||
}
|
||||
if flags.CorrosionBinaryPath != "" {
|
||||
if _, err := os.Stat(flags.CorrosionBinaryPath); err == nil {
|
||||
if s, herr := wireguard.FileSha256(flags.CorrosionBinaryPath); herr == nil {
|
||||
corrosionSha = s
|
||||
}
|
||||
}
|
||||
if flags.CooldBinaryPath != "" {
|
||||
if _, err := os.Stat(flags.CooldBinaryPath); err == nil {
|
||||
if s, herr := wireguard.FileSha256(flags.CooldBinaryPath); herr == nil {
|
||||
cooldSha = s
|
||||
}
|
||||
}
|
||||
if flags.CooldBinaryPath != "" {
|
||||
if _, err := os.Stat(flags.CooldBinaryPath); err == nil {
|
||||
if s, herr := wireguard.FileSha256(flags.CooldBinaryPath); herr == nil {
|
||||
cooldSha = s
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -67,10 +65,10 @@ func runPlan(ctx context.Context, cmd *cobra.Command, flags *InitFlags) error {
|
||||
ContainerPool: contPool,
|
||||
ContainerPrefix: flags.ContainerPrefix,
|
||||
ListenPort: flags.WGListenPort,
|
||||
InstallPodman: flags.InstallPodman,
|
||||
InstallPodman: true,
|
||||
PodmanNetworkName: flags.PodmanNetworkName,
|
||||
DefaultDenyContainers: flags.DefaultDenyContainers,
|
||||
InstallCoold: flags.InstallCoold,
|
||||
DefaultDenyContainers: !flags.SkipDefaultDeny,
|
||||
InstallCoold: true,
|
||||
CooldBinaryPath: flags.CooldBinaryPath,
|
||||
CorrosionBinaryPath: flags.CorrosionBinaryPath,
|
||||
CorrosionGossipPort: flags.CorrosionGossipPort,
|
||||
|
||||
Reference in New Issue
Block a user