Split the Android data binds: /data/app, /data/dalvik-cache and
/data/misc/apexdata/com.android.art/dalvik-cache are Android-system
caches (not the Termux app's private data), so both distro types get
them in the default mode. The Termux app dirs (apps, cache, $HOME)
under /data/data/com.termux stay normal-type-only.
Shared storage (/storage, /sdcard) is host-domain Android storage, not
the per-distro /data/data/com.termux, so it should be visible to both
distro types in the default mode. Only the Termux app data binds stay
normal-type-only.
Apply image manifest Env in every mode (isolated, minimal, and
termux-type containers, which previously ignored it). Inherit Android
host vars only in the default mode (Termux, neither isolated nor
minimal); factor the var list into a shared ANDROID_HOST_ENV_VARS.
Termux-type containers no longer bind the host's /data/data/com.termux
or shared storage: non-isolated binds only Android system directories,
isolated/minimal binds no host directories. Normal-type bindings are
unchanged.
push always forced verified HTTPS for custom registries (it discarded
the base resolved by get_auth_token), so it could neither reach an
HTTP-only registry nor an HTTPS registry with an untrusted certificate,
and it lacked the --allow-insecure escape hatch install has.
Resolve the scheme/base via get_auth_token(insecure=...) and thread that
base + the insecure flag through every request (_blob_exists,
_upload_blob_bytes, _upload_blob_file, _put_manifest now take base +
insecure and use opener(insecure) instead of registry_base_url +
auth_opener). get_auth_token is the gatekeeper, so push inherits the
same handling as install: by default a bad certificate or an HTTP-only
registry raises a meaningful error pointing at --allow-insecure; with
the flag, a bad cert is reached over unverified HTTPS and an HTTP-only
registry falls back to an http base. Docker Hub stays verified-HTTPS.
Add --allow-insecure to the parser, push command, help page, and bash/
zsh/fish completions. Update and extend the push tests for the new base
argument and the insecure threading.
Push made each HTTP request exactly once, so a transient blip during a
blob HEAD, upload, or manifest PUT aborted the whole push — unlike the
install/pull path, which now retries via the shared retry_http policy.
Wrap every push request with retry_http so both behave identically.
_blob_exists, _upload_blob_bytes, _upload_blob_file, and _put_manifest
now retry transient failures (5xx, connection resets) up to 5x with a
logged notice; deterministic ones (4xx) fail fast. An upload retry
re-runs the whole session: a fresh POST yields a new upload Location,
then the monolithic content-addressed PUT re-sends the bytes (or
re-streams the file from the start) — safe to repeat. _blob_exists
still maps 404 to "absent" without retrying.
The existing 401/403 -> push_denied_msg translation is preserved: those
are 4xx, never retried, so they reach the handler immediately as before.
The OCI/Docker pull path made each HTTP request exactly once, so a
transient blip during auth, manifest, or layer fetch aborted the whole
install — unlike download_file, which retried. Share one retry policy
between both.
Extract the retry behaviour from download_file into download.retry_http
plus is_retryable_http_error: transient failures (5xx, connection
resets, timeouts, DNS) are retried up to 5x with a 5s delay and a logged
notice; deterministic ones (4xx except 408/429, TLS cert failures,
plaintext-HTTP replies) fail fast. download_file now drives retry_http
and keeps its friendly-message translation in an outer handler.
Wrap every registry request with retry_http: Docker Hub auth, the
custom-registry /v2/ probe and bearer-token exchange, manifest and
image-config fetches, and each layer blob (retried whole, fresh sha256
per attempt; a hash mismatch stays a hard error). The expected 401
bearer challenge is a 4xx, so it is not retried and still flows to the
challenge handler; 401/403/404 still translate to auth_denied_msg /
"Image not found".
download_file retried transient failures silently, so a flaky download
looked hung until it either succeeded or exhausted every attempt. Emit a
[*] info line naming the attempt number and the underlying error before
each retry delay. Deterministic fail-fast errors (4xx, bad cert,
plaintext-HTTP) raise immediately and log no retry line; the message is
suppressed under --quiet like the other download progress lines.
download_file retried every URLError/OSError, so a deterministic HTTP
client error (404 Not Found, 403 Forbidden, …) was retried up to
max_retries times with delays before surfacing a raw error. A 404 will
not become a 200 on retry, so fail immediately with a meaningful
"HTTP <code> <reason>" message. 408 and 429 (the standard "retry later"
codes) and 5xx/network errors still fall through to the retry loop.
The docker pull path already fails fast: download_blob/_get_manifest/
get_auth_token have no retry loop and pull_image translates 404 to
"Image not found".
Surface a clear error instead of a raw [SSL: CERTIFICATE_VERIFY_FAILED]
when an HTTPS endpoint presents an untrusted/expired/self-signed cert (or
a hostname mismatch), for both Docker registry pulls and plain URL (tarball)
downloads. The download path now fails fast on such errors rather than
looping through every retry first.
Extend --allow-insecure to also skip TLS certificate verification on HTTPS
endpoints. For a custom registry the scheme is now resolved by probing
HTTPS (with verification disabled) first and falling back to plain HTTP,
so one flag covers both bad-cert HTTPS registries and HTTP-only registries
(Docker's --insecure-registry model). To carry the resolved scheme,
get_auth_token now returns (token, base_url) threaded through the pull path.
The generic TLS classifiers and the unverified SSL context live in
helpers/download.py and are shared by the registry transport. push is
unchanged (verified HTTPS only).
Turn a raw `[SSL: WRONG_VERSION_NUMBER]` into a meaningful, actionable error
when an HTTPS endpoint is actually served over plain HTTP — for both Docker
registry pulls and direct URL (rootfs/OCI archive) downloads.
Move the OpenSSL-reason classifier into helpers/download.py as the shared
is_plaintext_http_tls_error(); the Docker transport imports it. The registry
auth probe now treats a plaintext TLS handshake error as a conclusive
HTTP-only signal (no second request needed), keeping the active /v2/ HTTP
re-probe only as a fallback, so the friendly --allow-insecure hint no longer
depends on that probe succeeding. download_file() detects the same condition
and fails fast with a clear message (retry with the http:// URL) instead of
looping through retries and then surfacing the raw SSL error.
Enforce HTTPS for custom registry pulls by default. When the HTTPS /v2/
probe fails at the connection/TLS level, re-probe over plain HTTP to tell
an HTTP-only registry apart from an unreachable one: if it answers over
HTTP, fail with a message pointing the user at --allow-insecure instead of
a generic network error. With --allow-insecure, all registry traffic for
that install (auth probe, manifest, config, layer blobs) goes over HTTP.
Docker Hub stays HTTPS regardless; push and build are unchanged (the
insecure flag defaults to False throughout the transport stack).
Python's tarfile.extractfile() silently follows LNKTYPE (hardlink) and
SYMTYPE (symlink) members to their targets within the archive. Without
an explicit isreg() check, a crafted outer OCI tar could include a
hardlink blob member (e.g. blobs/sha256/<layer_hex>) that shadows the
legitimate regular-file entry in member_map. extractfile() would then
return a different member's content — with no digest verification to
catch the swap — causing the wrong blob to be applied or parsed as JSON.
Add member.isreg() guards in _oci_read_json and _oci_cache_layer before
any call to extractfile(), rejecting hardlinks and symlinks. The
existing `if fobj is None` check did not cover this because extractfile()
on a hardlink returns non-None by following the link internally.
Extend make_oci_archive in the test builder with outer_extra_members so
hostile members can be injected into the outer archive for testing. Add
three new tests that cover hardlink/symlink shadowing of a layer blob and
of index.json.
`$` matches before a single trailing `\n` in Python's `re` module, so
`is_valid_name("foo\n")` was incorrectly returning True. Switching to
`\Z` anchors the match at the true end of the string.
The previous rootfs check only proved that a member named a rootfs path,
not that a rootfs directory actually resulted. An archive could still
report success while leaving a broken, rootfs-less container: e.g. a
stray file or symlink where the rootfs directory belongs, or rootfs
entries that all fail to resolve (dangling hardlinks).
Tighten the guarantee so a restore either yields a real rootfs directory
or is rejected without leaving a broken container:
- Defer the destructive clear to the first rootfs member that actually
materialises content (resolve/skip dangling hardlinks and unknown
types first), so an archive whose rootfs entries don't resolve never
clears the existing container.
- Buffer the manifest and write it only on success, so a failed restore
never clobbers the installed container's metadata.
- After extraction, require a real directory at the rootfs path (not a
file, not a symlink). If absent, remove the partial result so nothing
rootfs-less is left behind.
The old rootfs is still cleared before extraction (no doubled disk use),
keeping restore friendly to space-constrained Termux devices.
A well-named archive with no rootfs — manifest-only, empty, a truncated
backup cut off before the rootfs, or the wrong file whose members all
get skipped — was "restored" silently: the existing rootfs was cleared
on the first member and the run reported success, leaving a destroyed or
phantom container.
Make restore atomic on the rootfs check. Defer the destructive steps
(clearing the old rootfs, writing the manifest) until the first rootfs
member is seen; buffer manifest.json until that commit point. An archive
that never yields a rootfs member therefore leaves the target untouched
and is rejected with a clear error.
`restore` derived the container name per-member and tracked them in a
dict/set, so an archive with members for several distinct names would
lock, clear, and restore all of them. `backup` only ever writes a single
container, so such archives are only hand-crafted or legacy and silently
overwrite more than the user asked for.
Fix the first valid member as the sole target and reject any member that
names a different container. Also clamp a hardlink's source to that
target so a crafted linkname can't read out of an unrelated container.
Add a Tests workflow (separate from publish.yml) that runs the offline
pytest suite on push and pull_request across Python 3.9/3.11/3.13, with
unit, integration, and security tests as separate steps. Live tests stay
skipped (RUN_LIVE_TESTS unset), so no proot or network access is required.
Replace the strict-xfail (which assumed the pre-_safe_resolve escape) with
passing tests that assert containment: an absolute-target symlink written
through it — both in a single archive and via the realistic cross-layer
OCI vector — resolves inside the rootfs and never reaches the host. Fails
loudly if the symlinked-parent clamping regresses.
The tar extractors created symlinks verbatim from attacker-controlled
member.linkname, but later members' destinations were built with a
plain os.path.join and written via os.makedirs/open, which follow any
symlink an earlier member planted. A crafted archive could ship
`evil -> /` (absolute, or `../../`) followed by `evil/secret` and the
write would land on the host. Absolute symlink targets are legitimate
inside a container (proot remaps guest '/' to the rootfs), so they
can't be rejected — the host extractor must instead refuse to follow
them.
Add _safe_resolve(): a securejoin-style resolver that follows existing
symlink components but clamps every hop inside the rootfs (absolute
targets re-root at the rootfs, '..' can't ascend past it, symlink loops
budgeted). This blocks the escape and matches proot's runtime view, so
legitimate absolute/usrmerge symlinks still resolve to the correct
in-rootfs location.
Applied at every write chokepoint:
- helpers/tar_extract.py (install + apply_layer/OCI layer apply):
resolve the destination parent; never follow the final component;
drop an existing symlink before a directory write; hard links store
validated relative parts and re-resolve both endpoints at copy time
so a symlink planted after the link member can't redirect the
deferred copy.
- commands/restore.py (backup restore): same via _safe_dest(), clamped
to the container dir, plus clamped hard-link read sources.
- helpers/build_engine/copy_step.py (COPY / ADD-tar materialisation):
resolve the parent so an ADD'd tar can't escape during a build.
Cross-layer attacks are covered too: _safe_resolve inspects the actual
on-disk rootfs, so a symlink in one layer and the write in a later one
is still contained.
It appears that PROOT_NO_SECCOMP is broken for a long time and its usage
can rather make things even worse. Don't mention this variable in any of
documentation.
resolve_rootfs_path was treating l2s symlink targets as guest-absolute
paths and prepending rootfs, producing a double-rooted path that doesn't
exist. This caused _check_shell_available to falsely conclude the shell
was missing and emit a misleading "use 'proot-distro run'" error when
the shell binary was stored as a proot link2symlink hardlink.
Fix by detecting l2s targets (via the existing resolve_l2s_target helper)
before the guest-path branch and stripping the rootfs prefix to recover
the correct guest-relative path for the next loop iteration.
Fixes https://github.com/termux/proot-distro/issues/675
Applying the archived mode immediately broke extraction when a parent
directory lacked owner write/exec, since subsequent file and subdir
writes into it would fail with EACCES. Widen such dirs to `mode |
S_IRWXU` during extraction and re-apply the real mode in reverse
insertion order afterwards so children get sealed before parents.
Issue https://github.com/termux/proot-distro/issues/673
Without -i, the caller's shell environment leaks into proot when the
printed command is executed, including variables that child_env
intentionally excludes. env -i starts from an empty environment,
matching the semantics of os.execvpe exactly.
Bind-mount /proc/sys/kernel/overflowuid and overflowgid with the
standard nobody/nogroup value so tools that read them work correctly
on Android where these entries may be inaccessible.
Sylirre