smashimo/oh-my-pi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: unify builtins in one place

Browse Source
This commit is contained in:
can1357
2026-08-08 09:58:34 +02:00
parent 08819b279c
commit 2ee9943563
454 changed files with 59110 additions and 67520 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yml
+4 -2
View File
@@ -163,14 +163,16 @@ jobs:
# Clippy scope mirrors `cargo clippy --workspace` (libraries only, no
# test targets) plus the strict/default split: crates with
# `[lints] workspace = true` get the workspace policy, the vendored
# brush fork is exempt (same as run-rs-task.ts's cargo excludes).
# brush-core fork is exempt (same as run-rs-task.ts's cargo excludes).
# pi-builtins lands in the default-lints bucket below: it opts out of
# pedantic/nursery in its own manifest but is still linted.
- name: Clippy (workspace lint policy on opted-in crates)
run: |
bazelisk query "kind('rust_library|rust_shared_library', //crates/pi-ast/... + //crates/pi-iso/... + //crates/pi-natives/... + //crates/pi-shell/... + //crates/pi-voice/... + //crates/pi-walker/...)" \
| xargs bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build --config=clippy-strict --
- name: Clippy (default lints elsewhere)
run: |
bazelisk query "kind('rust_library|rust_shared_library', //crates/... - (//crates/pi-ast/... + //crates/pi-iso/... + //crates/pi-natives/... + //crates/pi-shell/... + //crates/pi-voice/... + //crates/pi-walker/...) - //crates/vendor/brush-core/... - //crates/vendor/brush-builtins/...)" \
bazelisk query "kind('rust_library|rust_shared_library', //crates/... - (//crates/pi-ast/... + //crates/pi-iso/... + //crates/pi-natives/... + //crates/pi-shell/... + //crates/pi-voice/... + //crates/pi-walker/...) - //crates/vendor/brush-core/...)" \
| xargs bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build --config=clippy --
- name: Rustfmt
run: bazelisk --bazelrc="${{ steps.cache.outputs.rc }}" build --config=rustfmt //crates/...
Cargo.lock
Generated
+86 -855
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+1 -2
View File
@@ -12,7 +12,6 @@ repository = "https://github.com/can1357/oh-my-pi"
[patch.crates-io]
brush-core = { path = "crates/vendor/brush-core" }
brush-builtins = { path = "crates/vendor/brush-builtins" }
[profile.release]
opt-level = 3
@@ -181,7 +180,7 @@ pi-shell = { path = "crates/pi-shell" }
pi-voice = { path = "crates/pi-voice" }
pi-walker = { path = "crates/pi-walker" }
brush-core = { path = "crates/vendor/brush-core" }
brush-builtins = { path = "crates/vendor/brush-builtins" }
pi-builtins = { path = "crates/pi-builtins" }
# ──────────────────────────────────────────────────────────────────────────────
# Async Runtime & Concurrency
MODULE.bazel
+1 -1
View File
@@ -145,7 +145,7 @@ crate.from_cargo(
],
# The root manifest covers all workspace members, including the vendored
# brush fork (members so their sources render hermetically; the
# [patch.crates-io] entries redirect brush-builtins' registry dep to them).
# [patch.crates-io] entry redirects brush-core's registry dep to it).
manifests = ["//:Cargo.toml"],
)
MODULE.bazel.lock
Generated
+19 -269
View File
File diff suppressed because one or more lines are too long
README.md
+4 -12
View File
@@ -171,7 +171,7 @@ _[Watch the capture ↗](https://omp.sh/clips/web.mp4)_
### 09 · Unapologetically native. Even on Windows.
Other agents shell out to rg, grep, find, and bash. On many machines those binaries don't exist, and on the ones where they do, every call costs a fork-exec round-trip. omp links the real implementations into the process. ripgrep, glob, find: in-process. brush is the bash — with sessions that survive across calls, and 46 vendored coreutils (ls, sed, sort, xargs, even jq via jaq) that run as in-process builtins, zero fork/exec. The same omp binary runs on macOS, Linux, and Windows — no WSL bridge.
Other agents shell out to rg, grep, find, and bash. On many machines those binaries don't exist, and on the ones where they do, every call costs a fork-exec round-trip. omp links the real implementations into the process. ripgrep, glob, find: in-process. brush is the bash — with sessions that survive across calls, and 58 command-line utilities (ls, sed, sort, xargs, even jq) ported into the builtins crate and run in-process, zero fork/exec. The same omp binary runs on macOS, Linux, and Windows — no WSL bridge.
### 10 · Code review with priorities and a verdict
@@ -427,9 +427,9 @@ Vuln lookups answer with vendor data, not blog summaries.
## Roughly **~80,000** lines of Rust, doing the work other harnesses shell out for.
Nine crates, one platform-tagged N-API addon. Search, shell, AST, highlight, PTY, desktop control, image decode, BPE counting — all in-process on the libuv pool. No fork/exec on the hot path. Another ~77k lines ride along vendored: the brush bash fork, a jq engine (jaq), and 46 uutils coreutils compiled straight into the shell.
Six crates, one platform-tagged N-API addon. Search, shell, AST, highlight, PTY, desktop control, image decode, BPE counting — all in-process on the libuv pool. No fork/exec on the hot path. Another ~80k lines ride along vendored: the brush bash fork, plus 58 command-line utilities — coreutils, findutils, sed, jq, ripgrep-backed grep, fd, diff, moreutils — ported into the builtins crate and compiled straight into the shell.
- Crates: `pi-natives`, `pi-shell`, `pi-ast`, `pi-iso`, `pi-voice`, `pi-walker`, `pi-uu-grep`, `pi-uu-diff`, `pi-uutils-ctx`
- Crates: `pi-natives`, `pi-shell`, `pi-ast`, `pi-iso`, `pi-voice`, `pi-walker`
- Platforms: `linux-x64`, `linux-arm64`, `darwin-x64`, `darwin-arm64`, `win32-x64` — x64 ships dual AVX2 and baseline binaries
Per crate, code lines only:
@@ -440,11 +440,8 @@ Per crate, code lines only:
| pi-natives | The N-API surface — every module in the table below | 25,000 |
| pi-walker | Parallel ignore-aware walker + scan cache shared by grep · glob · workspace · shell | 5,200 |
| pi-iso | Workspace isolation · apfs · btrfs · zfs · reflink · overlayfs · projfs · rcopy | 3,300 |
| pi-uu-grep | ripgrep-backed grep, run as an in-process shell builtin | 3,300 |
| pi-ast | tree-sitter + ast-grep matching, block resolution, structural summaries | 2,900 |
| pi-voice | Audio capture/playback · Opus · live WebRTC | 1,000 |
| pi-uu-diff | Structured diff builtin backed by similar | 500 |
| pi-uutils-ctx | Thread-local stdio/cwd/env so builtins run concurrently without a fork | 300 |
Inside `pi-natives`, the per-module breakdown (glue and tests omitted):
@@ -630,13 +627,8 @@ For architecture and contribution guidelines, see [packages/coding-agent/DEVELOP
| **[pi-iso](crates/pi-iso)** | Task isolation backend resolver: APFS clones, btrfs/zfs reflinks, overlayfs, projfs, rcopy |
| **[pi-voice](crates/pi-voice)** | Audio capture/playback, Opus codecs, and live WebRTC streaming primitives |
| **[pi-walker](crates/pi-walker)** | Parallel ignore-aware filesystem walker with the scan cache shared by grep, glob, and workspace |
| **[pi-uu-grep](crates/pi-uu-grep)** | ripgrep-library-backed grep executed as an in-process shell builtin |
| **[pi-uu-diff](crates/pi-uu-diff)** | Structured diff builtin backed by the similar crate |
| **[pi-uutils-ctx](crates/pi-uutils-ctx)** | Thread-local stdio/cwd/env context so in-process builtins run concurrently |
| **[brush-core](crates/vendor/brush-core)** | Vendored fork of [brush-shell](https://github.com/reubeno/brush) for embedded bash execution |
| **[brush-builtins](crates/vendor/brush-builtins)** | Vendored bash builtins (cd, echo, test, printf, read, export, etc.) |
| **[jaq](crates/vendor/jaq)** | Vendored jq-compatible JSON query engine, run as an in-process builtin |
| **uu-\* family** ([crates/vendor](crates/vendor)) | 46 vendored uutils coreutils (ls, sed, sort, xargs, …) executed in-process, no fork/exec |
| **[pi-builtins](crates/pi-builtins)** | Bash builtins (cd, echo, test, printf, read, export, …) plus 67 in-process command-line utilities |
## Contributing
crates/vendor/brush-builtins/BUILD.bazel → crates/pi-builtins/BUILD.bazel
+68 -7
View File
@@ -5,14 +5,15 @@ package(default_visibility = ["//visibility:public"])
exports_files(["Cargo.toml"])
# Vendored fork, promoted to a workspace member so Bazel treats it as
# first-party (crate_universe's path-dep rendering is machine-local and breaks
# lockfile portability). Feature list mirrors cargo's resolved default set.
# First-party: the brush builtins fork plus the in-process command-line
# utilities the shell ships. Feature list mirrors cargo's resolved default set.
rust_library(
name = "brush-builtins",
name = "pi-builtins",
srcs = glob(["src/**/*.rs"]),
aliases = aliases(),
compile_data = glob(["src/*.txt"]),
crate_features = [
"base",
"builtin.alias",
"builtin.bg",
"builtin.bind",
@@ -66,19 +67,79 @@ rust_library(
"builtin.unalias",
"builtin.unset",
"builtin.wait",
"util.b2sum",
"util.base32",
"util.base64",
"util.basename",
"util.cat",
"util.cksum",
"util.cmp",
"util.combine",
"util.comm",
"util.cut",
"util.date",
"util.diff",
"util.dirname",
"util.errno",
"util.fd",
"util.find",
"util.grep",
"util.head",
"util.hostname",
"util.ifne",
"util.isutf8",
"util.jq",
"util.ln",
"util.ls",
"util.md5sum",
"util.mkdir",
"util.mktemp",
"util.mv",
"util.nproc",
"util.paste",
"util.printenv",
"util.readlink",
"util.realpath",
"util.rm",
"util.sed",
"util.seq",
"util.sha1sum",
"util.sha224sum",
"util.sha256sum",
"util.sha384sum",
"util.sha512sum",
"util.sort",
"util.sponge",
"util.stat",
"util.tac",
"util.tail",
"util.tee",
"util.touch",
"util.tr",
"util.truncate",
"util.ts",
"util.uname",
"util.uniq",
"util.wc",
"util.which",
"util.whoami",
"util.xargs",
"util.yes",
"utils",
],
crate_name = "brush_builtins",
crate_name = "pi_builtins",
edition = "2024",
proc_macro_deps = all_crate_deps(proc_macro = True),
version = "0.2.0",
deps = all_crate_deps(normal = True) + [
"//crates/pi-walker",
"//crates/vendor/brush-core",
],
)
rust_test(
name = "brush-builtins_test",
crate = ":brush-builtins",
name = "pi-builtins_test",
crate = ":pi-builtins",
edition = "2024",
proc_macro_deps = all_crate_deps(proc_macro_dev = True),
deps = all_crate_deps(normal_dev = True),
crates/vendor/brush-builtins/Cargo.lock → crates/pi-builtins/Cargo.lock
Generated
View File
crates/pi-builtins/Cargo.toml
+451
View File
@@ -0,0 +1,451 @@
# Shell builtins for the embedded brush shell.
#
# Two layers live here:
# * the POSIX/bash builtins (`cd`, `echo`, `test`, `printf`, `read`, …), a
# locally-patched fork of the upstream `brush-builtins` crate, and
# * the in-process command-line utilities the shell ships (`cat`, `grep`,
# `sed`, `ls`, `jq`, `ps`, `top`, `kill`, …), one module per command.
#
# The utilities used to be ~50 separate vendored crates driven through a
# thread-local I/O shim; they now run as ordinary builtins against the explicit
# `host::Host` view of the shell (see `src/host.rs`). See LICENSE for the
# third-party notices that come with the ported code.
[package]
name = "pi-builtins"
version.workspace = true
edition.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
rust-version = "1.88.0"
description = "Builtins and in-process command-line utilities for the embedded brush shell"
build = false
autolib = false
autobins = false
autoexamples = false
autotests = false
autobenches = false
[lib]
name = "pi_builtins"
path = "src/lib.rs"
bench = false
[dependencies]
brush-core = "^0.5.0"
brush-parser = "^0.4.0"
cfg-if = "1.0.4"
chrono = "0.4.44"
clap = { version = "4.6.0", features = ["cargo", "color", "derive", "env", "wrap_help"] }
fancy-regex = "0.17"
futures = "0.3.32"
itertools = "0.14.0"
parking_lot = "0.12.5"
strum = "0.28.0"
strum_macros = "0.28.0"
thiserror = "2.0.18"
tracing = "0.1.44"
# ── Utility builtin dependencies ──────────────────────────────────────────────
ansi-width = "0.1.0"
argmax = "0.3.1"
bigdecimal = "0.4"
binary-heap-plus = "0.5.0"
bstr = "1.12.0"
bytecount = { version = "0.6.8", features = ["runtime-dispatch-simd"] }
codesnake = "0.2"
compare = "0.1.0"
faccess = "0.2.4"
filetime = "0.2.23"
flume = "0.11"
foldhash = "0.2.0"
fs_extra = "1.3.0"
glob = "0.3.1"
globset = "0.4"
grep-cli = "0.1"
grep-matcher = "0.1"
grep-pcre2 = "0.1"
grep-printer = "0.3"
grep-regex = "0.1"
grep-searcher = "0.1"
hex = "0.4.3"
hifijson = "0.2.0"
hostname = "0.4"
ignore = "0.4"
indicatif = "0.18.0"
jaq-core = "2.1.1"
jaq-json = "1.1.1"
jaq-std = "2.1.0"
jiff = { version = "0.2.18", features = ["tzdb-bundle-platform"] }
lscolors = { version = "0.21.0", default-features = false, features = ["gnu_legacy"] }
memchr = "2.7.4"
nom = "8.0.0"
num-bigint = "0.4"
num-traits = "0.2"
onig = { version = "6.4", default-features = false }
os_display = "0.1.4"
parse_datetime = "0.14"
pi-walker = { path = "../pi-walker" }
platform-info = "2.0.3"
rand = { version = "0.10.0", features = ["std_rng"] }
rayon = "1.10"
regex = "1.11"
rustc-hash = "2.1.1"
self_cell = "1.0.4"
serde_json = "1"
similar = "3.1.0"
tempfile = "3.15.0"
tokio-util = "0.7"
unicode-width = "0.2.0"
libc = "0.2.172"
memmap2 = "0.9"
uutils_term_grid = "0.8"
yansi = "1.0.1"
[dependencies.uucore]
version = "0.8.0"
default-features = false
features = [
"backup-control",
"checksum",
"colors",
"encoding",
"entries",
"extendedbigdecimal",
"fast-inc",
"format",
"fs",
"fsext",
"hardware",
"i18n-charmap",
"i18n-collator",
"i18n-datetime",
"i18n-decimal",
"libc",
"lines",
"mode",
"parser",
"pipes",
"process",
"quoting-style",
"ranges",
"ringbuffer",
"signals",
"sum",
"time",
"version-cmp",
"wide",
"update-control",
]
[target."cfg(any(unix, windows))".dependencies.tokio]
version = "1.50.0"
features = ["io-util", "macros", "process", "rt", "rt-multi-thread", "signal", "sync"]
[target.'cfg(target_family = "wasm")'.dependencies.tokio]
version = "1.52.1"
features = ["io-util", "macros", "rt"]
[target.'cfg(any(target_os = "linux", target_os = "android"))'.dependencies]
procfs = "0.18.0"
[target."cfg(unix)".dependencies]
dns-lookup = "3.0.0"
nix = { version = "0.31.2", features = ["fs", "poll", "process", "resource", "signal", "term", "user"] }
notify = { version = "=8.2.0", features = ["macos_kqueue"] }
rlimit = "0.11.0"
rustix = { version = "1.1.4", features = ["fs", "process", "time"] }
uucore = { version = "0.8.0", features = ["fsxattr", "safe-traversal"] }
[target."cfg(windows)".dependencies]
notify = "=8.2.0"
rustix = { version = "1.1.4", features = ["fs", "process", "time"] }
windows-sys = { version = "0.61", features = [
"Win32_Foundation",
"Win32_Networking_WinSock",
"Win32_Storage_FileSystem",
"Win32_System_IO",
"Win32_System_SystemInformation",
"Win32_System_WindowsProgramming",
] }
[dev-dependencies]
anyhow = "1.0.102"
pretty_assertions = { version = "1.4.1", features = ["unstable"] }
tempfile = "3"
[features]
default = ["base", "utils"]
# Every upstream POSIX/bash builtin.
base = [
"builtin.alias",
"builtin.bg",
"builtin.bind",
"builtin.break",
"builtin.builtin",
"builtin.caller",
"builtin.cd",
"builtin.colon",
"builtin.command",
"builtin.compgen",
"builtin.compopt",
"builtin.complete",
"builtin.continue",
"builtin.declare",
"builtin.dirs",
"builtin.dot",
"builtin.echo",
"builtin.enable",
"builtin.eval",
"builtin.exec",
"builtin.exit",
"builtin.export",
"builtin.false",
"builtin.fc",
"builtin.fg",
"builtin.getopts",
"builtin.hash",
"builtin.help",
"builtin.history",
"builtin.jobs",
"builtin.kill",
"builtin.let",
"builtin.mapfile",
"builtin.popd",
"builtin.printf",
"builtin.pushd",
"builtin.pwd",
"builtin.read",
"builtin.return",
"builtin.set",
"builtin.shift",
"builtin.shopt",
"builtin.suspend",
"builtin.test",
"builtin.times",
"builtin.trap",
"builtin.true",
"builtin.type",
"builtin.ulimit",
"builtin.umask",
"builtin.unalias",
"builtin.unset",
"builtin.wait",
]
"builtin.alias" = []
"builtin.bg" = []
"builtin.bind" = []
"builtin.break" = []
"builtin.builtin" = []
"builtin.caller" = []
"builtin.cd" = []
"builtin.colon" = []
"builtin.command" = []
"builtin.compgen" = []
"builtin.complete" = []
"builtin.compopt" = []
"builtin.continue" = []
"builtin.declare" = []
"builtin.dirs" = []
"builtin.dot" = []
"builtin.echo" = []
"builtin.enable" = []
"builtin.eval" = []
"builtin.exec" = []
"builtin.exit" = []
"builtin.export" = []
"builtin.false" = []
"builtin.fc" = []
"builtin.fg" = []
"builtin.getopts" = []
"builtin.hash" = []
"builtin.help" = []
"builtin.history" = []
"builtin.jobs" = []
# The non-unix existence probe scans the process table.
"builtin.kill" = ["util.procs"]
"builtin.let" = []
"builtin.mapfile" = []
"builtin.popd" = []
"builtin.printf" = []
"builtin.pushd" = []
"builtin.pwd" = []
"builtin.read" = []
"builtin.return" = []
"builtin.set" = []
"builtin.shift" = []
"builtin.shopt" = []
"builtin.suspend" = []
"builtin.test" = []
"builtin.times" = []
"builtin.trap" = []
"builtin.true" = []
"builtin.type" = []
"builtin.ulimit" = []
"builtin.umask" = []
"builtin.unalias" = []
"builtin.unset" = []
"builtin.wait" = []
# ── Utility builtins ──────────────────────────────────────────────────────────
# One feature per ported utility so a single utility can be built (and checked)
# in isolation. `utils` turns on the whole set and is part of `default`.
"util.b2sum" = ["util.cksum"]
"util.base32" = []
"util.base64" = ["util.base32"]
"util.basename" = []
"util.cat" = []
"util.cksum" = []
"util.cmp" = []
"util.comm" = []
"util.cut" = []
"util.date" = []
"util.diff" = []
"util.dirname" = []
"util.fd" = []
"util.find" = []
"util.grep" = []
"util.head" = []
"util.hostname" = []
"util.jq" = []
"util.ln" = []
"util.ls" = []
"util.md5sum" = ["util.cksum"]
"util.mkdir" = []
"util.mktemp" = []
"util.combine" = []
"util.errno" = []
"util.ifne" = []
"util.isutf8" = []
"util.sponge" = []
"util.ts" = []
"util.mv" = []
"util.nproc" = []
"util.paste" = []
"util.nohup" = ["util.procs"]
"util.pgrep" = ["util.proc-match"]
"util.pidwait" = ["util.proc-match"]
"util.pkill" = ["util.proc-match"]
# Shared process-matching engine behind pgrep/pkill/pidwait.
"util.proc-match" = ["builtin.kill", "util.procs"]
# Shared process-table snapshot behind ps/top/pgrep/pkill/pidwait/kill.
"util.procs" = []
"util.ps" = ["util.procs"]
"util.sleep" = []
"util.timeout" = ["util.procs"]
"util.top" = ["util.procs"]
"util.printenv" = []
"util.readlink" = []
"util.realpath" = []
# Shares the PCRE2 JIT probe with `grep`.
"util.rg" = ["util.grep"]
"util.rm" = []
"util.sed" = []
"util.seq" = []
"util.sha1sum" = ["util.cksum"]
"util.sha224sum" = ["util.cksum"]
"util.sha256sum" = ["util.cksum"]
"util.sha384sum" = ["util.cksum"]
"util.sha512sum" = ["util.cksum"]
"util.sort" = []
"util.stat" = []
"util.tac" = []
"util.tail" = ["util.tac"]
"util.tee" = []
"util.touch" = []
"util.tr" = []
"util.truncate" = []
"util.uname" = []
"util.uniq" = []
"util.wc" = []
"util.which" = []
"util.whoami" = []
"util.xargs" = []
"util.yes" = []
utils = [
"util.b2sum",
"util.base32",
"util.base64",
"util.basename",
"util.cat",
"util.cksum",
"util.cmp",
"util.comm",
"util.cut",
"util.date",
"util.diff",
"util.dirname",
"util.fd",
"util.find",
"util.grep",
"util.head",
"util.hostname",
"util.jq",
"util.ln",
"util.ls",
"util.md5sum",
"util.mkdir",
"util.mktemp",
"util.combine",
"util.errno",
"util.ifne",
"util.isutf8",
"util.sponge",
"util.ts",
"util.mv",
"util.nproc",
"util.paste",
"util.nohup",
"util.pgrep",
"util.pidwait",
"util.pkill",
"util.proc-match",
"util.procs",
"util.ps",
"util.sleep",
"util.timeout",
"util.top",
"util.printenv",
"util.readlink",
"util.realpath",
"util.rg",
"util.rm",
"util.sed",
"util.seq",
"util.sha1sum",
"util.sha224sum",
"util.sha256sum",
"util.sha384sum",
"util.sha512sum",
"util.sort",
"util.stat",
"util.tac",
"util.tail",
"util.tee",
"util.touch",
"util.tr",
"util.truncate",
"util.uname",
"util.uniq",
"util.wc",
"util.which",
"util.whoami",
"util.xargs",
"util.yes",
]
# Deliberately NOT `lints.workspace = true`: most of this crate is ported
# third-party code (brush's builtins, uutils, jaq), and holding 70k lines of
# it to the workspace's pedantic/nursery bar would bury real warnings. rustc
# warnings still apply and are kept at zero.
[lints.clippy]
all = { level = "allow", priority = -1 }
nursery = { level = "allow", priority = -1 }
pedantic = { level = "allow", priority = -1 }
cargo = { level = "allow", priority = -1 }
[lints.rust]
unfulfilled_lint_expectations = { level = "allow", priority = -1 }
crates/pi-builtins/LICENSE
+186
View File
@@ -0,0 +1,186 @@
This crate began as a locally-patched copy of brush-builtins from the brush shell
project (https://github.com/reubeno/brush), used under the MIT License, and still
tracks it for the POSIX/bash builtins.
Copyright (c) 2024 reuben olinsky
Local modifications to the upstream builtins include: implementing
previously-stubbed builtins, the `wait` builtin (PID targeting plus the
`-f`/`-n`/`-p` flags), and cancellation support for the `read` builtin via poll.
It additionally hosts the in-process command-line utility builtins (`src/host.rs`
plus one module per command). Those are ports of third-party programs, previously
carried as separate vendored crates and now consolidated here; the host plumbing
is ours, the utility logic is theirs. Their notices are reproduced in full
below.
================================================================================
uutils coreutils (https://github.com/uutils/coreutils), MIT
--------------------------------------------------------------------------------
Covers: base32, base64, basename, cat, cksum (shared checksum machinery),
b2sum, md5sum, sha1sum, sha224sum, sha256sum, sha384sum, sha512sum, comm, cut,
date, dirname, head, hostname, ln, ls, mkdir, mktemp, mv, nproc, paste,
printenv, readlink, realpath, rm, seq, sort, stat, tac, tail, tee, touch, tr,
truncate, uname, uniq, wc, whoami, yes.
Copyright (c) uutils developers
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================================================
uutils findutils (https://github.com/uutils/findutils), MIT
--------------------------------------------------------------------------------
Covers: find, xargs.
Copyright (c) Google Inc.
Copyright (c) uutils developers
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================================================
uutils sed (https://github.com/uutils/sed), MIT
--------------------------------------------------------------------------------
Covers: sed.
MIT License
Copyright (c) 2025 Diomidis Spinellis
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================================================
jaq (https://github.com/01mf02/jaq), MIT
--------------------------------------------------------------------------------
Covers: jq (the command-line front end; the interpreter itself remains an
external `jaq-core`/`jaq-std`/`jaq-json` dependency).
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
================================================================================
rust-utf8 (https://github.com/SimonSapin/rust-utf8), MIT
--------------------------------------------------------------------------------
Covers: the incremental UTF-8 decoder inside `src/wc.rs`.
Copyright (c) Simon Sapin and many others
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
================================================================================
Original work in this crate (`src/host.rs`, `src/proc_snapshot.rs`,
`src/proc_match.rs`, `cmp`, `diff`, `fd`, `grep`/`rg`, `which`, `ps`, `top`,
`pgrep`, `pkill`, `pidwait`, `sleep`, `timeout`, `nohup`, and the
moreutils-inspired `ts`, `sponge`, `ifne`, `isutf8`, `combine`, `errno`) is
MIT-licensed under the workspace license:
MIT License
Copyright (c) 2025 Mario Zechner
Copyright (c) 2025-2026 Can Bölük
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
crates/pi-builtins/README.md
+69
View File
@@ -0,0 +1,69 @@
# pi-builtins
Every builtin the embedded shell installs. Two layers:
1. **Shell builtins** — `cd`, `echo`, `test`, `printf`, `read`, `export`, `trap`,
`wait`, … A locally-patched fork of
[`brush-builtins`](https://github.com/reubeno/brush) (MIT), tracking upstream
except where noted in `LICENSE`.
2. **In-process command-line utilities** — `cat`, `grep`/`rg`, `sed`, `ls`,
`find`, `sort`, `jq`, `fd`, `diff`, `xargs`, `ps`, `top`, `kill`, the
moreutils set, and ~50 more. One module per command, in `src/<command>.rs`.
The second layer exists so the shell never has to fork: a long-lived embedded
shell resolves these names itself, on every platform, whether or not the host
has the real binaries. They were previously ~50 separate vendored crates driven
through a thread-local I/O shim; consolidating them here removed the shim.
## The `Host` contract
`src/host.rs` is the whole story for a utility builtin. A utility is a `clap`
argument model plus a synchronous body:
```rust
pub(crate) trait Utility: clap::Parser + Send + Sync + 'static {
const NAME: &'static str;
const USAGE_ERROR: u8 = 1;
fn run(self, host: &mut Host) -> i32;
}
```
`Host` is the shell as the utility sees it, threaded explicitly rather than
through process globals or thread locals: the command's own stdio (`host.stdout`,
`host.stderr`, `host.stdin`), the shell's working directory (`host.resolve(path)`
— mandatory for every path argument, since the host process's current directory
is unrelated), the exported environment (`host.var`, `host.env`), cancellation
(`host.is_cancelled`), a child-process launcher that inherits all of the above
(`host.child_env()`), and the accumulated exit status (`host.fail`,
`host.exit_code`).
`host::util::<U, SE>()` wraps a `Utility` into a registration that handles, once
for all of them: process-substitution arguments (`diff <(a) <(b)`),
`--help`/`--version` on stdout with status 0, usage errors on stderr, execution
on a blocking thread, the shell's cancellation token, and panic containment.
Utilities that are genuinely async — `sleep`, `timeout`, `ps`, `top`, `pgrep`,
`kill`, `nohup` — implement `brush_core::builtins::Command` directly instead.
## Registration
Three entry points, so the embedding shell decides what to install:
| Function | Contents |
| --- | --- |
| `default_builtins(set)` | The POSIX/bash builtins, per `BuiltinSet`. |
| `utility_builtins()` | The coreutils-style commands, which shadow system binaries. |
| `process_builtins()` | `ps`, `top`, `pgrep`, `pkill`, `pidwait`, `sleep`, `timeout`, `nohup`. |
Every command is behind a cargo feature (`builtin.<name>`, `util.<name>`), so a
single one can be built and tested in isolation:
```console
$ cargo test -p pi-builtins --no-default-features --features base,util.sed
```
## Licensing
MIT throughout, but the ported utilities carry upstream notices — uutils
coreutils, uutils findutils, uutils sed, jaq, and rust-utf8. See `LICENSE`, which
reproduces each in full and lists which commands it covers.
crates/vendor/brush-builtins/src/alias.rs → crates/pi-builtins/src/alias.rs
View File
crates/pi-builtins/src/b2sum.rs
+69
View File
@@ -0,0 +1,69 @@
//! `b2sum` builtin: compute and check BLAKE2b digests.
//!
//! Ported from uutils coreutils 0.8.0.
use brush_core::{ShellExtensions, builtins::Registration};
use clap::ArgMatches;
use uucore::checksum::{AlgoKind, BlakeLength, parse_blake_length};
use crate::{
cksum,
host::{Host, Utility, matches_parser, util},
};
/// Parsed `b2sum` invocation.
pub(crate) struct B2sum {
matches: ArgMatches,
}
matches_parser!(B2sum, app);
impl Utility for B2sum {
const NAME: &'static str = "b2sum";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
let length = self
.matches
.get_one::<String>("length")
.map(|value| parse_blake_length(AlgoKind::Blake2b, BlakeLength::String(value)))
.transpose();
let length = match length {
Ok(length) => length,
Err(error) => {
host.error(error, 1);
return 1;
},
};
cksum::run(host, AlgoKind::Blake2b, self.matches, length)
}
}
fn app() -> clap::Command {
cksum::command(B2sum::NAME, true)
}
/// Creates the `b2sum` builtin registration.
pub(crate) fn b2sum_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<B2sum, SE>()
}
#[cfg(test)]
mod tests {
use super::B2sum;
use crate::host::run_util;
#[test]
fn length_selects_the_blake2b_output_size() {
let (code, capture) = run_util::<B2sum>(&["-l", "8"], "abc", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "6b -\n");
}
#[test]
fn length_must_be_a_multiple_of_eight() {
let (code, capture) = run_util::<B2sum>(&["-l", "7"], "", "/");
assert_eq!(code, 1);
assert!(capture.err().contains("multiple of 8"), "{}", capture.err());
}
}
crates/vendor/uu-base32/src/base_common.rs → crates/pi-builtins/src/base32.rs
+213 -130
View File
@@ -1,30 +1,78 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
// spell-checker:ignore hexupper lsbf msbf unpadded nopad aGVsbG8sIHdvcmxkIQ
//! `base32` builtin: encode or decode data using the RFC 4648 base32 alphabet.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
fmt,
ffi::OsString,
fs::File,
io::{self, BufRead, BufReader, Write},
path::Path,
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::ArgMatches;
use crate::host::{Host, Utility, matches_parser, util};
const ABOUT: &str = "encode/decode data and print to standard output\nWith no FILE, or when FILE is -, read standard input.\n\nThe data are encoded as described for the base32 alphabet in RFC 4648.\nWhen decoding, the input may contain newlines in addition to the bytes of the formal base32 alphabet. Use --ignore-garbage to attempt to recover from any other non-alphabet bytes in the encoded stream.";
/// Parsed `base32` invocation.
pub(crate) struct Base32 {
matches: ArgMatches,
}
matches_parser!(Base32, app);
impl Utility for Base32 {
const NAME: &'static str = "base32";
fn run(self, host: &mut Host) -> i32 {
run_base(&self.matches, Format::Base32, host)
}
}
fn app() -> Command {
base_app(Base32::NAME, ABOUT, "base32 [OPTION]... [FILE]")
}
/// Creates the `base32` builtin registration.
pub(crate) fn base32_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Base32, SE>()
}
#[derive(Debug)]
struct BaseError(String);
impl BaseError {
fn new(message: impl Into<String>) -> Self {
Self(message.into())
}
}
impl fmt::Display for BaseError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str(&self.0)
}
}
impl From<io::Error> for BaseError {
fn from(error: io::Error) -> Self {
Self(error.to_string())
}
}
type BaseResult<T> = Result<T, BaseError>;
use clap::{Arg, ArgAction, Command};
use uucore::{
display::Quotable,
encoding::{
BASE2LSBF, BASE2MSBF, Base32Wrapper, Base58Wrapper, Base64SimdWrapper, EncodingWrapper,
Format, SupportsFastDecodeAndEncode, Z85Wrapper,
for_base_common::{BASE32, BASE32HEX, BASE64URL, HEXUPPER_PERMISSIVE},
Base32Wrapper, Base58Wrapper, Base64SimdWrapper, EncodingWrapper, Format,
SupportsFastDecodeAndEncode, Z85Wrapper, BASE2LSBF, BASE2MSBF,
},
error::{FromIo, UResult, USimpleError, UUsageError, strip_errno},
format_usage,
};
pub const BASE_CMD_PARSE_ERROR: i32 = 1;
const BASE_CMD_PARSE_ERROR: i32 = 1;
/// Encoded output will be formatted in lines of this length (the last line can
/// be shorter)
@@ -32,37 +80,34 @@ pub const BASE_CMD_PARSE_ERROR: i32 = 1;
/// Other implementations default to 76
///
/// This default is only used if no "-w"/"--wrap" argument is passed
pub const WRAP_DEFAULT: usize = 76;
const WRAP_DEFAULT: usize = 76;
// Fixed to 8 KiB (equivalent to `std::sys::io::DEFAULT_BUF_SIZE` on most
// targets)
pub const DEFAULT_BUF_SIZE: usize = 8 * 1024;
const DEFAULT_BUF_SIZE: usize = 8 * 1024;
pub struct Config {
pub decode: bool,
pub ignore_garbage: bool,
pub wrap_cols: Option<usize>,
pub to_read: Option<OsString>,
struct Config {
decode: bool,
ignore_garbage: bool,
wrap_cols: Option<usize>,
to_read: Option<OsString>,
}
pub mod options {
pub static DECODE: &str = "decode";
pub static WRAP: &str = "wrap";
pub static IGNORE_GARBAGE: &str = "ignore-garbage";
pub static FILE: &str = "file";
mod options {
pub(super) static DECODE: &str = "decode";
pub(super) static WRAP: &str = "wrap";
pub(super) static IGNORE_GARBAGE: &str = "ignore-garbage";
pub(super) static FILE: &str = "file";
}
impl Config {
pub fn from(options: &clap::ArgMatches) -> UResult<Self> {
fn from(options: &clap::ArgMatches) -> BaseResult<Self> {
let to_read = match options.get_many::<OsString>(options::FILE) {
Some(mut values) => {
let name = values.next().unwrap();
if let Some(extra_op) = values.next() {
return Err(UUsageError::new(
BASE_CMD_PARSE_ERROR,
format!("extra operand {}", extra_op.quote()),
));
return Err(BaseError::new(format!("extra operand {}", extra_op.quote())));
}
if name == "-" {
@@ -78,10 +123,7 @@ impl Config {
.get_one::<String>(options::WRAP)
.map(|num| {
num.parse::<usize>().map_err(|_| {
USimpleError::new(
BASE_CMD_PARSE_ERROR,
format!("invalid wrap size: {}", num.quote()),
)
BaseError::new(format!("invalid wrap size: {}", num.quote()))
})
})
.transpose()?;
@@ -95,16 +137,14 @@ impl Config {
}
}
pub fn base_app(about: String, usage: String) -> Command {
let cmd = Command::new("")
.version(uucore::crate_version!())
/// Builds the shared command-line model used by base32 and base64.
pub(crate) fn base_app(name: &'static str, about: &'static str, usage: &'static str) -> Command {
Command::new(name)
.version("0.8.0")
.about(about)
.override_usage(format_usage(&usage))
.infer_long_args(true);
uucore::clap_localization::configure_localized_command(cmd)
// Format arguments.
.override_usage(crate::host::format_usage(usage))
.infer_long_args(true)
.arg(
// pi-uutils: macOS base64 accepts `-D` as a non-visible decode alias.
Arg::new(options::DECODE)
.short('d')
.short_alias('D')
@@ -127,13 +167,10 @@ pub fn base_app(about: String, usage: String) -> Command {
.long(options::WRAP)
.value_name("COLS")
.help(format!(
"wrap encoded lines after COLS character (default {WRAP_DEFAULT}, 0 to disable \
wrapping)"
"wrap encoded lines after COLS character (default {WRAP_DEFAULT}, 0 to disable wrapping)"
))
.overrides_with(options::WRAP),
)
// "multiple" arguments are used to check whether there is more than one
// file passed in.
.arg(
Arg::new(options::FILE)
.index(1)
@@ -143,27 +180,49 @@ pub fn base_app(about: String, usage: String) -> Command {
)
}
pub fn get_input(config: &Config) -> UResult<Box<dyn BufRead>> {
match &config.to_read {
Some(name) => {
let file = File::open(pi_uutils_ctx::resolve(Path::new(name)))
.map_err_context(|| name.maybe_quote().to_string())?;
Ok(Box::new(BufReader::with_capacity(DEFAULT_BUF_SIZE, file)))
/// Runs the shared base-encoding implementation against the selected format.
pub(crate) fn run_base(matches: &ArgMatches, format: Format, host: &mut Host) -> i32 {
let config = match Config::from(matches) {
Ok(config) => config,
Err(error) => {
host.error(error, BASE_CMD_PARSE_ERROR);
return BASE_CMD_PARSE_ERROR;
},
None => {
// pi-uutils: stdin belongs to this invocation, never the host process.
Ok(Box::new(BufReader::with_capacity(DEFAULT_BUF_SIZE, pi_uutils_ctx::stdin())))
};
let result = if let Some(name) = config.to_read.clone() {
match File::open(host.resolve(&name)) {
Ok(file) => {
let mut input = BufReader::with_capacity(DEFAULT_BUF_SIZE, file);
handle_input(&mut input, &mut host.stdout, format, config)
},
Err(error) => Err(BaseError::new(format!("{}: {error}", name.maybe_quote()))),
}
} else {
let mut input = BufReader::with_capacity(DEFAULT_BUF_SIZE, &mut host.stdin);
handle_input(&mut input, &mut host.stdout, format, config)
};
match result {
Ok(()) => host.exit_code(),
Err(error) => {
host.error(error, 1);
1
},
}
}
pub fn handle_input<R: BufRead>(input: &mut R, format: Format, config: Config) -> UResult<()> {
fn handle_input<R: BufRead>(
input: &mut R,
output: &mut dyn Write,
format: Format,
config: Config,
) -> BaseResult<()> {
// Always allow padding for Base64 to avoid a full pre-scan of the input.
let supports_fast_decode_and_encode =
get_supports_fast_decode_and_encode(format, config.decode, true);
let supports_fast_decode_and_encode_ref = supports_fast_decode_and_encode.as_ref();
// pi-uutils: all output is scoped to this invocation.
let mut stdout_lock = pi_uutils_ctx::stdout().lock();
let result = match (format, config.decode) {
// Base58 must process the entire input as one big integer; keep the
// historical behavior of buffering everything for this format only.
@@ -171,18 +230,18 @@ pub fn handle_input<R: BufRead>(input: &mut R, format: Format, config: Config) -
let mut buffered = Vec::new();
input
.read_to_end(&mut buffered)
.map_err(|err| USimpleError::new(1, format_read_error(&err)))?;
.map_err(|err| BaseError::new(format_read_error(&err)))?;
if config.decode {
fast_decode::fast_decode_buffer(
buffered,
&mut stdout_lock,
output,
supports_fast_decode_and_encode_ref,
config.ignore_garbage,
)
} else {
fast_encode::fast_encode_buffer(
buffered,
&mut stdout_lock,
output,
supports_fast_decode_and_encode_ref,
config.wrap_cols,
)
@@ -191,13 +250,13 @@ pub fn handle_input<R: BufRead>(input: &mut R, format: Format, config: Config) -
// Streaming path for all other encodings keeps memory bounded.
(_, true) => fast_decode::fast_decode_stream(
input,
&mut stdout_lock,
output,
supports_fast_decode_and_encode_ref,
config.ignore_garbage,
),
(_, false) => fast_encode::fast_encode_stream(
input,
&mut stdout_lock,
output,
supports_fast_decode_and_encode_ref,
config.wrap_cols,
),
@@ -205,14 +264,14 @@ pub fn handle_input<R: BufRead>(input: &mut R, format: Format, config: Config) -
// Ensure any pending stdout buffer is flushed even if decoding failed; GNU
// basenc keeps already-decoded bytes visible before reporting the error.
match (result, stdout_lock.flush()) {
match (result, output.flush()) {
(res, Ok(())) => res,
(Ok(_), Err(err)) => Err(err.into()),
(Err(original), Err(_)) => Err(original),
}
}
pub fn get_supports_fast_decode_and_encode(
fn get_supports_fast_decode_and_encode(
format: Format,
decode: bool,
has_padding: bool,
@@ -232,35 +291,30 @@ pub fn get_supports_fast_decode_and_encode(
HEXUPPER_PERMISSIVE,
BASE16_VALID_DECODING_MULTIPLE,
BASE16_UNPADDED_MULTIPLE,
// spell-checker:disable-next-line
b"0123456789ABCDEFabcdef",
)),
Format::Base2Lsbf => Box::from(EncodingWrapper::new(
BASE2LSBF,
BASE2_VALID_DECODING_MULTIPLE,
BASE2_UNPADDED_MULTIPLE,
// spell-checker:disable-next-line
b"01",
)),
Format::Base2Msbf => Box::from(EncodingWrapper::new(
BASE2MSBF,
BASE2_VALID_DECODING_MULTIPLE,
BASE2_UNPADDED_MULTIPLE,
// spell-checker:disable-next-line
b"01",
)),
Format::Base32 => Box::from(Base32Wrapper::new(
BASE32,
BASE32_VALID_DECODING_MULTIPLE,
BASE32_UNPADDED_MULTIPLE,
// spell-checker:disable-next-line
b"ABCDEFGHIJKLMNOPQRSTUVWXYZ234567=",
)),
Format::Base32Hex => Box::from(Base32Wrapper::new(
BASE32HEX,
BASE32_VALID_DECODING_MULTIPLE,
BASE32_UNPADDED_MULTIPLE,
// spell-checker:disable-next-line
b"0123456789ABCDEFGHIJKLMNOPQRSTUV=",
)),
Format::Base64 => {
@@ -281,7 +335,6 @@ pub fn get_supports_fast_decode_and_encode(
BASE64URL,
BASE64_VALID_DECODING_MULTIPLE,
BASE64_UNPADDED_MULTIPLE,
// spell-checker:disable-next-line
b"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789=_-",
)),
Format::Z85 => Box::from(Z85Wrapper {}),
@@ -289,7 +342,7 @@ pub fn get_supports_fast_decode_and_encode(
}
}
pub mod fast_encode {
mod fast_encode {
use std::{
cmp::min,
collections::VecDeque,
@@ -297,12 +350,9 @@ pub mod fast_encode {
num::NonZeroUsize,
};
use uucore::{
encoding::SupportsFastDecodeAndEncode,
error::{UResult, USimpleError},
};
use uucore::encoding::SupportsFastDecodeAndEncode;
use crate::base_common::WRAP_DEFAULT;
use super::{BaseError, BaseResult, WRAP_DEFAULT};
struct LineWrapping {
line_length: NonZeroUsize,
@@ -314,8 +364,9 @@ pub mod fast_encode {
supports_fast_decode_and_encode: &dyn SupportsFastDecodeAndEncode,
read_buffer: &[u8],
encoded_buffer: &mut VecDeque<u8>,
) -> UResult<()> {
supports_fast_decode_and_encode.encode_to_vec_deque(read_buffer, encoded_buffer)?;
) -> BaseResult<()> {
supports_fast_decode_and_encode.encode_to_vec_deque(read_buffer, encoded_buffer)
.map_err(|err| BaseError::new(err.to_string()))?;
Ok(())
}
@@ -402,12 +453,12 @@ pub mod fast_encode {
}
// End of helper functions
pub fn fast_encode_buffer(
pub(super) fn fast_encode_buffer(
input: Vec<u8>,
output: &mut dyn Write,
supports_fast_decode_and_encode: &dyn SupportsFastDecodeAndEncode,
wrap: Option<usize>,
) -> UResult<()> {
) -> BaseResult<()> {
// Based on performance testing
const ENCODE_IN_CHUNKS_OF_SIZE_MULTIPLE: usize = 1_024;
@@ -488,7 +539,8 @@ pub mod fast_encode {
{
// Encode all remaining unencoded bytes, placing them in `encoded_buffer`
supports_fast_decode_and_encode
.encode_to_vec_deque(leftover_buffer.make_contiguous(), &mut encoded_buffer)?;
.encode_to_vec_deque(leftover_buffer.make_contiguous(), &mut encoded_buffer)
.map_err(|err| BaseError::new(err.to_string()))?;
// Write all data in `encoded_buffer` to output
// `is_cleanup` triggers special cleanup-only logic
@@ -508,13 +560,13 @@ pub mod fast_encode {
/// - `None`: the default wrap width is applied.
///
/// Remaining bytes are encoded and flushed at the end. I/O or encoding
/// failures are propagated via `UResult`.
pub fn fast_encode_stream(
/// failures are propagated through the shared result type.
pub(super) fn fast_encode_stream(
input: &mut dyn BufRead,
output: &mut dyn Write,
supports_fast_decode_and_encode: &dyn SupportsFastDecodeAndEncode,
wrap: Option<usize>,
) -> UResult<()> {
) -> BaseResult<()> {
const ENCODE_IN_CHUNKS_OF_SIZE_MULTIPLE: usize = 1_024;
let encode_in_chunks_of_size =
@@ -541,7 +593,7 @@ pub mod fast_encode {
loop {
let read_buffer = input
.fill_buf()
.map_err(|err| USimpleError::new(1, super::format_read_error(&err)))?;
.map_err(|err| BaseError::new(super::format_read_error(&err)))?;
if read_buffer.is_empty() {
break;
}
@@ -606,7 +658,8 @@ pub mod fast_encode {
}
// Encode any remaining bytes and flush
supports_fast_decode_and_encode.encode_to_vec_deque(&leftover_buffer, &mut encoded_buffer)?;
supports_fast_decode_and_encode.encode_to_vec_deque(&leftover_buffer, &mut encoded_buffer)
.map_err(|err| BaseError::new(err.to_string()))?;
write_to_output(&mut line_wrapping, &mut encoded_buffer, output, true, wrap == Some(0))?;
@@ -614,13 +667,12 @@ pub mod fast_encode {
}
}
pub mod fast_decode {
mod fast_decode {
use std::io::{self, BufRead, Write};
use uucore::{
encoding::SupportsFastDecodeAndEncode,
error::{UResult, USimpleError},
};
use uucore::encoding::SupportsFastDecodeAndEncode;
use super::{BaseError, BaseResult};
// Start of helper functions
fn alphabet_lookup(alphabet: &[u8]) -> [bool; 256] {
@@ -639,8 +691,9 @@ pub mod fast_decode {
supports_fast_decode_and_encode: &dyn SupportsFastDecodeAndEncode,
read_buffer_filtered: &[u8],
decoded_buffer: &mut Vec<u8>,
) -> UResult<()> {
supports_fast_decode_and_encode.decode_into_vec(read_buffer_filtered, decoded_buffer)?;
) -> BaseResult<()> {
supports_fast_decode_and_encode.decode_into_vec(read_buffer_filtered, decoded_buffer)
.map_err(|err| BaseError::new(err.to_string()))?;
Ok(())
}
@@ -660,7 +713,7 @@ pub mod fast_decode {
supports_fast_decode_and_encode: &dyn SupportsFastDecodeAndEncode,
decoded_buffer: &mut Vec<u8>,
output: &mut dyn Write,
) -> UResult<()> {
) -> BaseResult<()> {
// While at least one full decode block is buffered, keep draining
// it and never yield more than block_limit per chunk.
while buffer.len() >= valid_multiple {
@@ -686,12 +739,12 @@ pub mod fast_decode {
}
// End of helper functions
pub fn fast_decode_buffer(
pub(super) fn fast_decode_buffer(
input: Vec<u8>,
output: &mut dyn Write,
supports_fast_decode_and_encode: &dyn SupportsFastDecodeAndEncode,
ignore_garbage: bool,
) -> UResult<()> {
) -> BaseResult<()> {
const DECODE_IN_CHUNKS_OF_SIZE_MULTIPLE: usize = 1_024;
let alphabet = supports_fast_decode_and_encode.alphabet();
@@ -723,7 +776,7 @@ pub mod fast_decode {
} else if ignore_garbage {
continue;
} else {
return Err(USimpleError::new(1, "error: invalid input"));
return Err(BaseError::new("error: invalid input"));
}
if supports_partial_decode {
@@ -768,23 +821,24 @@ pub mod fast_decode {
let final_chunk = owned_chunk.as_deref().unwrap_or(&buffer);
supports_fast_decode_and_encode.decode_into_vec(final_chunk, &mut decoded_buffer)?;
supports_fast_decode_and_encode.decode_into_vec(final_chunk, &mut decoded_buffer)
.map_err(|err| BaseError::new(err.to_string()))?;
write_to_output(&mut decoded_buffer, output)?;
if had_invalid_tail {
return Err(USimpleError::new(1, "error: invalid input"));
return Err(BaseError::new("error: invalid input"));
}
}
Ok(())
}
pub fn fast_decode_stream(
pub(super) fn fast_decode_stream(
input: &mut dyn BufRead,
output: &mut dyn Write,
supports_fast_decode_and_encode: &dyn SupportsFastDecodeAndEncode,
ignore_garbage: bool,
) -> UResult<()> {
) -> BaseResult<()> {
const DECODE_IN_CHUNKS_OF_SIZE_MULTIPLE: usize = 1_024;
let alphabet = supports_fast_decode_and_encode.alphabet();
@@ -803,7 +857,7 @@ pub mod fast_decode {
loop {
let read_buffer = input
.fill_buf()
.map_err(|err| USimpleError::new(1, super::format_read_error(&err)))?;
.map_err(|err| BaseError::new(super::format_read_error(&err)))?;
let read_len = read_buffer.len();
if read_len == 0 {
break;
@@ -839,7 +893,7 @@ pub mod fast_decode {
buffer.drain(..decode_in_chunks_of_size);
}
}
return Err(USimpleError::new(1, "error: invalid input"));
return Err(BaseError::new("error: invalid input"));
}
if supports_partial_decode {
@@ -887,11 +941,12 @@ pub mod fast_decode {
let final_chunk = owned_chunk.as_deref().unwrap_or(&buffer);
supports_fast_decode_and_encode.decode_into_vec(final_chunk, &mut decoded_buffer)?;
supports_fast_decode_and_encode.decode_into_vec(final_chunk, &mut decoded_buffer)
.map_err(|err| BaseError::new(err.to_string()))?;
write_to_output(&mut decoded_buffer, output)?;
if had_invalid_tail {
return Err(USimpleError::new(1, "error: invalid input"));
return Err(BaseError::new("error: invalid input"));
}
}
@@ -900,40 +955,72 @@ pub mod fast_decode {
}
fn format_read_error(error: &io::Error) -> String {
format!("read error: {}", strip_errno(error))
format!("read error: {}", error)
}
/// Determines if the input buffer contains any padding ('=') ignoring trailing
/// whitespace.
#[cfg(test)]
fn read_and_has_padding<R: io::Read>(input: &mut R) -> UResult<(bool, Vec<u8>)> {
let mut buf = Vec::new();
fn read_and_has_padding<R: io::Read>(input: &mut R) -> BaseResult<(bool, Vec<u8>)> {
let mut buffer = Vec::new();
input
.read_to_end(&mut buf)
.map_err(|err| USimpleError::new(1, format_read_error(&err)))?;
// Treat the stream as padded if any '=' exists (GNU coreutils continues
// decoding even when padding bytes are followed by more data).
let has_padding = buf.contains(&b'=');
Ok((has_padding, buf))
.read_to_end(&mut buffer)
.map_err(|error| BaseError::new(format_read_error(&error)))?;
let has_padding = buffer.contains(&b'=');
Ok((has_padding, buffer))
}
#[cfg(test)]
mod tests {
use std::io::Cursor;
use std::fs;
use crate::base_common::read_and_has_padding;
use super::{Base32, read_and_has_padding};
use crate::host::run_util;
#[test]
fn test_has_padding() {
let test_cases = vec![
fn encodes_stdin() {
let (code, capture) = run_util::<Base32>(&[], "hello", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "NBSWY3DP\n");
}
#[test]
fn decodes_stdin() {
let (code, capture) = run_util::<Base32>(&["--decode"], "NBSWY3DP\n", "/");
assert_eq!(code, 0);
assert_eq!(capture.stdout(), b"hello");
}
#[test]
fn wraps_encoded_output_at_requested_width() {
let (code, capture) = run_util::<Base32>(&["--wrap", "4"], "hello", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "NBSW\nY3DP\n");
}
#[test]
fn resolves_file_operand_against_shell_cwd() {
let cwd = tempfile::tempdir().unwrap();
fs::write(cwd.path().join("input"), b"hello").unwrap();
let (code, capture) = run_util::<Base32>(&["input"], "", cwd.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "NBSWY3DP\n");
}
#[test]
fn rejects_invalid_input() {
let (code, capture) = run_util::<Base32>(&["--decode"], "!", "/");
assert_eq!(code, 1);
assert_eq!(capture.err(), "base32: error: invalid input\n");
}
#[test]
fn detects_padding_anywhere_in_input() {
let test_cases = [
("aGVsbG8sIHdvcmxkIQ==", true),
("aGVsbG8sIHdvcmxkIQ== ", true),
("aGVsbG8sIHdvcmxkIQ==\n", true),
("aGVsbG8sIHdvcmxkIQ== \n", true),
("aGVsbG8sIHdvcmxkIQ=", true),
("aGVsbG8sIHdvcmxkIQ= ", true),
("MTIzNA==MTIzNA", true),
("MTIzNA==\nMTIzNA", true),
("aGVsbG8sIHdvcmxkIQ \n", false),
@@ -941,12 +1028,8 @@ mod tests {
];
for (input, expected) in test_cases {
let mut cursor = Cursor::new(input.as_bytes());
assert_eq!(
read_and_has_padding(&mut cursor).unwrap().0,
expected,
"Failed for input: '{input}'"
);
let (has_padding, _) = read_and_has_padding(&mut input.as_bytes()).unwrap();
assert_eq!(has_padding, expected, "failed for input: '{input}'");
}
}
}
crates/pi-builtins/src/base64.rs
+106
View File
@@ -0,0 +1,106 @@
//! `base64` builtin: encode or decode data using the Base64 alphabet.
//!
//! Ported from uutils coreutils 0.8.0.
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{ArgMatches, Command};
use uucore::encoding::Format;
use crate::{
base32::{base_app, run_base},
host::{Host, Utility, matches_parser, util},
};
const ABOUT: &str = "encode/decode data and print to standard output\nWith no FILE, or when FILE is -, read standard input.\n\nThe data are encoded as described for the base64 alphabet in RFC 3548.\nWhen decoding, the input may contain newlines in addition to the bytes of the formal base64 alphabet. Use --ignore-garbage to attempt to recover from any other non-alphabet bytes in the encoded stream.";
/// Parsed `base64` invocation.
pub(crate) struct Base64 {
matches: ArgMatches,
}
matches_parser!(Base64, app);
impl Utility for Base64 {
const NAME: &'static str = "base64";
fn run(self, host: &mut Host) -> i32 {
run_base(&self.matches, Format::Base64, host)
}
}
/// The `base64` argument model.
fn app() -> Command {
base_app(Base64::NAME, ABOUT, "base64 [OPTION]... [FILE]")
}
/// Creates the `base64` builtin registration.
pub(crate) fn base64_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Base64, SE>()
}
#[cfg(test)]
mod tests {
use super::Base64;
use crate::host::run_util;
#[test]
fn macos_decode_alias_round_trips_and_gnu_alias_still_works() {
let (code, encoded) = run_util::<Base64>(&[], "hello", "/");
assert_eq!(code, 0);
assert_eq!(encoded.out(), "aGVsbG8=\n");
assert_eq!(encoded.err(), "");
let (code, decoded) = run_util::<Base64>(&["-D"], &encoded.out(), "/");
assert_eq!(code, 0);
assert_eq!(decoded.out(), "hello");
assert_eq!(decoded.err(), "");
let (code, decoded) = run_util::<Base64>(&["-d"], &encoded.out(), "/");
assert_eq!(code, 0);
assert_eq!(decoded.out(), "hello");
assert_eq!(decoded.err(), "");
}
#[test]
fn wrap_controls_encoded_line_width() {
let (code, capture) = run_util::<Base64>(&["-w", "4"], "hello", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "aGVs\nbG8=\n");
assert_eq!(capture.err(), "");
}
#[test]
fn ignore_garbage_recovers_non_alphabet_bytes() {
let (code, capture) = run_util::<Base64>(&["-d", "-i"], "aG$Vs$bG8=\n", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "hello");
assert_eq!(capture.err(), "");
}
#[test]
fn dash_operand_reads_standard_input() {
let (code, capture) = run_util::<Base64>(&["-"], "hello", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "aGVsbG8=\n");
assert_eq!(capture.err(), "");
}
#[test]
fn invalid_encoded_input_exits_one() {
let (code, capture) = run_util::<Base64>(&["-d"], "!", "/");
assert_eq!(code, 1);
assert_eq!(capture.out(), "");
assert_eq!(capture.err(), "base64: error: invalid input\n");
}
#[test]
fn file_operand_resolves_against_shell_working_directory() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("input"), b"hello").unwrap();
let (code, capture) = run_util::<Base64>(&["input"], "ignored", dir.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "aGVsbG8=\n");
assert_eq!(capture.err(), "");
}
}
crates/pi-builtins/src/basename.rs
+208
View File
@@ -0,0 +1,208 @@
//! `basename` builtin: print a path with leading directory components (and
//! optionally a trailing suffix) removed.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{ffi::OsString, io::Write, path::PathBuf};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command, builder::ValueParser};
use uucore::{display::Quotable, line_ending::LineEnding};
use crate::host::{Host, Utility, format_usage, matches_parser, os_bytes, util};
mod options {
pub static MULTIPLE: &str = "multiple";
pub static NAME: &str = "name";
pub static SUFFIX: &str = "suffix";
pub static ZERO: &str = "zero";
}
/// Parsed `basename` invocation.
pub(crate) struct Basename {
matches: ArgMatches,
}
matches_parser!(Basename, app);
impl Utility for Basename {
const NAME: &'static str = "basename";
fn run(self, host: &mut Host) -> i32 {
let line_ending = LineEnding::from_zero_flag(self.matches.get_flag(options::ZERO));
let mut names = self
.matches
.get_many::<OsString>(options::NAME)
.unwrap_or_default()
.collect::<Vec<_>>();
if names.is_empty() {
host.error("missing operand", 1);
return 1;
}
// A suffix operand (or -a) switches from the two-operand "simple format"
// (`basename PATH SUFFIX`) to "strip this suffix from every operand".
let explicit_suffix = self.matches.get_one::<OsString>(options::SUFFIX);
let suffix = if explicit_suffix.is_some() || self.matches.get_flag(options::MULTIPLE) {
explicit_suffix.cloned().unwrap_or_default()
} else {
match names.len() {
1 => OsString::default(),
2 => names.pop().cloned().unwrap_or_default(),
_ => {
host.error(format!("extra operand {}", names[2].quote()), 1);
return 1;
},
}
};
for name in names {
let Some(stripped) = basename(name, &suffix) else {
host.error(format!("invalid argument {}", name.quote()), 1);
return 1;
};
if host.stdout.write_all(&stripped).is_err() || write!(host.stdout, "{line_ending}").is_err()
{
return 1;
}
}
if host.stdout.flush().is_err() {
return 1;
}
host.exit_code()
}
}
/// The final path component of `fullname`, minus `suffix`.
///
/// Returns bytes rather than an `OsString` because the result is written
/// straight out, and `None` when the operand has no byte representation on this
/// platform.
fn basename(fullname: &OsString, suffix: &OsString) -> Option<Vec<u8>> {
let fullname_bytes = os_bytes(fullname)?;
// `a/.` names the directory `a`, whose basename is `.`.
if fullname_bytes.ends_with(b"/.") {
return Some(b".".into());
}
let path = PathBuf::from(fullname);
let Some(last) = path.components().next_back() else {
return Some(Vec::new());
};
let name = last.as_os_str();
let name_bytes = os_bytes(name)?;
// Stripping the whole component would leave nothing, so GNU keeps it.
if name == suffix.as_os_str() {
return Some(name_bytes.into());
}
let suffix_bytes = os_bytes(suffix)?;
Some(name_bytes.strip_suffix(suffix_bytes).unwrap_or(name_bytes).into())
}
/// The `basename` argument model.
fn app() -> Command {
Command::new(Basename::NAME)
.version("0.8.0")
.about(
"Print NAME with any leading directory components removed\nIf specified, also remove a \
trailing SUFFIX",
)
.override_usage(format_usage("basename [-z] NAME [SUFFIX]\n basename OPTION... NAME..."))
.infer_long_args(true)
.arg(
Arg::new(options::MULTIPLE)
.short('a')
.long(options::MULTIPLE)
.help("support multiple arguments and treat each as a NAME")
.action(ArgAction::SetTrue)
.overrides_with(options::MULTIPLE),
)
.arg(
Arg::new(options::NAME)
.action(ArgAction::Append)
.value_parser(ValueParser::os_string())
.value_hint(clap::ValueHint::AnyPath)
.hide(true)
.trailing_var_arg(true),
)
.arg(
Arg::new(options::SUFFIX)
.short('s')
.long(options::SUFFIX)
.value_name("SUFFIX")
.value_parser(ValueParser::os_string())
.help("remove a trailing SUFFIX; implies -a")
.overrides_with(options::SUFFIX),
)
.arg(
Arg::new(options::ZERO)
.short('z')
.long(options::ZERO)
.help("end each output line with NUL, not newline")
.action(ArgAction::SetTrue)
.overrides_with(options::ZERO),
)
}
/// Creates the `basename` builtin registration.
pub(crate) fn basename_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Basename, SE>()
}
#[cfg(test)]
mod tests {
use super::Basename;
use crate::host::run_util;
fn basename(argv: &[&str]) -> (i32, String) {
let (code, capture) = run_util::<Basename>(argv, "", "/");
(code, capture.out())
}
#[test]
fn strips_directory_components() {
assert_eq!(basename(&["/usr/local/bin/omp"]), (0, "omp\n".to_string()));
}
#[test]
fn simple_format_takes_the_second_operand_as_a_suffix() {
assert_eq!(basename(&["src/host.rs", ".rs"]), (0, "host\n".to_string()));
}
#[test]
fn suffix_option_applies_to_every_operand() {
assert_eq!(basename(&["-s", ".rs", "a.rs", "b.rs"]), (0, "a\nb\n".to_string()));
}
#[test]
fn suffix_is_kept_when_it_is_the_whole_component() {
assert_eq!(basename(&["-s", ".rs", ".rs"]), (0, ".rs\n".to_string()));
}
#[test]
fn trailing_dot_component_names_the_parent() {
assert_eq!(basename(&["a/."]), (0, ".\n".to_string()));
}
#[test]
fn zero_flag_switches_the_terminator() {
assert_eq!(basename(&["-z", "/a/b"]), (0, "b\0".to_string()));
}
#[test]
fn missing_operand_is_a_usage_error() {
let (code, capture) = run_util::<Basename>(&[], "", "/");
assert_eq!(code, 1);
assert_eq!(capture.err(), "basename: missing operand\n");
}
#[test]
fn third_operand_without_multiple_is_a_usage_error() {
let (code, capture) = run_util::<Basename>(&["a", "b", "c"], "", "/");
assert_eq!(code, 1);
assert_eq!(capture.err(), "basename: extra operand 'c'\n");
}
}
crates/vendor/brush-builtins/src/bg.rs → crates/pi-builtins/src/bg.rs
View File
crates/vendor/brush-builtins/src/bind.rs → crates/pi-builtins/src/bind.rs
View File
crates/vendor/brush-builtins/src/break_.rs → crates/pi-builtins/src/break_.rs
View File
crates/vendor/brush-builtins/src/builder.rs → crates/pi-builtins/src/builder.rs
View File
crates/vendor/brush-builtins/src/builtin_.rs → crates/pi-builtins/src/builtin_.rs
View File
crates/vendor/brush-builtins/src/caller.rs → crates/pi-builtins/src/caller.rs
View File
crates/pi-builtins/src/cat.rs
+814
View File
@@ -0,0 +1,814 @@
//! `cat` builtin: concatenate files to standard output.
//!
//! Ported from uutils coreutils 0.8.0.
#[cfg(unix)]
use std::os::unix::fs::FileTypeExt;
use std::{
ffi::OsString,
fs::{File, metadata},
io::{self, BufWriter, ErrorKind, Read, Write},
path::Path,
};
use brush_core::{ShellExtensions, builtins::Registration, openfiles::OpenFile};
use clap::{Arg, ArgAction, ArgMatches, Command};
use memchr::memchr2;
use thiserror::Error;
use uucore::{display::Quotable, fast_inc::fast_inc_one};
use crate::host::{Host, Stdin, Utility, format_usage, matches_parser, util};
/// Linux splice support.
#[cfg(any(target_os = "linux", target_os = "android"))]
mod splice {
use std::io::{self, ErrorKind};
use std::os::fd::{AsFd, BorrowedFd};
use rustix::io::{read, write};
use uucore::pipes::{MAX_ROOTLESS_PIPE_SIZE, pipe, splice, splice_exact};
use super::{CatError, CatResult, FdReadable, InputHandle};
const BUF_SIZE: usize = 1024 * 16;
/// Moves input between real descriptors without copying through userspace.
///
/// `false` means the input reached EOF; `true` asks the caller to resume with
/// buffered copying because splice was unavailable for this descriptor pair.
#[inline]
pub(super) fn write_fast_using_splice<R: FdReadable>(
handle: &InputHandle<R>,
write_fd: BorrowedFd<'_>,
) -> CatResult<bool> {
let Some(read_fd) = handle.reader.try_borrow_as_fd() else {
return Ok(true);
};
if splice(&read_fd, &write_fd, MAX_ROOTLESS_PIPE_SIZE).is_ok() {
// fcntl improves throughput. It is harmless when stdout is not a pipe.
let _ = rustix::pipe::fcntl_setpipe_size(write_fd, MAX_ROOTLESS_PIPE_SIZE);
loop {
match splice(&read_fd, &write_fd, MAX_ROOTLESS_PIPE_SIZE) {
Ok(1..) => {},
Ok(0) => return Ok(false),
Err(error) if error.kind() == ErrorKind::BrokenPipe => {
return Err(CatError::BrokenPipe);
},
Err(_) => return Ok(true),
}
}
} else if let Ok((pipe_rd, pipe_wr)) = pipe() {
// Neither endpoint is a pipe, so broker through an intermediate pipe.
loop {
match splice(&read_fd, &pipe_wr, MAX_ROOTLESS_PIPE_SIZE) {
Ok(0) => return Ok(false),
Ok(n) => {
if let Err(error) = splice_exact(&pipe_rd, &write_fd, n) {
if error.kind() == ErrorKind::BrokenPipe {
return Err(CatError::BrokenPipe);
}
// Preserve bytes already moved into the broker pipe, then
// let the caller continue with buffered copying.
copy_exact(&pipe_rd, &write_fd, n)?;
return Ok(true);
}
},
Err(_) => return Ok(true),
}
}
} else {
Ok(true)
}
}
/// Moves exactly `num_bytes` bytes between the two descriptors.
fn copy_exact(
read_fd: &impl AsFd,
write_fd: &impl AsFd,
num_bytes: usize,
) -> io::Result<()> {
let mut left = num_bytes;
let mut buf = [0; BUF_SIZE];
while left > 0 {
let n = read(read_fd, &mut buf)?;
assert_ne!(n, 0, "unexpected end of pipe");
let mut written = 0;
while written < n {
match write(write_fd, &buf[written..n])? {
0 => unreachable!("fd should be writable"),
w => written += w,
}
}
left -= n;
}
Ok(())
}
}
// Allocate 32 digits for the line number. An estimate is that we can print
// about 1e8 lines/second, so 32 digits lasts for billions of universe lifetimes.
const LINE_NUMBER_BUF_SIZE: usize = 32;
struct LineNumber {
buf: [u8; LINE_NUMBER_BUF_SIZE],
print_start: usize,
num_start: usize,
num_end: usize,
}
// Manually incrementing the digits is significantly faster than formatting a
// `usize` each time. The buffer starts as " 1\t".
impl LineNumber {
fn new() -> Self {
let mut buf = [b'0'; LINE_NUMBER_BUF_SIZE];
let init_str = " 1\t";
let print_start = buf.len() - init_str.len();
let num_start = buf.len() - 2;
let num_end = buf.len() - 1;
buf[print_start..].copy_from_slice(init_str.as_bytes());
Self { buf, print_start, num_start, num_end }
}
fn increment(&mut self) {
fast_inc_one(&mut self.buf, &mut self.num_start, self.num_end);
self.print_start = self.print_start.min(self.num_start);
}
#[inline]
fn to_str(&self) -> &[u8] {
&self.buf[self.print_start..]
}
fn write(&self, writer: &mut impl Write) -> io::Result<()> {
writer.write_all(self.to_str())
}
}
#[derive(Error, Debug)]
enum CatError {
/// Wrapper around `io::Error`.
#[error("{}", strip_errno(.0))]
Io(io::Error),
/// The downstream reader closed its pipe; this ends the copy quietly.
#[error("broken pipe")]
BrokenPipe,
/// Unknown file type; it is not a regular file, socket, or known device.
#[error("unknown filetype: {ft_debug}")]
UnknownFiletype { ft_debug: String },
#[error("Is a directory")]
IsDirectory,
#[cfg(unix)]
#[error("No such device or address")]
NoSuchDeviceOrAddress,
#[error("Too many levels of symbolic links")]
TooManySymlinks,
}
impl From<io::Error> for CatError {
fn from(error: io::Error) -> Self {
if error.kind() == ErrorKind::BrokenPipe {
Self::BrokenPipe
} else {
Self::Io(error)
}
}
}
fn strip_errno(error: &io::Error) -> String {
let mut message = error.to_string();
if let Some(position) = message.find(" (os error ") {
message.truncate(position);
}
message
}
type CatResult<T> = Result<T, CatError>;
#[derive(PartialEq)]
enum NumberingMode {
None,
NonEmpty,
All,
}
struct OutputOptions {
/// Line numbering mode.
number: NumberingMode,
/// Suppress repeated empty output lines.
squeeze_blank: bool,
/// Display TAB characters as `tab`.
show_tabs: bool,
/// Show end of lines.
show_ends: bool,
/// Use ^ and M- notation, except for LF and TAB.
show_nonprint: bool,
}
impl OutputOptions {
fn tab(&self) -> &'static str {
if self.show_tabs { "^I" } else { "\t" }
}
fn end_of_line(&self) -> &'static str {
if self.show_ends { "$\n" } else { "\n" }
}
/// We can write fast when no output augmentation is requested.
fn can_write_fast(&self) -> bool {
!(self.show_tabs
|| self.show_nonprint
|| self.show_ends
|| self.squeeze_blank
|| self.number != NumberingMode::None)
}
}
/// State that persists between files on the augmented-output path.
struct OutputState {
line_number: LineNumber,
at_line_start: bool,
skipped_carriage_return: bool,
one_blank_kept: bool,
}
trait FdReadable: Read {
#[cfg(any(target_os = "linux", target_os = "android"))]
fn try_borrow_as_fd(&self) -> Option<std::os::fd::BorrowedFd<'_>> {
None
}
}
impl FdReadable for File {
#[cfg(any(target_os = "linux", target_os = "android"))]
fn try_borrow_as_fd(&self) -> Option<std::os::fd::BorrowedFd<'_>> {
use std::os::fd::AsFd;
Some(self.as_fd())
}
}
impl FdReadable for &mut Stdin {}
/// An input stream and whether it is connected to an interactive terminal.
struct InputHandle<R: FdReadable> {
reader: R,
is_interactive: bool,
}
/// Concrete enum of recognized file types.
enum InputType {
Directory,
File,
StdIn,
SymLink,
#[cfg(unix)]
BlockDevice,
#[cfg(unix)]
CharacterDevice,
#[cfg(unix)]
Fifo,
#[cfg(unix)]
Socket,
}
mod options {
pub static FILE: &str = "file";
pub static SHOW_ALL: &str = "show-all";
pub static NUMBER_NONBLANK: &str = "number-nonblank";
pub static SHOW_NONPRINTING_ENDS: &str = "e";
pub static SHOW_ENDS: &str = "show-ends";
pub static NUMBER: &str = "number";
pub static SQUEEZE_BLANK: &str = "squeeze-blank";
pub static SHOW_NONPRINTING_TABS: &str = "t";
pub static SHOW_TABS: &str = "show-tabs";
pub static SHOW_NONPRINTING: &str = "show-nonprinting";
pub static IGNORED_U: &str = "ignored-u";
}
/// Parsed `cat` invocation.
pub(crate) struct Cat {
matches: ArgMatches,
}
matches_parser!(Cat, app);
impl Utility for Cat {
const NAME: &'static str = "cat";
fn run(self, host: &mut Host) -> i32 {
let number_mode = if self.matches.get_flag(options::NUMBER_NONBLANK) {
NumberingMode::NonEmpty
} else if self.matches.get_flag(options::NUMBER) {
NumberingMode::All
} else {
NumberingMode::None
};
let show_nonprint = [
options::SHOW_ALL,
options::SHOW_NONPRINTING_ENDS,
options::SHOW_NONPRINTING_TABS,
options::SHOW_NONPRINTING,
]
.iter()
.any(|value| self.matches.get_flag(value));
let show_ends = [options::SHOW_ENDS, options::SHOW_ALL, options::SHOW_NONPRINTING_ENDS]
.iter()
.any(|value| self.matches.get_flag(value));
let show_tabs = [options::SHOW_ALL, options::SHOW_TABS, options::SHOW_NONPRINTING_TABS]
.iter()
.any(|value| self.matches.get_flag(value));
let options = OutputOptions {
number: number_mode,
squeeze_blank: self.matches.get_flag(options::SQUEEZE_BLANK),
show_tabs,
show_ends,
show_nonprint,
};
#[allow(clippy::unwrap_used, reason = "clap provides '-' by default")]
let files = self.matches.get_many::<OsString>(options::FILE).unwrap();
cat_files(files, &options, host)
}
}
/// The `cat` argument model.
fn app() -> Command {
Command::new(Cat::NAME)
.version("0.8.0")
.override_usage(format_usage("cat [OPTION]... [FILE]..."))
.about(
"Concatenate FILE(s), or standard input, to standard output\nWith no FILE, or when FILE \
is -, read standard input.",
)
.infer_long_args(true)
.args_override_self(true)
.arg(
Arg::new(options::FILE)
.hide(true)
.action(ArgAction::Append)
.value_parser(clap::value_parser!(OsString))
.default_value("-")
.value_hint(clap::ValueHint::FilePath),
)
.arg(
Arg::new(options::SHOW_ALL)
.short('A')
.long(options::SHOW_ALL)
.help("equivalent to -vET")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::NUMBER_NONBLANK)
.short('b')
.long(options::NUMBER_NONBLANK)
.help("number nonempty output lines, overrides -n")
// This must not override NUMBER: clap overriding is symmetric,
// while `-b -n` must still use `-b` semantics.
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SHOW_NONPRINTING_ENDS)
.short('e')
.help("equivalent to -vE")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SHOW_ENDS)
.short('E')
.long(options::SHOW_ENDS)
.help("display $ at end of each line")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::NUMBER)
.short('n')
.long(options::NUMBER)
.help("number all output lines")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SQUEEZE_BLANK)
.short('s')
.long(options::SQUEEZE_BLANK)
.help("suppress repeated empty output lines")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SHOW_NONPRINTING_TABS)
.short('t')
.help("equivalent to -vT")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SHOW_TABS)
.short('T')
.long(options::SHOW_TABS)
.help("display TAB characters at ^I")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SHOW_NONPRINTING)
.short('v')
.long(options::SHOW_NONPRINTING)
.help("use ^ and M- notation, except for LF (\\n) and TAB (\\t)")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::IGNORED_U)
.short('u')
.help("(ignored)")
.action(ArgAction::SetTrue),
)
}
fn cat_handle<R: FdReadable>(
handle: &mut InputHandle<R>,
options: &OutputOptions,
state: &mut OutputState,
stdout: &mut OpenFile,
) -> CatResult<()> {
if options.can_write_fast() {
write_fast(handle, stdout)
} else {
write_lines(handle, options, state, stdout)
}
}
fn cat_path(
path: &OsString,
options: &OutputOptions,
state: &mut OutputState,
host: &mut Host,
) -> CatResult<()> {
// Resolve every operand at the boundary, but retain `path` for diagnostics.
let resolved = host.resolve(path);
match get_input_type(path, &resolved)? {
InputType::StdIn => {
let (stdin, stdout) = (&mut host.stdin, &mut host.stdout);
let mut handle = InputHandle { reader: stdin, is_interactive: false };
cat_handle(&mut handle, options, state, stdout)
},
InputType::Directory => Err(CatError::IsDirectory),
#[cfg(unix)]
InputType::Socket => Err(CatError::NoSuchDeviceOrAddress),
_ => {
let file = File::open(resolved)?;
let mut handle = InputHandle { reader: file, is_interactive: false };
cat_handle(&mut handle, options, state, &mut host.stdout)
},
}
}
fn cat_files<'a, I>(files: I, options: &OutputOptions, host: &mut Host) -> i32
where
I: IntoIterator<Item = &'a OsString>,
{
let mut state = OutputState {
line_number: LineNumber::new(),
at_line_start: true,
skipped_carriage_return: false,
one_blank_kept: false,
};
for path in files {
match cat_path(path, options, &mut state, host) {
Ok(()) => {},
Err(CatError::BrokenPipe) => return host.exit_code(),
Err(error) => host.error(format!("{}: {error}", path.maybe_quote()), 1),
}
}
if state.skipped_carriage_return {
let _ = host.stdout.write_all(b"\r");
}
host.exit_code()
}
/// Classifies the input at `resolved`; `path` is retained to recognize `-`.
fn get_input_type(path: &OsString, resolved: &Path) -> CatResult<InputType> {
if path == "-" {
return Ok(InputType::StdIn);
}
let file_type = match metadata(resolved) {
Ok(metadata) => metadata.file_type(),
Err(error) => {
if let Some(raw_error) = error.raw_os_error() {
// ELOOP differs on Darwin and FreeBSD.
#[cfg(not(any(target_os = "macos", target_os = "freebsd")))]
let too_many_symlink_code = 40;
#[cfg(any(target_os = "macos", target_os = "freebsd"))]
let too_many_symlink_code = 62;
if raw_error == too_many_symlink_code {
return Err(CatError::TooManySymlinks);
}
}
return Err(CatError::from(error));
},
};
match file_type {
#[cfg(unix)]
file_type if file_type.is_block_device() => Ok(InputType::BlockDevice),
#[cfg(unix)]
file_type if file_type.is_char_device() => Ok(InputType::CharacterDevice),
#[cfg(unix)]
file_type if file_type.is_fifo() => Ok(InputType::Fifo),
#[cfg(unix)]
file_type if file_type.is_socket() => Ok(InputType::Socket),
file_type if file_type.is_dir() => Ok(InputType::Directory),
file_type if file_type.is_file() => Ok(InputType::File),
file_type if file_type.is_symlink() => Ok(InputType::SymLink),
_ => Err(CatError::UnknownFiletype { ft_debug: format!("{file_type:?}") }),
}
}
/// Writes a handle to stdout with no output transformation.
fn write_fast<R: FdReadable>(
handle: &mut InputHandle<R>,
stdout: &mut OpenFile,
) -> CatResult<()> {
#[cfg(any(target_os = "linux", target_os = "android"))]
{
// Splice is safe only when the in-process output stream is backed by a
// real descriptor. Never substitute the host process's fd 1.
let fall_back = match stdout.try_borrow_as_fd() {
Ok(stdout_fd) => splice::write_fast_using_splice(handle, stdout_fd)?,
Err(_) => true,
};
if !fall_back {
return Ok(());
}
}
let mut buf = [0; 1024 * 64];
loop {
match handle.reader.read(&mut buf) {
Ok(0) => break,
Ok(n) => stdout.write_all(&buf[..n])?,
Err(error) if error.kind() == ErrorKind::Interrupted => {},
Err(error) => return Err(error.into()),
}
}
stdout.flush()?;
Ok(())
}
/// Outputs a handle line by line with the requested transformations.
fn write_lines<R: FdReadable>(
handle: &mut InputHandle<R>,
options: &OutputOptions,
state: &mut OutputState,
stdout: &mut OpenFile,
) -> CatResult<()> {
let mut in_buf = [0; 1024 * 31];
// A 32K output buffer greatly improves performance.
let mut writer = BufWriter::with_capacity(32 * 1024, stdout);
loop {
let n = match handle.reader.read(&mut in_buf) {
Ok(0) => break,
Ok(n) => n,
Err(error) if error.kind() == ErrorKind::Interrupted => continue,
Err(error) => return Err(error.into()),
};
let in_buf = &in_buf[..n];
let mut pos = 0;
while pos < n {
if in_buf[pos] == b'\n' {
write_new_line(&mut writer, options, state, handle.is_interactive)?;
state.at_line_start = true;
pos += 1;
continue;
}
if state.skipped_carriage_return {
writer.write_all(b"\r")?;
state.skipped_carriage_return = false;
state.at_line_start = false;
}
state.one_blank_kept = false;
if state.at_line_start && options.number != NumberingMode::None {
state.line_number.write(&mut writer)?;
state.line_number.increment();
}
let offset = write_end(&mut writer, &in_buf[pos..], options)?;
if offset + pos == in_buf.len() {
state.at_line_start = false;
break;
}
if in_buf[pos + offset] == b'\r' {
state.skipped_carriage_return = true;
} else {
assert_eq!(in_buf[pos + offset], b'\n');
write_end_of_line(
&mut writer,
options.end_of_line().as_bytes(),
handle.is_interactive,
)?;
state.at_line_start = true;
}
pos += offset + 1;
}
// Flush before a pipe read can block so available output stays visible.
writer.flush()?;
}
Ok(())
}
/// Writes a newline, accounting for delayed carriage returns and numbering.
fn write_new_line<W: Write>(
writer: &mut W,
options: &OutputOptions,
state: &mut OutputState,
is_interactive: bool,
) -> CatResult<()> {
if state.skipped_carriage_return {
if options.show_ends {
writer.write_all(b"^M")?;
} else {
writer.write_all(b"\r")?;
}
state.skipped_carriage_return = false;
write_end_of_line(writer, options.end_of_line().as_bytes(), is_interactive)?;
return Ok(());
}
if !state.at_line_start || !options.squeeze_blank || !state.one_blank_kept {
state.one_blank_kept = true;
if state.at_line_start && options.number == NumberingMode::All {
state.line_number.write(writer)?;
state.line_number.increment();
}
write_end_of_line(writer, options.end_of_line().as_bytes(), is_interactive)?;
}
Ok(())
}
fn write_end<W: Write>(
writer: &mut W,
in_buf: &[u8],
options: &OutputOptions,
) -> io::Result<usize> {
if options.show_nonprint {
write_nonprint_to_end(in_buf, writer, options.tab().as_bytes())
} else if options.show_tabs {
write_tab_to_end(in_buf, writer)
} else {
write_to_end(in_buf, writer)
}
}
// Write all symbols until newline, carriage return, or the buffer end. The
// nonprinting path need not stop at carriage return because it always emits ^M.
fn write_to_end<W: Write>(in_buf: &[u8], writer: &mut W) -> io::Result<usize> {
if let Some(position) = memchr2(b'\n', b'\r', in_buf) {
writer.write_all(&in_buf[..position])?;
Ok(position)
} else {
writer.write_all(in_buf)?;
Ok(in_buf.len())
}
}
fn write_tab_to_end<W: Write>(mut in_buf: &[u8], writer: &mut W) -> io::Result<usize> {
let mut count = 0;
loop {
if let Some(position) = in_buf
.iter()
.position(|byte| *byte == b'\n' || *byte == b'\t' || *byte == b'\r')
{
writer.write_all(&in_buf[..position])?;
if in_buf[position] == b'\t' {
writer.write_all(b"^I")?;
in_buf = &in_buf[position + 1..];
count += position + 1;
} else {
return Ok(count + position);
}
} else {
writer.write_all(in_buf)?;
return Ok(in_buf.len() + count);
}
}
}
fn write_nonprint_to_end<W: Write>(
in_buf: &[u8],
writer: &mut W,
tab: &[u8],
) -> io::Result<usize> {
let mut count = 0;
for byte in in_buf.iter().copied() {
if byte == b'\n' {
break;
}
match byte {
9 => writer.write_all(tab),
0..=8 | 10..=31 => writer.write_all(&[b'^', byte + 64]),
32..=126 => writer.write_all(&[byte]),
127 => writer.write_all(b"^?"),
128..=159 => writer.write_all(&[b'M', b'-', b'^', byte - 64]),
160..=254 => writer.write_all(&[b'M', b'-', byte - 128]),
_ => writer.write_all(b"M-^?"),
}?;
count += 1;
}
Ok(count)
}
fn write_end_of_line<W: Write>(
writer: &mut W,
end_of_line: &[u8],
is_interactive: bool,
) -> CatResult<()> {
writer.write_all(end_of_line)?;
if is_interactive {
writer.flush()?;
}
Ok(())
}
/// Creates the `cat` builtin registration.
pub(crate) fn cat_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Cat, SE>()
}
#[cfg(test)]
mod tests {
use std::{fs, io::{BufWriter, sink}};
use tempfile::tempdir;
use super::{Cat, LineNumber, write_nonprint_to_end, write_tab_to_end};
use crate::host::run_util;
#[test]
fn test_write_tab_to_end_with_newline() {
let mut writer = BufWriter::with_capacity(1024 * 64, sink());
assert_eq!(write_tab_to_end(b"a\tb\tc\n", &mut writer).unwrap(), 5);
}
#[test]
fn test_write_tab_to_end_no_newline() {
let mut writer = BufWriter::with_capacity(1024 * 64, sink());
assert_eq!(write_tab_to_end(b"a\tb\tc", &mut writer).unwrap(), 5);
}
#[test]
fn test_write_nonprint_to_end_new_line() {
let mut writer = BufWriter::with_capacity(1024 * 64, sink());
write_nonprint_to_end(b"\n", &mut writer, b"").unwrap();
assert_eq!(writer.buffer().len(), 0);
}
#[test]
fn test_write_nonprint_to_end_9() {
let mut writer = BufWriter::with_capacity(1024 * 64, sink());
write_nonprint_to_end(&[9], &mut writer, b"tab").unwrap();
assert_eq!(writer.buffer(), b"tab");
}
#[test]
fn test_write_nonprint_to_end_0_to_8() {
for byte in 0_u8..=8 {
let mut writer = BufWriter::with_capacity(1024 * 64, sink());
write_nonprint_to_end(&[byte], &mut writer, b"").unwrap();
assert_eq!(writer.buffer(), [b'^', byte + 64]);
}
}
#[test]
fn test_write_nonprint_to_end_10_to_31() {
for byte in 11_u8..=31 {
let mut writer = BufWriter::with_capacity(1024 * 64, sink());
write_nonprint_to_end(&[byte], &mut writer, b"").unwrap();
assert_eq!(writer.buffer(), [b'^', byte + 64]);
}
}
#[test]
fn test_incrementing_string() {
let mut number = LineNumber::new();
assert_eq!(b" 1\t", number.to_str());
number.increment();
assert_eq!(b" 2\t", number.to_str());
for _ in 3..=100 {
number.increment();
}
assert_eq!(b" 100\t", number.to_str());
for _ in 101..=1_000_000 {
number.increment();
}
assert_eq!(b"1000000\t", number.to_str());
number.increment();
assert_eq!(b"1000001\t", number.to_str());
}
#[test]
fn missing_operand_reports_and_later_operand_is_processed() {
let directory = tempdir().unwrap();
fs::write(directory.path().join("present"), b"remaining\n").unwrap();
let (code, capture) = run_util::<Cat>(&["missing", "present"], "", directory.path());
assert_eq!(code, 1);
assert_eq!(capture.out(), "remaining\n");
assert_eq!(capture.err(), "cat: missing: No such file or directory\n");
}
}
crates/vendor/brush-builtins/src/cd.rs → crates/pi-builtins/src/cd.rs
View File
crates/pi-builtins/src/cksum.rs
+1957
View File
File diff suppressed because it is too large Load Diff
crates/pi-shell/src/cmp.rs → crates/pi-builtins/src/cmp.rs
+174 -143
View File
@@ -1,13 +1,21 @@
//! Context-safe, in-process `cmp` implementation for the embedded shell.
//! `cmp` builtin: compare two files byte by byte.
//!
//! Ported from the `pi-shell` in-process implementation.
use std::{
ffi::{OsStr, OsString},
fs::{self, File},
io::{self, BufRead, BufReader, Cursor, Read, Seek, SeekFrom, Write},
path::Path,
sync::{
atomic::{AtomicBool, Ordering},
},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use pi_uutils_ctx::format_usage;
use crate::host::{Host, Stdin, Utility, format_usage, matches_parser, util};
const OPT_PRINT_BYTES: &str = "print-bytes";
const OPT_IGNORE_INITIAL: &str = "ignore-initial";
@@ -36,13 +44,13 @@ struct Options {
skip2: u64,
}
enum InputReader {
enum InputReader<'a> {
File(File),
Stdin(pi_uutils_ctx::CtxStdin),
Stdin(&'a mut Stdin),
Bytes(Cursor<Vec<u8>>),
}
impl Read for InputReader {
impl Read for InputReader<'_> {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
match self {
Self::File(file) => file.read(buf),
@@ -52,7 +60,7 @@ impl Read for InputReader {
}
}
impl InputReader {
impl InputReader<'_> {
fn skip(&mut self, count: u64) -> io::Result<()> {
match self {
Self::File(file) => {
@@ -70,40 +78,36 @@ impl InputReader {
}
}
struct Input {
reader: BufReader<InputReader>,
struct Input<'a> {
reader: BufReader<InputReader<'a>>,
regular_len: Option<u64>,
}
/// Runs `cmp` against invocation-scoped stdin/stdout/stderr and shell-relative
/// paths.
pub fn run(argv: Vec<OsString>) -> i32 {
let matches = match command().try_get_matches_from(argv) {
Ok(matches) => matches,
Err(err) => {
let code = err.exit_code();
let rendered = err.to_string();
if err.use_stderr() {
let _ = write!(pi_uutils_ctx::stderr(), "{rendered}");
} else {
let _ = write!(pi_uutils_ctx::stdout(), "{rendered}");
}
return code;
},
};
/// Parsed `cmp` invocation.
pub(crate) struct Cmp {
matches: ArgMatches,
}
match compare(&matches) {
Ok(code) => code,
Err(message) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "cmp: {message}");
2
},
matches_parser!(Cmp, command);
impl Utility for Cmp {
const NAME: &'static str = "cmp";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
match compare(&self.matches, host) {
Ok(code) => code,
Err(message) => {
host.error(message, 2);
2
},
}
}
}
fn command() -> Command {
Command::new("cmp")
.version(concat!("cmp (pi-shell) ", env!("CARGO_PKG_VERSION")))
Command::new(Cmp::NAME)
.version("cmp (pi-shell) 17.2.11")
.about("Compare two files byte by byte.")
.override_usage(format_usage("cmp [OPTION]... FILE1 [FILE2 [SKIP1 [SKIP2]]]"))
.disable_help_flag(true)
@@ -188,7 +192,7 @@ fn command() -> Command {
.arg(Arg::new(ARG_SKIP2).value_name("SKIP2"))
}
fn compare(matches: &ArgMatches) -> Result<i32, String> {
fn compare(matches: &ArgMatches, host: &mut Host) -> Result<i32, String> {
let name1 = matches.get_one::<OsString>(ARG_FILE1).unwrap();
let default_stdin = OsString::from("-");
let name2 = matches
@@ -213,36 +217,50 @@ fn compare(matches: &ArgMatches) -> Result<i32, String> {
skip1,
skip2,
};
let path1 = (name1 != OsStr::new("-")).then(|| host.resolve(name1));
let path2 = (name2 != OsStr::new("-")).then(|| host.resolve(name2));
let cancel = host.cancel_flag();
let mut input1 = open_input(name1, options.no_follow)?;
let mut input2 = open_input(name2, options.no_follow)?;
if options.size_first
&& let (Some(len1), Some(len2)) = (input1.regular_len, input2.regular_len)
&& len1 != len2
{
if !options.quiet {
writeln!(
pi_uutils_ctx::stdout(),
"{} {} differ: size",
display_name(name1),
display_name(name2)
)
.map_err(io_message)?;
}
return Ok(1);
if name1 == OsStr::new("-") {
let input1 = stdin_input(&mut host.stdin);
let input2 = open_input(name2, path2.as_deref().unwrap(), options.no_follow)?;
compare_inputs(
input1,
input2,
name1,
name2,
options,
&mut host.stdout,
&mut host.stderr,
&cancel,
)
} else if name2 == OsStr::new("-") {
let input1 = open_input(name1, path1.as_deref().unwrap(), options.no_follow)?;
let input2 = stdin_input(&mut host.stdin);
compare_inputs(
input1,
input2,
name1,
name2,
options,
&mut host.stdout,
&mut host.stderr,
&cancel,
)
} else {
let input1 = open_input(name1, path1.as_deref().unwrap(), options.no_follow)?;
let input2 = open_input(name2, path2.as_deref().unwrap(), options.no_follow)?;
compare_inputs(
input1,
input2,
name1,
name2,
options,
&mut host.stdout,
&mut host.stderr,
&cancel,
)
}
input1
.reader
.get_mut()
.skip(options.skip1)
.map_err(|err| input_error(name1, &err))?;
input2
.reader
.get_mut()
.skip(options.skip2)
.map_err(|err| input_error(name2, &err))?;
compare_readers(&mut input1.reader, &mut input2.reader, name1, name2, options)
}
fn skips(matches: &ArgMatches) -> Result<(u64, u64), String> {
@@ -313,25 +331,22 @@ fn parse_count(value: &str) -> Result<u64, String> {
.ok_or_else(|| format!("byte count '{value}' is too large"))
}
fn open_input(name: &OsStr, no_follow: bool) -> Result<Input, String> {
if name == OsStr::new("-") {
return Ok(Input {
reader: BufReader::with_capacity(
BUFFER_SIZE,
InputReader::Stdin(pi_uutils_ctx::stdin()),
),
regular_len: None,
});
fn stdin_input(stdin: &mut Stdin) -> Input<'_> {
Input {
reader: BufReader::with_capacity(BUFFER_SIZE, InputReader::Stdin(stdin)),
regular_len: None,
}
let path = pi_uutils_ctx::resolve(name);
}
fn open_input<'a>(name: &OsStr, path: &Path, no_follow: bool) -> Result<Input<'a>, String> {
let metadata = if no_follow {
fs::symlink_metadata(&path)
fs::symlink_metadata(path)
} else {
fs::metadata(&path)
fs::metadata(path)
}
.map_err(|err| input_error(name, &err))?;
if no_follow && metadata.file_type().is_symlink() {
let target = fs::read_link(&path).map_err(|err| input_error(name, &err))?;
let target = fs::read_link(path).map_err(|err| input_error(name, &err))?;
return Ok(Input {
reader: BufReader::with_capacity(
BUFFER_SIZE,
@@ -348,12 +363,63 @@ fn open_input(name: &OsStr, no_follow: bool) -> Result<Input, String> {
Ok(Input { reader: BufReader::with_capacity(BUFFER_SIZE, InputReader::File(file)), regular_len })
}
fn compare_inputs(
mut input1: Input<'_>,
mut input2: Input<'_>,
name1: &OsStr,
name2: &OsStr,
options: Options,
stdout: &mut impl Write,
stderr: &mut impl Write,
cancel: &AtomicBool,
) -> Result<i32, String> {
if options.size_first
&& let (Some(len1), Some(len2)) = (input1.regular_len, input2.regular_len)
&& len1 != len2
{
if !options.quiet {
writeln!(
stdout,
"{} {} differ: size",
display_name(name1),
display_name(name2)
)
.map_err(io_message)?;
}
return Ok(1);
}
input1
.reader
.get_mut()
.skip(options.skip1)
.map_err(|err| input_error(name1, &err))?;
input2
.reader
.get_mut()
.skip(options.skip2)
.map_err(|err| input_error(name2, &err))?;
compare_readers(
&mut input1.reader,
&mut input2.reader,
name1,
name2,
options,
stdout,
stderr,
cancel,
)
}
fn compare_readers(
input1: &mut impl BufRead,
input2: &mut impl BufRead,
name1: &OsStr,
name2: &OsStr,
options: Options,
stdout: &mut impl Write,
stderr: &mut impl Write,
cancel: &AtomicBool,
) -> Result<i32, String> {
let mut byte_number = 1u64;
let mut line_number = 1u64;
@@ -361,7 +427,7 @@ fn compare_readers(
let mut different = false;
loop {
if pi_uutils_ctx::is_cancelled() {
if cancel.load(Ordering::Relaxed) {
return Err("interrupted".to_string());
}
if options.limit.is_some_and(|limit| compared >= limit) {
@@ -377,7 +443,7 @@ fn compare_readers(
if !options.quiet {
let eof_name = if left.is_empty() { name1 } else { name2 };
writeln!(
pi_uutils_ctx::stderr(),
stderr,
"cmp: EOF on {} after byte {}",
display_name(eof_name),
compared
@@ -405,6 +471,7 @@ fn compare_readers(
name1,
name2,
options,
stdout,
)?;
if !options.verbose && !options.hex {
return Ok(1);
@@ -431,8 +498,8 @@ fn report_difference(
name1: &OsStr,
name2: &OsStr,
options: Options,
out: &mut impl Write,
) -> Result<(), String> {
let mut out = pi_uutils_ctx::stdout();
if options.hex {
return writeln!(out, "{:08x} {left:02x} {right:02x}", byte_number - 1).map_err(io_message);
}
@@ -440,12 +507,12 @@ fn report_difference(
write!(out, "{byte_number} {left:3o}").map_err(io_message)?;
if options.print_bytes {
write!(out, " ").map_err(io_message)?;
write_visible_byte_padded(&mut out, left).map_err(io_message)?;
write_visible_byte_padded(out, left).map_err(io_message)?;
}
write!(out, " {right:3o}").map_err(io_message)?;
if options.print_bytes {
write!(out, " ").map_err(io_message)?;
write_visible_byte(&mut out, right).map_err(io_message)?;
write_visible_byte(out, right).map_err(io_message)?;
}
return writeln!(out).map_err(io_message);
}
@@ -459,9 +526,9 @@ fn report_difference(
.map_err(io_message)?;
if options.print_bytes {
write!(out, " is {left:3o} ").map_err(io_message)?;
write_visible_byte(&mut out, left).map_err(io_message)?;
write_visible_byte(out, left).map_err(io_message)?;
write!(out, " {right:3o} ").map_err(io_message)?;
write_visible_byte(&mut out, right).map_err(io_message)?;
write_visible_byte(out, right).map_err(io_message)?;
}
writeln!(out).map_err(io_message)
}
@@ -508,56 +575,21 @@ fn io_message(err: io::Error) -> String {
err.to_string()
}
/// Creates the `cmp` builtin registration.
pub(crate) fn cmp_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Cmp, SE>()
}
#[cfg(test)]
mod tests {
use std::{
collections::HashMap,
ffi::OsString,
fs,
io::{Cursor, Write},
path::PathBuf,
sync::{Arc, atomic::AtomicBool},
};
use std::{fs, path::PathBuf};
use parking_lot::Mutex;
use pi_uutils_ctx::ScopeIo;
use super::Cmp;
use crate::host::run_util;
use super::run;
#[derive(Clone)]
struct SharedWriter(Arc<Mutex<Vec<u8>>>);
impl Write for SharedWriter {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
self.0.lock().write(buf)
}
fn flush(&mut self) -> std::io::Result<()> {
self.0.lock().flush()
}
}
fn run_in(cwd: PathBuf, stdin: &[u8], args: &[&str]) -> (i32, String, String) {
let stdout = Arc::new(Mutex::new(Vec::new()));
let stderr = Arc::new(Mutex::new(Vec::new()));
let io = ScopeIo {
stdin: Box::new(Cursor::new(stdin.to_vec())),
stdin_fd: None,
stdin_is_search_input: false,
stdout: Box::new(SharedWriter(Arc::clone(&stdout))),
stderr: Box::new(SharedWriter(Arc::clone(&stderr))),
cwd,
env: HashMap::new(),
cancel: Arc::new(AtomicBool::new(false)),
};
let argv = std::iter::once("cmp")
.chain(args.iter().copied())
.map(OsString::from)
.collect();
let code = pi_uutils_ctx::scope(io, || run(argv));
let stdout = String::from_utf8(stdout.lock().clone()).unwrap();
let stderr = String::from_utf8(stderr.lock().clone()).unwrap();
(code, stdout, stderr)
fn run_in(cwd: PathBuf, stdin: &str, args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Cmp>(args, stdin, cwd);
(code, capture.out(), capture.err())
}
fn tempdir() -> (tempfile::TempDir, PathBuf) {
@@ -573,8 +605,8 @@ mod tests {
fs::write(root.join("b"), b"same").unwrap();
fs::write(root.join("c"), b"different").unwrap();
assert_eq!(run_in(root.clone(), b"", &["-s", "a", "b"]), (0, String::new(), String::new()));
assert_eq!(run_in(root, b"", &["-s", "a", "c"]), (1, String::new(), String::new()));
assert_eq!(run_in(root.clone(), "", &["-s", "a", "b"]), (0, String::new(), String::new()));
assert_eq!(run_in(root, "", &["-s", "a", "c"]), (1, String::new(), String::new()));
}
#[test]
@@ -583,7 +615,7 @@ mod tests {
fs::write(root.join("a"), b"one\ntwo\n").unwrap();
fs::write(root.join("b"), b"one\ntXo\n").unwrap();
let (code, stdout, stderr) = run_in(root, b"", &["a", "b"]);
let (code, stdout, stderr) = run_in(root, "", &["a", "b"]);
assert_eq!(code, 1);
assert_eq!(stdout, "a b differ: char 6, line 2\n");
assert_eq!(stderr, "");
@@ -595,10 +627,10 @@ mod tests {
fs::write(root.join("file"), b"abc").unwrap();
assert_eq!(
run_in(root.clone(), b"xxabc", &["-i", "0:2", "file"]),
run_in(root.clone(), "xxabc", &["-i", "0:2", "file"]),
(0, String::new(), String::new())
);
assert_eq!(run_in(root, b"abZ", &["-n", "0x2", "file"]), (0, String::new(), String::new()));
assert_eq!(run_in(root, "abZ", &["-n", "0x2", "file"]), (0, String::new(), String::new()));
}
#[test]
@@ -607,29 +639,28 @@ mod tests {
fs::write(root.join("a"), b"abc").unwrap();
fs::write(root.join("b"), b"axd").unwrap();
let (code, stdout, stderr) = run_in(root.clone(), b"", &["-l", "a", "b"]);
let (code, stdout, stderr) = run_in(root.clone(), "", &["-l", "a", "b"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (1, "2 142 170\n3 143 144\n", ""));
let (code, stdout, stderr) = run_in(root, b"", &["-x", "a", "b"]);
let (code, stdout, stderr) = run_in(root, "", &["-x", "a", "b"]);
assert_eq!(
(code, stdout.as_str(), stderr.as_str()),
(1, "00000001 62 78\n00000002 63 64\n", "")
);
}
#[cfg(unix)]
#[test]
fn no_follow_compares_symlink_targets() {
let (_dir, root) = tempdir();
fs::write(root.join("a"), b"same").unwrap();
fs::write(root.join("b"), b"same").unwrap();
#[cfg(unix)]
{
std::os::unix::fs::symlink("a", root.join("left")).unwrap();
std::os::unix::fs::symlink("b", root.join("right")).unwrap();
let (code, stdout, stderr) = run_in(root, b"", &["-h", "left", "right"]);
assert_eq!(code, 1);
assert_eq!(stdout, "left right differ: char 1, line 1\n");
assert_eq!(stderr, "");
}
std::os::unix::fs::symlink("a", root.join("left")).unwrap();
std::os::unix::fs::symlink("b", root.join("right")).unwrap();
let (code, stdout, stderr) = run_in(root, "", &["-h", "left", "right"]);
assert_eq!(code, 1);
assert_eq!(stdout, "left right differ: char 1, line 1\n");
assert_eq!(stderr, "");
}
}
crates/vendor/brush-builtins/src/colon.rs → crates/pi-builtins/src/colon.rs
View File
crates/pi-builtins/src/combine.rs
+381
View File
@@ -0,0 +1,381 @@
//! `combine` builtin: boolean set operations on the lines of two files.
//!
//! Ported from the moreutils-inspired in-process implementation in `pi-shell`.
//! Keeping the utility in process lets it use the shell's scoped streams,
//! working directory, and cancellation rather than spawning an external tool.
//!
//! `combine FILE1 OP FILE2` accepts the case-insensitive operators `and`, `not`,
//! `or`, and `xor`. `-` names stdin, but only for one operand. Lines remain raw
//! byte strings; membership ignores a trailing newline, while output preserves
//! each original line exactly.
use std::{
collections::HashSet,
ffi::{OsStr, OsString},
fs::File,
io::{BufRead, BufReader, Write},
sync::atomic::{AtomicBool, Ordering},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command, value_parser};
use crate::host::{Host, Utility, format_usage, matches_parser, util};
const ARG_FILE1: &str = "file1";
const ARG_OP: &str = "op";
const ARG_FILE2: &str = "file2";
#[derive(Clone, Copy, PartialEq)]
enum Op {
And,
Not,
Or,
Xor,
}
enum Error {
Cancelled,
Msg(String),
}
/// Parsed `combine` invocation.
pub(crate) struct Combine {
matches: ArgMatches,
}
matches_parser!(Combine, command);
impl Utility for Combine {
const NAME: &'static str = "combine";
fn run(self, host: &mut Host) -> i32 {
let file1 = self
.matches
.get_one::<OsString>(ARG_FILE1)
.expect("required")
.clone();
let op = self.matches.get_one::<String>(ARG_OP).expect("required");
let file2 = self
.matches
.get_one::<OsString>(ARG_FILE2)
.expect("required")
.clone();
match execute(&file1, op, &file2, host) {
Ok(()) => 0,
Err(Error::Cancelled) => 130,
Err(Error::Msg(message)) => {
host.error(message, 1);
1
},
}
}
}
/// The `combine` argument model.
fn command() -> Command {
Command::new(Combine::NAME)
.version("combine (pi-shell) 17.2.11")
.about("Combine the lines of two files using boolean operations.")
.override_usage(format_usage("combine FILE1 and|not|or|xor FILE2"))
.disable_help_flag(true)
.disable_version_flag(true)
.arg(Arg::new("help").long("help").action(ArgAction::Help))
.arg(
Arg::new(ARG_FILE1)
.value_name("FILE1")
.required(true)
.value_parser(value_parser!(OsString)),
)
.arg(Arg::new(ARG_OP).value_name("OP").required(true))
.arg(
Arg::new(ARG_FILE2)
.value_name("FILE2")
.required(true)
.value_parser(value_parser!(OsString)),
)
}
fn execute(file1: &OsStr, op: &str, file2: &OsStr, host: &mut Host) -> Result<(), Error> {
let op = match op.to_ascii_lowercase().as_str() {
"and" => Op::And,
"not" => Op::Not,
"or" => Op::Or,
"xor" => Op::Xor,
other => {
return Err(Error::Msg(format!(
"unknown operation '{other}' (expected and, not, or, xor)"
)));
},
};
let dash = OsStr::new("-");
if file1 == dash && file2 == dash {
return Err(Error::Msg("only one file can be stdin".into()));
}
// Open both up front so a missing FILE2 fails before stdin is consumed.
let input1 = open_input(file1, host)?;
let input2 = open_input(file2, host)?;
let cancel = host.cancel_flag();
match (input1, input2) {
(Some(input1), Some(input2)) => operate(
BufReader::new(input1),
file1,
BufReader::new(input2),
file2,
op,
&cancel,
&mut host.stdout,
),
(None, Some(input2)) => operate(
BufReader::new(&mut host.stdin),
file1,
BufReader::new(input2),
file2,
op,
&cancel,
&mut host.stdout,
),
(Some(input1), None) => operate(
BufReader::new(input1),
file1,
BufReader::new(&mut host.stdin),
file2,
op,
&cancel,
&mut host.stdout,
),
(None, None) => unreachable!("two stdin operands were rejected above"),
}
}
fn operate(
mut input1: impl BufRead,
file1: &OsStr,
mut input2: impl BufRead,
file2: &OsStr,
op: Op,
cancel: &AtomicBool,
out: &mut impl Write,
) -> Result<(), Error> {
match op {
Op::And | Op::Not => {
// Membership side must be fully loaded before streaming FILE1.
let lines2 = read_lines(&mut input2, file2, cancel)?;
let set2: HashSet<&[u8]> = lines2.iter().map(|line| key(line)).collect();
let keep_member = op == Op::And;
each_line(&mut input1, file1, cancel, |line| {
if set2.contains(key(line)) == keep_member {
write_line(out, line)?;
}
Ok(())
})?;
},
Op::Or => {
each_line(&mut input1, file1, cancel, |line| write_line(out, line))?;
each_line(&mut input2, file2, cancel, |line| write_line(out, line))?;
},
Op::Xor => {
let lines1 = read_lines(&mut input1, file1, cancel)?;
let lines2 = read_lines(&mut input2, file2, cancel)?;
let set1: HashSet<&[u8]> = lines1.iter().map(|line| key(line)).collect();
let set2: HashSet<&[u8]> = lines2.iter().map(|line| key(line)).collect();
for line in &lines1 {
if !set2.contains(key(line)) {
write_line(out, line)?;
}
}
for line in &lines2 {
if !set1.contains(key(line)) {
write_line(out, line)?;
}
}
},
}
out.flush().map_err(|err| Error::Msg(err.to_string()))?;
Ok(())
}
fn open_input(name: &OsStr, host: &Host) -> Result<Option<File>, Error> {
if name == OsStr::new("-") {
return Ok(None);
}
let path = host.resolve(name);
let file = File::open(path).map_err(|err| Error::Msg(input_error(name, &err.to_string())))?;
Ok(Some(file))
}
/// Streams `reader` line by line (trailing `\n` retained when present),
/// polling for cancellation between lines.
fn each_line(
reader: &mut dyn BufRead,
name: &OsStr,
cancel: &AtomicBool,
mut f: impl FnMut(&[u8]) -> Result<(), Error>,
) -> Result<(), Error> {
let mut line = Vec::new();
loop {
if cancel.load(Ordering::Relaxed) {
return Err(Error::Cancelled);
}
line.clear();
let n = reader
.read_until(b'\n', &mut line)
.map_err(|err| Error::Msg(input_error(name, &err.to_string())))?;
if n == 0 {
return Ok(());
}
f(&line)?;
}
}
fn read_lines(
reader: &mut dyn BufRead,
name: &OsStr,
cancel: &AtomicBool,
) -> Result<Vec<Vec<u8>>, Error> {
let mut lines = Vec::new();
each_line(reader, name, cancel, |line| {
lines.push(line.to_vec());
Ok(())
})?;
Ok(lines)
}
/// Membership key: the line with any trailing newline stripped, so `foo`
/// (no newline) matches `foo\n`.
fn key(line: &[u8]) -> &[u8] {
line.strip_suffix(b"\n").unwrap_or(line)
}
fn write_line(out: &mut impl Write, line: &[u8]) -> Result<(), Error> {
out.write_all(line)
.map_err(|err| Error::Msg(err.to_string()))
}
fn input_error(name: &OsStr, err: &str) -> String {
format!("{}: {}", name.to_string_lossy(), err)
}
/// Creates the `combine` builtin registration.
pub(crate) fn combine_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Combine, SE>()
}
#[cfg(test)]
mod tests {
use std::{fs, path::Path};
use super::Combine;
use crate::host::run_util;
fn run_in(cwd: &Path, stdin: &str, args: &[&str]) -> (i32, Vec<u8>, String) {
let (code, capture) = run_util::<Combine>(args, stdin, cwd);
(code, capture.stdout(), capture.err())
}
fn fixture() -> tempfile::TempDir {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("one"), b"a\nb\na\nc\n").unwrap();
fs::write(dir.path().join("two"), b"a\nc\nd\n").unwrap();
dir
}
#[test]
fn and_keeps_file1_order_and_duplicates() {
let dir = fixture();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "and", "two"]);
assert_eq!((code, stdout.as_slice(), stderr.as_str()), (0, b"a\na\nc\n".as_slice(), ""));
}
#[test]
fn not_removes_file2_members() {
let dir = fixture();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "not", "two"]);
assert_eq!((code, stdout.as_slice(), stderr.as_str()), (0, b"b\n".as_slice(), ""));
}
#[test]
fn or_concatenates_both_files() {
let dir = fixture();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "or", "two"]);
assert_eq!(
(code, stdout.as_slice(), stderr.as_str()),
(0, b"a\nb\na\nc\na\nc\nd\n".as_slice(), "")
);
}
#[test]
fn xor_emits_exclusive_lines_from_both_sides() {
let dir = fixture();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "XOR", "two"]);
assert_eq!((code, stdout.as_slice(), stderr.as_str()), (0, b"b\nd\n".as_slice(), ""));
}
#[test]
fn dash_reads_file1_from_stdin() {
let dir = fixture();
let (code, stdout, stderr) =
run_in(dir.path(), "a\nb\na\nc\n", &["-", "and", "two"]);
assert_eq!((code, stdout.as_slice(), stderr.as_str()), (0, b"a\na\nc\n".as_slice(), ""));
}
#[test]
fn both_sides_stdin_is_rejected() {
let dir = tempfile::tempdir().unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["-", "and", "-"]);
assert_eq!(code, 1);
assert!(stdout.is_empty());
assert_eq!(stderr, "combine: only one file can be stdin\n");
}
#[test]
fn non_utf8_lines_survive_byte_exact() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("one"), b"\xff\xfe\n\x80ok\n").unwrap();
fs::write(dir.path().join("two"), b"\xff\xfe\n").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "and", "two"]);
assert_eq!((code, stdout.as_slice(), stderr.as_str()), (0, b"\xff\xfe\n".as_slice(), ""));
}
#[test]
fn missing_file_reports_error_exit_1() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("one"), b"a\n").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "and", "nope"]);
assert_eq!(code, 1);
assert!(stdout.is_empty());
assert!(stderr.starts_with("combine: nope: "), "stderr: {stderr}");
}
#[test]
fn unknown_op_is_usage_error_exit_1() {
let dir = fixture();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "nand", "two"]);
assert_eq!(code, 1);
assert!(stdout.is_empty());
assert_eq!(stderr, "combine: unknown operation 'nand' (expected and, not, or, xor)\n");
}
#[test]
fn wrong_arg_count_is_usage_error_exit_1() {
let dir = tempfile::tempdir().unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["only-one"]);
assert_eq!(code, 1);
assert!(stdout.is_empty());
assert!(stderr.contains("Usage"), "stderr: {stderr}");
}
#[test]
fn last_line_without_newline_matches_and_is_emitted_as_is() {
// `b` without a trailing newline still counts as a line, matches
// `b\n` in the other file, and is emitted without adding a newline.
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("one"), b"a\nb").unwrap();
fs::write(dir.path().join("two"), b"b\n").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["one", "and", "two"]);
assert_eq!((code, stdout.as_slice(), stderr.as_str()), (0, b"b".as_slice(), ""));
}
}
crates/pi-builtins/src/comm.rs
+422
View File
@@ -0,0 +1,422 @@
//! `comm` builtin: compare two sorted files line by line.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
cmp::Ordering,
ffi::{OsStr, OsString},
fs::{self, File},
io::{self, BufRead, BufReader, BufWriter, Read, Write},
path::Path,
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use uucore::{display::Quotable, line_ending::LineEnding};
use crate::host::{Host, Utility, format_usage, matches_parser, util};
mod options {
pub const COLUMN_1: &str = "1";
pub const COLUMN_2: &str = "2";
pub const COLUMN_3: &str = "3";
pub const DELIMITER: &str = "output-delimiter";
pub const FILE_1: &str = "FILE1";
pub const FILE_2: &str = "FILE2";
pub const TOTAL: &str = "total";
pub const ZERO_TERMINATED: &str = "zero-terminated";
pub const CHECK_ORDER: &str = "check-order";
pub const NO_CHECK_ORDER: &str = "nocheck-order";
}
#[derive(Clone, Copy)]
enum FileNumber {
One,
Two,
}
impl FileNumber {
fn as_str(self) -> &'static str {
match self {
Self::One => "1",
Self::Two => "2",
}
}
}
struct OrderChecker {
last_line: Vec<u8>,
file_num: FileNumber,
check_order: bool,
has_error: bool,
}
impl OrderChecker {
fn new(file_num: FileNumber, check_order: bool) -> Self {
Self { last_line: Vec::new(), file_num, check_order, has_error: false }
}
fn verify_order(&mut self, line: &[u8], stderr: &mut impl Write) -> bool {
if self.last_line.is_empty() {
self.last_line = line.to_vec();
return true;
}
let ordered = line >= self.last_line.as_slice();
if !ordered && !self.has_error {
let _ = writeln!(stderr, "comm: file {} is not in sorted order", self.file_num.as_str());
self.has_error = true;
}
self.last_line.clear();
self.last_line.extend_from_slice(line);
ordered || !self.check_order
}
}
struct LineReader<'a> {
line_ending: u8,
input: Box<dyn BufRead + 'a>,
}
impl<'a> LineReader<'a> {
fn new(input: Box<dyn BufRead + 'a>, line_ending: LineEnding) -> Self {
Self { input, line_ending: line_ending.into() }
}
fn read_line(&mut self, buf: &mut Vec<u8>) -> io::Result<usize> {
let result = self.input.read_until(self.line_ending, buf)?;
if result != 0 && !buf.ends_with(&[self.line_ending]) {
buf.push(self.line_ending);
}
Ok(result)
}
}
fn files_identical(path1: &Path, path2: &Path) -> io::Result<bool> {
let m1 = fs::metadata(path1)?;
let m2 = fs::metadata(path2)?;
if !m1.is_file() || !m2.is_file() || m1.len() != m2.len() {
return Ok(false);
}
let mut a = BufReader::new(File::open(path1)?);
let mut b = BufReader::new(File::open(path2)?);
let mut ba = [0; 8192];
let mut bb = [0; 8192];
loop {
let na = loop {
match a.read(&mut ba) {
Err(e) if e.kind() == io::ErrorKind::Interrupted => {},
r => break r?,
}
};
let nb = loop {
match b.read(&mut bb) {
Err(e) if e.kind() == io::ErrorKind::Interrupted => {},
r => break r?,
}
};
if na != nb || ba[..na] != bb[..nb] {
return Ok(false);
}
if na == 0 {
return Ok(true);
}
}
}
fn write_delimited(writer: &mut impl Write, delim: &[u8], line: &[u8]) -> io::Result<()> {
writer.write_all(delim)?;
writer.write_all(line)
}
fn read_context(reader: &mut LineReader<'_>, buf: &mut Vec<u8>, name: &OsStr) -> Result<usize, String> {
reader.read_line(buf).map_err(|e| format!("{}: {e}", name.maybe_quote()))
}
fn compare(
a: &mut LineReader<'_>,
b: &mut LineReader<'_>,
name1: &OsStr,
name2: &OsStr,
delim: &str,
opts: &ArgMatches,
identical: bool,
stdout: &mut impl Write,
stderr: &mut impl Write,
) -> Result<bool, String> {
let col2 = delim.repeat(usize::from(!opts.get_flag(options::COLUMN_1)));
let col3 = delim.repeat(
usize::from(!opts.get_flag(options::COLUMN_1))
+ usize::from(!opts.get_flag(options::COLUMN_2)),
);
let mut writer = BufWriter::new(stdout);
let (mut ra, mut rb) = (Vec::new(), Vec::new());
let mut na = read_context(a, &mut ra, name1)?;
let mut nb = read_context(b, &mut rb, name2)?;
let (mut n1, mut n2, mut n3) = (0usize, 0usize, 0usize);
let explicit = opts.get_flag(options::CHECK_ORDER);
let should_check = !opts.get_flag(options::NO_CHECK_ORDER) && (explicit || !identical);
let (mut c1, mut c2) = (
OrderChecker::new(FileNumber::One, explicit),
OrderChecker::new(FileNumber::Two, explicit),
);
let mut delayed_error = false;
while na != 0 || nb != 0 {
let ord = match (na, nb) {
(0, _) => Ordering::Greater,
(_, 0) => Ordering::Less,
_ => ra.cmp(&rb),
};
match ord {
Ordering::Less => {
if should_check && !c1.verify_order(&ra, stderr) {
break;
}
if !opts.get_flag(options::COLUMN_1) {
writer.write_all(&ra).map_err(|e| format!("write error: {e}"))?;
}
ra.clear();
na = read_context(a, &mut ra, name1)?;
n1 += 1;
},
Ordering::Greater => {
if should_check && !c2.verify_order(&rb, stderr) {
break;
}
if !opts.get_flag(options::COLUMN_2) {
write_delimited(&mut writer, col2.as_bytes(), &rb)
.map_err(|e| format!("write error: {e}"))?;
}
rb.clear();
nb = read_context(b, &mut rb, name2)?;
n2 += 1;
},
Ordering::Equal => {
if should_check
&& (!c1.verify_order(&ra, stderr) || !c2.verify_order(&rb, stderr))
{
break;
}
if !opts.get_flag(options::COLUMN_3) {
write_delimited(&mut writer, col3.as_bytes(), &ra)
.map_err(|e| format!("write error: {e}"))?;
}
ra.clear();
rb.clear();
na = read_context(a, &mut ra, name1)?;
nb = read_context(b, &mut rb, name2)?;
n3 += 1;
},
}
if (c1.has_error || c2.has_error) && !explicit {
delayed_error = true;
}
}
if opts.get_flag(options::TOTAL) {
let ending = LineEnding::from_zero_flag(opts.get_flag(options::ZERO_TERMINATED));
write!(writer, "{n1}{delim}{n2}{delim}{n3}{delim}total{ending}")
.map_err(|e| format!("write error: {e}"))?;
}
writer.flush().map_err(|e| format!("write error: {e}"))?;
if should_check && (c1.has_error || c2.has_error) {
if delayed_error {
let _ = writeln!(stderr, "comm: input is not in sorted order");
}
Ok(false)
} else {
Ok(true)
}
}
fn open_file<'a>(
name: &OsStr,
resolved: &Path,
stdin: Option<&'a mut dyn Read>,
ending: LineEnding,
) -> io::Result<LineReader<'a>> {
if name == "-" {
return Ok(LineReader::new(Box::new(BufReader::new(stdin.expect("stdin operand"))), ending));
}
if fs::metadata(resolved)?.is_dir() {
return Err(io::Error::other("is a directory"));
}
Ok(LineReader::new(Box::new(BufReader::new(File::open(resolved)?)), ending))
}
/// Parsed `comm` invocation.
pub(crate) struct Comm {
matches: ArgMatches,
}
matches_parser!(Comm, app);
impl Utility for Comm {
const NAME: &'static str = "comm";
fn run(self, host: &mut Host) -> i32 {
let name1 = self.matches.get_one::<OsString>(options::FILE_1).unwrap();
let name2 = self.matches.get_one::<OsString>(options::FILE_2).unwrap();
if name1 == "-" && name2 == "-" {
host.error("standard input is specified twice", 1);
return 1;
}
let path1 = host.resolve(name1);
let path2 = host.resolve(name2);
let delimiters: Vec<_> = self
.matches
.get_many::<String>(options::DELIMITER)
.unwrap()
.collect();
if delimiters[1..].iter().any(|d| *d != delimiters[0]) {
host.error("multiple conflicting output delimiters specified", 1);
return 1;
}
let delim = if delimiters[0].is_empty() { "\0" } else { delimiters[0] };
let identical = if name1 == "-" || name2 == "-" {
false
} else {
files_identical(&path1, &path2).unwrap_or(false)
};
let ending = LineEnding::from_zero_flag(self.matches.get_flag(options::ZERO_TERMINATED));
let opened: Result<_, (&OsStr, io::Error)> = if name1 == "-" {
open_file(name2, &path2, None, ending)
.map_err(|e| (name2.as_os_str(), e))
.and_then(|f2| {
open_file(name1, &path1, Some(&mut host.stdin), ending)
.map(|f1| (f1, f2))
.map_err(|e| (name1.as_os_str(), e))
})
} else if name2 == "-" {
open_file(name1, &path1, None, ending)
.map_err(|e| (name1.as_os_str(), e))
.and_then(|f1| {
open_file(name2, &path2, Some(&mut host.stdin), ending)
.map(|f2| (f1, f2))
.map_err(|e| (name2.as_os_str(), e))
})
} else {
open_file(name1, &path1, None, ending)
.map_err(|e| (name1.as_os_str(), e))
.and_then(|f1| {
open_file(name2, &path2, None, ending)
.map(|f2| (f1, f2))
.map_err(|e| (name2.as_os_str(), e))
})
};
let (mut f1, mut f2) = match opened {
Ok(files) => files,
Err((name, e)) => {
let _ = writeln!(host.stderr, "comm: {}: {e}", name.maybe_quote());
return 1;
},
};
match compare(
&mut f1,
&mut f2,
name1,
name2,
delim,
&self.matches,
identical,
&mut host.stdout,
&mut host.stderr,
) {
Ok(true) => 0,
Ok(false) => 1,
Err(e) => {
let _ = writeln!(host.stderr, "comm: {e}");
1
},
}
}
}
fn app() -> Command {
Command::new(Comm::NAME)
.version("0.8.0")
.about("Compare sorted files FILE1 and FILE2 line by line.")
.override_usage(format_usage("comm [OPTION]... FILE1 FILE2"))
.infer_long_args(true)
.args_override_self(true)
.arg(Arg::new(options::COLUMN_1).short('1').help("suppress column 1 (lines unique to FILE1)").action(ArgAction::SetTrue))
.arg(Arg::new(options::COLUMN_2).short('2').help("suppress column 2 (lines unique to FILE2)").action(ArgAction::SetTrue))
.arg(Arg::new(options::COLUMN_3).short('3').help("suppress column 3 (lines that appear in both files)").action(ArgAction::SetTrue))
.arg(Arg::new(options::DELIMITER).long(options::DELIMITER).help("separate columns with STR").value_name("STR").default_value("\t").allow_hyphen_values(true).action(ArgAction::Append).hide_default_value(true))
.arg(Arg::new(options::ZERO_TERMINATED).long(options::ZERO_TERMINATED).short('z').overrides_with(options::ZERO_TERMINATED).help("line delimiter is NUL, not newline").action(ArgAction::SetTrue))
.arg(Arg::new(options::FILE_1).required(true).value_hint(clap::ValueHint::FilePath).value_parser(clap::value_parser!(OsString)))
.arg(Arg::new(options::FILE_2).required(true).value_hint(clap::ValueHint::FilePath).value_parser(clap::value_parser!(OsString)))
.arg(Arg::new(options::TOTAL).long(options::TOTAL).help("output a summary").action(ArgAction::SetTrue))
.arg(Arg::new(options::CHECK_ORDER).long(options::CHECK_ORDER).help("check that input is correctly sorted, even if all input lines are pairable").action(ArgAction::SetTrue))
.arg(Arg::new(options::NO_CHECK_ORDER).long(options::NO_CHECK_ORDER).help("do not check that input is correctly sorted").action(ArgAction::SetTrue).conflicts_with(options::CHECK_ORDER))
}
/// Creates the `comm` builtin registration.
pub(crate) fn comm_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Comm, SE>()
}
#[cfg(test)]
mod tests {
use std::fs;
use super::Comm;
use crate::host::run_util;
#[test]
fn compares_three_columns_and_resolves_relative_paths() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a"), "a\nb\nd\n").unwrap();
fs::write(dir.path().join("b"), "b\nc\nd\n").unwrap();
let (code, capture) = run_util::<Comm>(&["a", "b"], "", dir.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "a\n\t\tb\n\tc\n\t\td\n");
}
#[test]
fn accepts_one_stdin_operand_and_column_suppression() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("b"), "b\nc\n").unwrap();
let (code, capture) = run_util::<Comm>(&["-1", "-", "b"], "a\nb\n", dir.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "\tb\nc\n");
}
#[test]
fn rejects_stdin_twice() {
let (code, capture) = run_util::<Comm>(&["-", "-"], "", "/");
assert_eq!(code, 1);
assert_eq!(capture.err(), "comm: standard input is specified twice\n");
}
#[test]
fn supports_delimiter_total_and_zero_termination() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a"), b"a\0b\0").unwrap();
fs::write(dir.path().join("b"), b"b\0c\0").unwrap();
let (code, capture) = run_util::<Comm>(
&["-z", "--output-delimiter=|", "--total", "a", "b"],
"",
dir.path(),
);
assert_eq!(code, 0);
assert_eq!(capture.stdout(), b"a\0||b\0|c\01|1|1|total\0");
}
#[test]
fn check_order_reports_unsorted_input() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a"), "b\na\n").unwrap();
fs::write(dir.path().join("b"), "b\na\n").unwrap();
let (code, capture) = run_util::<Comm>(&["--check-order", "a", "b"], "", dir.path());
assert_eq!(code, 1);
assert!(capture.err().contains("file 1 is not in sorted order"));
}
#[test]
fn nocheck_order_accepts_unsorted_input() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a"), "b\na\n").unwrap();
fs::write(dir.path().join("b"), "b\na\n").unwrap();
let (code, _) = run_util::<Comm>(&["--nocheck-order", "a", "b"], "", dir.path());
assert_eq!(code, 0);
}
}
crates/vendor/brush-builtins/src/command.rs → crates/pi-builtins/src/command.rs
View File
crates/vendor/brush-builtins/src/complete.rs → crates/pi-builtins/src/complete.rs
View File
crates/vendor/brush-builtins/src/continue_.rs → crates/pi-builtins/src/continue_.rs
View File
crates/vendor/uu-cut/src/cut.rs → crates/pi-builtins/src/cut.rs
+458 -200
View File
@@ -1,33 +1,330 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
// spell-checker:ignore (ToDO) delim sourcefiles undelimited
//! `cut` builtin: print selected byte, character, or field columns.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
ffi::OsString,
fs::File,
io::{BufRead, BufReader, BufWriter, Read, Write},
path::Path,
io::{self, BufRead, BufReader, BufWriter, Read, Write},
};
use bstr::io::BufReadExt;
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command, builder::ValueParser};
use uucore::{display::Quotable, line_ending::LineEnding, ranges::Range};
use crate::host::{Host, Utility, format_usage, matches_parser, os_bytes, util};
/// Parsed `cut` invocation.
pub(crate) struct Cut {
matches: ArgMatches,
}
matches_parser!(Cut, app);
impl Utility for Cut {
const NAME: &'static str = "cut";
fn rewrite_argv(argv: Vec<OsString>) -> Result<Vec<OsString>, String> {
// GNU cut accepts `-d=` as a delimiter spelling. Clap otherwise parses it
// as an empty value assigned to `-d`.
Ok(argv
.into_iter()
.map(|arg| if arg == "-d=" { OsString::from("--delimiter==") } else { arg })
.collect())
}
fn run(self, host: &mut Host) -> i32 {
if let Err(error) = cut_main(&self.matches, host) {
host.error(error, 1);
return 1;
}
host.exit_code()
}
}
mod matcher {
use memchr::{memchr, memchr2};
// Find the next matching byte sequence positions
// Return (first, last) where haystack[first..last] corresponds to the matched
// pattern
pub trait Matcher {
fn next_match(&self, haystack: &[u8]) -> Option<(usize, usize)>;
}
// Matches for the exact byte sequence pattern
pub struct ExactMatcher<'a> {
needle: &'a [u8],
}
impl<'a> ExactMatcher<'a> {
pub fn new(needle: &'a [u8]) -> Self {
assert!(!needle.is_empty());
Self { needle }
}
}
impl Matcher for ExactMatcher<'_> {
fn next_match(&self, haystack: &[u8]) -> Option<(usize, usize)> {
let mut pos = 0usize;
loop {
let match_idx = memchr(self.needle[0], &haystack[pos..])?;
let match_idx = match_idx + pos; // account for starting from pos
if self.needle.len() == 1 || haystack[match_idx + 1..].starts_with(&self.needle[1..]) {
return Some((match_idx, match_idx + self.needle.len()));
}
pos = match_idx + 1;
}
}
}
// Matches for any number of SPACE or TAB
pub struct WhitespaceMatcher {}
impl Matcher for WhitespaceMatcher {
fn next_match(&self, haystack: &[u8]) -> Option<(usize, usize)> {
let match_idx = memchr2(b' ', b'\t', haystack)?;
let mut skip = match_idx + 1;
while skip < haystack.len() {
match haystack[skip] {
b' ' | b'\t' => skip += 1,
_ => break,
}
}
Some((match_idx, skip))
}
}
#[cfg(test)]
mod matcher_tests {
use super::*;
#[test]
fn test_exact_matcher_single_byte() {
let matcher = ExactMatcher::new(":".as_bytes());
assert_eq!(matcher.next_match("".as_bytes()), None);
assert_eq!(matcher.next_match(":".as_bytes()), Some((0, 1)));
assert_eq!(matcher.next_match(":abcxyz".as_bytes()), Some((0, 1)));
assert_eq!(matcher.next_match("abc:xyz".as_bytes()), Some((3, 4)));
assert_eq!(matcher.next_match("abcxyz:".as_bytes()), Some((6, 7)));
assert_eq!(matcher.next_match("abcxyz".as_bytes()), None);
}
#[test]
fn test_exact_matcher_multi_bytes() {
let matcher = ExactMatcher::new("<>".as_bytes());
assert_eq!(matcher.next_match("".as_bytes()), None);
assert_eq!(matcher.next_match("<>".as_bytes()), Some((0, 2)));
assert_eq!(matcher.next_match("<>abcxyz".as_bytes()), Some((0, 2)));
assert_eq!(matcher.next_match("abc<>xyz".as_bytes()), Some((3, 5)));
assert_eq!(matcher.next_match("abcxyz<>".as_bytes()), Some((6, 8)));
assert_eq!(matcher.next_match("abcxyz".as_bytes()), None);
}
#[test]
fn test_whitespace_matcher_single_space() {
let matcher = WhitespaceMatcher {};
assert_eq!(matcher.next_match("".as_bytes()), None);
assert_eq!(matcher.next_match(" ".as_bytes()), Some((0, 1)));
assert_eq!(matcher.next_match("\tabcxyz".as_bytes()), Some((0, 1)));
assert_eq!(matcher.next_match("abc\txyz".as_bytes()), Some((3, 4)));
assert_eq!(matcher.next_match("abcxyz ".as_bytes()), Some((6, 7)));
assert_eq!(matcher.next_match("abcxyz".as_bytes()), None);
}
#[test]
fn test_whitespace_matcher_multi_spaces() {
let matcher = WhitespaceMatcher {};
assert_eq!(matcher.next_match("".as_bytes()), None);
assert_eq!(matcher.next_match(" \t ".as_bytes()), Some((0, 3)));
assert_eq!(matcher.next_match("\t\tabcxyz".as_bytes()), Some((0, 2)));
assert_eq!(matcher.next_match("abc \txyz".as_bytes()), Some((3, 5)));
assert_eq!(matcher.next_match("abcxyz ".as_bytes()), Some((6, 8)));
assert_eq!(matcher.next_match("abcxyz".as_bytes()), None);
}
}
}
mod searcher {
use super::matcher::Matcher;
// Generic searcher that relies on a specific matcher
pub struct Searcher<'a, 'b, M: Matcher> {
matcher: &'a M,
haystack: &'b [u8],
position: usize,
}
impl<'a, 'b, M: Matcher> Searcher<'a, 'b, M> {
pub fn new(matcher: &'a M, haystack: &'b [u8]) -> Self {
Self { matcher, haystack, position: 0 }
}
}
// Iterate over field delimiters
// Returns (first, last) positions of each sequence, where
// `haystack[first..last]` corresponds to the delimiter.
impl<M: Matcher> Iterator for Searcher<'_, '_, M> {
type Item = (usize, usize);
fn next(&mut self) -> Option<Self::Item> {
let (first, last) = self.matcher.next_match(&self.haystack[self.position..])?;
let result = (first + self.position, last + self.position);
self.position += last;
Some(result)
}
}
#[cfg(test)]
mod exact_searcher_tests {
use super::{super::matcher::ExactMatcher, *};
#[test]
fn test_normal() {
let matcher = ExactMatcher::new("a".as_bytes());
let iter = Searcher::new(&matcher, "a.a.a".as_bytes());
let items: Vec<(usize, usize)> = iter.collect();
assert_eq!(vec![(0, 1), (2, 3), (4, 5)], items);
}
#[test]
fn test_empty() {
let matcher = ExactMatcher::new("a".as_bytes());
let iter = Searcher::new(&matcher, "".as_bytes());
let items: Vec<(usize, usize)> = iter.collect();
assert!(items.is_empty());
}
fn test_multibyte(line: &[u8], expected: &[(usize, usize)]) {
let matcher = ExactMatcher::new("ab".as_bytes());
let iter = Searcher::new(&matcher, line);
let items: Vec<(usize, usize)> = iter.collect();
assert_eq!(expected, items);
}
#[test]
fn test_multibyte_normal() {
test_multibyte("...ab...ab...".as_bytes(), &[(3, 5), (8, 10)]);
}
#[test]
fn test_multibyte_needle_head_at_end() {
test_multibyte("a".as_bytes(), &[]);
}
#[test]
fn test_multibyte_starting_needle() {
test_multibyte("ab...ab...".as_bytes(), &[(0, 2), (5, 7)]);
}
#[test]
fn test_multibyte_trailing_needle() {
test_multibyte("...ab...ab".as_bytes(), &[(3, 5), (8, 10)]);
}
#[test]
fn test_multibyte_first_byte_false_match() {
test_multibyte("aA..aCaC..ab..aD".as_bytes(), &[(10, 12)]);
}
#[test]
fn test_searcher_with_exact_matcher() {
let matcher = ExactMatcher::new("<>".as_bytes());
let haystack = "<><>a<>b<><>cd<><>".as_bytes();
let mut searcher = Searcher::new(&matcher, haystack);
assert_eq!(searcher.next(), Some((0, 2)));
assert_eq!(searcher.next(), Some((2, 4)));
assert_eq!(searcher.next(), Some((5, 7)));
assert_eq!(searcher.next(), Some((8, 10)));
assert_eq!(searcher.next(), Some((10, 12)));
assert_eq!(searcher.next(), Some((14, 16)));
assert_eq!(searcher.next(), Some((16, 18)));
assert_eq!(searcher.next(), None);
assert_eq!(searcher.next(), None);
}
}
#[cfg(test)]
mod whitespace_searcher_tests {
use super::{super::matcher::WhitespaceMatcher, *};
#[test]
fn test_space() {
let matcher = WhitespaceMatcher {};
let iter = Searcher::new(&matcher, " . . ".as_bytes());
let items: Vec<(usize, usize)> = iter.collect();
assert_eq!(vec![(0, 1), (2, 3), (4, 5)], items);
}
#[test]
fn test_tab() {
let matcher = WhitespaceMatcher {};
let iter = Searcher::new(&matcher, "\t.\t.\t".as_bytes());
let items: Vec<(usize, usize)> = iter.collect();
assert_eq!(vec![(0, 1), (2, 3), (4, 5)], items);
}
#[test]
fn test_empty() {
let matcher = WhitespaceMatcher {};
let iter = Searcher::new(&matcher, "".as_bytes());
let items: Vec<(usize, usize)> = iter.collect();
assert!(items.is_empty());
}
fn test_multispace(line: &[u8], expected: &[(usize, usize)]) {
let matcher = WhitespaceMatcher {};
let iter = Searcher::new(&matcher, line);
let items: Vec<(usize, usize)> = iter.collect();
assert_eq!(expected, items);
}
#[test]
fn test_multispace_normal() {
test_multispace("... ... \t...\t ... \t ...".as_bytes(), &[
(3, 5),
(8, 10),
(13, 15),
(18, 21),
]);
}
#[test]
fn test_multispace_begin() {
test_multispace(" \t\t...".as_bytes(), &[(0, 3)]);
}
#[test]
fn test_multispace_end() {
test_multispace("...\t ".as_bytes(), &[(3, 6)]);
}
#[test]
fn test_searcher_with_whitespace_matcher() {
let matcher = WhitespaceMatcher {};
let haystack = "\t a b \t cd\t\t".as_bytes();
let mut searcher = Searcher::new(&matcher, haystack);
assert_eq!(searcher.next(), Some((0, 2)));
assert_eq!(searcher.next(), Some((3, 4)));
assert_eq!(searcher.next(), Some((5, 8)));
assert_eq!(searcher.next(), Some((10, 12)));
assert_eq!(searcher.next(), None);
assert_eq!(searcher.next(), None);
}
}
}
use matcher::{ExactMatcher, Matcher, WhitespaceMatcher};
use pi_uutils_ctx::format_usage;
use uucore::{
display::Quotable,
error::{FromIo, UResult, USimpleError},
line_ending::LineEnding,
os_str_as_bytes,
ranges::Range,
};
use self::searcher::Searcher;
mod matcher;
mod searcher;
use searcher::Searcher;
struct Options<'a> {
out_delimiter: Option<&'a [u8]>,
@@ -57,12 +354,6 @@ impl Default for Delimiter<'_> {
}
}
impl<'a> From<&'a OsString> for Delimiter<'a> {
fn from(s: &'a OsString) -> Self {
Self::Slice(os_str_as_bytes(s).unwrap())
}
}
fn list_to_ranges(list: &str, complement: bool) -> Result<Vec<Range>, String> {
if complement {
Range::from_list(list).map(|r| uucore::ranges::complement(&r))
@@ -76,7 +367,7 @@ fn cut_bytes<R: Read, W: Write>(
out: &mut W,
ranges: &[Range],
opts: &Options,
) -> UResult<()> {
) -> io::Result<()> {
let newline_char = opts.line_ending.into();
let mut buf_in = BufReader::new(reader);
let out_delim = opts.out_delimiter.unwrap_or(b"\t");
@@ -101,11 +392,7 @@ fn cut_bytes<R: Read, W: Write>(
Ok(true)
});
if let Err(e) = result {
return Err(USimpleError::new(1, e.to_string()));
}
Ok(())
result.map(|_| ())
}
/// Output delimiter is explicitly specified
@@ -117,7 +404,7 @@ fn cut_fields_explicit_out_delim<R: Read, W: Write, M: Matcher>(
only_delimited: bool,
newline_char: u8,
out_delim: &[u8],
) -> UResult<()> {
) -> io::Result<()> {
let mut buf_in = BufReader::new(reader);
let result = buf_in.for_byte_record_with_terminator(newline_char, |line| {
@@ -183,11 +470,7 @@ fn cut_fields_explicit_out_delim<R: Read, W: Write, M: Matcher>(
Ok(true)
});
if let Err(e) = result {
return Err(USimpleError::new(1, e.to_string()));
}
Ok(())
result.map(|_| ())
}
/// Output delimiter is the same as input delimiter
@@ -198,7 +481,7 @@ fn cut_fields_implicit_out_delim<R: Read, W: Write, M: Matcher>(
ranges: &[Range],
only_delimited: bool,
newline_char: u8,
) -> UResult<()> {
) -> io::Result<()> {
let mut buf_in = BufReader::new(reader);
let result = buf_in.for_byte_record_with_terminator(newline_char, |line| {
@@ -251,11 +534,7 @@ fn cut_fields_implicit_out_delim<R: Read, W: Write, M: Matcher>(
Ok(true)
});
if let Err(e) = result {
return Err(USimpleError::new(1, e.to_string()));
}
Ok(())
result.map(|_| ())
}
/// Streams and filters fields where the record terminator and
@@ -267,7 +546,7 @@ fn cut_fields_newline_char_delim<R: Read, W: Write>(
newline_char: u8,
out_delim: &[u8],
only_delimited: bool,
) -> UResult<()> {
) -> io::Result<()> {
let mut reader = BufReader::new(reader);
let mut line = Vec::new();
@@ -393,7 +672,7 @@ fn cut_fields<R: Read, W: Write>(
out: &mut W,
ranges: &[Range],
opts: &Options,
) -> UResult<()> {
) -> io::Result<()> {
let newline_char = opts.line_ending.into();
let field_opts = opts.field_opts.as_ref().unwrap(); // it is safe to unwrap() here - field_opts will always be Some() for cut_fields() call
match field_opts.delimiter {
@@ -445,102 +724,96 @@ fn cut_fields<R: Read, W: Write>(
}
}
// pi-uutils: route standard streams through the invocation context, and
// resolve each relative operand only when opening it while retaining the
// original operand for diagnostics.
fn cut_files<'a, I>(filenames: I, mode: &Mode)
fn cut_files<'a, I>(host: &mut Host, filenames: I, mode: &Mode)
where
I: IntoIterator<Item = &'a OsString>,
{
let inputs = filenames
.into_iter()
.map(|name| {
let path = if name == "-" { None } else { Some(host.resolve(name)) };
(name, path)
})
.collect::<Vec<_>>();
let mut stdin_read = false;
let mut out = BufWriter::new(pi_uutils_ctx::stdout());
let mut failed = false;
let mut out = BufWriter::new(&mut host.stdout);
for filename in filenames {
if filename == "-" {
if stdin_read {
continue;
}
let result = match mode {
Mode::Bytes(ranges, opts) | Mode::Characters(ranges, opts) => {
cut_bytes(pi_uutils_ctx::stdin(), &mut out, ranges, opts)
},
Mode::Fields(ranges, opts) => {
cut_fields(pi_uutils_ctx::stdin(), &mut out, ranges, opts)
},
};
if let Err(err) = result {
let _ = writeln!(pi_uutils_ctx::stderr(), "cut: {err}");
pi_uutils_ctx::set_exit_code(1);
}
stdin_read = true;
} else {
let result = File::open(pi_uutils_ctx::resolve(Path::new(filename)))
.map_err_context(|| filename.maybe_quote().to_string())
for (filename, path) in inputs {
let result = if let Some(path) = path {
File::open(path)
.map_err(|error| io::Error::new(error.kind(), format!("{}: {error}", filename.maybe_quote())))
.and_then(|file| match mode {
Mode::Bytes(ranges, opts) | Mode::Characters(ranges, opts) => {
cut_bytes(file, &mut out, ranges, opts)
},
Mode::Fields(ranges, opts) => cut_fields(file, &mut out, ranges, opts),
});
if let Err(err) = result {
let _ = writeln!(pi_uutils_ctx::stderr(), "cut: {err}");
pi_uutils_ctx::set_exit_code(1);
})
} else if stdin_read {
continue;
} else {
stdin_read = true;
match mode {
Mode::Bytes(ranges, opts) | Mode::Characters(ranges, opts) => {
cut_bytes(&mut host.stdin, &mut out, ranges, opts)
},
Mode::Fields(ranges, opts) => cut_fields(&mut host.stdin, &mut out, ranges, opts),
}
};
if let Err(error) = result {
let _ = writeln!(host.stderr, "cut: {error}");
failed = true;
}
}
if let Err(err) = out.flush().map_err_context(|| "write error".to_string()) {
let _ = writeln!(pi_uutils_ctx::stderr(), "cut: {err}");
pi_uutils_ctx::set_exit_code(1);
if let Err(error) = out.flush() {
let _ = writeln!(host.stderr, "cut: write error: {error}");
failed = true;
}
drop(out);
if failed {
host.fail(1);
}
}
/// Get delimiter and output delimiter from `-d`/`--delimiter` and
/// `--output-delimiter` options respectively Allow either delimiter to have a
/// value that is neither UTF-8 nor ASCII to align with GNU behavior
fn get_delimiters(matches: &ArgMatches) -> UResult<(Delimiter<'_>, Option<&[u8]>)> {
/// Gets input and output delimiters, accepting non-UTF-8 bytes like GNU `cut`.
fn get_delimiters(matches: &ArgMatches) -> Result<(Delimiter<'_>, Option<&[u8]>), String> {
let whitespace_delimited = matches.get_flag(options::WHITESPACE_DELIMITED);
let delim_opt = matches.get_one::<OsString>(options::DELIMITER);
let delim = match delim_opt {
Some(_) if whitespace_delimited => {
return Err(USimpleError::new(
1,
"invalid input: Only one of --delimiter (-d) or -w option can be specified",
));
return Err(
"invalid input: Only one of --delimiter (-d) or -w option can be specified".into(),
);
},
Some(os_string) => {
if os_string.is_empty() {
Delimiter::Slice(b"\0")
} else {
// For delimiter `-d` option value - allow both UTF-8 (possibly multi-byte)
// characters and Non UTF-8 (and not ASCII) single byte "characters", like
// `b"\xAD"` to align with GNU behavior
let bytes = os_str_as_bytes(os_string)?;
let bytes = os_bytes(os_string)
.ok_or_else(|| format!("invalid argument {}", os_string.maybe_quote()))?;
if os_string.to_str().is_some_and(|s| s.chars().count() > 1)
|| os_string.to_str().is_none() && bytes.len() > 1
{
return Err(USimpleError::new(1, "the delimiter must be a single character"));
return Err("the delimiter must be a single character".into());
}
Delimiter::from(os_string)
}
},
None => {
if whitespace_delimited {
Delimiter::Whitespace
} else {
Delimiter::default()
Delimiter::Slice(bytes)
}
},
None if whitespace_delimited => Delimiter::Whitespace,
None => Delimiter::default(),
};
let out_delim = matches
.get_one::<OsString>(options::OUTPUT_DELIMITER)
.map(|os_string| {
if os_string.is_empty() {
b"\0"
.map(|value| {
if value.is_empty() {
Ok(&b"\0"[..])
} else {
os_str_as_bytes(os_string).unwrap()
os_bytes(value)
.ok_or_else(|| format!("invalid argument {}", value.maybe_quote()))
}
});
})
.transpose()?;
Ok((delim, out_delim))
}
@@ -559,56 +832,14 @@ mod options {
pub const NOTHING: &str = "nothing";
}
// pi-uutils: replace the terminating uucore entry macro and localization-aware
// clap handler with an in-process entry point and literal English messages.
/// Run `cut` against the streams and working directory installed by
/// `pi-uutils-ctx`.
pub fn run(argv: Vec<OsString>) -> i32 {
// GNU cut accepts `-d=` as a delimiter spelling. Clap otherwise parses it
// as an empty value assigned to `-d`.
let argv = argv
.into_iter()
.map(|arg| {
if arg == "-d=" {
OsString::from("--delimiter==")
} else {
arg
}
})
.collect::<Vec<_>>();
let matches = match uu_app().try_get_matches_from(argv) {
Ok(matches) => matches,
Err(err) => {
let rendered = err.to_string();
if err.use_stderr() {
let _ = write!(pi_uutils_ctx::stderr(), "{rendered}");
return 1;
}
let _ = write!(pi_uutils_ctx::stdout(), "{rendered}");
return 0;
},
};
match cut_main(&matches) {
Ok(()) => pi_uutils_ctx::exit_code(),
Err(err) => {
let code = err.code();
let _ = writeln!(pi_uutils_ctx::stderr(), "cut: {err}");
if code == 0 { 1 } else { code }
},
}
}
fn cut_main(matches: &ArgMatches) -> UResult<()> {
fn cut_main(matches: &ArgMatches, host: &mut Host) -> Result<(), String> {
let complement = matches.get_flag(options::COMPLEMENT);
let only_delimited = matches.get_flag(options::ONLY_DELIMITED);
let (delimiter, out_delimiter) = get_delimiters(matches)?;
let line_ending = LineEnding::from_zero_flag(matches.get_flag(options::ZERO_TERMINATED));
// Only one, and only one of cutting mode arguments, i.e. `-b`, `-c`, `-f`,
// is expected. The number of those arguments is used for parsing a cutting
// mode and handling the error cases.
// is expected. Count occurrences because repeated modes are an error.
let mode_args_count = [
matches.indices_of(options::BYTES),
matches.indices_of(options::CHARACTERS),
@@ -624,72 +855,60 @@ fn cut_main(matches: &ArgMatches) -> UResult<()> {
matches.get_one::<String>(options::CHARACTERS),
matches.get_one::<String>(options::FIELDS),
) {
(1, Some(byte_ranges), None, None) => list_to_ranges(byte_ranges, complement).map(|ranges| {
(1, Some(ranges), None, None) => list_to_ranges(ranges, complement).map(|ranges| {
Mode::Bytes(ranges, Options { out_delimiter, line_ending, field_opts: None })
}),
(1, None, Some(char_ranges), None) => list_to_ranges(char_ranges, complement).map(|ranges| {
(1, None, Some(ranges), None) => list_to_ranges(ranges, complement).map(|ranges| {
Mode::Characters(ranges, Options { out_delimiter, line_ending, field_opts: None })
}),
(1, None, None, Some(field_ranges)) => {
list_to_ranges(field_ranges, complement).map(|ranges| {
Mode::Fields(ranges, Options {
out_delimiter,
line_ending,
field_opts: Some(FieldOptions { delimiter, only_delimited }),
})
(1, None, None, Some(ranges)) => list_to_ranges(ranges, complement).map(|ranges| {
Mode::Fields(ranges, Options {
out_delimiter,
line_ending,
field_opts: Some(FieldOptions { delimiter, only_delimited }),
})
},
}),
(2.., ..) => Err(
"invalid usage: expects no more than one of --fields (-f), --chars (-c) or --bytes (-b)"
.to_owned(),
),
_ => {
Err("invalid usage: expects one of --fields (-f), --chars (-c) or --bytes (-b)".to_owned())
},
_ => Err(
"invalid usage: expects one of --fields (-f), --chars (-c) or --bytes (-b)".to_owned(),
),
};
let mode_parse = match mode_parse {
Err(_) => mode_parse,
Ok(mode) => match mode {
Mode::Bytes(..) | Mode::Characters(..) if matches.contains_id(options::DELIMITER) => Err(
"invalid input: The '--delimiter' ('-d') option can only be used when printing a \
sequence of fields"
.to_owned(),
),
Mode::Bytes(..) | Mode::Characters(..)
if matches.get_flag(options::WHITESPACE_DELIMITED) =>
{
Err(
"invalid input: The '-w' option can only be used when printing a sequence of fields"
.to_owned(),
)
},
Mode::Bytes(..) | Mode::Characters(..) if matches.get_flag(options::ONLY_DELIMITED) => {
Err(
"invalid input: The '--only-delimited' ('-s') option can only be used when \
printing a sequence of fields"
.to_owned(),
)
},
_ => Ok(mode),
let mode = match mode_parse? {
Mode::Bytes(..) | Mode::Characters(..) if matches.contains_id(options::DELIMITER) => {
return Err(
"invalid input: The '--delimiter' ('-d') option can only be used when printing a sequence of fields"
.into(),
);
},
Mode::Bytes(..) | Mode::Characters(..)
if matches.get_flag(options::WHITESPACE_DELIMITED) =>
{
return Err(
"invalid input: The '-w' option can only be used when printing a sequence of fields".into(),
);
},
Mode::Bytes(..) | Mode::Characters(..) if matches.get_flag(options::ONLY_DELIMITED) => {
return Err(
"invalid input: The '--only-delimited' ('-s') option can only be used when printing a sequence of fields"
.into(),
);
},
mode => mode,
};
let mode = mode_parse.map_err(|e| USimpleError::new(1, e))?;
#[allow(clippy::unwrap_used, reason = "clap provides '-' by default")]
let files = matches.get_many::<OsString>(options::FILE).unwrap();
cut_files(files, &mode);
let files = matches
.get_many::<OsString>(options::FILE)
.expect("clap provides '-' by default");
cut_files(host, files, &mode);
Ok(())
}
pub fn uu_app() -> Command {
Command::new("cut")
.version(env!("CARGO_PKG_VERSION"))
fn app() -> Command {
Command::new(Cut::NAME)
.version("0.8.0")
.override_usage(format_usage("cut OPTION... [FILE]..."))
.about("Print specified byte or field columns from each line of stdin or input files")
.after_help(
@@ -794,3 +1013,42 @@ pub fn uu_app() -> Command {
.action(ArgAction::SetTrue),
)
}
/// Creates the `cut` builtin registration.
pub(crate) fn cut_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Cut, SE>()
}
#[cfg(test)]
mod integration_tests {
use super::Cut;
use crate::host::run_util;
#[test]
fn reads_default_operand_from_stdin() {
let (code, capture) = run_util::<Cut>(&["-f", "2", "-d", ":"], "left:right\n", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "right\n");
assert_eq!(capture.err(), "");
}
#[test]
fn resolves_file_operands_against_shell_cwd() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("input"), b"abcdef\n").unwrap();
let (code, capture) = run_util::<Cut>(&["-b", "2-4", "input"], "", dir.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "bcd\n");
assert_eq!(capture.err(), "");
}
#[test]
fn missing_mode_is_an_error() {
let (code, capture) = run_util::<Cut>(&[], "", "/");
assert_eq!(code, 1);
assert_eq!(
capture.err(),
"cut: invalid usage: expects one of --fields (-f), --chars (-c) or --bytes (-b)\n"
);
}
}
crates/pi-builtins/src/date.rs
+2052
View File
File diff suppressed because it is too large Load Diff
crates/vendor/brush-builtins/src/declare.rs → crates/pi-builtins/src/declare.rs
View File
crates/pi-builtins/src/diff.rs
+549
View File
@@ -0,0 +1,549 @@
//! `diff` builtin: compare files line by line using the `similar` library.
//!
//! Ported from `pi-uu-diff` 0.8.0.
use std::{
collections::BTreeSet,
ffi::{OsStr, OsString},
fs,
io::{Read, Write},
path::{Path, PathBuf},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{ArgAction, Parser};
use similar::TextDiff;
use crate::host::{Host, Utility, util};
/// Parsed `diff` invocation.
#[derive(Parser)]
#[command(
name = "diff",
version = "diff (pi-uu-diff) 0.8.0",
about = "Compare files line by line.",
override_usage = "diff [OPTION]... FILE1 FILE2",
infer_long_args = true
)]
pub(crate) struct Diff {
/// Output 3 lines of unified context (the default output format).
#[arg(short = 'u', action = ArgAction::SetTrue)]
_unified_flag: bool,
/// Output NUM lines of unified context.
#[arg(short = 'U', long = "unified", value_name = "NUM")]
unified: Option<usize>,
/// Report only when files differ.
#[arg(short = 'q', long = "brief", action = ArgAction::SetTrue)]
brief: bool,
/// Recursively compare subdirectories (always on for directories).
#[arg(short = 'r', long = "recursive", action = ArgAction::SetTrue)]
_recursive: bool,
/// Treat absent files as empty.
#[arg(short = 'N', long = "new-file", action = ArgAction::SetTrue)]
new_file: bool,
/// Use LABEL instead of a file name in a unified header.
#[arg(long = "label", value_name = "LABEL", action = ArgAction::Append)]
labels: Vec<OsString>,
/// Accepted for compatibility; output is never colorized.
#[arg(
long = "color",
value_name = "WHEN",
num_args = 0..=1,
require_equals = true,
default_missing_value = "auto"
)]
_color: Option<String>,
/// Files or directories to compare.
#[arg(required = true, num_args = 2, value_hint = clap::ValueHint::AnyPath)]
files: Vec<OsString>,
}
#[derive(Clone, Copy)]
struct Options<'a> {
context: usize,
brief: bool,
new_file: bool,
labels: &'a [OsString],
}
/// A classified operand and its resolved filesystem path.
enum Operand {
/// The builtin's standard input (`-`).
Stdin,
/// A regular (or other non-directory) file at the resolved path.
File(PathBuf),
/// A directory at the resolved path.
Dir(PathBuf),
/// A missing file tolerated by `-N` and compared as empty.
Absent,
}
impl Utility for Diff {
const NAME: &'static str = "diff";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
let opts = Options {
context: self.unified.unwrap_or(3),
brief: self.brief,
new_file: self.new_file,
labels: &self.labels,
};
match diff_main(&self.files, opts, host) {
Ok(code) => code,
Err(message) => {
host.error(message, 2);
2
},
}
}
}
fn diff_main(files: &[OsString], opts: Options<'_>, host: &mut Host) -> Result<i32, String> {
let (mut name_a, mut name_b) = (PathBuf::from(&files[0]), PathBuf::from(&files[1]));
let mut op_a = classify(&name_a, opts.new_file, host)?;
let mut op_b = classify(&name_b, opts.new_file, host)?;
// GNU: comparing a directory with a non-directory compares
// <dir>/<basename-of-other> with the other operand.
let a_is_dir = matches!(op_a, Operand::Dir(_));
let b_is_dir = matches!(op_b, Operand::Dir(_));
if a_is_dir != b_is_dir {
if matches!(op_a, Operand::Stdin) || matches!(op_b, Operand::Stdin) {
return Err("cannot compare '-' to a directory".to_string());
}
if a_is_dir {
name_a = descend(&name_a, &name_b)?;
op_a = classify(&name_a, opts.new_file, host)?;
} else {
name_b = descend(&name_b, &name_a)?;
op_b = classify(&name_b, opts.new_file, host)?;
}
}
let differed = if let (Operand::Dir(res_a), Operand::Dir(res_b)) = (&op_a, &op_b) {
diff_dirs(&name_a, res_a, &name_b, res_b, opts, host)?
} else {
let bytes_a = read_operand(&op_a, &name_a, host)?;
let bytes_b = read_operand(&op_b, &name_b, host)?;
diff_pair(&name_a, &bytes_a, &name_b, &bytes_b, opts, None, host)?
};
Ok(i32::from(differed))
}
/// Replaces a directory operand with `<dir>/<basename of other>` for the GNU
/// dir-vs-file comparison form.
fn descend(dir: &Path, other: &Path) -> Result<PathBuf, String> {
let base = other
.file_name()
.ok_or_else(|| format!("cannot compare {} to a directory", other.display()))?;
Ok(dir.join(base))
}
fn classify(name: &Path, new_file: bool, host: &Host) -> Result<Operand, String> {
if name.as_os_str() == OsStr::new("-") {
return Ok(Operand::Stdin);
}
// Keep `name` for diagnostics and headers; only filesystem access uses the
// path resolved against the shell working directory.
let resolved = host.resolve(name);
match fs::metadata(&resolved) {
Ok(meta) if meta.is_dir() => Ok(Operand::Dir(resolved)),
Ok(_) => Ok(Operand::File(resolved)),
Err(err) if err.kind() == std::io::ErrorKind::NotFound && new_file => Ok(Operand::Absent),
Err(err) => Err(format!("{}: {}", name.display(), io_msg(&err))),
}
}
fn read_operand(op: &Operand, name: &Path, host: &mut Host) -> Result<Vec<u8>, String> {
match op {
Operand::Stdin => {
let mut buf = Vec::new();
host.stdin
.read_to_end(&mut buf)
.map_err(|err| format!("-: {}", io_msg(&err)))?;
Ok(buf)
},
Operand::File(resolved) => {
fs::read(resolved).map_err(|err| format!("{}: {}", name.display(), io_msg(&err)))
},
Operand::Dir(_) => unreachable!("directories are handled by diff_dirs"),
Operand::Absent => Ok(Vec::new()),
}
}
/// Diffs one pair of already-read inputs. `prefix` is the `diff -r A/x B/x`
/// line emitted before per-pair output in directory mode.
fn diff_pair(
name_a: &Path,
bytes_a: &[u8],
name_b: &Path,
bytes_b: &[u8],
opts: Options<'_>,
prefix: Option<&str>,
host: &mut Host,
) -> Result<bool, String> {
if bytes_a == bytes_b {
return Ok(false);
}
let label_a = display_label(opts.labels.first(), name_a);
let label_b = display_label(opts.labels.get(1), name_b);
if opts.brief {
writeln!(host.stdout, "Files {label_a} and {label_b} differ").map_err(|e| io_msg(&e))?;
return Ok(true);
}
if is_binary(bytes_a) || is_binary(bytes_b) {
writeln!(host.stdout, "Binary files {label_a} and {label_b} differ")
.map_err(|e| io_msg(&e))?;
return Ok(true);
}
if let Some(line) = prefix {
writeln!(host.stdout, "{line}").map_err(|e| io_msg(&e))?;
}
let old = String::from_utf8_lossy(bytes_a);
let new = String::from_utf8_lossy(bytes_b);
let diff = TextDiff::from_lines(old.as_ref(), new.as_ref());
write!(
host.stdout,
"{}",
diff.unified_diff().context_radius(opts.context).header(&label_a, &label_b)
)
.map_err(|e| io_msg(&e))?;
Ok(true)
}
fn display_label(label: Option<&OsString>, name: &Path) -> String {
label.map_or_else(|| name.display().to_string(), |label| label.to_string_lossy().into_owned())
}
/// Recursively compares two directories over the sorted union of their entries,
/// GNU `diff -r` style.
fn diff_dirs(
name_a: &Path,
res_a: &Path,
name_b: &Path,
res_b: &Path,
opts: Options<'_>,
host: &mut Host,
) -> Result<bool, String> {
let mut names: BTreeSet<OsString> = BTreeSet::new();
for (dir_name, dir_res) in [(name_a, res_a), (name_b, res_b)] {
let entries = fs::read_dir(dir_res)
.map_err(|err| format!("{}: {}", dir_name.display(), io_msg(&err)))?;
for entry in entries {
let entry = entry.map_err(|err| format!("{}: {}", dir_name.display(), io_msg(&err)))?;
names.insert(entry.file_name());
}
}
let mut differed = false;
for name in names {
if host.is_cancelled() {
return Err("interrupted".to_string());
}
let (child_name_a, child_name_b) = (name_a.join(&name), name_b.join(&name));
// Resolve every recursively discovered display path through the host too;
// the process's current directory is unrelated to the shell's.
let child_res_a = host.resolve(&child_name_a);
let child_res_b = host.resolve(&child_name_b);
let meta_a = fs::metadata(&child_res_a).ok();
let meta_b = fs::metadata(&child_res_b).ok();
match (meta_a.as_ref(), meta_b.as_ref()) {
(Some(ma), Some(mb)) if ma.is_dir() && mb.is_dir() => {
differed |= diff_dirs(
&child_name_a,
&child_res_a,
&child_name_b,
&child_res_b,
opts,
host,
)?;
},
(Some(ma), Some(mb)) if ma.is_dir() != mb.is_dir() => {
let (dir, file) = if ma.is_dir() {
(&child_name_a, &child_name_b)
} else {
(&child_name_b, &child_name_a)
};
writeln!(
host.stdout,
"File {} is a directory while file {} is a regular file",
dir.display(),
file.display()
)
.map_err(|e| io_msg(&e))?;
differed = true;
},
(Some(_), Some(_)) => {
let bytes_a = fs::read(&child_res_a)
.map_err(|err| format!("{}: {}", child_name_a.display(), io_msg(&err)))?;
let bytes_b = fs::read(&child_res_b)
.map_err(|err| format!("{}: {}", child_name_b.display(), io_msg(&err)))?;
let prefix = format!("diff -r {} {}", child_name_a.display(), child_name_b.display());
differed |= diff_pair(
&child_name_a,
&bytes_a,
&child_name_b,
&bytes_b,
opts,
Some(&prefix),
host,
)?;
},
(Some(meta), None) | (None, Some(meta)) => {
let in_a = meta_b.is_none();
if opts.new_file && meta.is_file() {
let (present_name, present_res) = if in_a {
(&child_name_a, &child_res_a)
} else {
(&child_name_b, &child_res_b)
};
let bytes = fs::read(present_res)
.map_err(|err| format!("{}: {}", present_name.display(), io_msg(&err)))?;
let prefix =
format!("diff -r {} {}", child_name_a.display(), child_name_b.display());
let (ba, bb): (&[u8], &[u8]) = if in_a { (&bytes, &[]) } else { (&[], &bytes) };
differed |= diff_pair(
&child_name_a,
ba,
&child_name_b,
bb,
opts,
Some(&prefix),
host,
)?;
} else {
let present_dir = if in_a { name_a } else { name_b };
writeln!(
host.stdout,
"Only in {}: {}",
present_dir.display(),
Path::new(&name).display()
)
.map_err(|e| io_msg(&e))?;
differed = true;
}
},
(None, None) => {},
}
}
Ok(differed)
}
/// NUL byte within the first 8 KiB marks the input as binary, matching GNU
/// diff's heuristic for deciding between text and binary output.
fn is_binary(bytes: &[u8]) -> bool {
bytes.iter().take(8192).any(|&byte| byte == 0)
}
/// Renders an I/O error without Rust's ` (os error N)` suffix.
fn io_msg(err: &std::io::Error) -> String {
let msg = err.to_string();
match msg.find(" (os error") {
Some(idx) => msg[..idx].to_string(),
None => msg,
}
}
/// Creates the `diff` builtin registration.
pub(crate) fn diff_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Diff, SE>()
}
#[cfg(test)]
mod tests {
use std::{fs, path::Path};
use super::Diff;
use crate::host::run_util;
fn run_in(cwd: &Path, stdin: &str, args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Diff>(args, stdin, cwd);
(code, capture.out(), capture.err())
}
#[test]
fn identical_files_print_nothing_and_exit_zero() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "one\ntwo\n").unwrap();
fs::write(dir.path().join("b.txt"), "one\ntwo\n").unwrap();
assert_eq!(run_in(dir.path(), "", &["a.txt", "b.txt"]), (0, String::new(), String::new()));
}
#[test]
fn differing_files_emit_unified_diff_with_typed_headers() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "one\ntwo\nthree\n").unwrap();
fs::write(dir.path().join("b.txt"), "one\nTWO\nthree\n").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["a.txt", "b.txt"]);
assert_eq!(code, 1);
assert_eq!(stderr, "");
assert!(stdout.starts_with("--- a.txt\n+++ b.txt\n@@ "), "got: {stdout}");
assert!(stdout.contains("\n-two\n"), "got: {stdout}");
assert!(stdout.contains("\n+TWO\n"), "got: {stdout}");
assert!(stdout.contains("\n one\n"), "got: {stdout}");
assert!(stdout.contains("\n three\n"), "got: {stdout}");
}
#[test]
fn labels_override_typed_headers() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "old\n").unwrap();
fs::write(dir.path().join("b.txt"), "new\n").unwrap();
let (code, stdout, stderr) = run_in(
dir.path(),
"",
&["--label", "before", "--label", "after", "a.txt", "b.txt"],
);
assert_eq!(code, 1);
assert_eq!(stderr, "");
assert!(stdout.starts_with("--- before\n+++ after\n"), "got: {stdout}");
}
#[test]
fn unified_zero_drops_context_lines() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "one\ntwo\nthree\n").unwrap();
fs::write(dir.path().join("b.txt"), "one\nTWO\nthree\n").unwrap();
let (code, stdout, _) = run_in(dir.path(), "", &["-U", "0", "a.txt", "b.txt"]);
assert_eq!(code, 1);
assert!(!stdout.contains("\n one\n"), "got: {stdout}");
assert!(!stdout.contains("\n three\n"), "got: {stdout}");
assert!(stdout.contains("\n-two\n"), "got: {stdout}");
assert!(stdout.contains("\n+TWO\n"), "got: {stdout}");
}
#[test]
fn brief_reports_one_line_per_differing_pair() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "x\n").unwrap();
fs::write(dir.path().join("b.txt"), "y\n").unwrap();
assert_eq!(
run_in(dir.path(), "", &["-q", "a.txt", "b.txt"]),
(1, "Files a.txt and b.txt differ\n".to_string(), String::new())
);
}
#[test]
fn compatibility_flags_are_accepted_and_ignored() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "x\n").unwrap();
fs::write(dir.path().join("b.txt"), "y\n").unwrap();
let (code, stdout, stderr) =
run_in(dir.path(), "", &["-u", "-r", "--color=always", "a.txt", "b.txt"]);
assert_eq!(code, 1);
assert_eq!(stderr, "");
assert!(stdout.starts_with("--- a.txt\n+++ b.txt\n"), "got: {stdout}");
assert!(!stdout.contains('\u{1b}'), "got: {stdout}");
}
#[test]
fn binary_inputs_report_binary_difference() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.bin"), b"aa\x00bb").unwrap();
fs::write(dir.path().join("b.bin"), b"aa\x00cc").unwrap();
assert_eq!(
run_in(dir.path(), "", &["a.bin", "b.bin"]),
(1, "Binary files a.bin and b.bin differ\n".to_string(), String::new())
);
}
#[test]
fn missing_operand_file_is_trouble() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "x\n").unwrap();
assert_eq!(
run_in(dir.path(), "", &["a.txt", "nope.txt"]),
(2, String::new(), "diff: nope.txt: No such file or directory\n".to_string())
);
}
#[test]
fn missing_second_operand_is_usage_error() {
let dir = tempfile::tempdir().unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["only-one"]);
assert_eq!(code, 2);
assert_eq!(stdout, "");
assert!(stderr.contains("required"), "got: {stderr}");
}
#[test]
fn new_file_treats_missing_operand_as_empty() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "one\n").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["-N", "nope.txt", "a.txt"]);
assert_eq!(code, 1);
assert_eq!(stderr, "");
assert!(stdout.starts_with("--- nope.txt\n+++ a.txt\n"), "got: {stdout}");
assert!(stdout.contains("\n+one\n"), "got: {stdout}");
}
#[test]
fn dash_reads_builtin_stdin() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a.txt"), "one\ntwo\n").unwrap();
assert_eq!(
run_in(dir.path(), "one\ntwo\n", &["a.txt", "-"]),
(0, String::new(), String::new())
);
let (code, stdout, _) = run_in(dir.path(), "one\nTWO\n", &["a.txt", "-"]);
assert_eq!(code, 1);
assert!(stdout.starts_with("--- a.txt\n+++ -\n"), "got: {stdout}");
}
#[test]
fn directories_diff_recursively_with_only_in_lines() {
let dir = tempfile::tempdir().unwrap();
let (a, b) = (dir.path().join("a"), dir.path().join("b"));
fs::create_dir_all(a.join("sub")).unwrap();
fs::create_dir_all(b.join("sub")).unwrap();
fs::write(a.join("common.txt"), "same\n").unwrap();
fs::write(b.join("common.txt"), "same\n").unwrap();
fs::write(a.join("only.txt"), "left\n").unwrap();
fs::write(b.join("other.txt"), "right\n").unwrap();
fs::write(a.join("sub/inner.txt"), "old\n").unwrap();
fs::write(b.join("sub/inner.txt"), "new\n").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["a", "b"]);
assert_eq!(code, 1);
assert_eq!(stderr, "");
assert!(stdout.contains("Only in a: only.txt\n"), "got: {stdout}");
assert!(stdout.contains("Only in b: other.txt\n"), "got: {stdout}");
assert!(
stdout.contains(
"diff -r a/sub/inner.txt b/sub/inner.txt\n--- a/sub/inner.txt\n+++ b/sub/inner.txt\n"
),
"got: {stdout}"
);
assert!(stdout.contains("\n-old\n"), "got: {stdout}");
assert!(stdout.contains("\n+new\n"), "got: {stdout}");
assert!(!stdout.contains("common.txt"), "got: {stdout}");
}
#[test]
fn identical_directories_exit_zero() {
let dir = tempfile::tempdir().unwrap();
let (a, b) = (dir.path().join("a"), dir.path().join("b"));
fs::create_dir_all(&a).unwrap();
fs::create_dir_all(&b).unwrap();
fs::write(a.join("f.txt"), "same\n").unwrap();
fs::write(b.join("f.txt"), "same\n").unwrap();
assert_eq!(run_in(dir.path(), "", &["-r", "a", "b"]), (0, String::new(), String::new()));
}
#[test]
fn help_renders_to_builtin_stdout() {
let dir = tempfile::tempdir().unwrap();
let (code, capture) = run_util::<Diff>(&["--help"], "", dir.path());
assert_eq!(code, 0);
assert!(capture.out().contains("Usage:"));
assert!(capture.out().contains("Compare files line by line"));
assert_eq!(capture.err(), "");
}
}
crates/vendor/uu-dirname/src/dirname.rs → crates/pi-builtins/src/dirname.rs
+78 -122
View File
@@ -1,21 +1,69 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
// pi-uutils: modified for in-process embedding using pi-uutils-ctx streams.
//! `dirname` builtin: strip the last component from a file name.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{borrow::Cow, ffi::OsString, io::Write};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use pi_uutils_ctx::format_usage;
use uucore::error::{UResult, UUsageError};
use uucore::{display::Quotable, line_ending::LineEnding};
use crate::host::{Host, Utility, format_usage, matches_parser, os_bytes, util};
mod options {
pub const ZERO: &str = "zero";
pub const DIR: &str = "dir";
}
/// Parsed `dirname` invocation.
pub(crate) struct Dirname {
matches: ArgMatches,
}
matches_parser!(Dirname, app);
impl Utility for Dirname {
const NAME: &'static str = "dirname";
fn run(self, host: &mut Host) -> i32 {
let dirnames = self
.matches
.get_many::<OsString>(options::DIR)
.unwrap_or_default()
.collect::<Vec<_>>();
if dirnames.is_empty() {
host.error("missing operand", 1);
return 1;
}
let line_ending = LineEnding::from_zero_flag(self.matches.get_flag(options::ZERO));
for path in dirnames {
let Some(path_bytes) = os_bytes(path.as_os_str()) else {
host.error(
format!(
"invalid UTF-8 input {} encountered when converting to bytes on a platform that doesn't expose byte arguments",
path.quote()
),
1,
);
return 1;
};
let result = dirname_string_manipulation(path_bytes);
if host.stdout.write_all(&result).is_err()
|| write!(host.stdout, "{line_ending}").is_err()
{
return 1;
}
}
if host.stdout.flush().is_err() {
return 1;
}
0
}
}
/// Perform dirname as pure string manipulation per POSIX/GNU behavior.
///
/// dirname should NOT normalize paths. It does simple string manipulation:
@@ -96,67 +144,11 @@ fn dirname_string_manipulation(path_bytes: &[u8]) -> Cow<'_, [u8]> {
Cow::Borrowed(b".")
}
/// In-process builtin entry point. Unlike upstream's `uumain`, this parses the
/// arguments directly, renders clap help/usage/version to the context
/// streams, and maps the `UResult` to an exit code, so it is safe to run inside
/// the host shell process.
pub fn run(argv: Vec<OsString>) -> i32 {
let matches = match uu_app().try_get_matches_from(argv) {
Ok(matches) => matches,
Err(err) => {
let rendered = err.to_string();
if err.use_stderr() {
let _ = write!(pi_uutils_ctx::stderr(), "{rendered}");
return 1;
}
let _ = write!(pi_uutils_ctx::stdout(), "{rendered}");
return 0;
},
};
match dirname_main(&matches) {
Ok(()) => pi_uutils_ctx::exit_code(),
Err(err) => {
let code = err.code();
let _ = writeln!(pi_uutils_ctx::stderr(), "dirname: {err}");
if code == 0 { 1 } else { code }
},
}
}
fn dirname_main(matches: &ArgMatches) -> UResult<()> {
let dirnames: Vec<OsString> = matches
.get_many::<OsString>(options::DIR)
.unwrap_or_default()
.cloned()
.collect();
if dirnames.is_empty() {
return Err(UUsageError::new(1, "missing operand".to_string()));
}
let line_ending = if matches.get_flag(options::ZERO) {
b"\0" as &[u8]
} else {
b"\n" as &[u8]
};
let mut stdout = pi_uutils_ctx::stdout();
for path in &dirnames {
let path_bytes = uucore::os_str_as_bytes(path.as_os_str())?;
let result = dirname_string_manipulation(path_bytes);
stdout.write_all(&result)?;
stdout.write_all(line_ending)?;
}
Ok(())
}
pub fn uu_app() -> Command {
Command::new("dirname")
/// The `dirname` argument model.
fn app() -> Command {
Command::new(Dirname::NAME)
.about("Strip last component from file name")
.version(uucore::crate_version!())
.version("0.8.0")
.override_usage(format_usage("dirname [OPTION] NAME..."))
.args_override_self(true)
.infer_long_args(true)
@@ -180,60 +172,24 @@ pub fn uu_app() -> Command {
)
}
/// Creates the `dirname` builtin registration.
pub(crate) fn dirname_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Dirname, SE>()
}
#[cfg(test)]
mod tests {
use std::{collections::HashMap, path::PathBuf, sync::Arc};
use super::Dirname;
use crate::host::run_util;
use parking_lot::Mutex;
use pi_uutils_ctx::ScopeIo;
use super::*;
fn run_test(args: Vec<&str>) -> (i32, String, String) {
let stdout_buf = Arc::new(Mutex::new(Vec::new()));
let stderr_buf = Arc::new(Mutex::new(Vec::new()));
#[derive(Clone)]
struct SharedWriter {
buf: Arc<Mutex<Vec<u8>>>,
}
impl Write for SharedWriter {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
self.buf.lock().write(buf)
}
fn flush(&mut self) -> std::io::Result<()> {
self.buf.lock().flush()
}
}
let io = ScopeIo {
stdin: Box::new(std::io::empty()),
stdin_fd: None,
stdin_is_search_input: false,
stdout: Box::new(SharedWriter { buf: stdout_buf.clone() }),
stderr: Box::new(SharedWriter { buf: stderr_buf.clone() }),
cwd: PathBuf::from("."),
env: HashMap::new(),
cancel: Arc::new(std::sync::atomic::AtomicBool::new(false)),
};
let argv: Vec<OsString> = std::iter::once("dirname")
.chain(args)
.map(OsString::from)
.collect();
let code = pi_uutils_ctx::scope(io, || run(argv));
let out_str = String::from_utf8(stdout_buf.lock().clone()).unwrap();
let err_str = String::from_utf8(stderr_buf.lock().clone()).unwrap();
(code, out_str, err_str)
fn run_test(args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Dirname>(args, "", "/");
(code, capture.out(), capture.err())
}
#[test]
fn test_normal() {
let (code, stdout, stderr) = run_test(vec!["foo/bar"]);
let (code, stdout, stderr) = run_test(&["foo/bar"]);
assert_eq!(code, 0);
assert_eq!(stdout, "foo\n");
assert_eq!(stderr, "");
@@ -241,7 +197,7 @@ mod tests {
#[test]
fn test_trailing_slash() {
let (code, stdout, stderr) = run_test(vec!["foo/bar/"]);
let (code, stdout, stderr) = run_test(&["foo/bar/"]);
assert_eq!(code, 0);
assert_eq!(stdout, "foo\n");
assert_eq!(stderr, "");
@@ -249,7 +205,7 @@ mod tests {
#[test]
fn test_root() {
let (code, stdout, stderr) = run_test(vec!["/"]);
let (code, stdout, stderr) = run_test(&["/"]);
assert_eq!(code, 0);
assert_eq!(stdout, "/\n");
assert_eq!(stderr, "");
@@ -257,7 +213,7 @@ mod tests {
#[test]
fn test_multiple() {
let (code, stdout, stderr) = run_test(vec!["a/b", "c/d/e"]);
let (code, stdout, stderr) = run_test(&["a/b", "c/d/e"]);
assert_eq!(code, 0);
assert_eq!(stdout, "a\nc/d\n");
assert_eq!(stderr, "");
@@ -265,7 +221,7 @@ mod tests {
#[test]
fn test_zero_delimited() {
let (code, stdout, stderr) = run_test(vec!["-z", "a/b", "c/d/e"]);
let (code, stdout, stderr) = run_test(&["-z", "a/b", "c/d/e"]);
assert_eq!(code, 0);
assert_eq!(stdout, "a\0c/d\0");
assert_eq!(stderr, "");
@@ -273,7 +229,7 @@ mod tests {
#[test]
fn test_help() {
let (code, stdout, stderr) = run_test(vec!["--help"]);
let (code, stdout, stderr) = run_test(&["--help"]);
assert_eq!(code, 0);
assert!(stdout.contains("Usage:"));
assert!(stdout.contains("Strip last component"));
@@ -282,7 +238,7 @@ mod tests {
#[test]
fn test_invalid_arg() {
let (code, stdout, stderr) = run_test(vec!["--invalid-flag"]);
let (code, stdout, stderr) = run_test(&["--invalid-flag"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert!(stderr.contains("unexpected argument"));
@@ -290,7 +246,7 @@ mod tests {
#[test]
fn test_missing_operand() {
let (code, stdout, stderr) = run_test(vec![]);
let (code, stdout, stderr) = run_test(&[]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert!(stderr.contains("missing operand"));
crates/vendor/brush-builtins/src/dirs.rs → crates/pi-builtins/src/dirs.rs
View File
crates/vendor/brush-builtins/src/dot.rs → crates/pi-builtins/src/dot.rs
View File
crates/vendor/brush-builtins/src/echo.rs → crates/pi-builtins/src/echo.rs
View File
crates/vendor/brush-builtins/src/enable.rs → crates/pi-builtins/src/enable.rs
View File
crates/pi-shell/src/moreutils/errno.rs → crates/pi-builtins/src/errno.rs
+98 -127
View File
@@ -1,26 +1,23 @@
//! moreutils `errno` builtin: look up errno names, numbers, and descriptions.
//! `errno` builtin: look up errno names, numbers, and descriptions.
//!
//! Lookup semantics: a name argument prints its `NAME NUMBER Description`
//! line; an unknown name reports `errno: unknown errno NAME` on stderr and
//! exits 1. A numeric argument reverse-maps to the first-listed (canonical)
//! name; an unknown number prints nothing and exits 1. With multiple
//! arguments each is processed and the exit code is 1 if any lookup failed.
//! `--list` prints every table entry (aliases included) sorted by number
//! then name; `--search` prints entries whose description contains all the
//! given words case-insensitively.
//! Ported from pi-shell's in-process implementation of the moreutils-inspired
//! `errno` utility. A name prints its `NAME NUMBER Description` line; numeric
//! arguments reverse-map to the first-listed canonical name. Lists include
//! aliases and are sorted by number, then name.
use std::{ffi::OsString, io::Write};
use std::io::Write;
use clap::{Arg, ArgAction, Command};
use pi_uutils_ctx::format_usage;
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use crate::host::{Host, Utility, format_usage, matches_parser, util};
const OPT_LIST: &str = "list";
const OPT_SEARCH: &str = "search";
const ARG_QUERY: &str = "query";
/// Errno NAME -> number table. Duplicate numbers are allowed (aliases such
/// as `EWOULDBLOCK`); the first-listed name for a number is canonical for
/// reverse lookup.
/// Errno name-to-number table. Duplicate numbers are aliases; the first name
/// listed for a number is canonical for reverse lookup.
const ERRNOS: &[(&str, i32)] = &[
("EPERM", libc::EPERM),
("ENOENT", libc::ENOENT),
@@ -105,49 +102,47 @@ const ERRNOS: &[(&str, i32)] = &[
("EWOULDBLOCK", libc::EWOULDBLOCK),
];
/// Runs `errno` against invocation-scoped stdout/stderr.
pub fn run(argv: Vec<OsString>) -> i32 {
let matches = match command().try_get_matches_from(argv) {
Ok(matches) => matches,
Err(err) => {
let code = err.exit_code();
let rendered = err.to_string();
if err.use_stderr() {
let _ = write!(pi_uutils_ctx::stderr(), "{rendered}");
} else {
let _ = write!(pi_uutils_ctx::stdout(), "{rendered}");
}
return code;
},
};
let args: Vec<String> = matches
.get_many::<String>(ARG_QUERY)
.map(|values| values.cloned().collect())
.unwrap_or_default();
if matches.get_flag(OPT_LIST) {
return list_all();
}
if matches.get_flag(OPT_SEARCH) {
return search(&args);
}
if args.is_empty() {
let _ = writeln!(pi_uutils_ctx::stderr(), "errno: no errno name or number given");
return 1;
}
let mut failed = false;
for arg in &args {
if !lookup(arg) {
failed = true;
}
}
i32::from(failed)
/// Parsed `errno` invocation.
pub(crate) struct Errno {
matches: ArgMatches,
}
matches_parser!(Errno, command);
impl Utility for Errno {
const NAME: &'static str = "errno";
fn run(self, host: &mut Host) -> i32 {
let args: Vec<String> = self
.matches
.get_many::<String>(ARG_QUERY)
.map(|values| values.cloned().collect())
.unwrap_or_default();
if self.matches.get_flag(OPT_LIST) {
return list_all(host);
}
if self.matches.get_flag(OPT_SEARCH) {
return search(host, &args);
}
if args.is_empty() {
let _ = writeln!(host.stderr, "errno: no errno name or number given");
return 1;
}
let mut failed = false;
for arg in &args {
if !lookup(host, arg) {
failed = true;
}
}
i32::from(failed)
}
}
/// The `errno` argument model.
fn command() -> Command {
Command::new("errno")
Command::new(Errno::NAME)
.version(concat!("errno (pi-shell) ", env!("CARGO_PKG_VERSION")))
.about("Look up errno names and descriptions.")
.override_usage(format_usage("errno [-ls] [--] [name-or-number...]"))
@@ -177,7 +172,7 @@ fn command() -> Command {
)
}
/// Formats the OS description for an errno number, without std's
/// Formats the OS description for an errno number without std's
/// ` (os error N)` suffix.
fn description(number: i32) -> String {
let text = std::io::Error::from_raw_os_error(number).to_string();
@@ -187,17 +182,17 @@ fn description(number: i32) -> String {
}
}
fn print_entry(name: &str, number: i32) {
let _ = writeln!(pi_uutils_ctx::stdout(), "{name} {number} {}", description(number));
fn print_entry(host: &mut Host, name: &str, number: i32) {
let _ = writeln!(host.stdout, "{name} {number} {}", description(number));
}
/// Looks up one name or number argument; returns false on failure.
fn lookup(arg: &str) -> bool {
fn lookup(host: &mut Host, arg: &str) -> bool {
if let Ok(number) = arg.parse::<i32>() {
// Reverse lookup: first-listed name for the number is canonical.
match ERRNOS.iter().find(|(_, value)| *value == number) {
Some((name, value)) => {
print_entry(name, *value);
print_entry(host, name, *value);
true
},
None => false,
@@ -206,102 +201,71 @@ fn lookup(arg: &str) -> bool {
.iter()
.find(|(name, _)| name.eq_ignore_ascii_case(arg))
{
print_entry(name, *value);
print_entry(host, name, *value);
true
} else {
let _ = writeln!(pi_uutils_ctx::stderr(), "errno: unknown errno {arg}");
let _ = writeln!(host.stderr, "errno: unknown errno {arg}");
false
}
}
/// Prints every table entry (aliases included) sorted by number, then name.
fn list_all() -> i32 {
fn list_all(host: &mut Host) -> i32 {
let mut entries: Vec<(&str, i32)> = ERRNOS.to_vec();
entries.sort_unstable_by(|a, b| a.1.cmp(&b.1).then(a.0.cmp(b.0)));
for (name, number) in entries {
print_entry(name, number);
print_entry(host, name, number);
}
0
}
/// Prints entries whose description contains all words, case-insensitively.
fn search(words: &[String]) -> i32 {
/// Prints entries whose descriptions contain all words, case-insensitively.
fn search(host: &mut Host, words: &[String]) -> i32 {
let lowered: Vec<String> = words.iter().map(|word| word.to_lowercase()).collect();
let mut entries: Vec<(&str, i32)> = ERRNOS.to_vec();
entries.sort_unstable_by(|a, b| a.1.cmp(&b.1).then(a.0.cmp(b.0)));
for (name, number) in entries {
let text = description(number).to_lowercase();
if lowered.iter().all(|word| text.contains(word.as_str())) {
print_entry(name, number);
print_entry(host, name, number);
}
}
0
}
/// Creates the `errno` builtin registration.
pub(crate) fn errno_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Errno, SE>()
}
#[cfg(test)]
mod tests {
use std::{
collections::HashMap,
ffi::OsString,
io::{Cursor, Write},
sync::{Arc, atomic::AtomicBool},
};
use parking_lot::Mutex;
use pi_uutils_ctx::ScopeIo;
use super::run;
#[derive(Clone)]
struct SharedWriter(Arc<Mutex<Vec<u8>>>);
impl Write for SharedWriter {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
self.0.lock().write(buf)
}
fn flush(&mut self) -> std::io::Result<()> {
self.0.lock().flush()
}
}
use super::Errno;
use crate::host::run_util;
fn run_errno(args: &[&str]) -> (i32, String, String) {
let stdout = Arc::new(Mutex::new(Vec::new()));
let stderr = Arc::new(Mutex::new(Vec::new()));
let io = ScopeIo {
stdin: Box::new(Cursor::new(Vec::new())),
stdin_fd: None,
stdin_is_search_input: false,
stdout: Box::new(SharedWriter(Arc::clone(&stdout))),
stderr: Box::new(SharedWriter(Arc::clone(&stderr))),
cwd: std::env::temp_dir(),
env: HashMap::new(),
cancel: Arc::new(AtomicBool::new(false)),
};
let argv = std::iter::once("errno")
.chain(args.iter().copied())
.map(OsString::from)
.collect();
let code = pi_uutils_ctx::scope(io, || run(argv));
let stdout = String::from_utf8(stdout.lock().clone()).unwrap();
let stderr = String::from_utf8(stderr.lock().clone()).unwrap();
(code, stdout, stderr)
let (code, capture) = run_util::<Errno>(args, "", "/");
(code, capture.out(), capture.err())
}
#[test]
fn looks_up_name() {
let (code, stdout, stderr) = run_errno(&["ENOENT"]);
assert_eq!(code, 0);
assert!(stdout.starts_with("ENOENT 2 "), "stdout: {stdout:?}");
assert!(stdout.trim_end().len() > "ENOENT 2 ".len(), "missing description: {stdout:?}");
assert!(stdout.starts_with(&format!("ENOENT {} ", libc::ENOENT)), "stdout: {stdout:?}");
assert!(
stdout.trim_end().len() > format!("ENOENT {} ", libc::ENOENT).len(),
"missing description: {stdout:?}"
);
assert!(stderr.is_empty());
}
#[test]
fn reverse_lookup_by_number() {
let (code, stdout, _) = run_errno(&["2"]);
let number = libc::ENOENT.to_string();
let (code, stdout, _) = run_errno(&[&number]);
assert_eq!(code, 0);
assert!(stdout.starts_with("ENOENT 2 "), "stdout: {stdout:?}");
assert!(stdout.starts_with(&format!("ENOENT {} ", libc::ENOENT)), "stdout: {stdout:?}");
}
#[test]
@@ -325,18 +289,15 @@ mod tests {
let (code, stdout, _) = run_errno(&["-l"]);
assert_eq!(code, 0);
let lines: Vec<&str> = stdout.lines().collect();
let eperm = lines.iter().position(|line| line.starts_with("EPERM 1 "));
let enoent = lines.iter().position(|line| line.starts_with("ENOENT 2 "));
let eperm = lines
.iter()
.position(|line| line.starts_with(&format!("EPERM {} ", libc::EPERM)));
let enoent = lines
.iter()
.position(|line| line.starts_with(&format!("ENOENT {} ", libc::ENOENT)));
assert!(eperm.is_some(), "EPERM missing from list");
assert!(enoent.is_some(), "ENOENT missing from list");
assert!(eperm.unwrap() < enoent.unwrap(), "list not number-sorted");
let numbers: Vec<i32> = lines
.iter()
.map(|line| line.split(' ').nth(1).unwrap().parse().unwrap())
.collect();
let mut sorted = numbers.clone();
sorted.sort_unstable();
assert_eq!(numbers, sorted, "list not sorted by number");
}
#[test]
@@ -344,7 +305,9 @@ mod tests {
let (code, stdout, _) = run_errno(&["-s", "No", "SUCH"]);
assert_eq!(code, 0);
assert!(
stdout.lines().any(|line| line.starts_with("ENOENT 2 ")),
stdout
.lines()
.any(|line| line.starts_with(&format!("ENOENT {} ", libc::ENOENT))),
"search missed ENOENT: {stdout:?}"
);
}
@@ -353,8 +316,16 @@ mod tests {
fn multiple_args_aggregate_exit_code() {
let (code, stdout, stderr) = run_errno(&["ENOENT", "ENOSUCHTHING", "EPERM"]);
assert_eq!(code, 1);
assert!(stdout.lines().any(|line| line.starts_with("ENOENT 2 ")));
assert!(stdout.lines().any(|line| line.starts_with("EPERM 1 ")));
assert!(
stdout
.lines()
.any(|line| line.starts_with(&format!("ENOENT {} ", libc::ENOENT)))
);
assert!(
stdout
.lines()
.any(|line| line.starts_with(&format!("EPERM {} ", libc::EPERM)))
);
assert!(stderr.contains("unknown errno ENOSUCHTHING"));
}
crates/vendor/brush-builtins/src/eval.rs → crates/pi-builtins/src/eval.rs
View File
crates/vendor/brush-builtins/src/exec.rs → crates/pi-builtins/src/exec.rs
View File
crates/vendor/brush-builtins/src/exit.rs → crates/pi-builtins/src/exit.rs
View File
crates/vendor/brush-builtins/src/export.rs → crates/pi-builtins/src/export.rs
View File
crates/vendor/brush-builtins/src/factory.rs → crates/pi-builtins/src/factory.rs
+172 -1
View File
@@ -90,7 +90,7 @@ pub fn default_builtins<SE: brush_core::ShellExtensions>(
m.insert("help".into(), builtin::<help::HelpCommand, SE>());
#[cfg(feature = "builtin.jobs")]
m.insert("jobs".into(), builtin::<jobs::JobsCommand, SE>());
#[cfg(all(feature = "builtin.kill", unix))]
#[cfg(all(feature = "builtin.kill", any(unix, windows)))]
m.insert("kill".into(), builtin::<kill::KillCommand, SE>());
#[cfg(feature = "builtin.declare")]
m.insert("local".into(), decl_builtin::<declare::DeclareCommand, SE>());
@@ -180,3 +180,174 @@ pub fn default_builtins<SE: brush_core::ShellExtensions>(
m
}
/// Returns every in-process command-line utility builtin, as
/// `(name, registration)` pairs.
///
/// These are kept out of [`default_builtins`] because they shadow real system
/// binaries: the embedding shell decides whether to install them (and may
/// withhold the destructive ones — `rm`, `mv`, `ln`).
#[allow(clippy::too_many_lines, reason = "one line per utility")]
pub fn utility_builtins<SE: brush_core::ShellExtensions>()
-> Vec<(&'static str, builtins::Registration<SE>)> {
#[allow(unused_mut, reason = "empty when no utility features are enabled")]
let mut m = Vec::<(&'static str, builtins::Registration<SE>)>::new();
#[cfg(feature = "util.b2sum")]
m.push(("b2sum", b2sum::b2sum_builtin::<SE>()));
#[cfg(feature = "util.base32")]
m.push(("base32", base32::base32_builtin::<SE>()));
#[cfg(feature = "util.base64")]
m.push(("base64", base64::base64_builtin::<SE>()));
#[cfg(feature = "util.basename")]
m.push(("basename", basename::basename_builtin::<SE>()));
#[cfg(feature = "util.cat")]
m.push(("cat", cat::cat_builtin::<SE>()));
#[cfg(feature = "util.cmp")]
m.push(("cmp", cmp::cmp_builtin::<SE>()));
#[cfg(feature = "util.comm")]
m.push(("comm", comm::comm_builtin::<SE>()));
#[cfg(feature = "util.combine")]
m.push(("combine", combine::combine_builtin::<SE>()));
#[cfg(feature = "util.cut")]
m.push(("cut", cut::cut_builtin::<SE>()));
#[cfg(feature = "util.date")]
m.push(("date", date::date_builtin::<SE>()));
#[cfg(feature = "util.diff")]
m.push(("diff", diff::diff_builtin::<SE>()));
#[cfg(feature = "util.dirname")]
m.push(("dirname", dirname::dirname_builtin::<SE>()));
#[cfg(all(feature = "util.errno", unix))]
m.push(("errno", errno::errno_builtin::<SE>()));
#[cfg(feature = "util.fd")]
m.push(("fd", fd::fd_builtin::<SE>()));
#[cfg(feature = "util.find")]
m.push(("find", find::find_builtin::<SE>()));
#[cfg(feature = "util.grep")]
m.push(("grep", grep::grep_builtin::<SE>()));
#[cfg(feature = "util.rg")]
m.push(("rg", rg::rg_builtin::<SE>()));
#[cfg(feature = "util.head")]
m.push(("head", head::head_builtin::<SE>()));
#[cfg(feature = "util.hostname")]
m.push(("hostname", hostname::hostname_builtin::<SE>()));
#[cfg(feature = "util.ifne")]
m.push(("ifne", ifne::ifne_builtin::<SE>()));
#[cfg(feature = "util.isutf8")]
m.push(("isutf8", isutf8::isutf8_builtin::<SE>()));
#[cfg(feature = "util.jq")]
m.push(("jq", jq::jq_builtin::<SE>()));
#[cfg(feature = "util.ln")]
m.push(("ln", ln::ln_builtin::<SE>()));
#[cfg(feature = "util.ls")]
m.push(("ls", ls::ls_builtin::<SE>()));
#[cfg(feature = "util.md5sum")]
m.push(("md5sum", md5sum::md5sum_builtin::<SE>()));
#[cfg(feature = "util.mkdir")]
m.push(("mkdir", mkdir::mkdir_builtin::<SE>()));
#[cfg(feature = "util.mktemp")]
m.push(("mktemp", mktemp::mktemp_builtin::<SE>()));
#[cfg(feature = "util.mv")]
m.push(("mv", mv::mv_builtin::<SE>()));
#[cfg(feature = "util.nproc")]
m.push(("nproc", nproc::nproc_builtin::<SE>()));
#[cfg(feature = "util.paste")]
m.push(("paste", paste::paste_builtin::<SE>()));
#[cfg(feature = "util.printenv")]
m.push(("printenv", printenv::printenv_builtin::<SE>()));
#[cfg(feature = "util.readlink")]
m.push(("readlink", readlink::readlink_builtin::<SE>()));
#[cfg(feature = "util.realpath")]
m.push(("realpath", realpath::realpath_builtin::<SE>()));
#[cfg(feature = "util.rm")]
m.push(("rm", rm::rm_builtin::<SE>()));
#[cfg(feature = "util.sed")]
m.push(("sed", sed::sed_builtin::<SE>()));
#[cfg(feature = "util.seq")]
m.push(("seq", seq::seq_builtin::<SE>()));
#[cfg(feature = "util.sha1sum")]
m.push(("sha1sum", sha1sum::sha1sum_builtin::<SE>()));
#[cfg(feature = "util.sha224sum")]
m.push(("sha224sum", sha224sum::sha224sum_builtin::<SE>()));
#[cfg(feature = "util.sha256sum")]
m.push(("sha256sum", sha256sum::sha256sum_builtin::<SE>()));
#[cfg(feature = "util.sha384sum")]
m.push(("sha384sum", sha384sum::sha384sum_builtin::<SE>()));
#[cfg(feature = "util.sha512sum")]
m.push(("sha512sum", sha512sum::sha512sum_builtin::<SE>()));
#[cfg(feature = "util.sort")]
m.push(("sort", sort::sort_builtin::<SE>()));
#[cfg(feature = "util.sponge")]
m.push(("sponge", sponge::sponge_builtin::<SE>()));
#[cfg(feature = "util.stat")]
m.push(("stat", stat::stat_builtin::<SE>()));
#[cfg(feature = "util.tac")]
m.push(("tac", tac::tac_builtin::<SE>()));
#[cfg(feature = "util.tail")]
m.push(("tail", tail::tail_builtin::<SE>()));
#[cfg(feature = "util.tee")]
m.push(("tee", tee::tee_builtin::<SE>()));
#[cfg(feature = "util.touch")]
m.push(("touch", touch::touch_builtin::<SE>()));
#[cfg(feature = "util.tr")]
m.push(("tr", tr::tr_builtin::<SE>()));
#[cfg(feature = "util.truncate")]
m.push(("truncate", truncate::truncate_builtin::<SE>()));
#[cfg(feature = "util.ts")]
m.push(("ts", ts::ts_builtin::<SE>()));
#[cfg(feature = "util.uname")]
m.push(("uname", uname::uname_builtin::<SE>()));
#[cfg(feature = "util.uniq")]
m.push(("uniq", uniq::uniq_builtin::<SE>()));
#[cfg(feature = "util.wc")]
m.push(("wc", wc::wc_builtin::<SE>()));
#[cfg(feature = "util.which")]
m.push(("which", which::which_builtin::<SE>()));
#[cfg(feature = "util.whoami")]
m.push(("whoami", whoami::whoami_builtin::<SE>()));
#[cfg(feature = "util.xargs")]
m.push(("xargs", xargs::xargs_builtin::<SE>()));
#[cfg(feature = "util.yes")]
m.push(("yes", yes::yes_builtin::<SE>()));
m
}
/// Returns the process-inspection and process-control builtins:
/// `pgrep`, `pkill`, `pidwait`, `ps`, `top`, `sleep`, `timeout`, and `nohup`.
///
/// Kept separate from [`default_builtins`] because they shadow real system
/// binaries, and separate from [`utility_builtins`] because the embedding shell
/// installs them unconditionally — they exist so a long-lived embedded shell can
/// inspect and control its own children without forking.
pub fn process_builtins<SE: brush_core::ShellExtensions>()
-> Vec<(&'static str, builtins::Registration<SE>)> {
#[allow(unused_mut, reason = "empty when no process features are enabled")]
let mut m = Vec::<(&'static str, builtins::Registration<SE>)>::new();
#[cfg(feature = "util.nohup")]
// `nohup` detaches its operand into a new session so a backgrounded server
// survives the shell's kill-on-drop teardown; the wrapper flag keeps the
// shell from treating it as the job itself.
m.push((
"nohup",
builtin::<nohup::NohupCommand, SE>().transparent_background_wrapper(),
));
#[cfg(feature = "util.pgrep")]
m.push(("pgrep", builtin::<pgrep::PgrepCommand, SE>()));
#[cfg(feature = "util.pidwait")]
m.push(("pidwait", builtin::<pidwait::PidwaitCommand, SE>()));
#[cfg(feature = "util.pkill")]
m.push(("pkill", builtin::<pkill::PkillCommand, SE>()));
#[cfg(feature = "util.ps")]
m.push(("ps", builtin::<ps::PsCommand, SE>()));
#[cfg(feature = "util.sleep")]
m.push(("sleep", builtin::<sleep::SleepCommand, SE>()));
#[cfg(feature = "util.timeout")]
m.push(("timeout", builtin::<timeout::TimeoutCommand, SE>()));
#[cfg(feature = "util.top")]
m.push(("top", builtin::<top::TopCommand, SE>()));
m
}
crates/vendor/brush-builtins/src/false_.rs → crates/pi-builtins/src/false_.rs
View File
crates/vendor/brush-builtins/src/fc.rs → crates/pi-builtins/src/fc.rs
View File
crates/pi-shell/src/fd.rs → crates/pi-builtins/src/fd.rs
+110 -242
View File
@@ -1,4 +1,6 @@
//! In-process `fd` builtin backed by `pi_walker`, `globset`, and `regex`.
//!
//! Relocated from pi-shell's native implementation.
use std::{
collections::HashMap,
@@ -13,19 +15,14 @@ use std::{
time::{Duration, SystemTime, UNIX_EPOCH},
};
use brush_core::{
Error,
builtins::{BoxFuture, ContentOptions, ContentType, Registration},
commands::{CommandArg, ExecutionContext},
extensions::ShellExtensions,
openfiles::{OpenFile, OpenFiles, null},
results::ExecutionResult,
};
use clap::{ArgAction, Parser, ValueEnum, error::ErrorKind};
use brush_core::{ShellExtensions, builtins::Registration, openfiles::OpenFile};
use clap::{ArgAction, Parser, ValueEnum};
use globset::{GlobBuilder, GlobMatcher};
use pi_walker::CollectedEntry;
use regex::{Regex, RegexBuilder};
use crate::host::{Host, Utility, util};
#[derive(Parser, Debug)]
#[command(
name = "fd",
@@ -552,119 +549,60 @@ struct SearchState {
had_error: bool,
}
/// Creates the `fd` shell builtin registration.
pub fn fd_builtin<SE: ShellExtensions>() -> Registration<SE> {
fn execute<SE: ShellExtensions>(
context: ExecutionContext<'_, SE>,
args: Vec<CommandArg>,
) -> BoxFuture<'_, Result<ExecutionResult, Error>> {
Box::pin(run_fd(context, args))
}
impl Utility for FdCli {
const NAME: &'static str = "fd";
const USAGE_ERROR: u8 = 2;
Registration {
execute_func: execute::<SE>,
content_func: fd_content,
disabled: false,
special_builtin: false,
declaration_builtin: false,
transparent_background_wrapper: false,
}
}
fn run(mut self, host: &mut Host) -> i32 {
let quiet = self.quiet;
let base_dir = self
.base_directory
.as_deref()
.map_or_else(|| host.cwd().to_path_buf(), |path| host.resolve(path));
self.ignore_files = self
.ignore_files
.iter()
.map(|path| {
if path.is_absolute() {
host.resolve(path)
} else {
host.resolve(base_dir.join(path))
}
})
.collect();
let cancelled = host.cancel_flag();
async fn run_fd<SE: ShellExtensions>(
context: ExecutionContext<'_, SE>,
args: Vec<CommandArg>,
) -> Result<ExecutionResult, Error> {
let stdout = context.try_fd(OpenFiles::STDOUT_FD);
let stderr = context.try_fd(OpenFiles::STDERR_FD);
let cwd = context.shell.working_dir().to_path_buf();
let cancel = context.cancel_token();
let argv: Vec<OsString> = args
.iter()
.map(|arg| OsString::from(arg.to_string()))
.collect();
drop(context);
let cancel_flag = Arc::new(AtomicBool::new(false));
let thread_flag = Arc::clone(&cancel_flag);
let mut handle = tokio::task::spawn_blocking(move || {
let mut stdout = stdout.unwrap_or_else(null_sink);
let mut stderr = stderr.unwrap_or_else(null_sink);
run_fd_sync(argv, cwd, &mut stdout, &mut stderr, &thread_flag)
});
let code = match cancel {
Some(token) => {
let token_check = token.clone();
tokio::select! {
biased;
() = token.cancelled() => {
cancel_flag.store(true, Ordering::Relaxed);
let _ = (&mut handle).await;
130
},
result = &mut handle => {
if token_check.is_cancelled() { 130 } else { result.unwrap_or(1) }
},
}
},
None => handle.await.unwrap_or(1),
};
Ok(ExecutionResult::new(exit_status(code)))
}
fn run_fd_sync(
argv: Vec<OsString>,
cwd: PathBuf,
stdout: &mut OpenFile,
stderr: &mut OpenFile,
cancelled: &AtomicBool,
) -> i32 {
let cli = match FdCli::try_parse_from(argv) {
Ok(cli) => cli,
Err(err) => {
let rendered = err.to_string();
return match err.kind() {
ErrorKind::DisplayHelp | ErrorKind::DisplayVersion => {
let _ = write!(stdout, "{rendered}");
0
},
_ => {
let _ = write!(stderr, "{rendered}");
match search(self, base_dir, host, &cancelled) {
Ok(state) => {
if state.had_error {
2
},
};
},
};
let quiet = cli.quiet;
match search(cli, cwd, stdout, stderr, cancelled) {
Ok(state) => {
if state.had_error {
} else if quiet {
i32::from(state.matches == 0)
} else {
0
}
},
// A closed downstream reader (`fd … | head`) surfaces as BrokenPipe on
// stdout writes. Real fd dies silently from SIGPIPE; mirror that with
// exit 141 (128+SIGPIPE) and no diagnostic.
Err(err) if err.kind() == io::ErrorKind::BrokenPipe => 141,
Err(err) => {
let _ = writeln!(host.stderr, "fd: {err}");
2
} else if quiet {
i32::from(state.matches == 0)
} else {
0
}
},
// A closed downstream reader (`fd … | head`) surfaces as BrokenPipe on
// stdout writes. Real fd dies silently from SIGPIPE; mirror that with
// exit 141 (128+SIGPIPE) and no diagnostic.
Err(err) if err.kind() == io::ErrorKind::BrokenPipe => 141,
Err(err) => {
let _ = writeln!(stderr, "fd: {err}");
2
},
},
}
}
}
/// Creates the `fd` builtin registration.
pub(crate) fn fd_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<FdCli, SE>()
}
fn search(
cli: FdCli,
cwd: PathBuf,
stdout: &mut OpenFile,
stderr: &mut OpenFile,
base_dir: PathBuf,
host: &mut Host,
cancelled: &AtomicBool,
) -> io::Result<SearchState> {
if cli.list_details || !cli.exec.is_empty() || !cli.exec_batch.is_empty() || cli.batch_size != 0
@@ -676,8 +614,7 @@ fn search(
}
let _ = (cli.color, cli.hyperlink, cli.strip_cwd_prefix);
let base_dir = resolve_path(&cwd, cli.base_directory.as_deref());
let search_paths = resolve_search_paths(&cli, &base_dir)?;
let search_paths = resolve_search_paths(&cli, &base_dir, host)?;
let absolute_roots = search_paths
.iter()
.filter(|path| path.original.is_absolute())
@@ -728,14 +665,20 @@ fn search(
prune: cli.prune,
};
if let Some(state) =
try_search_fast(&cli, &search_paths, &config, max_results, stdout, stderr, cancelled)?
{
if let Some(state) = try_search_fast(
&cli,
&search_paths,
&config,
max_results,
&mut host.stdout,
&mut host.stderr,
cancelled,
)? {
return Ok(state);
}
let use_gitignore = !(no_ignore(&cli) || no_ignore_vcs(&cli));
let mut out = BufWriter::new(stdout);
let mut out = BufWriter::new(&mut host.stdout);
let mut state = SearchState { matches: 0, had_error: false };
for search_path in &search_paths {
if cancelled.load(Ordering::Relaxed) || max_results.is_some_and(|max| state.matches >= max) {
@@ -988,16 +931,16 @@ fn matches_walker_type_filter(
true
}
/// Builds a walker heartbeat closure that observes the shell cancel flag.
/// Builds a walker heartbeat closure that observes the host cancel flag.
///
/// The shell wrapper (`run_fd`) flips `cancelled` when it sees the runtime
/// cancellation token fire, then awaits the blocking task. Without this
/// closure, `pi_walker`'s per-entry heartbeat never checks the flag and a
/// cancelled walk keeps traversing until the whole tree is collected.
/// The shared utility adapter flips `cancelled` when the shell cancellation
/// token fires, then awaits the blocking task. Without this closure,
/// `pi_walker`'s per-entry heartbeat never checks the flag and a cancelled walk
/// keeps traversing until the whole tree is collected.
/// Returning [`io::ErrorKind::Interrupted`] surfaces as
/// [`WalkError::Interrupted`], which the callers translate to a silent break —
/// the shell wrapper owns the user-visible exit code (130), so no `fd:`
/// diagnostic is emitted.
/// [`pi_walker::WalkError::Interrupted`], which the callers translate to a
/// silent break — the shared adapter owns the user-visible exit code (130), so
/// no `fd:` diagnostic is emitted.
///
/// Regression cover for #3949 (fd) and #3933 (grep/rg — same class of defect).
fn cancel_heartbeat(cancelled: &AtomicBool) -> impl Fn() -> io::Result<()> + Sync + '_ {
@@ -1193,7 +1136,11 @@ const fn owner_side_matches(side: OwnerSide, actual: u32) -> bool {
}
}
fn resolve_search_paths(cli: &FdCli, base_dir: &Path) -> io::Result<Vec<SearchPath>> {
fn resolve_search_paths(
cli: &FdCli,
base_dir: &Path,
host: &Host,
) -> io::Result<Vec<SearchPath>> {
if !cli.search_paths.is_empty() && !cli.paths.is_empty() {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
@@ -1211,9 +1158,9 @@ fn resolve_search_paths(cli: &FdCli, base_dir: &Path) -> io::Result<Vec<SearchPa
.into_iter()
.map(|original| {
let resolved = if original.is_absolute() {
original.clone()
host.resolve(&original)
} else {
base_dir.join(&original)
host.resolve(base_dir.join(&original))
};
SearchPath { original, resolved }
})
@@ -1225,13 +1172,6 @@ struct SearchPath {
resolved: PathBuf,
}
fn resolve_path(cwd: &Path, path: Option<&Path>) -> PathBuf {
match path {
Some(path) if path.is_absolute() => path.to_path_buf(),
Some(path) => cwd.join(path),
None => cwd.to_path_buf(),
}
}
const fn include_hidden(cli: &FdCli) -> bool {
(cli.hidden || cli.unrestricted > 0) && !cli.no_hidden
@@ -1625,91 +1565,35 @@ fn remove_extension(value: &str) -> String {
}
}
fn null_sink() -> OpenFile {
null().unwrap_or_else(|_| OpenFile::from(io::stdout()))
}
#[allow(
clippy::unnecessary_wraps,
reason = "signature must match brush's CommandContentFunc fn pointer"
)]
fn fd_content(
_name: &str,
_content_type: ContentType,
_options: &ContentOptions,
) -> Result<String, Error> {
Ok("fd: fd [OPTIONS] [pattern] [path]...\n".to_string())
}
fn exit_status(code: i32) -> u8 {
u8::try_from(code.clamp(0, 255)).unwrap_or(1)
}
#[cfg(test)]
mod tests {
use std::{
env, fs,
io::Read,
sync::atomic::{AtomicBool, AtomicUsize, Ordering},
time::{SystemTime, UNIX_EPOCH},
fs,
sync::atomic::AtomicBool,
};
use brush_core::openfiles::OpenFile;
use clap::Parser;
use tempfile::{Builder, TempDir};
use super::{FdCli, cancel_heartbeat, run_fd_sync, search};
static COUNTER: AtomicUsize = AtomicUsize::new(0);
use super::{FdCli, cancel_heartbeat};
use crate::host::{Host, Utility, run_util};
/// Build a fresh temp directory containing a single matchable file plus a
/// filler file, so the walker has more than one entry to iterate. Both the
/// walker-level regressions below and the positive-path search test assert
/// against this seed.
fn seeded_tree(tag: &str) -> std::path::PathBuf {
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map_or(0, |d| d.as_nanos());
let root = env::temp_dir().join(format!(
"pi-shell-fd-cancel-{tag}-{}-{}-{}",
std::process::id(),
nanos,
COUNTER.fetch_add(1, Ordering::Relaxed),
));
fs::create_dir_all(&root).expect("temp tree should be created");
fs::write(root.join("haystack.txt"), b"needle\n").expect("seed file should be written");
fs::write(root.join("filler.txt"), b"filler\n").expect("filler file should be written");
root
}
/// Returns `(capture_path, writable_handle)`. The path is a fresh temp file
/// scoped to this test invocation; the handle wraps that same file so writes
/// go through the `OpenFile::File` variant and can later be read back.
fn capture_file(kind: &str) -> (std::path::PathBuf, fs::File) {
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map_or(0, |d| d.as_nanos());
let path = env::temp_dir().join(format!(
"pi-shell-fd-cancel-{kind}-{}-{}-{}",
std::process::id(),
nanos,
COUNTER.fetch_add(1, Ordering::Relaxed),
));
let file = fs::OpenOptions::new()
.create(true)
.read(true)
.write(true)
.truncate(true)
.open(&path)
.expect("capture file should open");
(path, file)
}
fn read_all(path: &std::path::Path) -> Vec<u8> {
let mut file = fs::File::open(path).expect("open capture");
let mut buf = Vec::new();
file.read_to_end(&mut buf).expect("read capture");
let _ = fs::remove_file(path);
buf
fn seeded_tree(tag: &str) -> TempDir {
let tree = Builder::new()
.prefix(&format!("brush-fd-cancel-{tag}-"))
.tempdir()
.expect("temp tree should be created");
fs::write(tree.path().join("haystack.txt"), b"needle\n")
.expect("seed file should be written");
fs::write(tree.path().join("filler.txt"), b"filler\n")
.expect("filler file should be written");
tree
}
/// Build the same `WalkRequest` `search`/`try_search_fast` build for the
@@ -1742,14 +1626,13 @@ mod tests {
// observing the flag — the exact bug #3949 reports.
let tree = seeded_tree("collect");
let cancelled = AtomicBool::new(true);
let err = walk_request(&tree)
let err = walk_request(tree.path())
.collect_with_heartbeat(cancel_heartbeat(&cancelled))
.expect_err("walker must surface the cancel flag as an error");
assert!(
matches!(err, pi_walker::WalkError::Interrupted(_)),
"heartbeat interruption should surface as WalkError::Interrupted, got {err:?}"
);
let _ = fs::remove_dir_all(&tree);
}
#[test]
@@ -1761,7 +1644,7 @@ mod tests {
let tree = seeded_tree("stream");
let cancelled = AtomicBool::new(true);
let visited = std::cell::Cell::new(0_usize);
let result = walk_request(&tree).for_each_entry_with_heartbeat(
let result = walk_request(tree.path()).for_each_entry_with_heartbeat(
cancel_heartbeat(&cancelled),
|_entry| {
visited.set(visited.get() + 1);
@@ -1779,7 +1662,6 @@ mod tests {
0,
"visitor must not receive any entry once the heartbeat has aborted the walk",
);
let _ = fs::remove_dir_all(&tree);
}
#[test]
@@ -1789,26 +1671,16 @@ mod tests {
// and its walker call (the outer pre-loop guard passes with
// cancelled=false), then finds the seeded match.
let tree = seeded_tree("normal");
let path = tree.to_str().expect("utf8 path");
let cli = FdCli::try_parse_from(["fd", "haystack", path]).expect("argv");
let (stdout_capture, stdout_file) = capture_file("stdout-ok");
let (stderr_capture, stderr_file) = capture_file("stderr-ok");
let mut stdout = OpenFile::from(stdout_file);
let mut stderr = OpenFile::from(stderr_file);
let cancelled = AtomicBool::new(false);
let state = search(cli, tree.clone(), &mut stdout, &mut stderr, &cancelled)
.expect("uncancelled search should succeed");
drop(stdout);
drop(stderr);
assert_eq!(state.matches, 1, "seeded haystack.txt should match once");
let out = read_all(&stdout_capture);
let err = read_all(&stderr_capture);
let path = tree.path().to_str().expect("utf8 path");
let (code, capture) = run_util::<FdCli>(&["haystack", path], "", tree.path());
assert_eq!(code, 0);
assert!(
String::from_utf8_lossy(&out).contains("haystack.txt"),
"stdout should list the match: {out:?}"
capture.out().contains("haystack.txt"),
"stdout should list the match: {:?}",
capture.stdout()
);
assert!(err.is_empty(), "stderr should stay clean on success: {err:?}");
let _ = fs::remove_dir_all(&tree);
assert!(capture.err().is_empty(), "stderr should stay clean: {:?}", capture.stderr());
}
#[test]
@@ -1818,20 +1690,16 @@ mod tests {
// silently from SIGPIPE; the builtin must map BrokenPipe to exit 141
// with no stderr diagnostic.
let tree = seeded_tree("epipe");
let path = tree.path().to_str().expect("utf8 path");
let cli = FdCli::try_parse_from(["fd", "haystack", path]).expect("argv");
let (mut host, capture) = Host::for_test("fd", "", tree.path());
let (reader, writer) = std::io::pipe().expect("pipe");
drop(reader); // downstream reader (e.g. `head`) already exited
let mut stdout = OpenFile::from(writer);
let (stderr_capture, stderr_file) = capture_file("stderr-epipe");
let mut stderr = OpenFile::from(stderr_file);
let cancelled = AtomicBool::new(false);
let argv = ["fd", "haystack", tree.to_str().expect("utf8 path")]
.map(std::ffi::OsString::from)
.to_vec();
let code = run_fd_sync(argv, tree.clone(), &mut stdout, &mut stderr, &cancelled);
drop(stderr);
host.stdout = OpenFile::from(writer);
let code = cli.run(&mut host);
assert_eq!(code, 141, "BrokenPipe must map to 128+SIGPIPE");
let err = read_all(&stderr_capture);
assert!(err.is_empty(), "stderr must stay clean on a broken pipe: {err:?}");
let _ = fs::remove_dir_all(&tree);
assert!(capture.err().is_empty(), "stderr must stay clean on a broken pipe");
}
}
crates/vendor/brush-builtins/src/fg.rs → crates/pi-builtins/src/fg.rs
View File
crates/pi-builtins/src/find.rs
+5121
View File
File diff suppressed because it is too large Load Diff
crates/vendor/brush-builtins/src/getopts.rs → crates/pi-builtins/src/getopts.rs
View File
crates/pi-uu-grep/src/lib.rs → crates/pi-builtins/src/grep.rs
+204 -372
View File
@@ -1,16 +1,7 @@
//! `grep` implemented as an in-process shell builtin on top of the ripgrep
//! libraries (`grep-regex` for the matcher, `grep-searcher` for line scanning),
//! with directory recursion via `pi-walker` and `--include` filtering via
//! `globset`. All I/O and path resolution is routed through `pi-uutils-ctx` so
//! the builtin writes to the command's redirected file descriptors and resolves
//! relative paths against the shell's working directory.
//! `grep` builtin implemented on top of the ripgrep libraries.
//!
//! Entry point: [`run`]. It never calls `std::process::exit`; clap
//! help/usage/error output is rendered to the context streams and an exit code
//! is returned following the GNU convention (0 = matched, 1 = no match,
//! 2 = error).
//! Matching uses `grep-regex`/`grep-searcher`; recursive walks use `pi-walker`.
mod rg;
use std::{
borrow::Cow,
@@ -18,9 +9,9 @@ use std::{
fs::File,
io::{self, BufWriter, Read, Write},
path::{Path, PathBuf},
sync::LazyLock,
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{ArgMatches, CommandFactory, FromArgMatches, Parser, ValueEnum, parser::ValueSource};
use globset::{Glob, GlobMatcher};
use grep_matcher::{LineTerminator, Matcher};
@@ -29,16 +20,17 @@ use grep_regex::{RegexMatcher, RegexMatcherBuilder};
use grep_searcher::{
BinaryDetection, Searcher, SearcherBuilder, Sink, SinkContext, SinkFinish, SinkMatch,
};
pub use rg::run as run_rg;
use crate::host::{Host, Utility, util};
/// PCRE2 JIT toggle: `OMP_PCRE2_JIT=1` forces JIT on, `0`/`false` forces it
/// off. Unset, JIT stays on everywhere except macOS, where PCRE2's SLJIT
/// executable allocator can fault while compiling patterns (issue #7399).
pub(crate) static PCRE2_JIT_ENABLED: LazyLock<bool> =
LazyLock::new(|| match std::env::var("OMP_PCRE2_JIT") {
Ok(v) if !v.is_empty() => v != "0" && !v.eq_ignore_ascii_case("false"),
pub(crate) fn pcre2_jit_enabled(host: &Host) -> bool {
match host.var("OMP_PCRE2_JIT") {
Some(value) if !value.is_empty() => value != "0" && !value.eq_ignore_ascii_case("false"),
_ => !cfg!(target_os = "macos"),
});
}
}
#[derive(Parser, Debug)]
#[command(
@@ -49,7 +41,7 @@ pub(crate) static PCRE2_JIT_ENABLED: LazyLock<bool> =
disable_version_flag = true,
args_override_self = true
)]
struct Cli {
struct GrepArgs {
/// Use PATTERN for matching (may be repeated; all patterns are OR-ed).
#[arg(short = 'e', long = "regexp", value_name = "PATTERN")]
patterns: Vec<String>,
@@ -279,6 +271,30 @@ struct Cli {
args: Vec<OsString>,
}
/// Parsed GNU `grep` invocation, including option occurrence order.
pub(crate) struct Grep {
cli: GrepArgs,
matches: ArgMatches,
}
impl CommandFactory for Grep {
fn command() -> clap::Command { GrepArgs::command() }
fn command_for_update() -> clap::Command { GrepArgs::command_for_update() }
}
impl FromArgMatches for Grep {
fn from_arg_matches(matches: &ArgMatches) -> Result<Self, clap::Error> {
Ok(Self { cli: GrepArgs::from_arg_matches(matches)?, matches: matches.clone() })
}
fn update_from_arg_matches(&mut self, matches: &ArgMatches) -> Result<(), clap::Error> {
self.cli.update_from_arg_matches(matches)?;
self.matches = matches.clone();
Ok(())
}
}
impl Parser for Grep {}
#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
enum BinaryFiles {
Binary,
@@ -435,7 +451,7 @@ fn resolve_file_list_modes(matches: &ArgMatches) -> (bool, bool) {
}
}
fn resolve_context(cli: &Cli, matches: &ArgMatches) -> (usize, usize) {
fn resolve_context(cli: &GrepArgs, matches: &ArgMatches) -> (usize, usize) {
let mut events = Vec::with_capacity(3);
if let (Some(index), Some(value)) = (last_index(matches, "after_context"), cli.after_context) {
events.push((index, false, value));
@@ -461,7 +477,7 @@ fn resolve_context(cli: &Cli, matches: &ArgMatches) -> (usize, usize) {
(before, after)
}
fn resolve_group_separator(cli: &Cli, matches: &ArgMatches) -> Option<Vec<u8>> {
fn resolve_group_separator(cli: &GrepArgs, matches: &ArgMatches) -> Option<Vec<u8>> {
let mut selected = (0, Some(b"--".to_vec()));
if let Some(separator) = &cli.group_separator {
choose_latest(
@@ -474,7 +490,7 @@ fn resolve_group_separator(cli: &Cli, matches: &ArgMatches) -> Option<Vec<u8>> {
selected.1
}
fn resolve_directory_action(cli: &Cli, matches: &ArgMatches) -> DirectoryAction {
fn resolve_directory_action(cli: &GrepArgs, matches: &ArgMatches) -> DirectoryAction {
let mut selected = (0, DirectoryAction::Read);
choose_latest(&mut selected, last_index(matches, "recursive"), DirectoryAction::Recurse);
choose_latest(
@@ -488,7 +504,7 @@ fn resolve_directory_action(cli: &Cli, matches: &ArgMatches) -> DirectoryAction
selected.1
}
fn resolve_follow_links(cli: &Cli, matches: &ArgMatches) -> pi_walker::FollowLinks {
fn resolve_follow_links(cli: &GrepArgs, matches: &ArgMatches) -> pi_walker::FollowLinks {
let mut selected = (0, pi_walker::FollowLinks::Roots);
choose_latest(&mut selected, last_index(matches, "recursive"), pi_walker::FollowLinks::Roots);
choose_latest(
@@ -513,7 +529,7 @@ fn resolve_follow_links(cli: &Cli, matches: &ArgMatches) -> pi_walker::FollowLin
selected.1
}
fn resolve_binary_files(cli: &Cli, matches: &ArgMatches) -> BinaryFiles {
fn resolve_binary_files(cli: &GrepArgs, matches: &ArgMatches) -> BinaryFiles {
// Preserve the builtin's historical byte-transparent default. Explicit
// GNU/BSD binary controls opt into detection.
let mut selected = (0, BinaryFiles::Text);
@@ -530,7 +546,7 @@ fn resolve_binary_files(cli: &Cli, matches: &ArgMatches) -> BinaryFiles {
selected.1
}
fn resolve_max_count(cli: &Cli) -> Result<Option<u64>, String> {
fn resolve_max_count(cli: &GrepArgs) -> Result<Option<u64>, String> {
match cli.max_count {
None | Some(-1) => Ok(None),
Some(value) if value >= 0 => u64::try_from(value)
@@ -541,54 +557,27 @@ fn resolve_max_count(cli: &Cli) -> Result<Option<u64>, String> {
}
fn option_takes_next_value(arg: &str) -> bool {
matches!(
arg,
"-e"
| "-f" | "-m"
| "-A" | "-B"
| "-C" | "-D"
| "-d" | "--regexp"
| "--file"
| "--max-count"
| "--after-context"
| "--before-context"
| "--context"
| "--label"
| "--group-separator"
| "--binary-files"
| "--devices"
| "--directories"
| "--include"
| "--exclude"
| "--exclude-from"
| "--exclude-dir"
| "--include-dir"
)
matches!(arg, "-e" | "-f" | "-m" | "-A" | "-B" | "-C" | "-D" | "-d"
| "--regexp" | "--file" | "--max-count" | "--after-context" | "--before-context"
| "--context" | "--label" | "--group-separator" | "--binary-files" | "--devices"
| "--directories" | "--include" | "--exclude" | "--exclude-from" | "--exclude-dir"
| "--include-dir")
}
fn normalize_context_args(argv: Vec<OsString>) -> Vec<OsString> {
let mut normalized = Vec::with_capacity(argv.len());
let mut literal = false;
let mut value_pending = false;
for (index, arg) in argv.into_iter().enumerate() {
if index == 0 || literal || value_pending {
value_pending = false;
normalized.push(arg);
continue;
}
let Some(text) = arg.to_str() else {
normalized.push(arg);
continue;
};
if text == "--" {
literal = true;
normalized.push(arg);
continue;
}
let Some(text) = arg.to_str() else { normalized.push(arg); continue };
if text == "--" { literal = true; normalized.push(arg); continue }
if let Some(digits) = text.strip_prefix('-')
&& !digits.is_empty()
&& digits.bytes().all(|byte| byte.is_ascii_digit())
&& !digits.is_empty() && digits.bytes().all(|byte| byte.is_ascii_digit())
{
normalized.push(OsString::from(format!("--context={digits}")));
continue;
@@ -686,8 +675,9 @@ fn build_default_matcher<P: AsRef<str>>(
/// Compile all patterns using the last-selected matcher mode.
fn build_matcher(
host: &Host,
patterns: &[String],
cli: &Cli,
cli: &GrepArgs,
mode: MatchMode,
ignore_case: bool,
) -> Result<CompiledMatcher, String> {
@@ -699,7 +689,7 @@ fn build_matcher(
.whole_line(cli.line_regexp)
.utf(true)
.ucp(true)
.jit_if_available(*PCRE2_JIT_ENABLED);
.jit_if_available(pcre2_jit_enabled(host));
return builder
.build_many(patterns)
.map(CompiledMatcher::Pcre)
@@ -980,6 +970,7 @@ fn display_path_for_operand(operand: &OsStr, resolved: &Path, path: &Path) -> Pa
#[allow(clippy::too_many_arguments)]
fn search_file_path<M: Matcher, W: Write>(
host: &mut Host,
operand: &OsStr,
resolved: &Path,
path: &Path,
@@ -999,7 +990,7 @@ fn search_file_path<M: Matcher, W: Write>(
*had_error = true;
if !opts.no_messages {
let _ = writeln!(
pi_uutils_ctx::stderr(),
host.stderr,
"grep: {}: {error}",
display_path.to_string_lossy()
);
@@ -1012,7 +1003,7 @@ fn search_file_path<M: Matcher, W: Write>(
*had_error = true;
if !opts.no_messages {
let _ = writeln!(
pi_uutils_ctx::stderr(),
host.stderr,
"grep: {}: {error}",
display_path.to_string_lossy()
);
@@ -1043,6 +1034,7 @@ fn grep_walk_request(root: &Path, follow_links: pi_walker::FollowLinks) -> pi_wa
/// Recursively search a directory operand while pruning excluded directories.
#[allow(clippy::too_many_arguments)]
fn search_dir<M: Matcher, W: Write>(
host: &mut Host,
operand: &OsStr,
resolved: &Path,
matcher: &M,
@@ -1056,9 +1048,11 @@ fn search_dir<M: Matcher, W: Write>(
let request = grep_walk_request(resolved, follow_links);
let mut any = false;
let had_error_state = std::cell::Cell::new(*had_error);
let cancel = host.cancel_flag();
let mut walk_err = host.stderr_clone();
let walk = request.for_each_entry_with_heartbeat(
|| {
if pi_uutils_ctx::is_cancelled() {
if cancel.load(std::sync::atomic::Ordering::Relaxed) {
Err(io::Error::from(io::ErrorKind::Interrupted))
} else {
Ok::<(), io::Error>(())
@@ -1081,6 +1075,7 @@ fn search_dir<M: Matcher, W: Write>(
}
let mut entry_had_error = had_error_state.get();
let matched = search_file_path(
host,
operand,
resolved,
entry.absolute_path.as_ref(),
@@ -1103,7 +1098,7 @@ fn search_dir<M: Matcher, W: Write>(
if !opts.no_messages {
let display_path = display_path_for_operand(operand, resolved, error.path);
let _ = writeln!(
pi_uutils_ctx::stderr(),
walk_err,
"grep: {}: {}",
display_path.to_string_lossy(),
error.error
@@ -1115,7 +1110,7 @@ fn search_dir<M: Matcher, W: Write>(
*had_error |= had_error_state.get();
match walk {
Ok(pi_walker::WalkStatus::Complete | pi_walker::WalkStatus::Stopped) => any,
Err(pi_walker::WalkError::Interrupted(_)) if pi_uutils_ctx::is_cancelled() => {
Err(pi_walker::WalkError::Interrupted(_)) if host.is_cancelled() => {
// The shell wrapper owns the user-visible cancellation status.
*had_error = true;
any
@@ -1123,7 +1118,7 @@ fn search_dir<M: Matcher, W: Write>(
Err(pi_walker::WalkError::Interrupted(error)) => {
*had_error = true;
if !opts.no_messages {
let _ = writeln!(pi_uutils_ctx::stderr(), "grep: {error}");
let _ = writeln!(host.stderr, "grep: {error}");
}
any
},
@@ -1132,7 +1127,7 @@ fn search_dir<M: Matcher, W: Write>(
if !opts.no_messages {
let display_path = display_path_for_operand(operand, resolved, &path);
let _ = writeln!(
pi_uutils_ctx::stderr(),
host.stderr,
"grep: {}: {message}",
display_path.to_string_lossy()
);
@@ -1142,12 +1137,12 @@ fn search_dir<M: Matcher, W: Write>(
}
}
fn read_auxiliary_file(path: &OsStr) -> Result<Vec<u8>, String> {
fn read_auxiliary_file(host: &mut Host, path: &OsStr) -> Result<Vec<u8>, String> {
let mut bytes = Vec::new();
let result = if path == OsStr::new("-") {
pi_uutils_ctx::stdin().read_to_end(&mut bytes)
host.stdin.read_to_end(&mut bytes)
} else {
File::open(pi_uutils_ctx::resolve(path)).and_then(|mut file| file.read_to_end(&mut bytes))
File::open(host.resolve(path)).and_then(|mut file| file.read_to_end(&mut bytes))
};
result
.map(|_| bytes)
@@ -1164,7 +1159,7 @@ fn pattern_file_lines(bytes: &[u8]) -> Vec<String> {
.collect()
}
fn resolve_patterns(cli: &Cli) -> Result<(Vec<String>, Vec<OsString>), String> {
fn resolve_patterns(host: &mut Host, cli: &GrepArgs) -> Result<(Vec<String>, Vec<OsString>), String> {
let has_explicit_patterns = !cli.patterns.is_empty() || !cli.pattern_files.is_empty();
let mut patterns = Vec::new();
let mut files = Vec::new();
@@ -1174,7 +1169,7 @@ fn resolve_patterns(cli: &Cli) -> Result<(Vec<String>, Vec<OsString>), String> {
patterns.extend(pattern.split('\n').map(str::to_owned));
}
for path in &cli.pattern_files {
patterns.extend(pattern_file_lines(&read_auxiliary_file(path)?));
patterns.extend(pattern_file_lines(&read_auxiliary_file(host, path)?));
}
files.clone_from(&cli.args);
return Ok((patterns, files));
@@ -1190,7 +1185,8 @@ fn resolve_patterns(cli: &Cli) -> Result<(Vec<String>, Vec<OsString>), String> {
}
fn collect_rule_specs(
cli: &Cli,
host: &mut Host,
cli: &GrepArgs,
matches: &ArgMatches,
) -> Result<(Vec<RuleSpec>, Vec<RuleSpec>), String> {
let mut files = Vec::new();
@@ -1206,7 +1202,7 @@ fn collect_rule_specs(
}
if let Some(indices) = matches.indices_of("exclude_from") {
for (index, path) in indices.zip(&cli.exclude_from) {
for pattern in pattern_file_lines(&read_auxiliary_file(path)?) {
for pattern in pattern_file_lines(&read_auxiliary_file(host, path)?) {
files.push(RuleSpec { index, include: false, pattern });
}
}
@@ -1238,12 +1234,12 @@ fn compile_rules(mut specs: Vec<RuleSpec>) -> Result<Vec<PathRule>, String> {
.collect()
}
fn build_path_rules(cli: &Cli, matches: &ArgMatches) -> Result<PathRules, String> {
let (files, dirs) = collect_rule_specs(cli, matches)?;
fn build_path_rules(host: &mut Host, cli: &GrepArgs, matches: &ArgMatches) -> Result<PathRules, String> {
let (files, dirs) = collect_rule_specs(host, cli, matches)?;
Ok(PathRules { files: compile_rules(files)?, dirs: compile_rules(dirs)? })
}
fn build_searcher(cli: &Cli, opts: &Options, max_count: Option<u64>) -> Searcher {
fn build_searcher(cli: &GrepArgs, opts: &Options, max_count: Option<u64>) -> Searcher {
let binary_detection = if cli.null_data || opts.binary_files == BinaryFiles::Text {
BinaryDetection::none()
} else if opts.binary_files == BinaryFiles::WithoutMatch {
@@ -1267,7 +1263,8 @@ fn build_searcher(cli: &Cli, opts: &Options, max_count: Option<u64>) -> Searcher
#[allow(clippy::too_many_arguments)]
fn execute_search<M: Matcher>(
cli: &Cli,
host: &mut Host,
cli: &GrepArgs,
matcher: &M,
files: &[OsString],
directory_action: DirectoryAction,
@@ -1277,7 +1274,7 @@ fn execute_search<M: Matcher>(
max_count: Option<u64>,
) -> i32 {
let mut searcher = build_searcher(cli, opts, max_count);
let mut out = BufWriter::new(pi_uutils_ctx::stdout());
let mut out = BufWriter::new(host.stdout_clone());
let mut any_match = false;
let mut had_error = false;
let mut processed_operand = false;
@@ -1286,7 +1283,7 @@ fn execute_search<M: Matcher>(
if opts.quiet && any_match {
break;
}
if processed_operand && pi_uutils_ctx::is_cancelled() {
if processed_operand && host.is_cancelled() {
had_error = true;
break;
}
@@ -1301,7 +1298,7 @@ fn execute_search<M: Matcher>(
match process_reader(
matcher,
&mut searcher,
pi_uutils_ctx::stdin(),
&mut host.stdin,
display,
opts,
&mut out,
@@ -1310,23 +1307,24 @@ fn execute_search<M: Matcher>(
Err(error) => {
had_error = true;
if !opts.no_messages {
let _ = writeln!(pi_uutils_ctx::stderr(), "grep: (standard input): {error}");
let _ = writeln!(host.stderr, "grep: (standard input): {error}");
}
},
}
if pi_uutils_ctx::is_cancelled() {
if host.is_cancelled() {
had_error = true;
break;
}
continue;
}
let resolved = pi_uutils_ctx::resolve(operand);
let resolved = host.resolve(operand);
match std::fs::metadata(&resolved) {
Ok(metadata) if metadata.is_dir() => match directory_action {
DirectoryAction::Recurse => {
if rules.allows_dir(Path::new(operand))
&& search_dir(
host,
operand.as_os_str(),
&resolved,
matcher,
@@ -1344,7 +1342,7 @@ fn execute_search<M: Matcher>(
DirectoryAction::Read => {
had_error = true;
let _ = writeln!(
pi_uutils_ctx::stderr(),
host.stderr,
"grep: {}: Is a directory",
operand.to_string_lossy()
);
@@ -1358,6 +1356,7 @@ fn execute_search<M: Matcher>(
continue;
}
if search_file_path(
host,
operand.as_os_str(),
&resolved,
&resolved,
@@ -1374,11 +1373,11 @@ fn execute_search<M: Matcher>(
had_error = true;
if !opts.no_messages {
let _ =
writeln!(pi_uutils_ctx::stderr(), "grep: {}: {error}", operand.to_string_lossy());
writeln!(host.stderr, "grep: {}: {error}", operand.to_string_lossy());
}
},
}
if pi_uutils_ctx::is_cancelled() {
if host.is_cancelled() {
had_error = true;
break;
}
@@ -1402,32 +1401,22 @@ fn execute_search<M: Matcher>(
}
}
fn report_clap_error(error: clap::Error) -> i32 {
let rendered = error.to_string();
if error.use_stderr() {
let _ = write!(pi_uutils_ctx::stderr(), "{rendered}");
2
} else {
let _ = write!(pi_uutils_ctx::stdout(), "{rendered}");
0
impl Utility for Grep {
const NAME: &'static str = "grep";
const USAGE_ERROR: u8 = 2;
fn rewrite_argv(argv: Vec<OsString>) -> Result<Vec<OsString>, String> {
Ok(normalize_context_args(argv))
}
}
/// Runs the in-process grep builtin and returns a GNU-compatible exit code.
pub fn run(argv: Vec<OsString>) -> i32 {
let matches = match Cli::command().try_get_matches_from(normalize_context_args(argv)) {
Ok(matches) => matches,
Err(error) => return report_clap_error(error),
};
let cli = match Cli::from_arg_matches(&matches) {
Ok(cli) => cli,
Err(error) => return report_clap_error(error),
};
fn run(self, host: &mut Host) -> i32 {
let cli = self.cli;
let matches = self.matches;
let (mut patterns, mut files) = match resolve_patterns(&cli) {
let (mut patterns, mut files) = match resolve_patterns(host, &cli) {
Ok(resolved) => resolved,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "grep: {error}");
let _ = writeln!(host.stderr, "grep: {error}");
return 2;
},
};
@@ -1443,18 +1432,19 @@ pub fn run(argv: Vec<OsString>) -> i32 {
let max_count = match resolve_max_count(&cli) {
Ok(max_count) => max_count,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "grep: {error}");
let _ = writeln!(host.stderr, "grep: {error}");
return 2;
},
};
let rules = match build_path_rules(&cli, &matches) {
let rules = match build_path_rules(host, &cli, &matches) {
Ok(rules) => rules,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "grep: {error}");
let _ = writeln!(host.stderr, "grep: {error}");
return 2;
},
};
let matcher = match build_matcher(
host,
&patterns,
&cli,
resolve_match_mode(&matches),
@@ -1462,7 +1452,7 @@ pub fn run(argv: Vec<OsString>) -> i32 {
) {
Ok(matcher) => matcher,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "grep: {error}");
let _ = writeln!(host.stderr, "grep: {error}");
return 2;
},
};
@@ -1501,6 +1491,7 @@ pub fn run(argv: Vec<OsString>) -> i32 {
match matcher {
CompiledMatcher::Rust(matcher) => execute_search(
host,
&cli,
&matcher,
&files,
@@ -1511,6 +1502,7 @@ pub fn run(argv: Vec<OsString>) -> i32 {
max_count,
),
CompiledMatcher::Pcre(matcher) => execute_search(
host,
&cli,
&matcher,
&files,
@@ -1522,298 +1514,138 @@ pub fn run(argv: Vec<OsString>) -> i32 {
),
}
}
}
/// Creates the GNU `grep` builtin registration.
pub(crate) fn grep_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Grep, SE>()
}
#[cfg(test)]
mod tests {
use std::{
collections::HashMap,
io::Cursor,
sync::{Arc, atomic::AtomicBool},
};
use parking_lot::Mutex;
use pi_uutils_ctx::{ScopeIo, scope};
use super::*;
use crate::host::{Host, run_util};
/// Sink that collects writes into a shared buffer for assertions.
struct SharedBuf(Arc<Mutex<Vec<u8>>>);
impl Write for SharedBuf {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.0.lock().extend_from_slice(buf);
Ok(buf.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
/// Run the `grep` builtin with `args` (no argv[0]) over `stdin`, returning
/// `(exit_code, stdout, stderr)`.
fn run_grep(args: &[&str], stdin: &str) -> (i32, String, String) {
run_grep_in(args, stdin, &std::env::temp_dir())
}
fn run_grep_in(args: &[&str], stdin: &str, cwd: &Path) -> (i32, String, String) {
let out = Arc::new(Mutex::new(Vec::new()));
let err = Arc::new(Mutex::new(Vec::new()));
let io = ScopeIo {
stdin: Box::new(Cursor::new(stdin.as_bytes().to_vec())),
stdin_fd: None,
stdin_is_search_input: true,
stdout: Box::new(SharedBuf(Arc::clone(&out))),
stderr: Box::new(SharedBuf(Arc::clone(&err))),
cwd: cwd.to_path_buf(),
env: HashMap::new(),
cancel: Arc::new(AtomicBool::new(false)),
};
let argv: Vec<OsString> = std::iter::once("grep")
.chain(args.iter().copied())
.map(OsString::from)
.collect();
let code = scope(io, || run(argv));
let stdout = String::from_utf8(out.lock().clone()).expect("utf8 stdout");
let stderr = String::from_utf8(err.lock().clone()).expect("utf8 stderr");
(code, stdout, stderr)
}
fn unique_tree(label: &str) -> PathBuf {
let tree = std::env::temp_dir().join(format!(
"pi-uu-grep-{label}-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|duration| duration.as_nanos())
.unwrap_or(0)
));
std::fs::create_dir_all(&tree).expect("temp tree should be created");
tree
fn run(args: &[&str], stdin: &str) -> (i32, String, String) {
let (code, capture) = run_util::<Grep>(args, stdin, "/");
(code, capture.out(), capture.err())
}
#[test]
fn max_count_accepts_compact_and_long_values() {
for option in ["-m1", "--max-count=1"] {
let (code, stdout, stderr) = run_grep(&[option, "hit"], "hit\nmiss\nhit\n");
assert_eq!(code, 0, "{option}: {stderr}");
assert_eq!(stdout, "hit\n", "{option}");
}
let (code, stdout, stderr) = run_grep(&["-m0", "hit"], "hit\n");
assert_eq!(code, 1, "{stderr}");
assert!(stdout.is_empty());
fn max_count_and_no_match_statuses_are_gnu_compatible() {
let (code, out, err) = run(&["-m1", "hit"], "hit\nmiss\nhit\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "hit\n");
let (code, out, _) = run(&["absent"], "hit\n");
assert_eq!(code, 1);
assert!(out.is_empty());
}
#[test]
fn pattern_file_combines_patterns_without_consuming_a_file_operand() {
let tree = unique_tree("patterns");
std::fs::write(tree.join("patterns"), "alpha\nbeta\n").expect("pattern file written");
std::fs::write(tree.join("haystack"), "alpha\ngamma\nbeta\n").expect("haystack written");
let (code, stdout, stderr) = run_grep_in(&["-f", "patterns", "haystack"], "", &tree);
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "alpha\nbeta\n");
let _ = std::fs::remove_dir_all(tree);
fn pattern_file_is_resolved_against_shell_cwd() {
let tree = tempfile::tempdir().unwrap();
std::fs::write(tree.path().join("patterns"), "alpha\nbeta\n").unwrap();
std::fs::write(tree.path().join("haystack"), "alpha\ngamma\nbeta\n").unwrap();
let (code, capture) = run_util::<Grep>(&["-f", "patterns", "haystack"], "", tree.path());
assert_eq!(code, 0, "{}", capture.err());
assert_eq!(capture.out(), "alpha\nbeta\n");
}
#[test]
fn perl_mode_supports_lookbehind() {
let (code, stdout, stderr) = run_grep(&["-P", "(?<=foo)bar"], "foobar\nbar\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "foobar\n");
let (code, out, err) = run(&["-P", "(?<=foo)bar"], "foobar\nbar\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "foobar\n");
}
#[test]
fn perl_mode_compiles_the_native_grep_crash_pattern() {
let pattern = r"(final_incremental_account_id_relationships|dim_customers_status_accepted_values|stg_orders_customer_id_relationships)(?!_[0-9a-f]{32})";
let (code, stdout, stderr) =
run_grep(&["-P", pattern], "dim_customers_status_accepted_values\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "dim_customers_status_accepted_values\n");
}
#[test]
fn byte_offsets_labels_and_nul_filename_separators_are_rendered() {
let (code, stdout, stderr) = run_grep(&["-bn", "hit"], "no\nhit\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "2:3:hit\n");
let (code, stdout, stderr) = run_grep(&["--label=pipe", "-HZ", "hit"], "hit\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout.as_bytes(), b"pipe\0hit\n");
}
#[test]
fn numeric_context_uses_the_configured_group_separator() {
fn compact_numeric_context_is_rewritten_before_clap() {
let input = "a\nhit\nb\ngap\nc\nhit\nd\n";
let (code, stdout, stderr) = run_grep(&["-1", "--group-separator=@", "hit"], input);
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "a\nhit\nb\n@\nc\nhit\nd\n");
let (code, out, err) = run(&["-1", "--group-separator=@", "hit"], input);
assert_eq!(code, 0, "{err}");
assert_eq!(out, "a\nhit\nb\n@\nc\nhit\nd\n");
}
#[test]
fn recursive_include_and_exclude_dir_rules_filter_the_walk() {
let tree = unique_tree("filters");
std::fs::write(tree.join("keep.rs"), "hit\n").expect("included file written");
std::fs::write(tree.join("drop.txt"), "hit\n").expect("excluded file written");
std::fs::create_dir(tree.join("vendor")).expect("excluded directory created");
std::fs::write(tree.join("vendor/hidden.rs"), "hit\n").expect("excluded file written");
let (code, stdout, stderr) =
run_grep_in(&["-r", "--include=*.rs", "--exclude-dir=vendor", "hit", "."], "", &tree);
assert_eq!(code, 0, "{stderr}");
assert!(stdout.contains("keep.rs:hit"), "{stdout:?}");
assert!(!stdout.contains("drop.txt"), "{stdout:?}");
assert!(!stdout.contains("hidden.rs"), "{stdout:?}");
let _ = std::fs::remove_dir_all(tree);
fn recursive_rules_filter_walk_and_preserve_relative_names() {
let tree = tempfile::tempdir().unwrap();
std::fs::write(tree.path().join("keep.rs"), "hit\n").unwrap();
std::fs::write(tree.path().join("drop.txt"), "hit\n").unwrap();
std::fs::create_dir(tree.path().join("vendor")).unwrap();
std::fs::write(tree.path().join("vendor/hidden.rs"), "hit\n").unwrap();
let (code, capture) = run_util::<Grep>(&["-r", "--include=*.rs", "--exclude-dir=vendor", "hit", "."], "", tree.path());
assert_eq!(code, 0, "{}", capture.err());
assert!(capture.out().contains("keep.rs:hit"));
assert!(!capture.out().contains("drop.txt"));
assert!(!capture.out().contains("hidden.rs"));
}
#[test]
fn unbalanced_paren_pattern_matches_literally() {
// Regression: `grep "fail)"` used to abort with `regex parse error:
// unopened group`. It must now match the literal text and exit 0.
let (code, stdout, stderr) = run_grep(&["-A", "1", "fail)"], "ok\n(1 fail)\nnext\n");
assert_eq!(code, 0, "stderr: {stderr}");
assert!(stderr.is_empty(), "no error expected, got: {stderr}");
assert!(stdout.contains("(1 fail)"), "matched line missing: {stdout}");
assert!(stdout.contains("next"), "after-context line missing: {stdout}");
}
#[test]
fn extended_flag_reports_parse_error() {
// -E opts into strict extended-regex syntax: the bad pattern is an error.
let (code, _stdout, stderr) = run_grep(&["-E", "fail)"], "fail)\n");
assert_eq!(code, 2);
assert!(stderr.contains("grep:"), "expected a grep error, got: {stderr}");
}
#[test]
fn valid_regex_still_applies() {
// A parseable pattern is used as a regex, not matched literally.
let (code, stdout, _err) = run_grep(&["fo+"], "foooo\nbar\n");
fn quiet_match_wins_over_later_error() {
let tree = tempfile::tempdir().unwrap();
std::fs::write(tree.path().join("hit"), "needle\n").unwrap();
let (code, capture) = run_util::<Grep>(&["-q", "needle", "hit", "missing"], "", tree.path());
assert_eq!(code, 0);
assert!(stdout.contains("foooo"));
assert!(!stdout.contains("bar"));
assert!(capture.out().is_empty());
assert!(capture.err().is_empty());
}
#[test]
fn default_mode_supports_gnu_basic_alternation() {
fn recursive_walk_observes_cancellation() {
let tree = tempfile::tempdir().unwrap();
std::fs::create_dir(tree.path().join("root")).unwrap();
std::fs::write(tree.path().join("root/file"), "hit\n").unwrap();
let parsed = Grep::try_parse_from(["grep", "-r", "hit", "root"]).unwrap();
let (mut host, capture) = Host::for_test("grep", Vec::new(), tree.path());
host.cancel_for_test();
assert_eq!(parsed.run(&mut host), 2);
assert!(capture.out().is_empty());
assert!(capture.err().is_empty());
}
#[test]
fn byte_offsets_labels_and_nul_separators_are_rendered() {
let (code, out, err) = run(&["-bn", "hit"], "no\nhit\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "2:3:hit\n");
let (code, out, err) = run(&["--label=pipe", "-HZ", "hit"], "hit\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out.as_bytes(), b"pipe\0hit\n");
}
#[test]
fn basic_mode_falls_back_per_pattern_but_extended_mode_is_strict() {
let (code, out, err) = run(&["-A", "1", "fail)"], "ok\n(1 fail)\nnext\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "(1 fail)\nnext\n");
let (code, _, err) = run(&["-E", "fail)"], "fail)\n");
assert_eq!(code, 2);
assert!(err.contains("grep:"));
let (code, out, err) =
run(&["-e", "fo+", "-e", "bar)", "-h"], "foooo\nbar)\nbaz\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "foooo\nbar)\n");
}
#[test]
fn gnu_basic_alternation_and_color_aliases_are_supported() {
let input = "\"tools.xdev\": {}\n\"tools.toolbox\": {}\n\"tools.other\": {}\n";
let (code, stdout, stderr) = run_grep(&["-c", r"tools.xdev\|tools.toolbox"], input);
assert_eq!(code, 0, "{stderr}");
assert!(stderr.is_empty(), "{stderr}");
assert_eq!(stdout, "2\n");
}
#[test]
fn multi_pattern_keeps_valid_alternative_as_regex() {
// Per-pattern fallback: valid `fo+` stays a regex while `bar)` is escaped.
let (code, stdout, err) = run_grep(&["-e", "fo+", "-e", "bar)", "-h"], "foooo\nbar)\nbaz\n");
assert_eq!(code, 0, "stderr: {err}");
assert!(stdout.contains("foooo"), "regex alternative should match: {stdout}");
assert!(stdout.contains("bar)"), "literal alternative should match: {stdout}");
assert!(!stdout.contains("baz"), "non-matching line leaked: {stdout}");
}
#[test]
fn color_flag_is_accepted_and_ignored() {
// Regression for #3755: the universal `alias grep='grep --color=auto'`
// must not break bare `grep` in shell pipelines.
for color in ["--color=auto", "--color=always", "--color=never", "--color", "--colour=auto"] {
let (code, stdout, stderr) = run_grep(&[color, "foo"], "foo\nbar\n");
assert_eq!(code, 0, "{color}: stderr: {stderr}");
assert!(stderr.is_empty(), "{color}: unexpected stderr: {stderr}");
assert_eq!(stdout, "foo\n", "{color}: matched lines: {stdout:?}");
let (code, out, err) = run(&["-c", r"tools.xdev\|tools.toolbox"], input);
assert_eq!(code, 0, "{err}");
assert_eq!(out, "2\n");
for color in ["--color=auto", "--color=always", "--color=never", "--color"] {
let (code, out, err) = run(&[color, "foo"], "foo\nbar\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "foo\n");
}
}
#[test]
fn version_flag_prints_and_exits_zero() {
// `grep --version` is the universal probe shells run; the builtin must
// not reject it with exit 2.
let (code, stdout, stderr) = run_grep(&["--version"], "");
assert_eq!(code, 0, "stderr: {stderr}");
assert!(stderr.is_empty(), "unexpected stderr: {stderr}");
assert!(
stdout.contains("grep") && stdout.contains("pi-uu-grep"),
"version output should identify the builtin, got: {stdout:?}"
);
}
/// Run `grep` with a pre-set cancel flag, mirroring how the shell wrapper
/// flips the flag when an abort/timeout fires while the blocking task is
/// still walking. Returns `(exit, stdout, stderr)`.
fn run_grep_cancelled(args: &[&str], cwd: &Path) -> (i32, String, String) {
let out = Arc::new(Mutex::new(Vec::new()));
let err = Arc::new(Mutex::new(Vec::new()));
let io = ScopeIo {
stdin: Box::new(io::empty()),
stdin_fd: None,
stdin_is_search_input: false,
stdout: Box::new(SharedBuf(Arc::clone(&out))),
stderr: Box::new(SharedBuf(Arc::clone(&err))),
cwd: cwd.to_path_buf(),
env: HashMap::new(),
cancel: Arc::new(AtomicBool::new(true)),
};
let argv: Vec<OsString> = std::iter::once("grep")
.chain(args.iter().copied())
.map(OsString::from)
.collect();
let code = scope(io, || run(argv));
let stdout = String::from_utf8(out.lock().clone()).expect("utf8 stdout");
let stderr = String::from_utf8(err.lock().clone()).expect("utf8 stderr");
(code, stdout, stderr)
}
#[test]
fn recursive_search_observes_scope_cancellation() {
// Regression for #3933: recursive grep used to pass a no-op heartbeat to
// pi_walker, so directory walks ignored the uutils cancel flag and the
// shell-side abort/timeout waited for the whole tree to be scanned.
// The walk must now bail out before scanning any file when the flag is
// already set, and it must do so without printing an "interrupted"
// diagnostic — the shell wrapper owns the user-visible status.
let tree = std::env::temp_dir().join(format!(
"pi-uu-grep-cancel-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
std::fs::create_dir_all(&tree).expect("temp tree should be created");
let walk_root = tree.join("walk-root");
std::fs::create_dir_all(&walk_root).expect("walk root should be created");
std::fs::write(walk_root.join("haystack.txt"), "match-me\n")
.expect("walked file should be written");
let later_file = tree.join("later.txt");
std::fs::write(&later_file, "match-me\n").expect("later file should be written");
let (code, stdout, stderr) = run_grep_cancelled(
&[
"-r",
"match-me",
walk_root.to_str().expect("utf8 path"),
later_file.to_str().expect("utf8 path"),
],
&tree,
);
// Walker must have observed the heartbeat before visiting the file,
// and the operand loop must not continue into the later regular file
// after cancellation is observed.
assert!(stdout.is_empty(), "cancelled walk should not output matches: {stdout:?}");
assert!(
stderr.is_empty(),
"cancelled walk should stay silent — diagnostic is the shell's job: {stderr:?}"
);
assert_eq!(code, 2, "interrupted directory walk should report had_error (exit 2)");
let _ = std::fs::remove_dir_all(&tree);
fn version_is_reported_on_stdout() {
let (code, out, err) = run(&["--version"], "");
assert_eq!(code, 0);
assert!(err.is_empty());
assert!(out.contains("grep") && out.contains("pi-uu-grep"));
}
}
crates/vendor/brush-builtins/src/hash.rs → crates/pi-builtins/src/hash.rs
View File
crates/pi-builtins/src/head.rs
+1481
View File
File diff suppressed because it is too large Load Diff
crates/vendor/brush-builtins/src/help.rs → crates/pi-builtins/src/help.rs
View File
crates/vendor/brush-builtins/src/history.rs → crates/pi-builtins/src/history.rs
View File
crates/pi-builtins/src/host.rs
+964
View File
@@ -0,0 +1,964 @@
//! Host plumbing for utility builtins (`cat`, `grep`, `sed`, `ls`, …).
//!
//! These builtins are ports of standalone command-line utilities: synchronous
//! programs that read `argv`, talk to fd 0/1/2, resolve relative paths against
//! the current directory, and exit with a status. [`Host`] hands them exactly
//! that view of the shell they run inside — as a value, threaded explicitly —
//! so no process-global or thread-local I/O state is involved: output lands on
//! the command's (possibly redirected or piped) file descriptors and relative
//! paths resolve against the *shell's* working directory rather than the host
//! process's.
//!
//! A utility implements [`Utility`]: a `clap` argument model plus a synchronous
//! [`Utility::run`] body. [`util`] wraps that into a [`Registration`] which
//!
//! 1. materializes process-substitution arguments (`diff <(a) <(b)`) into real
//! file descriptors,
//! 2. parses `argv`, rendering `--help`/`--version` on stdout and usage errors
//! on stderr with the utility's own exit status,
//! 3. runs the body on a blocking thread, so a slow utility never stalls the
//! async runtime and concurrent pipeline stages stay isolated,
//! 4. observes the shell's cancellation token (abort/`timeout`), and
//! 5. contains panics at the builtin boundary instead of taking down the
//! long-lived host process.
// The whole module is API consumed by the feature-gated utility modules; a build
// with no utility features enabled legitimately uses none of it.
#![allow(dead_code, reason = "consumed by the feature-gated utility modules")]
use std::{
cell::Cell,
collections::HashMap,
ffi::OsString,
io::{self, Read, Write},
marker::PhantomData,
panic::{AssertUnwindSafe, catch_unwind},
path::{Path, PathBuf},
time::Duration,
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
};
use brush_core::{
Error, ExecutionContext, ExecutionResult, ShellExtensions,
builtins::{self, Registration},
openfiles::{self, OpenFile, OpenFiles},
};
/// A command-line utility implemented as a shell builtin.
///
/// Implementors supply the `clap` argument model (via `derive(Parser)`, or
/// [`matches_parser!`] for builder-style definitions) and a synchronous body.
/// Register with [`util`].
pub(crate) trait Utility: clap::Parser + Send + Sync + 'static {
/// Program name, used in diagnostics (`sed: -e expression #1: …`).
const NAME: &'static str;
/// Exit status for a usage error. Most GNU utilities use 1; the
/// `ls`/`grep`/`cmp` families reserve 1 for "differences found" and use 2.
const USAGE_ERROR: u8 = 1;
/// Rewrites raw `argv` before clap parses it.
///
/// A few utilities accept syntax clap cannot model — GNU's obsolete
/// `head -5` count form, for instance. `argv[0]` is the command name.
/// Returning `Err(message)` reports `<name>: <message>` on stderr and exits
/// with [`Utility::USAGE_ERROR`]. The default is the identity.
fn rewrite_argv(argv: Vec<OsString>) -> Result<Vec<OsString>, String> {
Ok(argv)
}
/// Runs the utility to completion, returning its exit status.
///
/// Called on a blocking thread, so blocking reads, `rayon`, and long
/// filesystem walks are all fine. Long-running loops should poll
/// [`Host::is_cancelled`] so shell abort/`timeout` is observed promptly.
fn run(self, host: &mut Host) -> i32;
}
/// The shell as a utility builtin sees it: standard streams, working
/// directory, exported environment, cancellation, and accumulated exit status.
///
/// The three streams are public fields rather than accessors so a utility can
/// hold `&mut` borrows of two of them at once (reading stdin while writing
/// stdout is the common case).
pub(crate) struct Host {
/// Standard input. Reads observe cancellation, so a blocked pipe read
/// returns EOF on abort instead of hanging the shell.
pub stdin: Stdin,
/// Standard output; the null device when fd 1 is closed.
pub stdout: OpenFile,
/// Standard error; the null device when fd 2 is closed.
pub stderr: OpenFile,
name: String,
cwd: PathBuf,
env: HashMap<String, String>,
cancel: Arc<AtomicBool>,
exit_code: i32,
stdin_is_search_input: bool,
}
impl Host {
/// The name the utility was invoked as. Differs from [`Utility::NAME`] when
/// one implementation backs several builtins (`grep` and `rg`).
pub fn name(&self) -> &str {
&self.name
}
/// The shell working directory that relative paths resolve against.
pub fn cwd(&self) -> &Path {
&self.cwd
}
/// Resolves `path` against [`Host::cwd`]; absolute paths pass through.
///
/// Every path argument must go through this before touching the
/// filesystem: the host process's current directory is unrelated to the
/// shell's.
pub fn resolve(&self, path: impl AsRef<Path>) -> PathBuf {
let path = path.as_ref();
if path.is_absolute() {
path.to_path_buf()
} else {
self.cwd.join(path)
}
}
/// Looks up an exported shell variable.
///
/// The shell's exported variables are *not* present in the host process
/// environment, so `std::env::var` would miss them.
pub fn var(&self, key: &str) -> Option<&str> {
self.env.get(key).map(String::as_str)
}
/// The exported shell environment, for building a child process
/// environment (`env_clear().envs(host.env())`).
pub fn env(&self) -> impl Iterator<Item = (&str, &str)> {
self.env.iter().map(|(k, v)| (k.as_str(), v.as_str()))
}
/// Whether the host has asked this invocation to stop (shell abort or
/// `timeout`). Long internal loops — recursive directory walks in
/// particular — poll this so cancellation is observed without waiting for
/// stdin or for the whole work item to finish.
pub fn is_cancelled(&self) -> bool {
self.cancel.load(Ordering::Relaxed)
}
/// A cancellation flag that can be moved into worker threads and walker
/// callbacks.
pub fn cancel_flag(&self) -> Arc<AtomicBool> {
Arc::clone(&self.cancel)
}
/// Whether stdin is a shell pipe or custom stream, and so should be treated
/// as implicit input rather than a terminal. `rg PATTERN` uses this to
/// decide between searching stdin and searching `.`.
pub const fn stdin_is_search_input(&self) -> bool {
self.stdin_is_search_input
}
/// Records a non-zero exit status while processing continues (the
/// `cat a missing b` case: report, keep going, exit 1).
pub const fn fail(&mut self, code: i32) {
if code != 0 {
self.exit_code = code;
}
}
/// The status accumulated via [`Host::fail`]; 0 when nothing failed.
pub const fn exit_code(&self) -> i32 {
self.exit_code
}
/// Writes `<name>: <message>` to stderr and records exit status `code`.
pub fn error(&mut self, message: impl std::fmt::Display, code: i32) {
let _ = writeln!(self.stderr, "{}: {message}", self.name);
self.fail(code);
}
/// Duplicates stdout, for utilities that hand a writer to a helper thread.
pub fn stdout_clone(&self) -> OpenFile {
self.stdout.clone()
}
/// Duplicates stderr, for utilities that hand a writer to a helper thread.
pub fn stderr_clone(&self) -> OpenFile {
self.stderr.clone()
}
/// A launcher for child processes started by this utility.
///
/// Owned and `Clone`, so it can move into worker threads and into helper
/// types that never see the `Host` itself — `sort --compress-program` spawns
/// its compressor from inside the temp-file abstraction, for instance.
pub fn child_env(&self) -> ChildEnv {
ChildEnv {
cwd: self.cwd.clone(),
env: Arc::new(
self
.env
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect(),
),
stderr: self.stderr.clone(),
}
}
/// Runs `command` with stdin from the null device and stdout/stderr piped
/// back into this host's streams, returning the child's exit status.
///
/// The host's streams are in-process `Write` handles (pipes or in-memory
/// buffers), not inheritable descriptors, and the process's own fd 0/1/2
/// belong to the TUI — a child must never inherit stdio. Child stdout
/// streams through on the calling thread while a helper thread drains
/// stderr into a buffer, which is forwarded once the child exits.
///
/// Callers remain responsible for `current_dir` and the child environment
/// (`env_clear().envs(host.env())`).
pub fn run_captured(
&mut self,
command: &mut std::process::Command,
) -> io::Result<std::process::ExitStatus> {
command
.stdin(std::process::Stdio::null())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped());
let mut child = command.spawn()?;
let mut child_err = child.stderr.take();
let stderr_thread = std::thread::spawn(move || {
let mut buf = Vec::new();
if let Some(err) = child_err.as_mut() {
let _ = err.read_to_end(&mut buf);
}
buf
});
if let Some(mut out) = child.stdout.take() {
let _ = io::copy(&mut out, &mut self.stdout);
}
let status = child.wait();
if let Ok(buf) = stderr_thread.join() {
let _ = self.stderr.write_all(&buf);
}
status
}
}
/// A shell-faithful launcher for child processes started by a utility builtin.
///
/// Carries the three things a child must inherit from the *shell* rather than
/// from the host process: the working directory, the exported environment
/// (which is also what `PATH` lookup resolves against, so a program installed
/// only on the shell's `PATH` is found), and a duplicate of the command's
/// standard error.
///
/// That last one matters more than it looks: the host process's fd 2 belongs to
/// the TUI, so a child left with inherited stderr writes straight into the
/// rendered frame. [`ChildEnv::command`] therefore always pipes stderr, and
/// [`ChildEnv::forward_stderr`] drains it to the command's own fd 2.
#[derive(Clone)]
pub(crate) struct ChildEnv {
cwd: PathBuf,
env: Arc<Vec<(String, String)>>,
stderr: OpenFile,
}
impl ChildEnv {
/// Builds a `Command` for `program` with the shell's working directory and
/// environment, and with stderr piped.
///
/// Stdin and stdout are left untouched for the caller to wire; they default
/// to inherited, so a caller that leaves them alone MUST redirect them.
pub fn command(&self, program: impl AsRef<std::ffi::OsStr>) -> std::process::Command {
let mut command = std::process::Command::new(program);
command
.current_dir(&self.cwd)
.env_clear()
.envs(self.env.iter().map(|(k, v)| (k, v)))
.stderr(std::process::Stdio::piped());
command
}
/// Drains a child's piped stderr into the command's standard error on a
/// helper thread.
///
/// The returned handle should be joined once the child has exited, so the
/// diagnostic lands before the utility reports its own result. Dropping the
/// handle detaches the thread, which is only correct if nothing downstream
/// depends on the ordering.
pub fn forward_stderr(
&self,
mut child_stderr: std::process::ChildStderr,
) -> std::thread::JoinHandle<()> {
let mut stderr = self.stderr.clone();
std::thread::spawn(move || {
let _ = io::copy(&mut child_stderr, &mut stderr);
})
}
}
/// Standard input for a utility builtin: the command's fd 0 plus the
/// cancellation flag.
///
/// On unix, when fd 0 is a real descriptor, reads wait for readiness in short
/// slices so an abort or `timeout` is observed even when input never arrives on
/// a blocked pipe; the utility then sees EOF and unwinds cleanly rather than
/// leaving a detached thread writing to descriptors the host has moved on from.
pub(crate) struct Stdin {
file: OpenFile,
#[cfg_attr(not(unix), allow(dead_code, reason = "readiness polling is unix-only"))]
fd: Option<i32>,
cancel: Arc<AtomicBool>,
}
impl Stdin {
/// Mirror of `std::io::Stdin::lock`; the handle is already the lockable
/// target, so this is the identity.
pub const fn lock(&mut self) -> &mut Self {
self
}
/// The underlying open file, for utilities that need to inspect fd 0
/// (`is_terminal`) or hand it to a child process.
pub const fn file(&self) -> &OpenFile {
&self.file
}
}
impl Read for Stdin {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
if self.cancel.load(Ordering::Relaxed) {
return Ok(0);
}
#[cfg(unix)]
if let Some(fd) = self.fd {
loop {
if self.cancel.load(Ordering::Relaxed) {
return Ok(0);
}
let mut pfd = libc::pollfd { fd, events: libc::POLLIN, revents: 0 };
// SAFETY: one `pollfd` valid for the call; `fd` is owned by the
// live `OpenFile` held in this struct.
let ready = unsafe { libc::poll(&mut pfd, 1, 200) };
if ready < 0 {
let err = io::Error::last_os_error();
if err.kind() == io::ErrorKind::Interrupted {
continue;
}
return Err(err);
}
if ready > 0 {
break;
}
}
}
self.file.read(buf)
}
}
thread_local! {
/// Depth of active utility bodies on this thread. The native crash hook
/// reads this from inside a panic (see [`panic_scope_active`]) to decide
/// whether the panic is about to be caught; a `Cell` is used because the
/// panicking code may hold other borrows, and a `RefCell` borrow there
/// would panic again and abort the process.
static PANIC_SCOPE_DEPTH: Cell<usize> = const { Cell::new(0) };
}
/// Whether a utility builtin body is running on the current thread.
///
/// A panic raised here is, by construction, about to be caught at the builtin
/// boundary, so the native crash hook treats it as recoverable and keeps it out
/// of the user-facing crash report.
#[must_use]
pub fn panic_scope_active() -> bool {
PANIC_SCOPE_DEPTH.with(|depth| depth.get() > 0)
}
static RAYON_GLOBAL_POOL_AVAILABLE: AtomicBool = AtomicBool::new(!cfg!(target_os = "windows"));
/// Records whether utility builtins may use Rayon's process-global worker pool
/// without risking lazy initialization under Windows commit pressure.
pub fn set_rayon_global_pool_available(available: bool) {
RAYON_GLOBAL_POOL_AVAILABLE.store(available, Ordering::SeqCst);
}
/// Whether utility builtins may enter Rayon's process-global worker pool.
#[must_use]
pub fn rayon_global_pool_available() -> bool {
RAYON_GLOBAL_POOL_AVAILABLE.load(Ordering::SeqCst)
}
/// Indents all but the first line of a usage string by 7 spaces, aligning
/// continuation lines under clap's `Usage: ` prefix.
pub(crate) fn format_usage(usage: &str) -> String {
debug_assert!(
!usage.contains("{}"),
"usage strings must name the command explicitly, not via a '{{}}' placeholder"
);
usage.replace('\n', "\n ")
}
/// Borrows an `OsStr` as raw bytes.
///
/// Unix strings are arbitrary byte sequences, so this is free there. On Windows
/// only well-formed UTF-16 has a UTF-8 byte view, so an ill-formed value yields
/// `None`; callers report that as an invalid argument.
pub(crate) fn os_bytes(value: &std::ffi::OsStr) -> Option<&[u8]> {
#[cfg(unix)]
{
use std::os::unix::ffi::OsStrExt;
Some(value.as_bytes())
}
#[cfg(not(unix))]
{
value.to_str().map(str::as_bytes)
}
}
/// Borrows an `OsStr` as raw bytes, substituting replacement characters for
/// anything unrepresentable. For diagnostics, where losing a byte beats failing.
pub(crate) fn os_bytes_lossy(value: &std::ffi::OsStr) -> std::borrow::Cow<'_, [u8]> {
match os_bytes(value) {
Some(bytes) => std::borrow::Cow::Borrowed(bytes),
None => std::borrow::Cow::Owned(value.to_string_lossy().into_owned().into_bytes()),
}
}
/// Parses a GNU-style duration: a decimal number with an optional `s`/`m`/`h`/`d`
/// suffix, as accepted by `sleep` and `timeout`.
pub(crate) fn parse_duration(input: &str) -> Option<Duration> {
let trimmed = input.trim();
if trimmed.is_empty() {
return None;
}
let (number, multiplier) = match trimmed.chars().last()? {
's' => (&trimmed[..trimmed.len() - 1], 1.0),
'm' => (&trimmed[..trimmed.len() - 1], 60.0),
'h' => (&trimmed[..trimmed.len() - 1], 3600.0),
'd' => (&trimmed[..trimmed.len() - 1], 86400.0),
ch if ch.is_ascii_alphabetic() => return None,
_ => (trimmed, 1.0),
};
let value = number.parse::<f64>().ok()?;
if value.is_sign_negative() {
return None;
}
let millis = value * multiplier * 1000.0;
if !millis.is_finite() || millis < 0.0 {
return None;
}
Some(Duration::from_millis(millis.round() as u64))
}
/// Shell-quotes `arg` when rebuilding a command line for a child process.
///
/// `timeout` and `nohup` reconstruct the command they were handed so it can be
/// re-parsed by a shell; anything that could be re-split or re-expanded must be
/// quoted first.
pub(crate) fn quote_arg(arg: &str) -> String {
if arg.is_empty() {
return "''".to_string();
}
let safe = arg
.chars()
.all(|ch| ch.is_ascii_alphanumeric() || matches!(ch, '-' | '_' | '.' | '/' | ':' | '+'));
if safe {
return arg.to_string();
}
let escaped = arg.replace('\'', "'\"'\"'");
format!("'{escaped}'")
}
/// Reads a boolean "disable" flag for the uutils builtins from the session
/// environment (preferred) then the process environment, mirroring the nohup
/// builtin gate. Truthy = present and not "", "0", or "false".
/// Returns the [`Registration`] for a [`Utility`].
pub(crate) fn util<U: Utility, SE: ShellExtensions>() -> Registration<SE> {
builtins::builtin::<Util<U>, SE>()
}
/// Adapter turning a [`Utility`] into a brush builtin.
///
/// Holds the raw argument vector rather than a parsed `U`: process-substitution
/// arguments can only be materialized once the shell is in hand, which happens
/// in [`builtins::Command::execute`], and parse failures must be reported on the
/// utility's own terms (help on stdout, usage errors with the utility's exit
/// status) rather than through brush's generic usage-error path.
pub(crate) struct Util<U: Utility> {
argv: Vec<String>,
_marker: PhantomData<fn() -> U>,
}
impl<U: Utility> clap::FromArgMatches for Util<U> {
fn from_arg_matches(_matches: &clap::ArgMatches) -> Result<Self, clap::Error> {
Ok(Self { argv: Vec::new(), _marker: PhantomData })
}
fn update_from_arg_matches(&mut self, _matches: &clap::ArgMatches) -> Result<(), clap::Error> {
Ok(())
}
}
impl<U: Utility> clap::CommandFactory for Util<U> {
fn command() -> clap::Command {
U::command()
}
fn command_for_update() -> clap::Command {
U::command_for_update()
}
}
impl<U: Utility> clap::Parser for Util<U> {}
impl<U: Utility> builtins::Command for Util<U> {
type Error = Error;
fn new<I>(args: I) -> Result<Self, clap::Error>
where
I: IntoIterator<Item = String>,
{
Ok(Self { argv: args.into_iter().collect(), _marker: PhantomData })
}
async fn execute<SE: ShellExtensions>(
&self,
context: ExecutionContext<'_, SE>,
) -> Result<ExecutionResult, Self::Error> {
run_utility::<U, SE>(context, self.argv.clone()).await
}
}
/// Drives a utility from raw arguments to an exit status.
async fn run_utility<U: Utility, SE: ShellExtensions>(
context: ExecutionContext<'_, SE>,
argv: Vec<String>,
) -> Result<ExecutionResult, Error> {
// Capture everything owned *before* the first await so the returned future
// stays `Send`: the borrowed `ExecutionContext` (and its `&mut Shell`) is
// dropped before we await the blocking task.
#[cfg_attr(not(unix), expect(unused_mut, reason = "rewritten only on unix"))]
let mut argv: Vec<OsString> = argv.into_iter().map(OsString::from).collect();
#[cfg(unix)]
let process_substitution_fds = materialize_process_substitution_fds(&context, &mut argv)?;
let argv = match U::rewrite_argv(argv) {
Ok(argv) => argv,
Err(message) => {
let _ = writeln!(context.stderr(), "{}: {message}", U::NAME);
return Ok(ExecutionResult::new(U::USAGE_ERROR));
},
};
let parsed = match U::try_parse_from(&argv) {
Ok(parsed) => parsed,
Err(err) => {
// clap reports `--help` and `--version` as errors; those belong on
// stdout with a success status, everything else on stderr.
let rendered = err.to_string();
if err.use_stderr() {
let _ = write!(context.stderr(), "{rendered}");
return Ok(ExecutionResult::new(U::USAGE_ERROR));
}
let _ = write!(context.stdout(), "{rendered}");
return Ok(ExecutionResult::success());
},
};
let mut host = build_host(&context, U::NAME)?;
let cancel = context.cancel_token();
let cancel_flag = host.cancel_flag();
drop(context);
let mut handle = tokio::task::spawn_blocking(move || {
#[cfg(unix)]
let _process_substitution_fds = process_substitution_fds;
run_caught::<U>(parsed, &mut host)
});
// Respect shell abort/`timeout`. On cancel we set the host's cancel flag,
// which makes a blocked stdin read return EOF; the utility unwinds cleanly
// (flushing what it already produced) and the blocking task completes. We
// await that completion before returning so no detached thread keeps
// writing to the command's (possibly redirected) descriptors.
let code = match cancel {
Some(token) => {
let token_check = token.clone();
tokio::select! {
biased;
() = token.cancelled() => {
cancel_flag.store(true, Ordering::Relaxed);
let _ = (&mut handle).await;
130
},
result = &mut handle => {
// If the token already fired, the task only finished because
// our cancel flag unblocked it — report interrupted.
if token_check.is_cancelled() { 130 } else { result.unwrap_or(1) }
},
}
},
None => handle.await.unwrap_or(1),
};
Ok(ExecutionResult::new((code & 0xff) as u8))
}
/// Runs a utility body, containing any panic at the builtin boundary.
///
/// A port that panics (an `unwrap` on a `BrokenPipe`, say) must not take down
/// the long-lived host process. With `panic = "unwind"` the panic unwinds to
/// here, where it becomes a non-zero exit plus a concise note on the command's
/// own stderr.
fn run_caught<U: Utility>(parsed: U, host: &mut Host) -> i32 {
struct Guard;
impl Drop for Guard {
fn drop(&mut self) {
PANIC_SCOPE_DEPTH.with(|depth| depth.set(depth.get().saturating_sub(1)));
}
}
PANIC_SCOPE_DEPTH.with(|depth| depth.set(depth.get() + 1));
let _guard = Guard;
match catch_unwind(AssertUnwindSafe(|| parsed.run(host))) {
Ok(code) => code,
Err(_) => {
let _ = writeln!(host.stderr, "{}: internal error", U::NAME);
1
},
}
}
/// Snapshots the command's streams, working directory, and exported
/// environment into an owned [`Host`] that can move to a blocking thread.
fn build_host<SE: ShellExtensions>(
context: &ExecutionContext<'_, SE>,
name: &str,
) -> Result<Host, Error> {
let stdin = context.try_fd(OpenFiles::STDIN_FD);
// On unix, capture the raw stdin fd so reads can poll it for cancellation;
// the `OpenFile` is kept alive by the `Stdin` below, so the fd stays valid.
#[cfg(unix)]
let stdin_fd: Option<i32> = {
use std::os::fd::AsRawFd;
stdin
.as_ref()
.and_then(|file| file.try_borrow_as_fd().ok())
.map(|fd| fd.as_raw_fd())
};
#[cfg(not(unix))]
let stdin_fd: Option<i32> = None;
let stdin_is_search_input = stdin
.as_ref()
.is_some_and(|file| matches!(file, OpenFile::PipeReader(_) | OpenFile::Stream(_)));
let mut env = HashMap::new();
for (key, var) in context.shell.env().iter_exported() {
if var.value().is_set() {
env.insert(key.clone(), var.value().to_cow_str(context.shell).into_owned());
}
}
let invoked = if context.command_name.is_empty() {
name.to_string()
} else {
context.command_name.clone()
};
// One flag, shared: the adapter flips it on cancellation, and a blocked
// `Stdin::read` must observe the very same flag or it never wakes.
let cancel = Arc::new(AtomicBool::new(false));
Ok(Host {
stdin: Stdin {
file: or_null(stdin)?,
fd: stdin_fd,
cancel: Arc::clone(&cancel),
},
stdout: or_null(context.try_fd(OpenFiles::STDOUT_FD))?,
stderr: or_null(context.try_fd(OpenFiles::STDERR_FD))?,
name: invoked,
cwd: context.shell.working_dir().to_path_buf(),
env,
cancel,
exit_code: 0,
stdin_is_search_input,
})
}
/// Substitutes the null device for a closed descriptor, so a utility reading
/// from or writing to it sees EOF / discards output instead of failing.
fn or_null(file: Option<OpenFile>) -> Result<OpenFile, Error> {
match file {
Some(file) => Ok(file),
None => openfiles::null(),
}
}
/// Recognizes brush's process-substitution arguments (`/dev/fd/<shell fd>`).
#[cfg(unix)]
fn process_substitution_fd(arg: &std::ffi::OsStr) -> Option<brush_core::ShellFd> {
arg.to_str()?
.strip_prefix("/dev/fd/")?
.parse::<brush_core::ShellFd>()
.ok()
}
/// Rewrites `/dev/fd/<shell fd>` arguments to real descriptors of the host
/// process, returning the owned descriptors that must stay alive for the
/// duration of the utility.
///
/// Brush allocates process-substitution pipes in its own descriptor table, so
/// the shell fd number in the argument is meaningless to `open`.
#[cfg(unix)]
fn materialize_process_substitution_fds<SE: ShellExtensions>(
context: &ExecutionContext<'_, SE>,
argv: &mut [OsString],
) -> Result<Vec<std::os::fd::OwnedFd>, Error> {
use std::os::fd::AsRawFd;
let mut fds = Vec::new();
for arg in argv {
let Some(shell_fd) = process_substitution_fd(arg) else {
continue;
};
let Some(file) = context.try_fd(shell_fd) else {
continue;
};
let fd = file.try_borrow_as_fd()?.try_clone_to_owned()?;
*arg = OsString::from(format!("/dev/fd/{}", fd.as_raw_fd()));
fds.push(fd);
}
Ok(fds)
}
/// Implements `clap::Parser` for a builder-style utility: `$ty` stores the
/// `ArgMatches` produced by `$app` in a field named `matches`.
///
/// Ports whose upstream argument model is built with `clap::Command::new(…)`
/// use this instead of rewriting dozens of arguments into `derive(Parser)`
/// form. Brush still renders `--help`, usage, and man content from `$app`.
#[allow(unused_macros, reason = "used by utility modules, which are feature-gated")]
macro_rules! matches_parser {
($ty:ident, $app:path) => {
impl clap::FromArgMatches for $ty {
fn from_arg_matches(matches: &clap::ArgMatches) -> Result<Self, clap::Error> {
Ok(Self { matches: matches.clone() })
}
fn update_from_arg_matches(
&mut self,
matches: &clap::ArgMatches,
) -> Result<(), clap::Error> {
self.matches = matches.clone();
Ok(())
}
}
impl clap::CommandFactory for $ty {
fn command() -> clap::Command {
$app()
}
fn command_for_update() -> clap::Command {
$app()
}
}
impl clap::Parser for $ty {}
};
}
#[allow(unused_imports, reason = "used by utility modules, which are feature-gated")]
pub(crate) use matches_parser;
#[cfg(test)]
mod testing {
//! In-memory [`Host`] construction for unit tests.
use parking_lot::Mutex;
use super::{
Arc, AtomicBool, HashMap, Host, OpenFile, OsString, PathBuf, Read, Stdin, Utility, Write, io,
openfiles, run_caught,
};
/// Captured in-memory output from [`Host::for_test`].
pub(crate) struct Capture {
stdout: Arc<Mutex<Vec<u8>>>,
stderr: Arc<Mutex<Vec<u8>>>,
}
impl Capture {
/// Raw bytes the utility wrote to stdout.
pub fn stdout(&self) -> Vec<u8> {
self.stdout.lock().clone()
}
/// Raw bytes the utility wrote to stderr.
pub fn stderr(&self) -> Vec<u8> {
self.stderr.lock().clone()
}
/// Stdout as a lossy string, for readable assertions.
pub fn out(&self) -> String {
String::from_utf8_lossy(&self.stdout()).into_owned()
}
/// Stderr as a lossy string, for readable assertions.
pub fn err(&self) -> String {
String::from_utf8_lossy(&self.stderr()).into_owned()
}
}
impl Host {
/// Builds a host backed by in-memory streams.
///
/// Returns the host plus a [`Capture`] over the same buffers, so a test
/// can run a utility and then assert on what it wrote.
pub(crate) fn for_test(
name: &str,
stdin: impl Into<Vec<u8>>,
cwd: impl Into<PathBuf>,
) -> (Self, Capture) {
let capture = Capture {
stdout: Arc::new(Mutex::new(Vec::new())),
stderr: Arc::new(Mutex::new(Vec::new())),
};
let cancel = Arc::new(AtomicBool::new(false));
let host = Self {
stdin: Stdin {
file: OpenFile::Stream(Box::new(MemStream::reader(stdin.into()))),
fd: None,
cancel: Arc::clone(&cancel),
},
stdout: OpenFile::Stream(Box::new(MemStream::writer(Arc::clone(
&capture.stdout,
)))),
stderr: OpenFile::Stream(Box::new(MemStream::writer(Arc::clone(
&capture.stderr,
)))),
name: name.to_string(),
cwd: cwd.into(),
env: HashMap::new(),
cancel,
exit_code: 0,
stdin_is_search_input: false,
};
(host, capture)
}
/// Sets an exported variable on a test host.
pub(crate) fn set_test_var(&mut self, key: &str, value: &str) {
self.env.insert(key.to_string(), value.to_string());
}
/// Requests cancellation on a test host.
pub(crate) fn cancel_for_test(&self) {
self.cancel.store(true, super::Ordering::Relaxed);
}
}
/// Parses `argv` and runs `U` against an in-memory host, mirroring what the
/// registered builtin does: `argv[0]` is the command name, clap failures are
/// reported the same way, and panics are contained.
pub(crate) fn run_util<U: Utility>(
argv: &[&str],
stdin: &str,
cwd: impl Into<PathBuf>,
) -> (i32, Capture) {
let (mut host, capture) = Host::for_test(U::NAME, stdin.as_bytes().to_vec(), cwd);
let full: Vec<OsString> = std::iter::once(OsString::from(U::NAME))
.chain(argv.iter().map(OsString::from))
.collect();
let full = match U::rewrite_argv(full) {
Ok(full) => full,
Err(message) => {
let _ = writeln!(host.stderr, "{}: {message}", U::NAME);
return (i32::from(U::USAGE_ERROR), capture);
},
};
let code = match U::try_parse_from(&full) {
Ok(parsed) => run_caught::<U>(parsed, &mut host),
Err(err) => {
let rendered = err.to_string();
if err.use_stderr() {
let _ = write!(host.stderr, "{rendered}");
i32::from(U::USAGE_ERROR)
} else {
let _ = write!(host.stdout, "{rendered}");
0
}
},
};
(code, capture)
}
/// An in-memory [`openfiles::Stream`]: a cursor over fixed input, or an
/// appending writer over a shared buffer.
#[derive(Clone)]
struct MemStream {
input: Arc<Mutex<io::Cursor<Vec<u8>>>>,
output: Arc<Mutex<Vec<u8>>>,
}
impl MemStream {
fn reader(data: Vec<u8>) -> Self {
Self {
input: Arc::new(Mutex::new(io::Cursor::new(data))),
output: Arc::new(Mutex::new(Vec::new())),
}
}
fn writer(output: Arc<Mutex<Vec<u8>>>) -> Self {
Self { input: Arc::new(Mutex::new(io::Cursor::new(Vec::new()))), output }
}
}
impl Read for MemStream {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.input.lock().read(buf)
}
}
impl Write for MemStream {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.output.lock().extend_from_slice(buf);
Ok(buf.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl openfiles::Stream for MemStream {
fn clone_box(&self) -> Box<dyn openfiles::Stream> {
Box::new(self.clone())
}
#[cfg(unix)]
fn try_clone_to_owned(&self) -> Result<std::os::fd::OwnedFd, super::Error> {
Err(brush_core::error::ErrorKind::CannotConvertToNativeFd.into())
}
#[cfg(unix)]
fn try_borrow_as_fd(&self) -> Result<std::os::fd::BorrowedFd<'_>, super::Error> {
Err(brush_core::error::ErrorKind::CannotConvertToNativeFd.into())
}
}
}
#[cfg(test)]
#[allow(unused_imports, reason = "used by utility test modules, which are feature-gated")]
pub(crate) use testing::{Capture, run_util};
crates/pi-builtins/src/hostname.rs
+251
View File
@@ -0,0 +1,251 @@
//! `hostname` builtin: display or set the system's host name.
//!
//! Ported from uutils coreutils 0.8.0.
#[cfg(not(any(target_os = "freebsd", target_os = "openbsd")))]
use std::net::ToSocketAddrs;
use std::{collections::hash_set::HashSet, ffi::OsString, io::Write};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command, builder::ValueParser};
#[cfg(any(target_os = "freebsd", target_os = "openbsd"))]
use dns_lookup::lookup_host;
use crate::host::{Host, Utility, format_usage, matches_parser, util};
static OPT_DOMAIN: &str = "domain";
static OPT_IP_ADDRESS: &str = "ip-address";
static OPT_FQDN: &str = "fqdn";
static OPT_SHORT: &str = "short";
static OPT_HOST: &str = "host";
#[cfg(windows)]
mod wsa {
use std::io;
use windows_sys::Win32::Networking::WinSock::{WSACleanup, WSADATA, WSAStartup};
pub(super) struct WsaHandle(());
pub(super) fn start() -> io::Result<WsaHandle> {
let mut data = std::mem::MaybeUninit::<WSADATA>::uninit();
let err = unsafe { WSAStartup(0x0202, data.as_mut_ptr()) };
if err == 0 {
Ok(WsaHandle(()))
} else {
Err(io::Error::from_raw_os_error(err))
}
}
impl Drop for WsaHandle {
fn drop(&mut self) {
// This possibly returns an error but we can't handle it.
let _ = unsafe { WSACleanup() };
}
}
}
/// Parsed `hostname` invocation.
pub(crate) struct Hostname {
matches: ArgMatches,
}
matches_parser!(Hostname, app);
impl Utility for Hostname {
const NAME: &'static str = "hostname";
fn run(self, host: &mut Host) -> i32 {
#[cfg(windows)]
let _handle = match wsa::start() {
Ok(handle) => handle,
Err(err) => {
host.error(format!("failed to start Winsock: {err}"), 1);
return 1;
},
};
if self.matches.get_one::<OsString>(OPT_HOST).is_some() {
// The shared `hostname` dependency does not enable its process-global
// `set` feature, so an operand must fail explicitly rather than no-op.
host.error("setting the hostname is not supported by the in-process builtin", 1);
return 1;
}
match display_hostname(&self.matches, host) {
Ok(()) => host.exit_code(),
Err(message) => {
host.error(message, 1);
1
},
}
}
}
/// The `hostname` argument model.
fn app() -> Command {
Command::new(Hostname::NAME)
.version("0.8.0")
.about("Display or set the system's host name.")
.override_usage(format_usage("hostname [OPTION]... [HOSTNAME]"))
.infer_long_args(true)
.arg(
Arg::new(OPT_DOMAIN)
.short('d')
.long("domain")
.overrides_with_all([OPT_DOMAIN, OPT_IP_ADDRESS, OPT_FQDN, OPT_SHORT])
.help("Display the name of the DNS domain if possible")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_IP_ADDRESS)
.short('i')
.long("ip-address")
.overrides_with_all([OPT_DOMAIN, OPT_IP_ADDRESS, OPT_FQDN, OPT_SHORT])
.help("Display the network address(es) of the host")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_FQDN)
.short('f')
.long("fqdn")
.overrides_with_all([OPT_DOMAIN, OPT_IP_ADDRESS, OPT_FQDN, OPT_SHORT])
.help("Display the FQDN (Fully Qualified Domain Name) (default)")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_SHORT)
.short('s')
.long("short")
.overrides_with_all([OPT_DOMAIN, OPT_IP_ADDRESS, OPT_FQDN, OPT_SHORT])
.help("Display the short hostname (the portion before the first dot) if possible")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_HOST)
.value_parser(ValueParser::os_string())
.value_hint(clap::ValueHint::Hostname),
)
}
fn display_hostname(matches: &ArgMatches, host: &mut Host) -> Result<(), String> {
let hostname = hostname::get()
.map_err(|err| format!("failed to get hostname: {err}"))?
.to_string_lossy()
.into_owned();
if matches.get_flag(OPT_IP_ADDRESS) {
let addresses;
#[cfg(not(any(target_os = "freebsd", target_os = "openbsd")))]
{
let hostname = hostname + ":1";
addresses = hostname
.to_socket_addrs()
.map_err(|err| format!("failed to resolve socket addresses: {err}"))?;
}
// DNS reverse lookup via "hostname:1" does not work on FreeBSD and OpenBSD;
// use the dns-lookup crate instead.
#[cfg(any(target_os = "freebsd", target_os = "openbsd"))]
{
addresses = lookup_host(hostname.as_str())
.map_err(|err| format!("failed to lookup hostname: {err}"))?;
}
let mut hashset = HashSet::new();
let mut output = String::new();
for addr in addresses {
// XXX: not sure why this is necessary...
if !hashset.contains(&addr) {
let mut ip = addr.to_string();
if ip.ends_with(":1") {
let len = ip.len();
ip.truncate(len - 2);
}
output.push_str(&ip);
output.push(' ');
hashset.insert(addr);
}
}
let len = output.len();
if len > 0 {
writeln!(host.stdout, "{}", &output[0..len - 1])
.map_err(|err| err.to_string())?;
}
Ok(())
} else {
if matches.get_flag(OPT_SHORT) || matches.get_flag(OPT_DOMAIN) {
let mut it = hostname.char_indices().filter(|&ci| ci.1 == '.');
if let Some(ci) = it.next() {
if matches.get_flag(OPT_SHORT) {
writeln!(host.stdout, "{}", &hostname[0..ci.0])
.map_err(|err| err.to_string())?;
} else {
writeln!(host.stdout, "{}", &hostname[ci.0 + 1..])
.map_err(|err| err.to_string())?;
}
} else if matches.get_flag(OPT_SHORT) {
writeln!(host.stdout, "{hostname}").map_err(|err| err.to_string())?;
}
return Ok(());
}
writeln!(host.stdout, "{hostname}").map_err(|err| err.to_string())?;
Ok(())
}
}
/// Creates the `hostname` builtin registration.
pub(crate) fn hostname_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Hostname, SE>()
}
#[cfg(test)]
mod tests {
use super::Hostname;
use crate::host::run_util;
fn hostname(argv: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Hostname>(argv, "", "/");
(code, capture.out(), capture.err())
}
#[test]
fn bare_invocation_prints_a_nonempty_line() {
let (code, stdout, stderr) = hostname(&[]);
assert_eq!((code, stderr.as_str()), (0, ""));
assert!(stdout.ends_with('\n'));
assert!(!stdout.trim_end().is_empty());
}
#[test]
fn set_attempt_is_rejected() {
let (code, stdout, stderr) = hostname(&["new-name.example.com"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(
stderr,
"hostname: setting the hostname is not supported by the in-process builtin\n"
);
}
#[test]
fn short_is_dotless_prefix_of_full_hostname() {
let (code, short, stderr) = hostname(&["-s"]);
let (_, full, _) = hostname(&[]);
assert_eq!((code, stderr.as_str()), (0, ""));
let short = short.trim_end();
assert!(!short.contains('.'), "-s must strip everything after the first dot");
assert!(full.trim_end().starts_with(short));
}
#[test]
fn fqdn_flag_matches_default_display() {
let (code, fqdn, stderr) = hostname(&["-f"]);
let (_, bare, _) = hostname(&[]);
assert_eq!((code, stderr.as_str()), (0, ""));
assert_eq!(fqdn, bare);
}
}
crates/pi-builtins/src/ifne.rs
+313
View File
@@ -0,0 +1,313 @@
//! moreutils-inspired `ifne` builtin: run a command iff stdin is non-empty
//! (`-n` inverts the condition).
//!
//! This is one of the selected moreutils tools kept in-process so its standard
//! streams, working directory, environment, and cancellation come from the
//! invoking shell. The command is executed directly, without shell
//! interpretation.
use std::{
ffi::OsString,
io::{self, ErrorKind, Read, Write},
process::{Command, Stdio},
sync::atomic::{AtomicBool, Ordering},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command as ClapCommand, builder::ValueParser};
use crate::host::{Host, Utility, matches_parser, util};
const USAGE: &str = "usage: ifne [-n] command [args...]";
const CHUNK: usize = 64 * 1024;
/// Parsed `ifne` invocation.
pub(crate) struct Ifne {
matches: ArgMatches,
}
matches_parser!(Ifne, app);
impl Utility for Ifne {
const NAME: &'static str = "ifne";
fn run(self, host: &mut Host) -> i32 {
let invert = self.matches.get_flag("invert");
let command: Vec<OsString> = self
.matches
.get_many::<OsString>("command")
.unwrap_or_default()
.cloned()
.collect();
if command.is_empty() {
let _ = writeln!(host.stderr, "{USAGE}");
return 1;
}
// Probe stdin: one byte decides which mode acts. Check cancellation both
// before the potentially blocking read and after cancellation-induced EOF.
let mut first = [0u8; 1];
let got = loop {
if host.is_cancelled() {
return 130;
}
match host.stdin.read(&mut first) {
Ok(n) => break n,
Err(err) if err.kind() == ErrorKind::Interrupted => {
if host.is_cancelled() {
return 130;
}
},
Err(err) => {
host.error(format!("stdin: {err}"), 1);
return 1;
},
}
};
if got == 0 && host.is_cancelled() {
return 130;
}
let empty = got == 0;
if empty != invert {
if empty {
// Default mode, empty stdin: do nothing.
return 0;
}
// -n mode, non-empty stdin: pass stdin through, don't run the command.
let cancel = host.cancel_flag();
return match copy_cancellable(
&mut host.stdin,
&mut host.stdout,
Some(first[0]),
&cancel,
) {
Ok(()) => 0,
Err(CopyError::Cancelled) => 130,
Err(CopyError::Io(err)) => {
host.error(err, 1);
1
},
};
}
spawn_and_pump(host, &command, if empty { None } else { Some(first[0]) })
}
}
/// The `ifne` argument model.
fn app() -> ClapCommand {
ClapCommand::new(Ifne::NAME)
.disable_version_flag(true)
.override_usage("ifne [-n] command [args...]")
.arg(
Arg::new("invert")
.short('n')
.action(ArgAction::SetTrue)
.help("run the command when standard input is empty"),
)
.arg(
Arg::new("command")
.value_name("command [args...]")
.value_parser(ValueParser::os_string())
.allow_hyphen_values(true)
.trailing_var_arg(true)
.num_args(0..),
)
}
/// Spawns the child and pumps stdin into it while draining its stdout/stderr.
fn spawn_and_pump(host: &mut Host, command: &[OsString], first: Option<u8>) -> i32 {
let mut child = match Command::new(&command[0])
.args(&command[1..])
.current_dir(host.cwd())
.env_clear()
.envs(host.env())
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
{
Ok(child) => child,
Err(err) => {
host.error(format!("{}: {err}", command[0].to_string_lossy()), 127);
return 127;
},
};
let mut child_stdin = child.stdin.take().expect("piped stdin");
let mut child_stdout = child.stdout.take().expect("piped stdout");
let mut child_stderr = child.stderr.take().expect("piped stderr");
let cancel = host.cancel_flag();
// Drain both child output streams while pumping its input, so no pipe can
// fill and deadlock the others. The buffers are forwarded to the host after
// the child exits; its in-process streams must never be inherited directly.
let (out_buf, err_buf, pump) = std::thread::scope(|scope| {
let out = scope.spawn(move || {
let mut buf = Vec::new();
let _ = child_stdout.read_to_end(&mut buf);
buf
});
let err = scope.spawn(move || {
let mut buf = Vec::new();
let _ = child_stderr.read_to_end(&mut buf);
buf
});
// Ignore BrokenPipe: the child may exit before consuming its stdin
// (for example, `ifne head -1`).
let pump = match copy_cancellable(&mut host.stdin, &mut child_stdin, first, &cancel) {
Err(CopyError::Io(err)) if err.kind() != ErrorKind::BrokenPipe => {
Err(CopyError::Io(err))
},
Err(CopyError::Cancelled) => Err(CopyError::Cancelled),
_ => Ok(()),
};
drop(child_stdin); // EOF so the child terminates.
if matches!(pump, Err(CopyError::Cancelled)) {
let _ = child.kill();
}
(out.join().unwrap_or_default(), err.join().unwrap_or_default(), pump)
});
let status = child.wait();
let _ = host.stdout.write_all(&out_buf);
let _ = host.stderr.write_all(&err_buf);
match pump {
Err(CopyError::Cancelled) => return 130,
Err(CopyError::Io(err)) => {
host.error(err, 1);
return 1;
},
Ok(()) => {},
}
match status {
Ok(status) => exit_code(status),
Err(err) => {
host.error(err, 1);
1
},
}
}
enum CopyError {
Cancelled,
Io(io::Error),
}
/// Copies `first` (when present) then all of `src` into `dst` in chunks.
fn copy_cancellable(
src: &mut impl Read,
dst: &mut impl Write,
first: Option<u8>,
cancel: &AtomicBool,
) -> Result<(), CopyError> {
if let Some(byte) = first {
dst.write_all(&[byte]).map_err(CopyError::Io)?;
}
let mut buf = vec![0u8; CHUNK].into_boxed_slice();
loop {
if cancel.load(Ordering::Relaxed) {
return Err(CopyError::Cancelled);
}
match src.read(&mut buf) {
Ok(0) => return Ok(()),
Ok(n) => dst.write_all(&buf[..n]).map_err(CopyError::Io)?,
Err(err) if err.kind() == ErrorKind::Interrupted => {},
Err(err) => return Err(CopyError::Io(err)),
}
}
}
/// Maps a child exit status to its code, or `128 + signal` on Unix.
fn exit_code(status: std::process::ExitStatus) -> i32 {
if let Some(code) = status.code() {
return code;
}
#[cfg(unix)]
{
use std::os::unix::process::ExitStatusExt;
if let Some(signal) = status.signal() {
return 128 + signal;
}
}
1
}
/// Creates the `ifne` builtin registration.
pub(crate) fn ifne_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Ifne, SE>()
}
#[cfg(test)]
mod tests {
use super::Ifne;
use crate::host::run_util;
fn run_in(stdin: &str, args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Ifne>(args, stdin, std::env::temp_dir());
(code, capture.out(), capture.err())
}
#[cfg(unix)]
#[test]
fn nonempty_stdin_runs_command_with_stdin() {
let result = run_in("hello world\n", &["/bin/cat"]);
assert_eq!(result, (0, "hello world\n".to_string(), String::new()));
}
#[cfg(unix)]
#[test]
fn empty_stdin_skips_command() {
let result = run_in("", &["/bin/sh", "-c", "echo ran"]);
assert_eq!(result, (0, String::new(), String::new()));
}
#[cfg(unix)]
#[test]
fn invert_runs_command_on_empty_stdin() {
let result = run_in("", &["-n", "/bin/sh", "-c", "echo ran"]);
assert_eq!(result, (0, "ran\n".to_string(), String::new()));
}
#[cfg(unix)]
#[test]
fn invert_passes_nonempty_stdin_through() {
let result = run_in("data\n", &["-n", "/bin/sh", "-c", "echo ran"]);
assert_eq!(result, (0, "data\n".to_string(), String::new()));
}
#[cfg(unix)]
#[test]
fn child_exit_code_propagates() {
let result = run_in("x", &["/bin/sh", "-c", "exit 3"]);
assert_eq!(result, (3, String::new(), String::new()));
}
#[test]
fn unknown_command_exits_127() {
let (code, stdout, stderr) = run_in("x", &["definitely-not-a-command-xyz"]);
assert_eq!(code, 127);
assert_eq!(stdout, "");
assert!(stderr.starts_with("ifne: definitely-not-a-command-xyz: "), "stderr: {stderr}");
}
#[cfg(unix)]
#[test]
fn early_exiting_child_is_not_an_error() {
let big = "a".repeat(1 << 20);
let result = run_in(&big, &["/usr/bin/head", "-c", "1"]);
assert_eq!(result, (0, "a".to_string(), String::new()));
}
#[test]
fn missing_command_is_usage_error() {
let (code, stdout, stderr) = run_in("", &[]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert!(stderr.contains("usage: ifne"));
}
}
crates/pi-builtins/src/isutf8.rs
+395
View File
@@ -0,0 +1,395 @@
//! `isutf8` builtin: check whether files (or, with no operands, standard input)
//! are valid UTF-8.
//!
//! This moreutils-inspired implementation runs in process so it can validate
//! shell-relative files without spawning an external command. Diagnostic
//! coordinates follow moreutils semantics: `line` is 1-based (counting `\n`),
//! `char` is the 1-based character position within that line, and `byte` is the
//! 0-based file offset of the first byte of the invalid sequence. Input is
//! streamed in 64 KiB chunks; a multi-byte sequence split across a chunk
//! boundary carries its incomplete tail (at most 3 bytes) into the next chunk,
//! and an incomplete tail at EOF counts as invalid.
//!
//! Standard input is reported as `(standard input)`. With `--invert` the exit
//! status and `--list` output treat valid inputs as failures; the default
//! diagnostic is still printed for invalid inputs. Exit codes: 0 = every input
//! passes the (possibly inverted) predicate, 1 = at least one input fails it,
//! 2 = an I/O error opening or reading a file (remaining files are still
//! checked).
use std::{
ffi::{OsStr, OsString},
fs::File,
io::{self, Read, Write},
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command, builder::ValueParser};
use crate::host::{Host, Utility, format_usage, matches_parser, util};
const OPT_QUIET: &str = "quiet";
const OPT_LIST: &str = "list";
const OPT_INVERT: &str = "invert";
const ARG_FILES: &str = "files";
const CHUNK_SIZE: usize = 64 * 1024;
const STDIN_NAME: &str = "(standard input)";
enum Verdict {
Valid,
Invalid { line: u64, character: u64, byte: u64 },
Cancelled,
}
/// Parsed `isutf8` invocation.
pub(crate) struct Isutf8 {
matches: ArgMatches,
}
matches_parser!(Isutf8, command);
impl Utility for Isutf8 {
const NAME: &'static str = "isutf8";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
let quiet = self.matches.get_flag(OPT_QUIET);
let list = self.matches.get_flag(OPT_LIST);
let invert = self.matches.get_flag(OPT_INVERT);
let files: Vec<OsString> = self
.matches
.get_many::<OsString>(ARG_FILES)
.map_or_else(Vec::new, |values| values.cloned().collect());
let cancel = host.cancel_flag();
let mut any_failed = false;
let mut io_error = false;
if files.is_empty() {
match validate(&mut host.stdin, &cancel) {
Err(err) => {
host.error(format!("{STDIN_NAME}: {err}"), 2);
io_error = true;
},
Ok(Verdict::Cancelled) => return 130,
Ok(verdict) => {
any_failed = report_verdict(
host,
STDIN_NAME,
verdict,
quiet,
list,
invert,
);
},
}
} else {
for name in &files {
let display = display_name(name);
let result = if name == "-" {
validate(&mut host.stdin, &cancel)
} else {
File::open(host.resolve(name)).and_then(|mut file| validate(&mut file, &cancel))
};
let verdict = match result {
Err(err) => {
host.error(format!("{display}: {err}"), 2);
io_error = true;
continue;
},
Ok(Verdict::Cancelled) => return 130,
Ok(verdict) => verdict,
};
any_failed |= report_verdict(host, &display, verdict, quiet, list, invert);
}
}
if io_error { 2 } else { i32::from(any_failed) }
}
}
fn report_verdict(
host: &mut Host,
display: &str,
verdict: Verdict,
quiet: bool,
list: bool,
invert: bool,
) -> bool {
let valid = match verdict {
Verdict::Valid => true,
Verdict::Invalid { line, character, byte } => {
if !quiet && !list {
let _ = writeln!(
host.stdout,
"{display}: line {line}, char {character}, byte {byte}: invalid UTF-8 code"
);
}
false
},
Verdict::Cancelled => unreachable!("cancellation is handled by the caller"),
};
// An input fails when its validity matches the inversion flag.
let failed = valid == invert;
if failed && list && !quiet {
let _ = writeln!(host.stdout, "{display}");
}
failed
}
fn command() -> Command {
Command::new(Isutf8::NAME)
.version("isutf8 (pi-shell) 17.2.11")
.about("Check whether files are valid UTF-8.")
.override_usage(format_usage("isutf8 [-q|--quiet] [-l|--list] [-i|--invert] [FILE]..."))
.disable_help_flag(true)
.disable_version_flag(true)
.infer_long_args(true)
.arg(
Arg::new(OPT_QUIET)
.short('q')
.long(OPT_QUIET)
.help("suppress all output; report via exit status only")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_LIST)
.short('l')
.long(OPT_LIST)
.help("print only the names of files failing the check")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_INVERT)
.short('i')
.long(OPT_INVERT)
.help("invert the check: valid files fail")
.action(ArgAction::SetTrue),
)
.arg(Arg::new("help").long("help").action(ArgAction::Help))
.arg(
Arg::new(ARG_FILES)
.value_name("FILE")
.num_args(0..)
.value_parser(ValueParser::os_string()),
)
}
/// Streams `input` in [`CHUNK_SIZE`] chunks, carrying an incomplete multi-byte
/// tail (at most 3 bytes) across chunk boundaries.
fn validate(input: &mut impl Read, cancel: &Arc<AtomicBool>) -> io::Result<Verdict> {
let mut buf = vec![0u8; CHUNK_SIZE + 3];
let mut carry = 0usize; // bytes at buf[..carry] carried from the previous chunk
let mut offset = 0u64; // file offset of buf[0]
let mut line = 1u64;
let mut chars_in_line = 0u64; // complete chars decoded on the current line
loop {
if cancel.load(Ordering::Relaxed) {
return Ok(Verdict::Cancelled);
}
let read = input.read(&mut buf[carry..carry + CHUNK_SIZE])?;
let eof = read == 0;
let data_len = carry + read;
if data_len == 0 {
return Ok(Verdict::Valid);
}
let mut pos = 0usize;
while pos < data_len {
match std::str::from_utf8(&buf[pos..data_len]) {
Ok(_) => {
advance(&buf[pos..data_len], &mut line, &mut chars_in_line);
pos = data_len;
},
Err(err) => {
advance(&buf[pos..pos + err.valid_up_to()], &mut line, &mut chars_in_line);
pos += err.valid_up_to();
if err.error_len().is_some() || eof {
// Bad sequence, or an incomplete one truncated by EOF.
return Ok(Verdict::Invalid {
line,
character: chars_in_line + 1,
byte: offset + pos as u64,
});
}
break; // incomplete tail: carry it into the next chunk
},
}
}
if eof {
return Ok(Verdict::Valid);
}
// Slide the unconsumed tail (at most 3 bytes) to the front of the buffer.
buf.copy_within(pos..data_len, 0);
carry = data_len - pos;
offset += pos as u64;
}
}
/// Updates line/char counters over `text`, a slice already known to be valid
/// UTF-8 (chars are counted as non-continuation bytes, so no re-decode).
fn advance(text: &[u8], line: &mut u64, chars_in_line: &mut u64) {
match memchr::memrchr(b'\n', text) {
Some(last) => {
*line += memchr::memchr_iter(b'\n', text).count() as u64;
*chars_in_line = count_chars(&text[last + 1..]);
},
None => *chars_in_line += count_chars(text),
}
}
fn count_chars(bytes: &[u8]) -> u64 {
bytes.iter().filter(|&&byte| (byte & 0xc0) != 0x80).count() as u64
}
fn display_name(name: &OsStr) -> String {
if name == "-" {
STDIN_NAME.to_owned()
} else {
name.to_string_lossy().into_owned()
}
}
/// Creates the `isutf8` builtin registration.
pub(crate) fn isutf8_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Isutf8, SE>()
}
#[cfg(test)]
mod tests {
use std::fs;
use super::Isutf8;
use crate::host::run_util;
fn run_in(cwd: &std::path::Path, stdin: &str, args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Isutf8>(args, stdin, cwd);
(code, capture.out(), capture.err())
}
#[test]
fn valid_ascii_and_multibyte_pass_silently() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("ok"), "hello é 🎉\nplain ascii\n").unwrap();
assert_eq!(run_in(dir.path(), "", &["ok"]), (0, String::new(), String::new()));
}
#[test]
fn invalid_sequence_reports_line_char_and_byte() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("bad"), b"ab\xC3(\n").unwrap();
fs::write(dir.path().join("late"), b"a\nb\n\xFF").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["bad"]);
assert_eq!(code, 1);
assert_eq!(stdout, "bad: line 1, char 3, byte 2: invalid UTF-8 code\n");
assert_eq!(stderr, "");
let (code, stdout, _) = run_in(dir.path(), "", &["late"]);
assert_eq!(code, 1);
assert_eq!(stdout, "late: line 3, char 1, byte 4: invalid UTF-8 code\n");
}
#[test]
fn multibyte_sequence_straddling_chunk_boundary_is_valid() {
let dir = tempfile::tempdir().unwrap();
let mut bytes = vec![b'a'; 65535];
bytes.extend_from_slice("é".as_bytes());
fs::write(dir.path().join("straddle"), &bytes).unwrap();
assert_eq!(
run_in(dir.path(), "", &["straddle"]),
(0, String::new(), String::new())
);
}
#[test]
fn truncated_sequence_at_chunk_boundary_is_invalid() {
let dir = tempfile::tempdir().unwrap();
let mut bytes = vec![b'a'; 65535];
bytes.push(0xc3);
bytes.extend_from_slice(b"zzz");
fs::write(dir.path().join("cut"), &bytes).unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["cut"]);
assert_eq!(code, 1);
assert_eq!(stdout, "cut: line 1, char 65536, byte 65535: invalid UTF-8 code\n");
assert_eq!(stderr, "");
}
#[test]
fn truncated_multibyte_at_eof_is_invalid() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("eof"), b"abc\xE2\x82").unwrap();
let (code, stdout, _) = run_in(dir.path(), "", &["eof"]);
assert_eq!(code, 1);
assert_eq!(stdout, "eof: line 1, char 4, byte 3: invalid UTF-8 code\n");
}
#[test]
fn quiet_suppresses_output_but_keeps_status() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("bad"), b"\xFF").unwrap();
assert_eq!(
run_in(dir.path(), "", &["-q", "bad"]),
(1, String::new(), String::new())
);
}
#[test]
fn list_prints_failing_names_and_invert_flips_them() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("good"), "fine\n").unwrap();
fs::write(dir.path().join("bad"), b"\xFF\n").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["-l", "good", "bad"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (1, "bad\n", ""));
let (code, stdout, stderr) =
run_in(dir.path(), "", &["-l", "-i", "good", "bad"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (1, "good\n", ""));
}
#[test]
fn invert_flips_exit_status_without_list() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("good"), "fine\n").unwrap();
fs::write(dir.path().join("bad"), b"\xFF").unwrap();
assert_eq!(run_in(dir.path(), "", &["-q", "-i", "good"]).0, 1);
assert_eq!(run_in(dir.path(), "", &["-q", "-i", "bad"]).0, 0);
}
#[test]
fn stdin_is_validated_when_no_files_given() {
let dir = tempfile::tempdir().unwrap();
assert_eq!(
run_in(dir.path(), "héllo\n", &[]),
(0, String::new(), String::new())
);
assert_eq!(
run_in(dir.path(), "explicit\n", &["-"]),
(0, String::new(), String::new())
);
}
#[test]
fn missing_file_reports_io_error_and_continues() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("bad"), b"\xFF").unwrap();
let (code, stdout, stderr) = run_in(dir.path(), "", &["nope", "bad"]);
assert_eq!(code, 2);
assert_eq!(stdout, "bad: line 1, char 1, byte 0: invalid UTF-8 code\n");
assert!(stderr.starts_with("isutf8: nope: "), "stderr: {stderr}");
}
}
crates/vendor/brush-builtins/src/jobs.rs → crates/pi-builtins/src/jobs.rs
View File
crates/vendor/jaq/src/help.txt → crates/pi-builtins/src/jq-help.txt
View File
crates/pi-builtins/src/jq.rs
+1550
View File
File diff suppressed because it is too large Load Diff
crates/pi-builtins/src/kill.rs
+494
View File
@@ -0,0 +1,494 @@
//! The `kill` builtin, moved from `pi-shell`.
use std::io::Write;
use brush_core::{
ExecutionExitCode, ExecutionResult, builtins, sys, traps::TrapSignal, ExecutionContext,
};
use clap::Parser;
#[cfg(not(unix))]
use crate::proc_snapshot::{ProcInfo, ProcessStatus};
/// Signal a job or process.
#[derive(Parser)]
pub(crate) struct KillCommand {
/// Name of the signal to send.
#[arg(short = 's', value_name = "SIG_NAME")]
signal_name: Option<String>,
/// Number of the signal to send.
#[arg(short = 'n', value_name = "SIG_NUM")]
signal_number: Option<usize>,
/// List known signal names.
#[arg(short = 'l', short_alias = 'L')]
list_signals: bool,
// Interpretation of these depends on whether -l is present.
#[arg(allow_hyphen_values = true)]
args: Vec<String>,
/// Process/job operands given after the `--` end-of-options marker. clap
/// consumes `--` before `execute`, so these are captured separately and are
/// always operands — never signal specifications (preserves negative PIDs).
#[arg(last = true, allow_hyphen_values = true)]
post_marker_args: Vec<String>,
}
impl builtins::Command for KillCommand {
type Error = brush_core::Error;
#[allow(unknown_lints, reason = "unused_async_trait_impl is unknown to the pinned CI nightly")]
#[allow(
clippy::unused_async_trait_impl,
reason = "the builtin Command trait declares execute as async"
)]
async fn execute<SE: brush_core::ShellExtensions>(
&self,
context: ExecutionContext<'_, SE>,
) -> std::result::Result<ExecutionResult, Self::Error> {
let default_signal = if let Some(signal_name) = &self.signal_name {
if let Ok(signal) = KillSignal::parse(signal_name) {
signal
} else {
writeln!(
context.stderr(),
"{}: invalid signal name: {}",
context.command_name,
signal_name
)?;
return Ok(ExecutionExitCode::InvalidUsage.into());
}
} else {
KillSignal::parse("TERM")?
};
let mut signal = match self.signal_number {
Some(signal_number) => {
let Ok(signal_number) = i32::try_from(signal_number) else {
writeln!(
context.stderr(),
"{}: invalid signal number: {}",
context.command_name,
signal_number
)?;
return Ok(ExecutionExitCode::InvalidUsage.into());
};
if let Ok(signal) = KillSignal::parse(&signal_number.to_string()) {
signal
} else {
writeln!(
context.stderr(),
"{}: invalid signal number: {}",
context.command_name,
signal_number
)?;
return Ok(ExecutionExitCode::InvalidUsage.into());
}
},
None => default_signal,
};
// Interpret the pre-`--` args as an optional leading `-sigspec`, followed
// by PID/jobspec operands. Once a signal or operand has been seen, later
// hyphen-led arguments remain operands so negative process-group IDs survive.
let mut operands: Vec<&String> = Vec::new();
let mut options_done = self.signal_name.is_some() || self.signal_number.is_some();
let mut consumed_marker = false;
for arg in &self.args {
if !consumed_marker && arg == "--" {
consumed_marker = true;
options_done = true;
continue;
}
if !options_done && let Some(spec) = arg.strip_prefix('-').filter(|spec| !spec.is_empty())
{
signal = if let Ok(signal) = KillSignal::parse(spec) {
signal
} else {
writeln!(context.stderr(), "{}: invalid signal name", context.command_name)?;
return Ok(ExecutionExitCode::InvalidUsage.into());
};
options_done = true;
continue;
}
options_done = true;
operands.push(arg);
}
operands.extend(&self.post_marker_args);
if self.list_signals {
return print_kill_signals(&context, operands);
}
if operands.is_empty() {
writeln!(context.stderr(), "{}: invalid usage", context.command_name)?;
return Ok(ExecutionExitCode::InvalidUsage.into());
}
let mut had_failure = false;
for operand in operands {
if context.is_cancelled() {
return Ok(ExecutionExitCode::Interrupted.into());
}
if operand.starts_with('%') {
let Some(job) = context.shell.jobs_mut().resolve_job_spec(operand) else {
writeln!(context.stderr(), "{}: {}: no such job", context.command_name, operand)?;
had_failure = true;
continue;
};
#[cfg(unix)]
{
let mut targets: Vec<i32> = job
.process_ids()
.filter_map(|pid| {
// SAFETY: getpgid reads process-group metadata for a managed child.
let pgid = unsafe { libc::getpgid(pid) };
(pgid > 0).then_some(-pgid)
})
.collect();
if targets.is_empty()
&& let Some(pgid) = job.process_group_id()
{
targets.push(-pgid);
}
targets.sort_unstable();
targets.dedup();
if signal.sends_signal() && targets.iter().copied().any(kill_target_includes_host) {
writeln!(
context.stderr(),
"{}: {}: refusing to signal the shell process",
context.command_name,
operand
)?;
had_failure = true;
continue;
}
let succeeded = match signal {
KillSignal::Probe => targets.iter().copied().any(probe_kill_target),
KillSignal::Signal(signal) => {
let mut succeeded = false;
for target in targets {
if sys::signal::kill_process(target, signal).is_ok() {
succeeded = true;
}
}
succeeded
},
};
if !succeeded {
writeln!(
context.stderr(),
"{}: {}: failed to send signal",
context.command_name,
operand
)?;
had_failure = true;
}
}
#[cfg(windows)]
{
let job_group = job.process_group_id();
if signal.sends_signal()
&& (job_group.is_some_and(kill_job_group_includes_host)
|| job.process_ids().any(kill_pid_is_host))
{
writeln!(
context.stderr(),
"{}: {}: refusing to signal the shell process",
context.command_name,
operand
)?;
had_failure = true;
continue;
}
let expected_handles = job.external_process_count();
let handles = job.duplicate_kill_handles();
let mut succeeded = expected_handles != 0 && handles.len() == expected_handles;
for handle in &handles {
let handled = match signal {
KillSignal::Probe => brush_core::processes::process_handle_is_running(handle),
KillSignal::Signal(_) => {
brush_core::processes::terminate_process_handle(handle)
},
};
if !handled {
succeeded = false;
}
}
if !succeeded {
writeln!(
context.stderr(),
"{}: {}: failed to send signal",
context.command_name,
operand
)?;
had_failure = true;
}
}
#[cfg(all(not(unix), not(windows)))]
{
let job_group = job.process_group_id();
let representative = job.representative_pid();
if signal.sends_signal()
&& (job_group.is_some_and(kill_job_group_includes_host)
|| representative.is_some_and(kill_pid_is_host))
{
writeln!(
context.stderr(),
"{}: {}: refusing to signal the shell process",
context.command_name,
operand
)?;
had_failure = true;
continue;
}
match signal {
KillSignal::Probe => {
if !representative.is_some_and(probe_kill_target) {
writeln!(
context.stderr(),
"{}: {}: failed to send signal",
context.command_name,
operand
)?;
had_failure = true;
}
},
KillSignal::Signal(signal) => {
if let Err(err) = job.kill(signal) {
writeln!(
context.stderr(),
"{}: {}: {}",
context.command_name,
operand,
err
)?;
had_failure = true;
}
},
}
}
continue;
}
let pid = match brush_core::int_utils::parse(operand, 10) {
Ok(pid) => pid,
Err(err) => {
writeln!(context.stderr(), "{}: {}: {}", context.command_name, operand, err)?;
had_failure = true;
continue;
},
};
if signal.sends_signal() && kill_target_includes_host(pid) {
writeln!(
context.stderr(),
"{}: {}: refusing to signal the shell process",
context.command_name,
operand
)?;
had_failure = true;
continue;
}
match signal {
KillSignal::Probe => {
if !probe_kill_target(pid) {
writeln!(
context.stderr(),
"{}: {}: failed to send signal",
context.command_name,
operand
)?;
had_failure = true;
}
},
KillSignal::Signal(signal) => {
if let Err(err) = sys::signal::kill_process(pid, signal) {
writeln!(context.stderr(), "{}: {}: {}", context.command_name, operand, err)?;
had_failure = true;
}
},
}
}
if had_failure {
Ok(ExecutionResult::general_error())
} else {
Ok(ExecutionResult::success())
}
}
}
fn kill_pid_is_host(pid: i32) -> bool {
i32::try_from(std::process::id()).ok() == Some(pid)
}
#[cfg(not(unix))]
fn kill_job_group_includes_host(pgid: i32) -> bool {
kill_pid_is_host(pgid)
}
fn kill_target_includes_host(target: i32) -> bool {
if target == -1 || target == 0 || kill_pid_is_host(target) {
return true;
}
#[cfg(unix)]
{
// SAFETY: getpgrp has no arguments or memory access.
target.checked_neg() == Some(unsafe { libc::getpgrp() })
}
#[cfg(not(unix))]
{
false
}
}
#[cfg(unix)]
fn probe_kill_target(target: i32) -> bool {
// SAFETY: signal 0 only checks target existence and permission.
unsafe { libc::kill(target, 0) == 0 }
}
#[cfg(not(unix))]
fn probe_kill_target(target: i32) -> bool {
target > 0
&& ProcInfo::all().into_iter().any(|process| {
process.pid() == target && process.status() == ProcessStatus::Running
})
}
fn print_kill_signals<'a>(
context: &ExecutionContext<'_, impl brush_core::ShellExtensions>,
signals: impl IntoIterator<Item = &'a String>,
) -> std::result::Result<ExecutionResult, brush_core::Error> {
let mut result = ExecutionResult::success();
let mut signals = signals.into_iter().peekable();
if signals.peek().is_none() {
return brush_core::traps::format_signals(
context.stdout(),
TrapSignal::iterator().filter(|signal| !matches!(signal, TrapSignal::Exit)),
)
.map(|()| ExecutionResult::success());
}
for value in signals {
enum PrintedSignal {
Name(&'static str),
Number(i32),
}
let signal = if let Ok(number) = value.parse::<i32>() {
TrapSignal::try_from(number).map(|signal| {
PrintedSignal::Name(
signal
.as_str()
.strip_prefix("SIG")
.unwrap_or(signal.as_str()),
)
})
} else {
TrapSignal::try_from(value.as_str()).map(|signal| {
i32::try_from(signal)
.map_or(PrintedSignal::Name(signal.as_str()), PrintedSignal::Number)
})
};
match signal {
Ok(PrintedSignal::Name(name)) => writeln!(context.stdout(), "{name}")?,
Ok(PrintedSignal::Number(number)) => writeln!(context.stdout(), "{number}")?,
Err(err) => {
writeln!(context.stderr(), "{err}")?;
result = ExecutionResult::general_error();
},
}
}
Ok(result)
}
#[cfg(test)]
impl KillCommand {
fn listed_signals(&self) -> impl Iterator<Item = &String> {
let mut consumed_marker = false;
self.args
.iter()
.filter(move |arg| {
if !consumed_marker && *arg == "--" {
consumed_marker = true;
false
} else {
true
}
})
.chain(&self.post_marker_args)
}
}
#[cfg(test)]
mod tests {
use super::*;
fn listed(args: &[&str]) -> Vec<String> {
let cmd = KillCommand::try_parse_from(args).unwrap();
cmd.listed_signals().cloned().collect()
}
#[test]
fn lists_post_marker_operands() {
assert_eq!(listed(&["kill", "-l", "--", "9"]), ["9"]);
}
#[test]
fn lists_pre_and_post_marker_operands() {
assert_eq!(listed(&["kill", "-l", "TERM", "--", "9"]), ["TERM", "9"]);
}
#[test]
fn lists_pre_marker_operands_without_marker() {
assert_eq!(listed(&["kill", "-l", "TERM", "HUP"]), ["TERM", "HUP"]);
}
}
/// A `kill` signal argument: a real signal, or the "does this process
/// exist?" probe that signal 0 requests.
#[derive(Clone, Copy)]
enum KillSignal {
Probe,
Signal(TrapSignal),
}
impl KillSignal {
fn parse(value: &str) -> std::result::Result<Self, brush_core::Error> {
if let Ok(number) = value.parse::<i32>() {
if number == 0 {
Ok(Self::Probe)
} else {
TrapSignal::try_from(number).map(Self::Signal)
}
} else {
TrapSignal::try_from(value).map(Self::Signal)
}
}
const fn sends_signal(self) -> bool {
matches!(self, Self::Signal(_))
}
}
/// Resolves a signal name or number to its number.
///
/// Shared with `pkill`, which accepts the same `-SIGNAL` spellings.
#[allow(
dead_code,
reason = "shared with optional process-match builtins that may be feature-disabled"
)]
pub(crate) fn signal_number(value: &str) -> Option<i32> {
let value = value
.strip_prefix("SIG")
.or_else(|| value.strip_prefix("sig"))
.unwrap_or(value);
if let Ok(number) = value.parse::<i32>() {
#[cfg(target_os = "linux")]
return (0..=libc::SIGRTMAX()).contains(&number).then_some(number);
#[cfg(target_os = "macos")]
return (0..=31).contains(&number).then_some(number);
#[cfg(not(unix))]
return (0..=64).contains(&number).then_some(number);
}
match KillSignal::parse(value).ok()? {
KillSignal::Probe => Some(0),
KillSignal::Signal(signal) => i32::try_from(signal).ok(),
}
}
crates/vendor/brush-builtins/src/let_.rs → crates/pi-builtins/src/let_.rs
View File
crates/pi-builtins/src/lib.rs
+309
View File
@@ -0,0 +1,309 @@
//! Standard builtins.
#[cfg(feature = "builtin.alias")]
mod alias;
#[cfg(feature = "builtin.bg")]
mod bg;
#[cfg(feature = "builtin.bind")]
mod bind;
#[cfg(feature = "builtin.break")]
mod break_;
#[cfg(feature = "builtin.builtin")]
mod builtin_;
#[cfg(feature = "builtin.caller")]
mod caller;
#[cfg(feature = "builtin.cd")]
mod cd;
#[cfg(feature = "builtin.colon")]
mod colon;
#[cfg(feature = "builtin.command")]
mod command;
#[cfg(any(feature = "builtin.complete", feature = "builtin.compgen", feature = "builtin.compopt"))]
mod complete;
#[cfg(feature = "builtin.continue")]
mod continue_;
#[cfg(feature = "builtin.declare")]
mod declare;
#[cfg(feature = "builtin.dirs")]
mod dirs;
#[cfg(feature = "builtin.dot")]
mod dot;
#[cfg(feature = "builtin.echo")]
mod echo;
#[cfg(feature = "builtin.enable")]
mod enable;
#[cfg(feature = "builtin.eval")]
mod eval;
#[cfg(all(feature = "builtin.exec", unix))]
mod exec;
#[cfg(feature = "builtin.exit")]
mod exit;
#[cfg(feature = "builtin.export")]
mod export;
#[cfg(feature = "builtin.false")]
mod false_;
#[cfg(feature = "builtin.fc")]
mod fc;
#[cfg(feature = "builtin.fg")]
mod fg;
#[cfg(feature = "builtin.getopts")]
mod getopts;
#[cfg(feature = "builtin.hash")]
mod hash;
#[cfg(feature = "builtin.help")]
mod help;
#[cfg(feature = "builtin.history")]
mod history;
#[cfg(feature = "builtin.jobs")]
mod jobs;
// Needs either unix signals or the windows process table; no other target
// has a way to signal a process (upstream brush gated this to unix alone).
#[cfg(all(feature = "builtin.kill", any(unix, windows)))]
mod kill;
#[cfg(feature = "builtin.let")]
mod let_;
#[cfg(feature = "builtin.mapfile")]
mod mapfile;
#[cfg(feature = "builtin.popd")]
mod popd;
#[cfg(all(feature = "builtin.printf", any(unix, windows)))]
mod printf;
#[cfg(feature = "builtin.pushd")]
mod pushd;
#[cfg(feature = "builtin.pwd")]
mod pwd;
#[cfg(feature = "builtin.read")]
mod read;
#[cfg(feature = "builtin.return")]
mod return_;
#[cfg(feature = "builtin.set")]
mod set;
#[cfg(feature = "builtin.shift")]
mod shift;
#[cfg(feature = "builtin.shopt")]
mod shopt;
#[cfg(all(feature = "builtin.suspend", unix))]
mod suspend;
#[cfg(feature = "builtin.test")]
mod test;
#[cfg(feature = "builtin.times")]
mod times;
#[cfg(feature = "builtin.trap")]
mod trap;
#[cfg(feature = "builtin.true")]
mod true_;
#[cfg(feature = "builtin.type")]
mod type_;
#[cfg(all(feature = "builtin.ulimit", unix))]
mod ulimit;
#[cfg(all(feature = "builtin.umask", unix))]
mod umask;
#[cfg(feature = "builtin.unalias")]
mod unalias;
#[cfg(feature = "builtin.unset")]
mod unset;
#[cfg(feature = "builtin.wait")]
mod wait;
mod builder;
mod factory;
mod host;
mod unimp;
// ── Utility builtins ──────────────────────────────────────────────────────────
// Ports of the standalone command-line utilities the shell ships in-process.
// Each runs against the explicit `host::Host` view of the shell; see
// `src/host.rs` for the contract.
#[cfg(feature = "util.b2sum")]
mod b2sum;
#[cfg(feature = "util.base32")]
mod base32;
#[cfg(feature = "util.base64")]
mod base64;
#[cfg(feature = "util.basename")]
mod basename;
#[cfg(feature = "util.cat")]
mod cat;
/// Shared checksum machinery behind `md5sum`, `sha*sum`, and `b2sum`.
#[cfg(feature = "util.cksum")]
mod cksum;
#[cfg(feature = "util.md5sum")]
mod md5sum;
#[cfg(feature = "util.sha1sum")]
mod sha1sum;
#[cfg(feature = "util.sha224sum")]
mod sha224sum;
#[cfg(feature = "util.sha256sum")]
mod sha256sum;
#[cfg(feature = "util.sha384sum")]
mod sha384sum;
#[cfg(feature = "util.sha512sum")]
mod sha512sum;
#[cfg(feature = "util.cmp")]
mod cmp;
#[cfg(feature = "util.comm")]
mod comm;
#[cfg(feature = "util.combine")]
mod combine;
#[cfg(feature = "util.cut")]
mod cut;
#[cfg(feature = "util.date")]
mod date;
#[cfg(feature = "util.diff")]
mod diff;
#[cfg(feature = "util.dirname")]
mod dirname;
#[cfg(all(feature = "util.errno", unix))]
mod errno;
#[cfg(feature = "util.fd")]
mod fd;
#[cfg(feature = "util.find")]
mod find;
#[cfg(feature = "util.grep")]
mod grep;
#[cfg(feature = "util.head")]
mod head;
#[cfg(feature = "util.hostname")]
mod hostname;
#[cfg(feature = "util.ifne")]
mod ifne;
#[cfg(feature = "util.isutf8")]
mod isutf8;
#[cfg(feature = "util.jq")]
mod jq;
#[cfg(feature = "util.ln")]
mod ln;
#[cfg(feature = "util.ls")]
mod ls;
#[cfg(feature = "util.mkdir")]
mod mkdir;
#[cfg(feature = "util.mktemp")]
mod mktemp;
#[cfg(feature = "util.mv")]
mod mv;
#[cfg(feature = "util.nproc")]
mod nproc;
#[cfg(feature = "util.paste")]
mod paste;
#[cfg(feature = "util.nohup")]
mod nohup;
#[cfg(feature = "util.pgrep")]
mod pgrep;
#[cfg(feature = "util.pidwait")]
mod pidwait;
#[cfg(feature = "util.pkill")]
mod pkill;
/// Shared process-matching engine behind `pgrep`, `pkill`, and `pidwait`.
#[cfg(feature = "util.proc-match")]
mod proc_match;
/// Shared process-table snapshot behind the process builtins.
#[cfg(feature = "util.procs")]
mod proc_snapshot;
#[cfg(feature = "util.ps")]
mod ps;
#[cfg(feature = "util.sleep")]
mod sleep;
#[cfg(feature = "util.timeout")]
mod timeout;
#[cfg(feature = "util.top")]
mod top;
#[cfg(feature = "util.printenv")]
mod printenv;
#[cfg(feature = "util.readlink")]
mod readlink;
#[cfg(feature = "util.realpath")]
mod realpath;
#[cfg(feature = "util.rg")]
mod rg;
#[cfg(feature = "util.rm")]
mod rm;
#[cfg(feature = "util.sed")]
mod sed;
#[cfg(feature = "util.seq")]
mod seq;
#[cfg(feature = "util.sort")]
mod sort;
#[cfg(feature = "util.sponge")]
mod sponge;
#[cfg(feature = "util.stat")]
mod stat;
#[cfg(feature = "util.tac")]
mod tac;
#[cfg(feature = "util.tail")]
mod tail;
#[cfg(feature = "util.tee")]
mod tee;
#[cfg(feature = "util.touch")]
mod touch;
#[cfg(feature = "util.tr")]
mod tr;
#[cfg(feature = "util.truncate")]
mod truncate;
#[cfg(feature = "util.ts")]
mod ts;
#[cfg(feature = "util.uname")]
mod uname;
#[cfg(feature = "util.uniq")]
mod uniq;
#[cfg(feature = "util.wc")]
mod wc;
#[cfg(feature = "util.which")]
mod which;
#[cfg(feature = "util.whoami")]
mod whoami;
#[cfg(feature = "util.xargs")]
mod xargs;
#[cfg(feature = "util.yes")]
mod yes;
pub use builder::ShellBuilderExt;
pub use factory::{BuiltinSet, default_builtins, process_builtins, utility_builtins};
pub use host::{panic_scope_active, rayon_global_pool_available, set_rayon_global_pool_available};
/// The process table the process builtins read, and the liveness state of an
/// entry in it. Public so an embedding shell can inspect processes through the
/// same snapshot its `ps`/`pgrep`/`kill` builtins use.
#[cfg(feature = "util.procs")]
pub use proc_snapshot::{ProcInfo, ProcessStatus};
/// Macro to define a struct that represents a shell built-in flag argument that
/// can be enabled or disabled by specifying an option with a leading '+' or '-'
/// character.
///
/// # Arguments
///
/// - `$struct_name` - The identifier to be used for the struct to define.
/// - `$flag_char` - The character to use as the flag.
/// - `$desc` - The string description of the flag.
#[macro_export]
macro_rules! minus_or_plus_flag_arg {
($struct_name:ident, $flag_char:literal, $desc:literal) => {
#[derive(clap::Parser)]
pub(crate) struct $struct_name {
#[arg(short = $flag_char, name = concat!(stringify!($struct_name), "_enable"), action = clap::ArgAction::SetTrue, help = $desc)]
_enable: bool,
#[arg(long = concat!("+", $flag_char), name = concat!(stringify!($struct_name), "_disable"), action = clap::ArgAction::SetTrue, hide = true)]
_disable: bool,
}
impl From<$struct_name> for Option<bool> {
fn from(value: $struct_name) -> Self {
value.to_bool()
}
}
impl $struct_name {
#[allow(dead_code, reason = "may not be used in all macro instantiations")]
pub const fn is_some(&self) -> bool {
self._enable || self._disable
}
pub const fn to_bool(&self) -> Option<bool> {
match (self._enable, self._disable) {
(true, false) => Some(true),
(false, true) => Some(false),
_ => None,
}
}
}
};
}
crates/vendor/uu-ln/src/ln.rs → crates/pi-builtins/src/ln.rs
+189 -288
View File
@@ -1,19 +1,6 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
// spell-checker:ignore (ToDO) srcpath targetpath EEXIST
// pi-uutils: vendored from uutils/coreutils 0.8.0 and patched to run in-process
// as a shell builtin. Every filesystem syscall resolves its path operand
// against the shell working directory via `pi_uutils_ctx::resolve` AT THE CALL
// SITE, while the original operands are kept for display/error messages (GNU
// prints operands as typed) — and, crucially, for the CONTENT of symbolic
// links, which stays exactly as typed like GNU ln (only the location where the
// link is created gets resolved). All process-global stdio and the `-i` prompt
// are routed through `pi_uutils_ctx`, `translate!` strings are literalized, and
// the entry point no longer calls `std::process::exit`.
//! `ln` builtin: make hard and symbolic links.
//!
//! Ported from uutils coreutils 0.8.0.
#[cfg(any(unix, target_os = "redox"))]
use std::os::unix::fs::symlink;
@@ -24,23 +11,24 @@ use std::{
collections::HashSet,
ffi::OsString,
fs,
io::Write,
io::{Read, Write},
path::{Path, PathBuf},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use pi_uutils_ctx::format_usage;
use thiserror::Error;
use uucore::{
backup_control::{self, BackupMode},
display::Quotable,
error::{FromIo, UError, UResult, USimpleError, strip_errno},
fs::{
MissingHandling, ResolveMode, canonicalize, make_path_relative_to, paths_refer_to_same_file,
},
};
pub struct Settings {
use crate::host::{Host, Utility, format_usage, matches_parser, util};
struct Settings {
overwrite: OverwriteMode,
backup: BackupMode,
suffix: OsString,
@@ -54,14 +42,12 @@ pub struct Settings {
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum OverwriteMode {
enum OverwriteMode {
NoClobber,
Interactive,
Force,
}
// pi-uutils: the `translate!` message templates are literalized with the
// en-US strings from upstream's locales/en-US.ftl.
#[derive(Error, Debug)]
enum LnError {
#[error("target {} is not a directory", _0.quote())]
@@ -81,13 +67,14 @@ enum LnError {
#[error("{}: hard link not allowed for directory", _0.to_string_lossy())]
FailedToCreateHardLinkDir(PathBuf),
#[error("{0}")]
Message(String),
#[error("{0}")]
Io(#[from] std::io::Error),
}
impl UError for LnError {
fn code(&self) -> i32 {
1
}
}
mod options {
pub const FORCE: &str = "force";
@@ -105,29 +92,28 @@ mod options {
static ARG_FILES: &str = "files";
/// pi-uutils: replacement for uucore's `show_error!` — writes the diagnostic
/// to the context stderr instead of the process-global one. Errors that render
/// to an empty message (e.g. [`LnError::SomeLinksFailed`]) print nothing
/// rather than a dangling "ln: " prefix.
fn show_error(msg: impl std::fmt::Display) {
/// Parsed `ln` invocation.
pub(crate) struct Ln {
matches: ArgMatches,
}
matches_parser!(Ln, uu_app);
type LnResult<T> = Result<T, LnError>;
fn show_error(host: &mut Host, msg: impl std::fmt::Display) {
let rendered = msg.to_string();
if !rendered.is_empty() {
let _ = writeln!(pi_uutils_ctx::stderr(), "ln: {rendered}");
let _ = writeln!(host.stderr, "ln: {rendered}");
}
}
/// pi-uutils: replacement for uucore's `read_yes`, reading from the context
/// stdin one byte at a time (no buffering) so consecutive prompts don't
/// over-read into a later prompt's input. Returns true when the first character
/// of the line is `y`/`Y`.
fn read_yes() -> bool {
use std::io::Read as _;
let mut stdin = pi_uutils_ctx::stdin();
fn read_yes(host: &mut Host) -> bool {
let mut buf = [0u8; 1];
let mut first = None;
loop {
match stdin.read(&mut buf) {
Ok(0) => break, // EOF
match host.stdin.read(&mut buf) {
Ok(0) => break,
Ok(_) => {
if buf[0] == b'\n' {
break;
@@ -142,51 +128,27 @@ fn read_yes() -> bool {
matches!(first, Some(b'y' | b'Y'))
}
/// pi-uutils: replacement for uucore's `prompt_yes!` — writes
/// "ln: \<prompt\> " to the context stderr, then reads the answer from the
/// context stdin.
fn prompt_yes(prompt: impl std::fmt::Display) -> bool {
let mut err = pi_uutils_ctx::stderr();
let _ = write!(err, "ln: {prompt} ");
let _ = err.flush();
read_yes()
fn prompt_yes(host: &mut Host, prompt: impl std::fmt::Display) -> bool {
let _ = write!(host.stderr, "ln: {prompt} ");
let _ = host.stderr.flush();
read_yes(host)
}
/// In-process builtin entry point. Unlike upstream's `uumain`, this parses the
/// arguments directly (without the uucore clap-localization helper that would
/// terminate the process), renders clap help/usage/version to the context
/// streams, and maps the `UResult` to an exit code, so it is safe to run inside
/// the host shell process.
pub fn run(argv: Vec<OsString>) -> i32 {
let matches = match uu_app().try_get_matches_from(argv) {
Ok(matches) => matches,
Err(err) => {
let rendered = err.to_string();
if err.use_stderr() {
let _ = write!(pi_uutils_ctx::stderr(), "{rendered}");
return 1;
}
let _ = write!(pi_uutils_ctx::stdout(), "{rendered}");
return 0;
},
};
match ln_main(&matches) {
Ok(()) => pi_uutils_ctx::exit_code(),
Err(err) => {
let code = err.code();
// pi-uutils: `SomeLinksFailed` renders to an empty message
// (upstream prints the per-file diagnostics as it goes); don't
// emit a dangling "ln: " prefix for it.
let msg = err.to_string();
if !msg.is_empty() {
let _ = writeln!(pi_uutils_ctx::stderr(), "ln: {msg}");
}
if code == 0 { 1 } else { code }
},
impl Utility for Ln {
const NAME: &'static str = "ln";
fn run(self, host: &mut Host) -> i32 {
match ln_main(&self.matches, host) {
Ok(()) => host.exit_code(),
Err(err) => {
show_error(host, err);
1
},
}
}
}
fn ln_main(matches: &ArgMatches) -> UResult<()> {
fn ln_main(matches: &ArgMatches, host: &mut Host) -> LnResult<()> {
/* the list of files */
let paths: Vec<PathBuf> = matches
@@ -205,7 +167,8 @@ fn ln_main(matches: &ArgMatches) -> UResult<()> {
OverwriteMode::NoClobber
};
let backup_mode = backup_control::determine_backup_mode(matches)?;
let backup_mode = backup_control::determine_backup_mode(matches)
.map_err(|error| LnError::Message(error.to_string()))?;
let backup_suffix = backup_control::determine_backup_suffix(matches);
// When we have "-L" or "-L -P", false otherwise
@@ -226,10 +189,10 @@ fn ln_main(matches: &ArgMatches) -> UResult<()> {
verbose: matches.get_flag(options::VERBOSE),
};
exec(&paths[..], &settings)
exec(host, &paths[..], &settings)
}
pub fn uu_app() -> Command {
fn uu_app() -> Command {
let after_help = format!(
"In the 1st form, create a link to TARGET with the name LINK_NAME.\nIn the 2nd form, create \
a link to TARGET in the current directory.\nIn the 3rd and 4th forms, create links to each \
@@ -241,16 +204,14 @@ pub fn uu_app() -> Command {
);
Command::new("ln")
.version(uucore::crate_version!())
.version("0.8.0")
.about("Make links between files.")
.override_usage(format_usage(
"ln [OPTION]... [-T] TARGET LINK_NAME\nln [OPTION]... TARGET\nln [OPTION]... TARGET... \
DIRECTORY\nln [OPTION]... -t DIRECTORY TARGET...",
))
.infer_long_args(true)
// pi-uutils: free the `-h` short for the BSD `--no-dereference`
// alias below; neither GNU nor BSD ln has `-h` help, and `--help`
// keeps working via the explicit long-only arg.
// Free `-h` for the BSD `--no-dereference` alias; `--help` remains.
.disable_help_flag(true)
.arg(
Arg::new("help")
@@ -286,8 +247,7 @@ pub fn uu_app() -> Command {
.arg(
Arg::new(options::NO_DEREFERENCE)
.short('n')
// pi-uutils: BSD/macOS ln spells this flag `-h` (`ln -sfh` is
// common macOS muscle memory); hidden alias, GNU help shape.
// BSD/macOS spells `--no-dereference` as `-h`.
.short_alias('h')
.long(options::NO_DEREFERENCE)
.help("treat LINK_NAME as a normal file if it is a\nsymbolic link to a directory")
@@ -361,22 +321,27 @@ pub fn uu_app() -> Command {
)
}
fn exec(files: &[PathBuf], settings: &Settings) -> UResult<()> {
/// Creates the `ln` builtin registration.
pub(crate) fn ln_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Ln, SE>()
}
fn exec(host: &mut Host, files: &[PathBuf], settings: &Settings) -> LnResult<()> {
// Handle cases where we create links in a directory first.
if let Some(target_path) = &settings.target_dir {
// 4th form: a directory is specified by -t.
return link_files_in_dir(files, target_path, settings);
return link_files_in_dir(host, files, target_path, settings);
}
if !settings.no_target_dir {
if files.len() == 1 {
// 2nd form: the target directory is the current directory.
return link_files_in_dir(files, &PathBuf::from("."), settings);
return link_files_in_dir(host, files, &PathBuf::from("."), settings);
}
let last_file = &PathBuf::from(files.last().unwrap());
// pi-uutils: probe the destination via the resolved path.
if files.len() > 2 || pi_uutils_ctx::resolve(last_file).is_dir() {
if files.len() > 2 || host.resolve(last_file).is_dir() {
// 3rd form: create links in the last argument.
return link_files_in_dir(&files[0..files.len() - 1], last_file, settings);
return link_files_in_dir(host, &files[0..files.len() - 1], last_file, settings);
}
}
@@ -386,20 +351,21 @@ fn exec(files: &[PathBuf], settings: &Settings) -> UResult<()> {
return Err(LnError::MissingDestination(files[0].clone()).into());
}
if files.len() > 2 {
// pi-uutils: `uucore::execution_phrase()` reads the process argv,
// which is the host shell's; the builtin is always invoked as "ln".
return Err(LnError::ExtraOperand(files[2].clone().into(), "ln".to_string()).into());
return Err(LnError::ExtraOperand(files[2].clone().into(), "ln".to_string()));
}
assert!(!files.is_empty());
link(&files[0], &files[1], settings)
link(host, &files[0], &files[1], settings)
}
#[allow(clippy::cognitive_complexity)]
fn link_files_in_dir(files: &[PathBuf], target_dir: &Path, settings: &Settings) -> UResult<()> {
// pi-uutils: resolved target directory for every syscall below; the
// operand keeps its as-typed spelling for display and link-name building.
let target_dir_fs = pi_uutils_ctx::resolve(target_dir);
fn link_files_in_dir(
host: &mut Host,
files: &[PathBuf],
target_dir: &Path,
settings: &Settings,
) -> LnResult<()> {
// Keep the operand spelling for diagnostics and link-name construction.
let target_dir_fs = host.resolve(target_dir);
if !target_dir_fs.is_dir() {
return Err(LnError::TargetIsNotADirectory(target_dir.to_owned()).into());
}
@@ -409,33 +375,30 @@ fn link_files_in_dir(files: &[PathBuf], target_dir: &Path, settings: &Settings)
let mut all_successful = true;
for srcpath in files {
let targetpath = if settings.no_dereference && target_dir_fs.is_symlink() {
let remove_target = || {
// In that case, we don't want to do link resolution
// We need to clean the target
let remove_target = |host: &mut Host| {
// In that case, we don't want to do link resolution.
if target_dir_fs.is_file()
&& let Err(e) = fs::remove_file(&target_dir_fs)
{
show_error(format_args!("Could not update {}: {e}", target_dir.quote()));
show_error(host, format_args!("Could not update {}: {e}", target_dir.quote()));
}
#[cfg(windows)]
if target_dir_fs.is_dir() {
// Not sure why but on Windows, the symlink can be
// considered as a dir
// See test_ln::test_symlink_no_deref_dir
// On Windows a directory symlink can be considered a directory.
if let Err(e) = fs::remove_dir(&target_dir_fs) {
show_error(format_args!("Could not update {}: {e}", target_dir.quote()));
show_error(host, format_args!("Could not update {}: {e}", target_dir.quote()));
}
}
};
match settings.overwrite {
OverwriteMode::NoClobber => {},
OverwriteMode::Interactive => {
if prompt_yes(format_args!("replace {}?", target_dir.quote())) {
remove_target();
if prompt_yes(host, format_args!("replace {}?", target_dir.quote())) {
remove_target(host);
}
},
OverwriteMode::Force => {
remove_target();
remove_target(host);
},
}
target_dir.to_path_buf()
@@ -449,21 +412,21 @@ fn link_files_in_dir(files: &[PathBuf], target_dir: &Path, settings: &Settings)
None => target_dir.join(name),
}
} else {
show_error(format_args!("cannot stat {}: No such file or directory", srcpath.quote()));
show_error(host, format_args!("cannot stat {}: No such file or directory", srcpath.quote()));
all_successful = false;
continue;
};
if linked_destinations.contains(&targetpath) {
// If the target file was already created in this ln call, do not overwrite
show_error(format_args!(
show_error(host, format_args!(
"will not overwrite just-created {} with {}",
targetpath.quote(),
srcpath.quote()
));
all_successful = false;
} else if let Err(e) = link(srcpath, &targetpath, settings) {
show_error(format_args!("{e}"));
} else if let Err(e) = link(host, srcpath, &targetpath, settings) {
show_error(host, format_args!("{e}"));
all_successful = false;
}
@@ -476,14 +439,12 @@ fn link_files_in_dir(files: &[PathBuf], target_dir: &Path, settings: &Settings)
}
}
fn relative_path<'a>(src: &'a Path, dst: &Path) -> Cow<'a, Path> {
// pi-uutils: canonicalize from the resolved operands so `-r` computes the
// link text against the shell working directory (uucore's canonicalize
// would otherwise fall back to the process cwd for relative paths).
fn relative_path<'a>(host: &Host, src: &'a Path, dst: &Path) -> Cow<'a, Path> {
// Resolve before canonicalizing so `-r` computes against the shell cwd.
if let Ok(src_abs) =
canonicalize(pi_uutils_ctx::resolve(src), MissingHandling::Missing, ResolveMode::Physical)
canonicalize(host.resolve(src), MissingHandling::Missing, ResolveMode::Physical)
&& let Ok(dst_abs) = canonicalize(
pi_uutils_ctx::resolve(dst.parent().unwrap()),
host.resolve(dst.parent().unwrap()),
MissingHandling::Missing,
ResolveMode::Physical,
) {
@@ -492,25 +453,21 @@ fn relative_path<'a>(src: &'a Path, dst: &Path) -> Cow<'a, Path> {
src.into()
}
#[allow(clippy::cognitive_complexity)]
fn link(src: &Path, dst: &Path, settings: &Settings) -> UResult<()> {
fn link(host: &mut Host, src: &Path, dst: &Path, settings: &Settings) -> LnResult<()> {
let mut backup_path = None;
let source: Cow<'_, Path> = if settings.relative {
relative_path(src, dst)
relative_path(host, src, dst)
} else {
src.into()
};
// pi-uutils: resolved counterparts of both operands for every filesystem
// syscall below. `src`/`dst`/`source` keep the as-typed spelling for
// display — and `source` is what gets stored as the symlink CONTENT, so it
// must never be resolved.
let src_fs = pi_uutils_ctx::resolve(src);
let dst_fs = pi_uutils_ctx::resolve(dst);
// Resolve both filesystem operands, but never resolve `source`: it is the
// text stored inside a symbolic link.
let src_fs = host.resolve(src);
let dst_fs = host.resolve(dst);
if dst_fs.is_symlink() || dst_fs.exists() {
// pi-uutils: probe numbered backups from the resolved destination so
// the directory scan hits the shell's working directory.
// Probe numbered backups from the resolved destination.
backup_path = backup_control::get_backup_path(settings.backup, &dst_fs, &settings.suffix);
if settings.backup == BackupMode::Existing && !settings.symbolic {
// when ln --backup f f, it should detect that it is the same file
@@ -519,12 +476,13 @@ fn link(src: &Path, dst: &Path, settings: &Settings) -> UResult<()> {
}
}
if let Some(p) = &backup_path {
fs::rename(&dst_fs, p).map_err_context(|| format!("cannot backup {}", dst.quote()))?;
fs::rename(&dst_fs, p)
.map_err(|e| LnError::Message(format!("cannot backup {}: {e}", dst.quote())))?;
}
match settings.overwrite {
OverwriteMode::NoClobber => {},
OverwriteMode::Interactive => {
if !prompt_yes(format_args!("replace {}?", dst.quote())) {
if !prompt_yes(host, format_args!("replace {}?", dst.quote())) {
return Err(LnError::SomeLinksFailed.into());
}
@@ -552,25 +510,21 @@ fn link(src: &Path, dst: &Path, settings: &Settings) -> UResult<()> {
}
}
let res: UResult<()> = if settings.symbolic {
// pi-uutils: the link is created at the resolved location, but its
// content (`source`) stays exactly as typed, like GNU ln. uucore's
// io-error conversion renders EEXIST as "Already exists"; format the
// GNU-style diagnostic ("failed to create symbolic link 'x': File
// exists") from the raw OS error instead.
symlink(&source, &dst_fs).map_err(|e| {
USimpleError::new(
1,
format!("failed to create symbolic link {}: {}", dst.quote(), strip_errno(&e)),
)
let res: LnResult<()> = if settings.symbolic {
make_symlink(host, &source, &dst_fs).map_err(|e| {
LnError::Message(format!(
"failed to create symbolic link {}: {}",
dst.quote(),
strip_errno(&e)
))
})
} else {
// pi-uutils: hard links dereference their target, so the resolved
// source is what the syscalls get.
let source_fs = pi_uutils_ctx::resolve(&source);
// Hard links dereference their target, so syscalls get the resolved source.
let source_fs = host.resolve(&source);
let p = if settings.logical && source_fs.is_symlink() {
fs::canonicalize(&source_fs)
.map_err_context(|| format!("failed to access {}", source.quote()))?
fs::canonicalize(&source_fs).map_err(|e| {
LnError::Message(format!("failed to access {}: {e}", source.quote()))
})?
} else {
source_fs
};
@@ -579,36 +533,30 @@ fn link(src: &Path, dst: &Path, settings: &Settings) -> UResult<()> {
Err(_) if p.is_dir() => {
Err(LnError::FailedToCreateHardLinkDir(source.to_path_buf()).into())
},
// pi-uutils: same GNU-style rendering as the symlink arm (uucore
// would print "Already exists" for EEXIST).
Err(e) => Err(USimpleError::new(
1,
format!(
"failed to create hard link {} => {}: {}",
source.quote(),
dst.quote(),
strip_errno(&e)
),
)),
Err(e) => Err(LnError::Message(format!(
"failed to create hard link {} => {}: {}",
source.quote(),
dst.quote(),
strip_errno(&e)
))),
}
};
if let Err(e) = res {
if let Some(p) = &backup_path {
fs::rename(p, &dst_fs).map_err_context(|| format!("cannot backup {}", dst.quote()))?;
fs::rename(p, &dst_fs)
.map_err(|e| LnError::Message(format!("cannot backup {}: {e}", dst.quote())))?;
}
return Err(e);
}
if settings.verbose {
// pi-uutils: verbose output goes to the context stdout.
let mut out = pi_uutils_ctx::stdout();
let out = &mut host.stdout;
write!(out, "{} -> {}", dst.quote(), source.quote())?;
match backup_path {
Some(path) => {
// pi-uutils: `path` derives from the resolved (absolute)
// destination; rebuild a display path from the operand for
// the verbose message.
// Rebuild a display path from the operand because the backup path is resolved.
let backup_display = match (dst.parent(), path.file_name()) {
(Some(parent), Some(name)) if !parent.as_os_str().is_empty() => parent.join(name),
(_, Some(name)) => PathBuf::from(name),
@@ -622,12 +570,30 @@ fn link(src: &Path, dst: &Path, settings: &Settings) -> UResult<()> {
Ok(())
}
fn strip_errno(error: &std::io::Error) -> String {
let rendered = error.to_string();
rendered
.rsplit_once(" (os error ")
.map_or(rendered.as_str(), |(message, _)| message)
.to_string()
}
#[cfg(any(unix, target_os = "redox"))]
fn make_symlink<P1: AsRef<Path>, P2: AsRef<Path>>(
_host: &Host,
src: P1,
dst: P2,
) -> std::io::Result<()> {
symlink(src, dst)
}
#[cfg(windows)]
pub fn symlink<P1: AsRef<Path>, P2: AsRef<Path>>(src: P1, dst: P2) -> std::io::Result<()> {
// pi-uutils: the dir/file probe resolves the target against the shell
// working directory (upstream consults the process cwd); the stored link
// content is still the caller's as-typed `src`.
if pi_uutils_ctx::resolve(src.as_ref()).is_dir() {
fn make_symlink<P1: AsRef<Path>, P2: AsRef<Path>>(
host: &Host,
src: P1,
dst: P2,
) -> std::io::Result<()> {
if host.resolve(src.as_ref()).is_dir() {
symlink_dir(src, dst)
} else {
symlink_file(src, dst)
@@ -635,7 +601,11 @@ pub fn symlink<P1: AsRef<Path>, P2: AsRef<Path>>(src: P1, dst: P2) -> std::io::R
}
#[cfg(target_os = "wasi")]
fn symlink<P1: AsRef<Path>, P2: AsRef<Path>>(_src: P1, _dst: P2) -> std::io::Result<()> {
fn make_symlink<P1: AsRef<Path>, P2: AsRef<Path>>(
_host: &Host,
_src: P1,
_dst: P2,
) -> std::io::Result<()> {
Err(std::io::Error::new(
std::io::ErrorKind::Unsupported,
"symlinks not supported on this platform",
@@ -644,61 +614,22 @@ fn symlink<P1: AsRef<Path>, P2: AsRef<Path>>(_src: P1, _dst: P2) -> std::io::Res
#[cfg(test)]
mod tests {
use std::{collections::HashMap, io::Write, path::PathBuf, sync::Arc};
use std::{fs, path::PathBuf};
use parking_lot::Mutex;
use pi_uutils_ctx::ScopeIo;
use super::Ln;
use crate::host::run_util;
use super::*;
fn run_with_stdin(cwd: PathBuf, args: Vec<&str>, stdin: &[u8]) -> (i32, String, String) {
let stdout_buf = Arc::new(Mutex::new(Vec::new()));
let stderr_buf = Arc::new(Mutex::new(Vec::new()));
#[derive(Clone)]
struct SharedWriter {
buf: Arc<Mutex<Vec<u8>>>,
}
impl Write for SharedWriter {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
self.buf.lock().write(buf)
}
fn flush(&mut self) -> std::io::Result<()> {
self.buf.lock().flush()
}
}
let io = ScopeIo {
stdin: Box::new(std::io::Cursor::new(stdin.to_vec())),
stdin_fd: None,
stdin_is_search_input: false,
stdout: Box::new(SharedWriter { buf: stdout_buf.clone() }),
stderr: Box::new(SharedWriter { buf: stderr_buf.clone() }),
cwd,
env: HashMap::new(),
cancel: Arc::new(std::sync::atomic::AtomicBool::new(false)),
};
let argv: Vec<OsString> = std::iter::once("ln")
.chain(args)
.map(OsString::from)
.collect();
let code = pi_uutils_ctx::scope(io, || run(argv));
let out_str = String::from_utf8(stdout_buf.lock().clone()).unwrap();
let err_str = String::from_utf8(stderr_buf.lock().clone()).unwrap();
(code, out_str, err_str)
fn run_in(cwd: PathBuf, args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Ln>(args, "", cwd);
(code, capture.out(), capture.err())
}
fn run_in(cwd: PathBuf, args: Vec<&str>) -> (i32, String, String) {
run_with_stdin(cwd, args, b"")
fn run_with_stdin(cwd: PathBuf, args: &[&str], stdin: &str) -> (i32, String, String) {
let (code, capture) = run_util::<Ln>(args, stdin, cwd);
(code, capture.out(), capture.err())
}
/// Canonicalized temp dir (macOS tempdirs live behind /var -> /private/var,
/// which canonicalizing code paths would otherwise expand mid-assertion).
/// Canonicalized temp dir (macOS tempdirs live behind /var -> /private/var).
fn canonical_tempdir() -> (tempfile::TempDir, PathBuf) {
let dir = tempfile::tempdir().unwrap();
let canon = fs::canonicalize(dir.path()).unwrap();
@@ -707,18 +638,11 @@ mod tests {
#[cfg(unix)]
#[test]
fn symlink_relative_operands_create_in_scope_cwd_with_literal_content() {
fn relative_symlink_target_is_literal_while_link_path_resolves() {
let (_dir, root) = canonical_tempdir();
// Relative operands + scope cwd differing from the process cwd: only
// the call-site `pi_uutils_ctx::resolve` patch places the link in the
// tempdir — while the CONTENT must stay exactly as typed.
let (code, stdout, stderr) = run_in(root.clone(), vec!["-s", "target", "link"]);
let (code, stdout, stderr) = run_in(root.clone(), &["-s", "../a", "b"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "", ""));
let link = root.join("link");
assert!(link.is_symlink(), "link must be created inside the scope cwd");
assert_eq!(fs::read_link(&link).unwrap(), PathBuf::from("target"));
assert_eq!(fs::read_link(root.join("b")).unwrap(), PathBuf::from("../a"));
}
#[cfg(unix)]
@@ -728,13 +652,10 @@ mod tests {
fs::create_dir(root.join("dir_a")).unwrap();
fs::create_dir(root.join("dir_b")).unwrap();
std::os::unix::fs::symlink("dir_a", root.join("cur")).unwrap();
// macOS `ln -sfh`: BSD spells `--no-dereference` as `-h`. Without it,
// `cur` dereferences to `dir_a` and the link lands *inside* it.
let (code, stdout, stderr) = run_in(root.clone(), vec!["-sfh", "dir_b", "cur"]);
let (code, stdout, stderr) = run_in(root.clone(), &["-sfh", "dir_b", "cur"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "", ""));
assert_eq!(fs::read_link(root.join("cur")).unwrap(), PathBuf::from("dir_b"));
assert!(!root.join("dir_a").join("dir_b").exists(), "must not link inside the target dir");
assert!(!root.join("dir_a").join("dir_b").exists());
}
#[cfg(unix)]
@@ -744,10 +665,8 @@ mod tests {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("a"), b"payload").unwrap();
let (code, stdout, stderr) = run_in(root.clone(), vec!["a", "b"]);
let (code, stdout, stderr) = run_in(root.clone(), &["a", "b"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "", ""));
assert_eq!(fs::read(root.join("b")).unwrap(), b"payload");
assert_eq!(fs::metadata(root.join("a")).unwrap().nlink(), 2);
assert_eq!(
@@ -761,12 +680,11 @@ mod tests {
fn existing_destination_without_force_fails_with_file_exists() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("link"), b"old").unwrap();
let (code, stdout, stderr) = run_in(root.clone(), vec!["-s", "target", "link"]);
let (code, stdout, stderr) = run_in(root.clone(), &["-s", "target", "link"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "ln: failed to create symbolic link 'link': File exists\n");
assert_eq!(fs::read(root.join("link")).unwrap(), b"old", "destination must be untouched");
assert_eq!(fs::read(root.join("link")).unwrap(), b"old");
}
#[cfg(unix)]
@@ -774,8 +692,7 @@ mod tests {
fn force_overwrites_existing_destination() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("link"), b"old").unwrap();
let (code, stdout, stderr) = run_in(root.clone(), vec!["-sf", "target", "link"]);
let (code, stdout, stderr) = run_in(root.clone(), &["-sf", "target", "link"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "", ""));
assert_eq!(fs::read_link(root.join("link")).unwrap(), PathBuf::from("target"));
}
@@ -784,45 +701,34 @@ mod tests {
#[test]
fn verbose_symlink_prints_mapping_to_stdout() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root, vec!["-sv", "target", "link"]);
assert_eq!(code, 0);
assert_eq!(stdout, "'link' -> 'target'\n");
assert_eq!(stderr, "");
let (code, stdout, stderr) = run_in(root, &["-sv", "target", "link"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "'link' -> 'target'\n", ""));
}
#[cfg(unix)]
#[test]
fn interactive_prompt_reads_ctx_stdin() {
fn interactive_prompt_reads_host_stdin() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("link"), b"old").unwrap();
// Decline: destination untouched, some-links-failed exit code, no
// dangling "ln: " diagnostic beyond the prompt itself.
let (code, stdout, stderr) =
run_with_stdin(root.clone(), vec!["-si", "target", "link"], b"n\n");
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "ln: replace 'link'? ");
run_with_stdin(root.clone(), &["-si", "target", "link"], "n\n");
assert_eq!((code, stdout.as_str(), stderr.as_str()), (1, "", "ln: replace 'link'? "));
assert!(!root.join("link").is_symlink());
// Accept: existing file is replaced by the symlink.
let (code, _, stderr) = run_with_stdin(root.clone(), vec!["-si", "target", "link"], b"y\n");
assert_eq!(code, 0);
assert_eq!(stderr, "ln: replace 'link'? ");
let (code, _, stderr) =
run_with_stdin(root.clone(), &["-si", "target", "link"], "y\n");
assert_eq!((code, stderr.as_str()), (0, "ln: replace 'link'? "));
assert_eq!(fs::read_link(root.join("link")).unwrap(), PathBuf::from("target"));
}
#[cfg(unix)]
#[test]
fn relative_flag_computes_link_text_against_scope_cwd() {
fn relative_flag_computes_link_text_against_host_cwd() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("target"), b"x").unwrap();
fs::create_dir(root.join("sub")).unwrap();
let (code, stdout, stderr) = run_in(root.clone(), vec!["-sr", "target", "sub/link"]);
let (code, stdout, stderr) = run_in(root.clone(), &["-sr", "target", "sub/link"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "", ""));
assert_eq!(fs::read_link(root.join("sub").join("link")).unwrap(), PathBuf::from("../target"));
assert_eq!(fs::read_link(root.join("sub/link")).unwrap(), PathBuf::from("../target"));
}
#[cfg(unix)]
@@ -830,31 +736,26 @@ mod tests {
fn target_directory_flag_places_links_in_directory() {
let (_dir, root) = canonical_tempdir();
fs::create_dir(root.join("d")).unwrap();
let (code, stdout, stderr) = run_in(root.clone(), vec!["-s", "-t", "d", "x"]);
let (code, stdout, stderr) = run_in(root.clone(), &["-s", "-t", "d", "x"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "", ""));
assert_eq!(fs::read_link(root.join("d").join("x")).unwrap(), PathBuf::from("x"));
assert_eq!(fs::read_link(root.join("d/x")).unwrap(), PathBuf::from("x"));
}
#[test]
fn missing_destination_is_an_error() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root, vec!["-T", "only"]);
let (code, stdout, stderr) = run_in(root, &["-T", "only"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert!(
stderr.contains("missing destination file operand after 'only'"),
"stderr was: {stderr:?}"
);
assert!(stderr.contains("missing destination file operand after 'only'"));
}
#[test]
fn help_renders_to_scope_stdout() {
let (code, stdout, stderr) = run_in(PathBuf::from("."), vec!["--help"]);
fn help_renders_to_host_stdout() {
let (code, capture) = run_util::<Ln>(&["--help"], "", ".");
assert_eq!(code, 0);
assert!(stdout.contains("Usage:"));
assert!(stdout.contains("Make links between files."));
assert_eq!(stderr, "");
assert!(capture.out().contains("Usage:"));
assert!(capture.out().contains("Make links between files."));
assert_eq!(capture.err(), "");
}
}
crates/pi-builtins/src/ls.rs
+5262
View File
File diff suppressed because it is too large Load Diff
crates/vendor/brush-builtins/src/mapfile.rs → crates/pi-builtins/src/mapfile.rs
View File
crates/pi-builtins/src/md5sum.rs
+115
View File
@@ -0,0 +1,115 @@
//! `md5sum` builtin: compute and check MD5 digests.
//!
//! Ported from uutils coreutils 0.8.0.
use brush_core::{ShellExtensions, builtins::Registration};
use clap::ArgMatches;
use uucore::checksum::AlgoKind;
use crate::{
cksum,
host::{Host, Utility, matches_parser, util},
};
/// Parsed `md5sum` invocation.
pub(crate) struct Md5sum {
matches: ArgMatches,
}
matches_parser!(Md5sum, app);
impl Utility for Md5sum {
const NAME: &'static str = "md5sum";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
cksum::run(host, AlgoKind::Md5, self.matches, None)
}
}
fn app() -> clap::Command {
cksum::command(Md5sum::NAME, false)
}
/// Creates the `md5sum` builtin registration.
pub(crate) fn md5sum_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Md5sum, SE>()
}
#[cfg(test)]
mod tests {
use std::fs;
use super::Md5sum;
use crate::host::run_util;
#[test]
fn computes_stdin_digest() {
let (code, capture) = run_util::<Md5sum>(&[], "abc", "/");
assert_eq!(code, 0);
assert_eq!(capture.out(), "900150983cd24fb0d6963f7d28e17f72 -\n");
}
#[test]
fn resolves_operand_but_prints_user_supplied_name() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("data"), b"abc").unwrap();
let (code, capture) = run_util::<Md5sum>(&["data"], "", dir.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "900150983cd24fb0d6963f7d28e17f72 data\n");
}
#[test]
fn resolves_checklist_and_checked_paths() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("data"), b"abc").unwrap();
fs::write(
dir.path().join("checksums"),
b"900150983cd24fb0d6963f7d28e17f72 data\n",
)
.unwrap();
let (code, capture) = run_util::<Md5sum>(&["-c", "checksums"], "", dir.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "data: OK\n");
}
#[test]
fn status_suppresses_mismatch_output() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("data"), b"abc").unwrap();
let checklist = "00000000000000000000000000000000 data\n";
let (code, capture) = run_util::<Md5sum>(&["-c", "--status"], checklist, dir.path());
assert_eq!(code, 1);
assert_eq!(capture.out(), "");
assert_eq!(capture.err(), "");
}
#[test]
fn quiet_suppresses_success_output() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("data"), b"abc").unwrap();
let checklist = "900150983cd24fb0d6963f7d28e17f72 data\n";
let (code, capture) = run_util::<Md5sum>(&["-c", "--quiet"], checklist, dir.path());
assert_eq!(code, 0);
assert_eq!(capture.out(), "");
}
#[test]
fn strict_rejects_an_improper_line() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("data"), b"abc").unwrap();
let checklist = concat!(
"not a checksum line\n",
"900150983cd24fb0d6963f7d28e17f72 data\n",
);
let (code, capture) = run_util::<Md5sum>(&["-c", "--strict"], checklist, dir.path());
assert_eq!(code, 1);
assert_eq!(capture.out(), "data: OK\n");
assert!(capture.err().contains("1 line(s) are improperly formatted"));
}
}
crates/pi-builtins/src/mkdir.rs
+426
View File
@@ -0,0 +1,426 @@
//! `mkdir` builtin: create directories.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
ffi::OsString,
fmt,
io::{self, Write},
path::Path,
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command, builder::ValueParser, parser::ValuesRef};
use uucore::{display::Quotable, fs, mode};
#[cfg(all(unix, target_os = "linux"))]
use uucore::fsxattr;
use crate::host::{Host, Utility, format_usage, matches_parser, util};
const DEFAULT_PERM: u32 = 0o777;
mod options {
pub const MODE: &str = "mode";
pub const PARENTS: &str = "parents";
pub const VERBOSE: &str = "verbose";
pub const DIRS: &str = "dirs";
pub const SECURITY_CONTEXT: &str = "z";
pub const CONTEXT: &str = "context";
}
struct Config {
recursive: bool,
mode: u32,
verbose: bool,
}
#[cfg(windows)]
fn get_mode(_matches: &ArgMatches) -> Result<u32, String> {
Ok(DEFAULT_PERM)
}
#[cfg(not(windows))]
fn get_mode(matches: &ArgMatches) -> Result<u32, String> {
if let Some(mode_arg) = matches.get_one::<String>(options::MODE) {
mode::parse_chmod(DEFAULT_PERM, mode_arg, true, mode::get_umask())
} else {
// If no mode argument is specified, return the mode derived from umask.
Ok(!mode::get_umask() & DEFAULT_PERM)
}
}
#[derive(Debug)]
enum MkdirError {
Message(String),
Io { context: Option<String>, source: io::Error },
}
impl MkdirError {
fn io(source: io::Error) -> Self {
Self::Io { context: None, source }
}
#[cfg(all(unix, target_os = "linux"))]
fn io_with_context(source: io::Error, context: String) -> Self {
Self::Io { context: Some(context), source }
}
}
impl fmt::Display for MkdirError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Message(message) => formatter.write_str(message),
Self::Io { context, source } => {
if let Some(context) = context {
write!(formatter, "{context}: ")?;
}
formatter.write_str(&normalized_io_message(source))
},
}
}
}
fn normalized_io_message(error: &io::Error) -> String {
if error.raw_os_error().is_none() {
return error.to_string();
}
use io::ErrorKind::{
AddrInUse, AddrNotAvailable, AlreadyExists, BrokenPipe, ConnectionAborted,
ConnectionRefused, ConnectionReset, Interrupted, InvalidData, InvalidInput, NotConnected,
NotFound, PermissionDenied, TimedOut, UnexpectedEof, WouldBlock, WriteZero,
};
match error.kind() {
NotFound => "No such file or directory".into(),
PermissionDenied => "Permission denied".into(),
ConnectionRefused => "Connection refused".into(),
ConnectionReset => "Connection reset".into(),
ConnectionAborted => "Connection aborted".into(),
NotConnected => "Not connected".into(),
AddrInUse => "Address in use".into(),
AddrNotAvailable => "Address not available".into(),
BrokenPipe => "Broken pipe".into(),
AlreadyExists => "Already exists".into(),
WouldBlock => "Would block".into(),
InvalidInput => "Invalid input".into(),
InvalidData => "Invalid data".into(),
TimedOut => "Timed out".into(),
WriteZero => "Write zero".into(),
Interrupted => "Interrupted".into(),
UnexpectedEof => "Unexpected end of file".into(),
_ => error
.to_string()
.split_once(" (os error ")
.map_or_else(|| error.to_string(), |(message, _)| message.to_string()),
}
}
/// Parsed `mkdir` invocation.
pub(crate) struct Mkdir {
matches: ArgMatches,
}
matches_parser!(Mkdir, app);
impl Utility for Mkdir {
const NAME: &'static str = "mkdir";
fn run(self, host: &mut Host) -> i32 {
let dirs = self
.matches
.get_many::<OsString>(options::DIRS)
.unwrap_or_default();
let config = match get_mode(&self.matches) {
Ok(mode) => Config {
recursive: self.matches.get_flag(options::PARENTS),
mode,
verbose: self.matches.get_flag(options::VERBOSE),
},
Err(message) => {
host.error(message, 1);
return 1;
},
};
exec(dirs, &config, host);
host.exit_code()
}
}
fn app() -> Command {
Command::new(Mkdir::NAME)
.version("0.8.0")
.about("Create the given DIRECTORY(ies) if they do not exist")
.override_usage(format_usage("mkdir [OPTION]... DIRECTORY..."))
.infer_long_args(true)
.after_help("Each MODE is of the form [ugoa]*([-+=]([rwxXst]*|[ugo]))+|[-+=]?[0-7]+.")
.arg(
Arg::new(options::MODE)
.short('m')
.long(options::MODE)
.help("set file mode (not implemented on windows)")
.allow_hyphen_values(true)
.num_args(1),
)
.arg(
Arg::new(options::PARENTS)
.short('p')
.long(options::PARENTS)
.help("make parent directories as needed")
.overrides_with(options::PARENTS)
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::VERBOSE)
.short('v')
.long(options::VERBOSE)
.help("print a message for each printed directory")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::SECURITY_CONTEXT)
.short('Z')
.help("set SELinux security context of each created directory to the default type")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::CONTEXT)
.long(options::CONTEXT)
.value_name("CTX")
.help(
"like -Z, or if CTX is specified then set the SELinux or SMACK security context to \
CTX",
),
)
.arg(
Arg::new(options::DIRS)
.action(ArgAction::Append)
.num_args(1..)
.required(true)
.value_parser(ValueParser::os_string())
.value_hint(clap::ValueHint::DirPath),
)
}
fn exec(dirs: ValuesRef<'_, OsString>, config: &Config, host: &mut Host) {
for dir in dirs {
if let Err(error) = mkdir(Path::new(dir), config, host) {
host.error(error, 1);
}
}
}
/// Creates a directory at `path`, including parents when requested.
fn mkdir(path: &Path, config: &Config, host: &mut Host) -> Result<(), MkdirError> {
if path.as_os_str().is_empty() {
return Err(MkdirError::Message(
"cannot create directory '': No such file or directory".into(),
));
}
// `mkdir -p foo/.` succeeds, although `std::fs::create_dir("foo/.")` does not.
let path = fs::dir_strip_dot_for_creation(path);
create_dir(&path, false, config, host)
}
#[cfg(all(unix, target_os = "linux"))]
fn chmod(fs_path: &Path, display_path: &Path, mode: u32) -> Result<(), MkdirError> {
use std::{
fs::{Permissions, set_permissions},
os::unix::fs::PermissionsExt,
};
set_permissions(fs_path, Permissions::from_mode(mode)).map_err(|source| {
MkdirError::io_with_context(
source,
format!("cannot set permissions {}", display_path.quote()),
)
})
}
// Uses an iterative approach instead of recursion to avoid stack overflow with
// deep nesting.
fn create_dir(
path: &Path,
is_parent: bool,
config: &Config,
host: &mut Host,
) -> Result<(), MkdirError> {
let path_exists = host.resolve(path).exists();
if path_exists && !config.recursive {
return Err(MkdirError::Message(format!("{}: File exists", path.maybe_quote())));
}
if path == Path::new("") {
return Ok(());
}
if config.recursive {
let mut dirs_to_create = Vec::with_capacity(16);
let mut current = path;
while let Some(parent) = current.parent() {
if parent == Path::new("") {
break;
}
dirs_to_create.push(parent);
current = parent;
}
for dir in dirs_to_create.iter().rev() {
if !host.resolve(dir).exists() {
create_single_dir(dir, true, config, host)?;
}
}
}
create_single_dir(path, is_parent, config, host)
}
/// Restores the process umask when directory creation finishes or unwinds.
#[cfg(unix)]
struct UmaskGuard(rustix::fs::Mode);
#[cfg(unix)]
impl UmaskGuard {
fn set(new_mask: rustix::fs::Mode) -> Self {
let old_mask = rustix::process::umask(new_mask);
Self(old_mask)
}
}
#[cfg(unix)]
impl Drop for UmaskGuard {
fn drop(&mut self) {
rustix::process::umask(self.0);
}
}
#[cfg(unix)]
fn create_dir_with_mode(path: &Path, mode: u32) -> io::Result<()> {
use std::os::unix::fs::DirBuilderExt;
// GNU mkdir creates with the exact requested mode atomically by temporarily
// disabling the process umask.
let _guard = UmaskGuard::set(rustix::fs::Mode::empty());
std::fs::DirBuilder::new().mode(mode).create(path)
}
#[cfg(not(unix))]
fn create_dir_with_mode(path: &Path, _mode: u32) -> io::Result<()> {
std::fs::create_dir(path)
}
fn create_single_dir(
path: &Path,
is_parent: bool,
config: &Config,
host: &mut Host,
) -> Result<(), MkdirError> {
let fs_path = host.resolve(path);
#[cfg(all(unix, target_os = "linux"))]
let path_exists = fs_path.exists();
#[cfg(unix)]
let create_mode = if is_parent {
// Parents made by `-p` use the umask-derived mode with `u+wx` restored.
(!mode::get_umask() & 0o777) | 0o300
} else {
config.mode
};
#[cfg(not(unix))]
let create_mode = config.mode;
match create_dir_with_mode(&fs_path, create_mode) {
Ok(()) => {
if config.verbose {
writeln!(host.stdout, "mkdir: created directory {}", path.quote())
.map_err(MkdirError::io)?;
}
#[cfg(all(unix, target_os = "linux"))]
if !path_exists {
let acl_perm_bits = fsxattr::get_acl_perm_bits_from_xattr(&fs_path);
if acl_perm_bits != 0 {
chmod(&fs_path, path, create_mode | acl_perm_bits)?;
}
}
Ok(())
},
Err(_) if fs_path.is_dir() => {
let ends_with_parent_dir =
matches!(path.components().next_back(), Some(std::path::Component::ParentDir));
if config.verbose && is_parent && config.recursive && !ends_with_parent_dir {
writeln!(host.stdout, "mkdir: created directory {}", path.quote())
.map_err(MkdirError::io)?;
}
Ok(())
},
Err(source) => Err(MkdirError::io(source)),
}
}
/// Creates the `mkdir` builtin registration.
pub(crate) fn mkdir_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Mkdir, SE>()
}
#[cfg(test)]
mod tests {
use super::Mkdir;
use crate::host::run_util;
#[test]
fn creates_relative_operand_under_host_cwd() {
let cwd = tempfile::tempdir().unwrap();
let (code, capture) = run_util::<Mkdir>(&["child"], "", cwd.path());
assert_eq!(code, 0, "{}", capture.err());
assert!(cwd.path().join("child").is_dir());
}
#[test]
fn parents_creates_the_whole_path() {
let cwd = tempfile::tempdir().unwrap();
let (code, capture) = run_util::<Mkdir>(&["-p", "a/b/c"], "", cwd.path());
assert_eq!(code, 0, "{}", capture.err());
assert!(cwd.path().join("a/b/c").is_dir());
}
#[test]
fn verbose_uses_the_user_supplied_path() {
let cwd = tempfile::tempdir().unwrap();
let (code, capture) = run_util::<Mkdir>(&["-v", "shown"], "", cwd.path());
assert_eq!(code, 0, "{}", capture.err());
assert_eq!(capture.out(), "mkdir: created directory 'shown'\n");
}
#[test]
fn reports_existing_directory_and_continues() {
let cwd = tempfile::tempdir().unwrap();
std::fs::create_dir(cwd.path().join("exists")).unwrap();
let (code, capture) = run_util::<Mkdir>(&["exists", "created"], "", cwd.path());
assert_eq!(code, 1);
assert_eq!(capture.err(), "mkdir: exists: File exists\n");
assert!(cwd.path().join("created").is_dir());
}
#[cfg(unix)]
#[test]
fn explicit_mode_applies_to_leaf_but_parents_keep_owner_write_and_search() {
use std::os::unix::fs::PermissionsExt;
let cwd = tempfile::tempdir().unwrap();
let (code, capture) = run_util::<Mkdir>(&["-p", "-m", "000", "parent/leaf"], "", cwd.path());
assert_eq!(code, 0, "{}", capture.err());
let leaf_mode = std::fs::metadata(cwd.path().join("parent/leaf"))
.unwrap()
.permissions()
.mode()
& 0o777;
assert_eq!(leaf_mode, 0o000);
let parent_mode = std::fs::metadata(cwd.path().join("parent"))
.unwrap()
.permissions()
.mode()
& 0o777;
assert_eq!(parent_mode & 0o300, 0o300);
}
}
crates/pi-builtins/src/mktemp.rs
+805
View File
@@ -0,0 +1,805 @@
//! `mktemp` builtin: create and display a temporary file or directory from a
//! template.
//!
//! Ported from uutils coreutils 0.8.0.
#[cfg(unix)]
use std::fs;
#[cfg(unix)]
use std::os::unix::prelude::PermissionsExt;
use std::{
env,
ffi::{OsStr, OsString},
io::{self, ErrorKind, Write},
iter,
path::{MAIN_SEPARATOR, Path, PathBuf},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{
Arg, ArgAction, ArgMatches, Command,
builder::{TypedValueParser, ValueParserFactory},
};
use rand::{
RngExt as _, SeedableRng as _,
rngs::{self, SmallRng},
};
use tempfile::Builder;
use thiserror::Error;
use uucore::display::Quotable;
use crate::host::{Host, Utility, format_usage, matches_parser, os_bytes, util};
static DEFAULT_TEMPLATE: &str = "tmp.XXXXXXXXXX";
static OPT_DIRECTORY: &str = "directory";
static OPT_DRY_RUN: &str = "dry-run";
static OPT_QUIET: &str = "quiet";
static OPT_SUFFIX: &str = "suffix";
static OPT_TMPDIR: &str = "tmpdir";
static OPT_P: &str = "p";
static OPT_T: &str = "t";
static ARG_TEMPLATE: &str = "template";
#[cfg(not(windows))]
const TMPDIR_ENV_VAR: &str = "TMPDIR";
#[cfg(windows)]
const TMPDIR_ENV_VAR: &str = "TMP";
const FALLBACK_TMPDIR: &str = "/tmp";
#[derive(Error, Debug)]
enum MkTempError {
#[error("could not persist file {}", .0.quote())]
Persist(PathBuf),
#[error("with --suffix, template {} must end in X", .0.quote())]
MustEndInX(String),
#[error("too few X's in template {}", .0.quote())]
TooFewXs(String),
#[error("invalid template, {}, contains directory separator", .0.quote())]
PrefixContainsDirSeparator(String),
#[error("invalid suffix {}, contains directory separator", .0.quote())]
SuffixContainsDirSeparator(String),
#[error("invalid template, {}; with --tmpdir, it may not be absolute", .0.quote())]
InvalidTemplate(OsString),
#[error("too many templates")]
TooManyTemplates,
#[error("failed to create {} via template {}: No such file or directory", .0, .1.quote())]
NotFound(String, PathBuf),
#[error(transparent)]
Io(#[from] io::Error),
}
/// Options parsed from the command line.
///
/// This provides a layer of indirection between the application logic and
/// `clap`, allowing each to vary independently.
#[derive(Clone)]
struct Options {
/// Whether to create a temporary directory instead of a file.
directory: bool,
/// Whether to just print the name of a file that would have been created.
dry_run: bool,
/// Whether to suppress file creation error messages.
quiet: bool,
/// The directory in which to create the temporary file.
tmpdir: Option<PathBuf>,
/// The suffix to append to the temporary file, if any.
suffix: Option<OsString>,
/// Whether to treat the template argument as a single file path component.
treat_as_template: bool,
/// The template to use for the name of the temporary file.
template: OsString,
}
impl Options {
fn from(matches: &ArgMatches, host: &Host) -> Self {
let tmpdir = matches
.get_one::<Option<PathBuf>>(OPT_TMPDIR)
.or_else(|| matches.get_one::<Option<PathBuf>>(OPT_P))
.map(|dir| match dir {
// If the argument of -p/--tmpdir is non-empty, use it as the tmpdir.
Some(dir) => dir.clone(),
// Otherwise use $TMPDIR if set, else the system default.
None => get_tmpdir_env_or_default(host),
});
let (tmpdir, template) = match matches.get_one::<OsString>(ARG_TEMPLATE) {
// If no template argument is given, `--tmpdir` is implied.
None => (
Some(tmpdir.unwrap_or_else(|| get_tmpdir_env_or_default(host))),
OsString::from(DEFAULT_TEMPLATE),
),
Some(template) => {
let tmpdir = if let Some(tmpdir) = host.var(TMPDIR_ENV_VAR)
&& matches.get_flag(OPT_T)
{
Some(PathBuf::from(tmpdir))
} else if tmpdir.is_some() {
tmpdir
} else if matches.get_flag(OPT_T) || matches.contains_id(OPT_TMPDIR) {
Some(get_tmpdir_env_or_default(host))
} else {
None
};
(tmpdir, template.clone())
},
};
Self {
directory: matches.get_flag(OPT_DIRECTORY),
dry_run: matches.get_flag(OPT_DRY_RUN),
quiet: matches.get_flag(OPT_QUIET),
tmpdir,
suffix: matches.get_one::<OsString>(OPT_SUFFIX).cloned(),
treat_as_template: matches.get_flag(OPT_T),
template,
}
}
}
/// Parameters controlling the path and name of the temporary entry.
struct Params {
/// The directory that will contain the temporary entry.
directory: PathBuf,
/// The non-random prefix of the temporary entry.
prefix: String,
/// The number of random characters in the name.
num_rand_chars: usize,
/// The non-random suffix of the temporary entry.
suffix: String,
}
/// Finds the last contiguous block of at least three `X` characters.
fn find_last_contiguous_block_of_xs(s: &str) -> Option<(usize, usize)> {
let bytes = s.as_bytes();
let end = bytes.iter().rposition(|&b| b == b'X')?;
let mut start = end;
while start > 0 && bytes[start - 1] == b'X' {
start -= 1;
}
(end + 1 - start >= 3).then_some((start, end + 1))
}
impl Params {
fn from(options: Options) -> Result<Self, MkTempError> {
// `-t` follows GNU's permissive treatment of invalid UTF-8. Regular
// templates retain the upstream strict validation.
let mut template_str = if options.treat_as_template {
options.template.to_string_lossy().into_owned()
} else {
options
.template
.to_str()
.ok_or_else(|| {
MkTempError::InvalidTemplate("template contains invalid UTF-8".into())
})?
.to_string()
};
if options.suffix.is_some() && !template_str.ends_with('X') {
return Err(MkTempError::MustEndInX(template_str));
}
let (i, j) = match find_last_contiguous_block_of_xs(&template_str) {
Some(indices) => indices,
// BSD `mktemp -t PREFIX` treats PREFIX as a name prefix.
None if options.treat_as_template => {
template_str.push('.');
template_str.push_str("XXXXXXXXXX");
let j = template_str.len();
(j - 10, j)
},
None => return Err(MkTempError::TooFewXs(template_str)),
};
// Combine the option directory and the template prefix, then split the
// parent directory from the final file-name component.
let tmpdir = options.tmpdir;
let prefix_from_option = tmpdir.clone().unwrap_or_default();
let prefix_from_template = &template_str[..i];
let prefix_path = Path::new(&prefix_from_option).join(prefix_from_template);
if options.treat_as_template && prefix_from_template.contains(MAIN_SEPARATOR) {
return Err(MkTempError::PrefixContainsDirSeparator(template_str));
}
if tmpdir.is_some() && Path::new(prefix_from_template).is_absolute() {
return Err(MkTempError::InvalidTemplate(template_str.into()));
}
let (directory, prefix) = {
let prefix_str = prefix_path.to_string_lossy();
if prefix_str.ends_with(MAIN_SEPARATOR) {
(prefix_path, String::new())
} else {
let directory = prefix_path.parent().map_or_else(PathBuf::new, Path::to_path_buf);
let prefix = prefix_path
.file_name()
.map_or_else(String::new, |f| f.to_string_lossy().into_owned());
(directory, prefix)
}
};
// Combine a suffix embedded in the template with `--suffix`.
let suffix_from_option = options
.suffix
.map(|s| s.to_string_lossy().into_owned())
.unwrap_or_default();
let suffix_from_template = &template_str[j..];
let suffix = format!("{suffix_from_template}{suffix_from_option}");
if suffix.contains(MAIN_SEPARATOR) {
return Err(MkTempError::SuffixContainsDirSeparator(suffix));
}
Ok(Self { directory, prefix, num_rand_chars: j - i, suffix })
}
}
/// Parses an empty directory option as `None` and a non-empty one as a path.
#[derive(Clone, Debug)]
struct OptionalPathBufParser;
impl TypedValueParser for OptionalPathBufParser {
type Value = Option<PathBuf>;
fn parse_ref(
&self,
_cmd: &Command,
_arg: Option<&Arg>,
value: &OsStr,
) -> Result<Self::Value, clap::Error> {
if value.is_empty() {
Ok(None)
} else {
Ok(Some(PathBuf::from(value)))
}
}
}
impl ValueParserFactory for OptionalPathBufParser {
type Parser = Self;
fn value_parser() -> Self::Parser {
Self
}
}
/// Parsed `mktemp` invocation.
pub(crate) struct Mktemp {
matches: ArgMatches,
}
matches_parser!(Mktemp, app);
impl Utility for Mktemp {
const NAME: &'static str = "mktemp";
fn rewrite_argv(argv: Vec<OsString>) -> Result<Vec<OsString>, String> {
// Upstream replaces clap's generic positional overflow diagnostic with
// GNU mktemp's concise message.
if let Err(err) = app().try_get_matches_from(&argv)
&& err.kind() == clap::error::ErrorKind::TooManyValues
&& err.context().any(|(kind, value)| {
kind == clap::error::ContextKind::InvalidArg
&& value == &clap::error::ContextValue::String("[template]".into())
})
{
return Err(MkTempError::TooManyTemplates.to_string());
}
Ok(argv)
}
fn run(self, host: &mut Host) -> i32 {
let options = Options::from(&self.matches, host);
// Under POSIXLY_CORRECT the template must be the last argument. Clap's
// occurrence indices preserve that ordering after short-option expansion.
if host.var("POSIXLY_CORRECT").is_some()
&& self.matches.contains_id(ARG_TEMPLATE)
&& !template_is_last(&self.matches)
{
host.error(MkTempError::TooManyTemplates, 1);
return 1;
}
let dry_run = options.dry_run;
let quiet = options.quiet;
let make_dir = options.directory;
let Params { directory, prefix, num_rand_chars, suffix } = match Params::from(options) {
Ok(params) => params,
Err(err) => {
host.error(err, 1);
return 1;
},
};
let result = if dry_run {
Ok(dry_exec(&directory, &prefix, num_rand_chars, &suffix))
} else {
exec(host, &directory, &prefix, num_rand_chars, &suffix, make_dir)
};
let path = match result {
Ok(path) => path,
Err(_) if quiet => return 1,
Err(err) => {
host.error(err, 1);
return 1;
},
};
let Some(bytes) = os_bytes(path.as_os_str()) else {
host.error("failed to print directory name: path contains invalid text", 1);
return 1;
};
if host.stdout.write_all(bytes).is_err()
|| host.stdout.write_all(b"\n").is_err()
|| host.stdout.flush().is_err()
{
return 1;
}
0
}
}
/// Returns whether the template occurred after every option and option value.
fn template_is_last(matches: &ArgMatches) -> bool {
let Some(template_index) = matches.index_of(ARG_TEMPLATE) else {
return true;
};
[
OPT_DIRECTORY,
OPT_DRY_RUN,
OPT_QUIET,
OPT_SUFFIX,
OPT_TMPDIR,
OPT_P,
OPT_T,
]
.into_iter()
.filter_map(|id| matches.index_of(id))
.all(|index| index < template_index)
}
/// Builds the `mktemp` command-line model.
fn app() -> Command {
Command::new(Mktemp::NAME)
.version("0.8.0")
.about("Create a temporary file or directory.")
.override_usage(format_usage("mktemp [OPTION]... [TEMPLATE]"))
.infer_long_args(true)
.arg(
Arg::new(OPT_DIRECTORY)
.short('d')
.long(OPT_DIRECTORY)
.help("Make a directory instead of a file")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_DRY_RUN)
.short('u')
.long(OPT_DRY_RUN)
.help("do not create anything; merely print a name (unsafe)")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_QUIET)
.short('q')
.long(OPT_QUIET)
.help("Fail silently if an error occurs.")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_SUFFIX)
.long(OPT_SUFFIX)
.help(
"append SUFFIX to TEMPLATE; SUFFIX must not contain a path separator. This option \
is implied if TEMPLATE does not end with X.",
)
.value_name("SUFFIX")
.value_parser(clap::value_parser!(OsString)),
)
.arg(
Arg::new(OPT_P)
.short('p')
.help("short form of --tmpdir")
.value_name("DIR")
.num_args(1)
.value_parser(OptionalPathBufParser)
.value_hint(clap::ValueHint::DirPath),
)
.arg(
Arg::new(OPT_TMPDIR)
.long(OPT_TMPDIR)
.help(
"interpret TEMPLATE relative to DIR; if DIR is not specified, use $TMPDIR ($TMP on \
windows) if set, else /tmp. With this option, TEMPLATE must not be an absolute \
name; unlike with -t, TEMPLATE may contain slashes, but mktemp creates only the \
final component",
)
.value_name("DIR")
.num_args(0..=1)
.require_equals(true)
.overrides_with(OPT_P)
.value_parser(OptionalPathBufParser)
.value_hint(clap::ValueHint::DirPath),
)
.arg(
Arg::new(OPT_T)
.short('t')
.help(
"Generate a template (using the supplied prefix and TMPDIR (TMP on windows) if \
set) to create a filename template [deprecated]",
)
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(ARG_TEMPLATE)
.num_args(..=1)
.value_parser(clap::value_parser!(OsString)),
)
}
fn dry_exec(tmpdir: &Path, prefix: &str, rand: usize, suffix: &str) -> PathBuf {
let len = prefix.len() + suffix.len() + rand;
let mut buf = Vec::with_capacity(len);
buf.extend(prefix.as_bytes());
buf.extend(iter::repeat_n(b'X', rand));
buf.extend(suffix.as_bytes());
let bytes = &mut buf[prefix.len()..prefix.len() + rand];
SmallRng::try_from_rng(&mut rngs::SysRng)
.unwrap_or_else(|_| SmallRng::seed_from_u64(bytes.as_ptr() as usize as u64))
.fill(bytes);
for byte in bytes {
*byte = match *byte % 62 {
v @ 0..=9 => v + b'0',
v @ 10..=35 => v - 10 + b'a',
v @ 36..=61 => v - 36 + b'A',
_ => unreachable!(),
};
}
// Every byte was mapped into the ASCII alphanumeric range.
let buf = String::from_utf8(buf).unwrap();
tmpdir.join(buf)
}
/// Creates a temporary directory with owner-only permissions.
fn make_temp_dir(
dir: &Path,
display_dir: &Path,
prefix: &str,
rand: usize,
suffix: &str,
) -> Result<PathBuf, MkTempError> {
let mut builder = Builder::new();
builder.prefix(prefix).rand_bytes(rand).suffix(suffix);
#[cfg(not(windows))]
builder.permissions(fs::Permissions::from_mode(0o700));
match builder.tempdir_in(dir) {
Ok(directory) => Ok(directory.keep()),
Err(err) if err.kind() == ErrorKind::NotFound => {
let filename = format!("{prefix}{}{suffix}", "X".repeat(rand));
Err(MkTempError::NotFound(
"directory".to_string(),
display_dir.join(filename),
))
},
Err(err) => Err(err.into()),
}
}
/// Creates a temporary file with owner-only permissions.
fn make_temp_file(
dir: &Path,
display_dir: &Path,
prefix: &str,
rand: usize,
suffix: &str,
) -> Result<PathBuf, MkTempError> {
let mut builder = Builder::new();
builder.prefix(prefix).rand_bytes(rand).suffix(suffix);
match builder.tempfile_in(dir) {
Ok(file) => file.keep().map(|(_, path)| path).map_err(|err| {
let path = err.file.path();
let display_path = path
.file_name()
.map_or_else(|| display_dir.to_path_buf(), |name| display_dir.join(name));
MkTempError::Persist(display_path)
}),
Err(err) if err.kind() == ErrorKind::NotFound => {
let filename = format!("{prefix}{}{suffix}", "X".repeat(rand));
Err(MkTempError::NotFound("file".to_string(), display_dir.join(filename)))
},
Err(err) => Err(err.into()),
}
}
fn exec(
host: &Host,
dir: &Path,
prefix: &str,
rand: usize,
suffix: &str,
make_dir: bool,
) -> Result<PathBuf, MkTempError> {
// Only the filesystem-facing form is resolved. The returned path retains
// the spelling implied by the user's operands, which scripts consume.
let resolved_dir = host.resolve(dir);
let created = if make_dir {
make_temp_dir(&resolved_dir, dir, prefix, rand, suffix)?
} else {
make_temp_file(&resolved_dir, dir, prefix, rand, suffix)?
};
let filename = created.file_name().expect("tempfile path has a file name");
Ok(dir.join(filename))
}
/// Reads the shell's temporary-directory variable, falling back to the platform
/// default. An explicitly empty variable uses `/tmp`, matching GNU mktemp.
fn get_tmpdir_env_or_default(host: &Host) -> PathBuf {
match host.var(TMPDIR_ENV_VAR) {
Some(value) if value.is_empty() => PathBuf::from(FALLBACK_TMPDIR),
Some(value) => PathBuf::from(value),
None => env::temp_dir(),
}
}
/// Creates the `mktemp` builtin registration.
pub(crate) fn mktemp_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Mktemp, SE>()
}
#[cfg(test)]
mod tests {
use std::{
env,
ffi::OsString,
io::Write,
path::{Path, PathBuf},
};
use clap::Parser;
use super::Mktemp;
use crate::host::{Host, Utility};
fn canonical_tempdir() -> (tempfile::TempDir, PathBuf) {
let dir = tempfile::tempdir().unwrap();
let canonical = std::fs::canonicalize(dir.path()).unwrap();
(dir, canonical)
}
fn run_in(cwd: PathBuf, env: &[(&str, &str)], args: &[&str]) -> (i32, String, String) {
let (mut host, capture) = Host::for_test(Mktemp::NAME, "", cwd);
for (key, value) in env {
host.set_test_var(key, value);
}
let argv: Vec<OsString> = std::iter::once(OsString::from(Mktemp::NAME))
.chain(args.iter().map(OsString::from))
.collect();
let argv = match Mktemp::rewrite_argv(argv) {
Ok(argv) => argv,
Err(message) => {
host.error(message, i32::from(Mktemp::USAGE_ERROR));
return (i32::from(Mktemp::USAGE_ERROR), capture.out(), capture.err());
},
};
let code = match Mktemp::try_parse_from(argv) {
Ok(parsed) => parsed.run(&mut host),
Err(err) => {
let rendered = err.to_string();
if err.use_stderr() {
let _ = host.stderr.write_all(rendered.as_bytes());
i32::from(Mktemp::USAGE_ERROR)
} else {
let _ = host.stdout.write_all(rendered.as_bytes());
0
}
},
};
(code, capture.out(), capture.err())
}
fn tmpdir_env(dir: &Path) -> [(&str, &str); 1] {
[("TMPDIR", dir.to_str().unwrap())]
}
#[test]
fn default_invocation_creates_file_at_printed_path() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root.clone(), &tmpdir_env(&root), &[]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert!(printed.is_file());
assert_eq!(printed.parent(), Some(root.as_path()));
assert!(printed.file_name().unwrap().to_str().unwrap().starts_with("tmp."));
}
#[test]
fn directory_flag_creates_directory() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root.clone(), &tmpdir_env(&root), &["-d"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert!(printed.is_dir());
assert_eq!(printed.parent(), Some(root.as_path()));
}
#[test]
fn relative_tmpdir_resolves_against_host_cwd_but_prints_relative() {
let (_dir, root) = canonical_tempdir();
std::fs::create_dir(root.join("sub")).unwrap();
let (code, stdout, stderr) = run_in(root.clone(), &[], &["-p", "sub", "foo.XXXX"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert_eq!(printed.parent(), Some(Path::new("sub")));
assert!(root.join(&printed).is_file());
}
#[test]
fn relative_template_directory_resolves_against_host_cwd() {
let (_dir, root) = canonical_tempdir();
std::fs::create_dir(root.join("nested")).unwrap();
let (code, stdout, stderr) = run_in(root.clone(), &[], &["nested/foo.XXXX"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert_eq!(printed.parent(), Some(Path::new("nested")));
assert!(root.join(printed).is_file());
}
#[test]
fn host_tmpdir_wins_over_process_environment() {
let (_dir, root) = canonical_tempdir();
assert_ne!(env::temp_dir(), root);
let (code, stdout, stderr) = run_in(root.clone(), &tmpdir_env(&root), &[]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert_eq!(printed.parent(), Some(root.as_path()));
assert!(printed.is_file());
}
#[test]
fn too_few_xs_is_an_error() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root, &[], &["foo.XX"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "mktemp: too few X's in template 'foo.XX'\n");
}
#[test]
fn bsd_t_prefix_creates_file_in_tmpdir() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root.clone(), &tmpdir_env(&root), &["-t", "omp"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert!(printed.is_file());
assert_eq!(printed.parent(), Some(root.as_path()));
let name = printed.file_name().unwrap().to_str().unwrap();
assert!(name.starts_with("omp."));
assert_eq!(name.len(), "omp.".len() + 10);
}
#[test]
fn bsd_t_prefix_creates_directory_with_d_flag() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) =
run_in(root.clone(), &tmpdir_env(&root), &["-d", "-t", "pfx"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert!(printed.is_dir());
assert_eq!(printed.parent(), Some(root.as_path()));
}
#[test]
fn gnu_t_template_keeps_template_behavior() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) =
run_in(root.clone(), &tmpdir_env(&root), &["-t", "fooXXXX"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert!(printed.is_file());
assert_eq!(printed.parent(), Some(root.as_path()));
let name = printed.file_name().unwrap().to_str().unwrap();
assert!(name.starts_with("foo"));
assert_eq!(name.len(), "foo".len() + 4);
}
#[test]
fn template_without_xs_without_t_remains_an_error() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root, &[], &["prefix"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "mktemp: too few X's in template 'prefix'\n");
}
#[test]
fn dry_run_prints_nonexistent_path() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root.clone(), &tmpdir_env(&root), &["-u"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert_eq!(printed.parent(), Some(root.as_path()));
assert!(!printed.exists());
}
#[test]
fn suffix_is_appended_after_random_block() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) =
run_in(root.clone(), &[], &["--suffix=.txt", "-p", ".", "fooXXXX"]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
let printed = PathBuf::from(stdout.trim_end_matches('\n'));
assert!(root.join(&printed).is_file());
let name = printed.file_name().unwrap().to_str().unwrap();
assert!(name.starts_with("foo") && name.ends_with(".txt"));
}
#[test]
fn quiet_suppresses_creation_error_message_but_not_exit_code() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) =
run_in(root, &[], &["-q", "-p", "missing-dir", "foo.XXXX"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "");
}
#[test]
fn creation_error_keeps_relative_template_in_diagnostic() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) =
run_in(root, &[], &["-p", "missing-dir", "foo.XXXX"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(
stderr,
"mktemp: failed to create file via template 'missing-dir/foo.XXXX': No such file or directory\n"
);
}
#[test]
fn too_many_templates_uses_gnu_diagnostic() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root, &[], &["one.XXXX", "two.XXXX"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "mktemp: too many templates\n");
}
#[test]
fn posixly_correct_requires_template_last() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) =
run_in(root, &[("POSIXLY_CORRECT", "1")], &["foo.XXXX", "-d"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "mktemp: too many templates\n");
}
#[test]
fn help_renders_to_host_stdout() {
let (code, stdout, stderr) = run_in(PathBuf::from("."), &[], &["--help"]);
assert_eq!(code, 0);
assert!(stdout.contains("Usage:"));
assert!(stdout.contains("temporary file or directory"));
assert_eq!(stderr, "");
}
}
crates/vendor/uu-mv/src/mv.rs → crates/pi-builtins/src/mv.rs
+769 -289
View File
File diff suppressed because it is too large Load Diff
crates/pi-builtins/src/nohup.rs
+107
View File
@@ -0,0 +1,107 @@
//! The `nohup` command, moved from `pi-shell`.
//!
//! This builtin detaches a backgrounded operand into a new session so a server
//! survives the embedded shell's kill-on-drop teardown. A system `nohup` does
//! not escape the process-group kill, so this command intentionally shadows it.
//! Registration marks it as a transparent background wrapper, allowing brush to
//! spawn the operand directly with session reparenting.
use std::{future::Future, io::Write};
use brush_core::{
ExecutionContext, ExecutionExitCode, ExecutionResult, ProcessGroupPolicy, SourceInfo, builtins,
};
use clap::Parser;
use crate::host::quote_arg;
/// Runs an operand with the process-group policy required by `nohup`.
#[derive(Parser)]
#[command(disable_help_flag = true)]
pub(crate) struct NohupCommand {
#[arg(num_args = 0.., trailing_var_arg = true, allow_hyphen_values = true)]
command: Vec<String>,
}
impl builtins::Command for NohupCommand {
type Error = brush_core::Error;
fn execute<SE: brush_core::ShellExtensions>(
&self,
context: ExecutionContext<'_, SE>,
) -> impl Future<Output = std::result::Result<ExecutionResult, brush_core::Error>> + Send {
let command = self.command.clone();
async move {
if context.is_cancelled() {
return Ok(ExecutionExitCode::Interrupted.into());
}
// coreutils `nohup` with no operand fails with exit code 125.
if command.is_empty() {
return Ok(report_missing_operand(context.stderr()));
}
// `nohup <cmd>` (foreground) runs the operand directly and surfaces its
// exit status. Persistence across the host's teardown is a *background*
// concern that never reaches this builtin: brush's
// `transparent_background_wrapper` unwraps `nohup <server> &` to spawn the
// operand directly with session reparenting, double-forking it out of the
// shell's descendant tree. Like coreutils, we run the operand here; we only
// differ by not masking SIGHUP.
let command_line = rebuild_command_line(&command);
let mut params = context.params.clone();
params.process_group_policy = ProcessGroupPolicy::NewProcessGroup;
let source_info = SourceInfo::from("pi-natives:nohup");
context
.shell
.run_string(command_line, &source_info, &params)
.await
}
}
}
fn report_missing_operand(mut stderr: impl Write) -> ExecutionResult {
let _ = writeln!(stderr, "nohup: missing operand");
ExecutionResult::new(125)
}
fn rebuild_command_line(command: &[String]) -> String {
let mut command_line = String::new();
for (idx, arg) in command.iter().enumerate() {
if idx > 0 {
command_line.push(' ');
}
command_line.push_str(&quote_arg(arg));
}
command_line
}
#[cfg(test)]
mod tests {
use super::{rebuild_command_line, report_missing_operand};
#[test]
fn missing_operand_reports_diagnostic_and_exit_code() {
let mut stderr = Vec::new();
let result = report_missing_operand(&mut stderr);
assert_eq!(u8::from(result.exit_code), 125);
assert_eq!(stderr, b"nohup: missing operand\n");
}
#[test]
fn rebuilds_command_line_with_shell_quoting() {
let command = [
"printf".to_string(),
"%s %s".to_string(),
"two words".to_string(),
"it's".to_string(),
String::new(),
];
assert_eq!(
rebuild_command_line(&command),
"printf '%s %s' 'two words' 'it'\"'\"'s' ''"
);
}
}
crates/pi-builtins/src/nproc.rs
+206
View File
@@ -0,0 +1,206 @@
//! `nproc` builtin: print the number of processing units available.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{io::Write, thread};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use uucore::display::Quotable;
use crate::host::{Host, Utility, format_usage, matches_parser, util};
static OPT_ALL: &str = "all";
static OPT_IGNORE: &str = "ignore";
/// Parsed `nproc` invocation.
pub(crate) struct Nproc {
matches: ArgMatches,
}
matches_parser!(Nproc, app);
impl Utility for Nproc {
const NAME: &'static str = "nproc";
fn run(self, host: &mut Host) -> i32 {
let ignore = match self.matches.get_one::<String>(OPT_IGNORE) {
Some(numstr) => match numstr.trim().parse::<usize>() {
Ok(num) => num,
Err(error) => {
host.error(format!("{} is not a valid number: {error}", numstr.quote()), 1);
return 1;
},
},
None => 0,
};
let limit = match host.var("OMP_THREAD_LIMIT") {
// Use the OpenMP variable to limit the number of threads. A parse
// failure or zero means no limit.
Some(threads) => match threads.parse() {
Ok(0) | Err(_) => usize::MAX,
Ok(n) => n,
},
None => usize::MAX,
};
let mut cores = if self.matches.get_flag(OPT_ALL) {
num_cpus_all()
} else {
match host.var("OMP_NUM_THREADS") {
Some(threads) => {
// OMP_NUM_THREADS may be "x,y,z"; GNU nproc uses only the
// first value. A parse failure or zero falls back to CPU detection.
match threads.split_terminator(',').next() {
None => available_parallelism(),
Some(value) => match value.trim().parse() {
Ok(0) | Err(_) => available_parallelism(),
Ok(n) => n,
},
}
},
None => available_parallelism(),
}
};
cores = std::cmp::min(limit, cores);
if cores <= ignore {
cores = 1;
} else {
cores -= ignore;
}
if let Err(error) = writeln!(host.stdout, "{cores}") {
host.error(error, 1);
return 1;
}
0
}
}
fn app() -> Command {
Command::new("nproc")
.version("0.8.0")
.about(
"Print the number of cores available to the current process.\nIf the OMP_NUM_THREADS or \
OMP_THREAD_LIMIT environment variables are set, then\nthey will determine the minimum \
and maximum returned value respectively.",
)
.override_usage(format_usage("nproc [OPTIONS]..."))
.infer_long_args(true)
.arg(
Arg::new(OPT_ALL)
.long(OPT_ALL)
.help("print the number of cores available to the system")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_IGNORE)
.long(OPT_IGNORE)
.value_name("N")
.help("ignore up to N cores"),
)
}
#[cfg(unix)]
fn num_cpus_all() -> usize {
// In some situations, /proc and /sys are not mounted, and sysconf returns 1.
// However, we want to guarantee that `nproc --all` >= `nproc`.
unsafe { libc::sysconf(libc::_SC_NPROCESSORS_CONF) }
.try_into()
.ok()
.filter(|&n: &isize| n > 1)
.map_or_else(available_parallelism, |n| n as usize)
}
#[cfg(not(unix))]
fn num_cpus_all() -> usize {
available_parallelism()
}
/// Returns the available parallelism, falling back to one like GNU `nproc`.
fn available_parallelism() -> usize {
thread::available_parallelism().map_or(1, std::num::NonZeroUsize::get)
}
/// Creates the `nproc` builtin registration.
pub(crate) fn nproc_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Nproc, SE>()
}
#[cfg(test)]
mod tests {
use std::ffi::OsString;
use clap::Parser;
use super::*;
fn run_in(env: &[(&str, &str)], args: &[&str]) -> (i32, crate::host::Capture) {
let (mut host, capture) = Host::for_test("nproc", "", ".");
for (key, value) in env {
host.set_test_var(key, value);
}
let argv: Vec<OsString> = std::iter::once(OsString::from("nproc"))
.chain(args.iter().map(OsString::from))
.collect();
let parsed = Nproc::try_parse_from(argv).expect("test arguments should parse");
let code = parsed.run(&mut host);
(code, capture)
}
#[test]
fn host_omp_num_threads_forces_count() {
let (code, capture) = run_in(&[("OMP_NUM_THREADS", "3")], &[]);
assert_eq!((code, capture.out(), capture.err()), (0, "3\n".to_string(), String::new()));
}
#[test]
fn omp_thread_limit_caps_omp_num_threads() {
let (code, capture) = run_in(
&[("OMP_NUM_THREADS", "64"), ("OMP_THREAD_LIMIT", "2")],
&[],
);
assert_eq!((code, capture.out(), capture.err()), (0, "2\n".to_string(), String::new()));
}
#[test]
fn all_prints_positive_integer_and_ignores_omp_num_threads() {
let (code, capture) = run_in(&[("OMP_NUM_THREADS", "0")], &["--all"]);
assert_eq!(code, 0);
assert_eq!(capture.err(), "");
let n: usize = capture.out().trim_end().parse().expect("--all output is an integer");
assert!(n >= 1);
}
#[test]
fn process_environment_is_not_consulted() {
// A variable present only in the host process must not affect the shell
// builtin, whose exported environment lives on `Host`.
unsafe { std::env::set_var("OMP_NUM_THREADS", "1234") };
let (code, capture) = run_in(&[], &[]);
unsafe { std::env::remove_var("OMP_NUM_THREADS") };
assert_eq!((code, capture.err()), (0, String::new()));
assert_ne!(capture.out(), "1234\n");
let n: usize = capture.out().trim_end().parse().expect("output is an integer");
assert!(n >= 1);
}
#[test]
fn ignore_subtracts_and_floors_at_one() {
let (code, capture) = run_in(&[("OMP_NUM_THREADS", "8")], &["--ignore=3"]);
assert_eq!((code, capture.out()), (0, "5\n".to_string()));
let (code, capture) = run_in(&[("OMP_NUM_THREADS", "2")], &["--ignore=5"]);
assert_eq!((code, capture.out()), (0, "1\n".to_string()));
}
#[test]
fn invalid_ignore_value_is_an_error() {
let (code, capture) = run_in(&[], &["--ignore=bogus"]);
assert_eq!(code, 1);
assert_eq!(capture.out(), "");
assert!(capture.err().contains("is not a valid number"), "stderr: {}", capture.err());
}
}
crates/pi-builtins/src/paste.rs
+452
View File
@@ -0,0 +1,452 @@
//! `paste` builtin: merge corresponding or successive lines of files.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
cell::RefCell,
ffi::OsString,
fs::File,
io::{self, BufRead, BufReader, Read, Write},
iter::Cycle,
rc::Rc,
slice::Iter,
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use uucore::{display::Quotable, i18n::charmap::mb_char_len};
use crate::host::{Host, Stdin, Utility, format_usage, matches_parser, os_bytes, util};
mod options {
pub const DELIMITER: &str = "delimiters";
pub const SERIAL: &str = "serial";
pub const FILE: &str = "file";
pub const ZERO_TERMINATED: &str = "zero-terminated";
}
/// Parsed `paste` invocation.
pub(crate) struct Paste {
matches: ArgMatches,
}
matches_parser!(Paste, app);
impl Utility for Paste {
const NAME: &'static str = "paste";
fn run(self, host: &mut Host) -> i32 {
let serial = self.matches.get_flag(options::SERIAL);
let delimiters = self
.matches
.get_one::<OsString>(options::DELIMITER)
.expect("delimiter has a default")
.clone();
let files = self
.matches
.get_many::<OsString>(options::FILE)
.expect("file has a default")
.cloned()
.collect();
let line_ending = if self.matches.get_flag(options::ZERO_TERMINATED) {
b'\0'
} else {
b'\n'
};
match paste(host, files, serial, &delimiters, line_ending) {
Ok(()) => host.exit_code(),
Err(err) => {
host.error(err, 1);
1
},
}
}
}
/// The `paste` argument model.
fn app() -> Command {
Command::new(Paste::NAME)
.version("0.8.0")
.about("Merge lines of files")
.override_usage(format_usage("paste [OPTION]... [FILE]..."))
.infer_long_args(true)
.arg(
Arg::new(options::SERIAL)
.long(options::SERIAL)
.short('s')
.help("paste one file at a time instead of in parallel")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(options::DELIMITER)
.long(options::DELIMITER)
.short('d')
.help("reuse characters from LIST instead of TABs")
.value_name("LIST")
.default_value("\t")
.hide_default_value(true)
.value_parser(clap::value_parser!(OsString)),
)
.arg(
Arg::new(options::FILE)
.value_name("FILE")
.action(ArgAction::Append)
.default_value("-")
.value_hint(clap::ValueHint::FilePath)
.value_parser(clap::value_parser!(OsString)),
)
.arg(
Arg::new(options::ZERO_TERMINATED)
.long(options::ZERO_TERMINATED)
.short('z')
.help("line delimiter is NUL, not newline")
.action(ArgAction::SetTrue),
)
}
fn paste(
host: &mut Host,
filenames: Vec<OsString>,
serial: bool,
delimiters: &OsString,
line_ending: u8,
) -> Result<(), String> {
let delimiters = parse_delimiters(delimiters)?;
let mut prepared = Vec::with_capacity(filenames.len());
for filename in filenames {
if filename == "-" {
prepared.push(PreparedSource::StandardInput);
} else {
let file = File::open(host.resolve(&filename)).map_err(|err| {
format!("{}: {}", filename.to_string_lossy(), strip_errno(&err))
})?;
prepared.push(PreparedSource::File(BufReader::new(file)));
}
}
let stdin = Rc::new(RefCell::new(BufReader::new(&mut host.stdin)));
let mut sources = prepared
.into_iter()
.map(|source| match source {
PreparedSource::File(reader) => InputSource::File(reader),
PreparedSource::StandardInput => InputSource::StandardInput(Rc::clone(&stdin)),
})
.collect::<Vec<_>>();
let source_count = sources.len();
let stdout = &mut host.stdout;
if !serial && source_count == 1 {
return write_single_input_source(stdout, sources.pop().unwrap(), line_ending)
.map_err(|err| strip_errno(&err));
}
let mut delimiter_state = DelimiterState::new(&delimiters);
let mut output = Vec::new();
if serial {
for source in &mut sources {
output.clear();
loop {
if source.read_until(line_ending, &mut output).map_err(|err| strip_errno(&err))?
== 0
{
break;
}
remove_trailing_line_ending(line_ending, &mut output);
delimiter_state.write_delimiter(&mut output);
}
delimiter_state.remove_trailing_delimiter(&mut output);
stdout.write_all(&output).map_err(|err| strip_errno(&err))?;
stdout.write_all(&[line_ending]).map_err(|err| strip_errno(&err))?;
}
} else {
let mut eof = vec![false; source_count];
loop {
output.clear();
let mut eof_count = 0;
for (i, source) in sources.iter_mut().enumerate() {
if eof[i] {
eof_count += 1;
} else if source
.read_until(line_ending, &mut output)
.map_err(|err| strip_errno(&err))?
== 0
{
eof[i] = true;
eof_count += 1;
} else {
remove_trailing_line_ending(line_ending, &mut output);
}
delimiter_state.write_delimiter(&mut output);
}
if eof_count == source_count {
break;
}
delimiter_state.remove_trailing_delimiter(&mut output);
stdout.write_all(&output).map_err(|err| strip_errno(&err))?;
stdout.write_all(&[line_ending]).map_err(|err| strip_errno(&err))?;
delimiter_state.reset_to_first_delimiter();
}
}
Ok(())
}
fn write_single_input_source(
writer: &mut impl Write,
mut source: InputSource<'_>,
line_ending: u8,
) -> io::Result<()> {
let mut buffer = [0_u8; 8192];
let mut has_data = false;
let mut last_byte = line_ending;
loop {
let count = source.read(&mut buffer)?;
if count == 0 {
break;
}
has_data = true;
last_byte = buffer[count - 1];
writer.write_all(&buffer[..count])?;
}
if has_data && last_byte != line_ending {
writer.write_all(&[line_ending])?;
}
Ok(())
}
fn parse_delimiters(delimiters: &OsString) -> Result<Box<[Box<[u8]>]>, String> {
let bytes = os_bytes(delimiters).ok_or_else(|| {
format!(
"invalid UTF-8 input {} encountered when converting to bytes on a platform that doesn't expose byte arguments",
delimiters.quote()
)
})?;
let mut result = Vec::<Box<[u8]>>::with_capacity(bytes.len());
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'\\' {
i += 1;
if i >= bytes.len() {
return Err(format!(
"delimiter list ends with an unescaped backslash: {}",
delimiters.to_string_lossy()
));
}
match bytes[i] {
b'0' => result.push(Box::new([])),
b'\\' => result.push(Box::new(*b"\\")),
b'n' => result.push(Box::new(*b"\n")),
b't' => result.push(Box::new(*b"\t")),
b'b' => result.push(Box::new(*b"\x08")),
b'f' => result.push(Box::new(*b"\x0c")),
b'r' => result.push(Box::new(*b"\r")),
b'v' => result.push(Box::new(*b"\x0b")),
_ => {
let len = mb_char_len(&bytes[i..]).min(bytes.len() - i);
result.push(Box::from(&bytes[i..i + len]));
i += len;
continue;
},
}
i += 1;
} else {
let len = mb_char_len(&bytes[i..]).min(bytes.len() - i);
result.push(Box::from(&bytes[i..i + len]));
i += len;
}
}
Ok(result.into_boxed_slice())
}
fn remove_trailing_line_ending(line_ending: u8, output: &mut Vec<u8>) {
if output.last() == Some(&line_ending) {
output.pop();
}
}
enum DelimiterState<'a> {
NoDelimiters,
OneDelimiter(&'a [u8]),
MultipleDelimiters {
current: &'a [u8],
delimiters: &'a [Box<[u8]>],
iterator: Cycle<Iter<'a, Box<[u8]>>>,
},
}
impl<'a> DelimiterState<'a> {
fn new(delimiters: &'a [Box<[u8]>]) -> Self {
match delimiters {
[] => Self::NoDelimiters,
[only] if only.is_empty() => Self::NoDelimiters,
[only] => Self::OneDelimiter(only),
[first, ..] => Self::MultipleDelimiters {
current: first,
delimiters,
iterator: delimiters.iter().cycle(),
},
}
}
fn reset_to_first_delimiter(&mut self) {
if let Self::MultipleDelimiters { delimiters, iterator, .. } = self {
*iterator = delimiters.iter().cycle();
}
}
fn remove_trailing_delimiter(&self, output: &mut Vec<u8>) {
let len = match self {
Self::NoDelimiters => return,
Self::OneDelimiter(d) => d.len(),
Self::MultipleDelimiters { current, .. } => current.len(),
};
if len > 0 {
output.truncate(output.len().saturating_sub(len));
}
}
fn write_delimiter(&mut self, output: &mut Vec<u8>) {
match self {
Self::NoDelimiters => {},
Self::OneDelimiter(d) => output.extend_from_slice(d),
Self::MultipleDelimiters { current, iterator, .. } => {
let d = iterator.next().unwrap();
output.extend_from_slice(d);
*current = d;
},
}
}
}
enum PreparedSource {
File(BufReader<File>),
StandardInput,
}
enum InputSource<'a> {
File(BufReader<File>),
StandardInput(Rc<RefCell<BufReader<&'a mut Stdin>>>),
}
impl Read for InputSource<'_> {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
match self {
Self::File(reader) => reader.read(buf),
Self::StandardInput(stdin) => stdin.borrow_mut().read(buf),
}
}
}
impl BufRead for InputSource<'_> {
fn fill_buf(&mut self) -> io::Result<&[u8]> {
match self {
Self::File(reader) => reader.fill_buf(),
Self::StandardInput(_) => Err(io::Error::other(
"standard input does not support direct buffer access",
)),
}
}
fn consume(&mut self, amount: usize) {
if let Self::File(reader) = self {
reader.consume(amount);
}
}
fn read_until(&mut self, byte: u8, buf: &mut Vec<u8>) -> io::Result<usize> {
match self {
Self::File(reader) => reader.read_until(byte, buf),
Self::StandardInput(stdin) => stdin.borrow_mut().read_until(byte, buf),
}
}
}
fn strip_errno(error: &io::Error) -> String {
let mut message = error.to_string();
if let Some(position) = message.find(" (os error ") {
message.truncate(position);
}
message
}
/// Creates the `paste` builtin registration.
pub(crate) fn paste_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Paste, SE>()
}
#[cfg(test)]
mod tests {
use std::{fs, path::Path};
use super::Paste;
use crate::host::run_util;
fn paste(argv: &[&str], stdin: &str, cwd: &Path) -> (i32, Vec<u8>, String) {
let (code, capture) = run_util::<Paste>(argv, stdin, cwd);
(code, capture.stdout(), capture.err())
}
#[test]
fn merges_files_in_parallel_and_cycles_delimiters() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a"), "a1\na2\n").unwrap();
fs::write(dir.path().join("b"), "b1\nb2\n").unwrap();
fs::write(dir.path().join("c"), "c1\nc2\n").unwrap();
assert_eq!(
paste(&["-d", ",:", "a", "b", "c"], "", dir.path()),
(0, b"a1,b1:c1\na2,b2:c2\n".to_vec(), String::new())
);
}
#[test]
fn serial_mode_pastes_each_file_separately() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("a"), "a1\na2\n").unwrap();
fs::write(dir.path().join("b"), "b1\nb2\n").unwrap();
assert_eq!(
paste(&["-s", "-d", ",", "a", "b"], "", dir.path()),
(0, b"a1,a2\nb1,b2\n".to_vec(), String::new())
);
}
#[test]
fn repeated_stdin_operands_consume_successive_lines() {
assert_eq!(
paste(&["-", "-"], "first\nsecond\n", Path::new("/")),
(0, b"first\tsecond\n".to_vec(), String::new())
);
}
#[test]
fn zero_terminated_mode_uses_nul_for_input_and_output() {
assert_eq!(
paste(&["-z", "-d", ","], "a\0b", Path::new("/")),
(0, b"a\0b\0".to_vec(), String::new())
);
}
#[test]
fn zero_delimiter_escape_joins_without_separator() {
assert_eq!(
paste(&["-s", "-d", "\\0"], "a\nb\n", Path::new("/")),
(0, b"ab\n".to_vec(), String::new())
);
}
#[test]
fn reports_trailing_backslash_in_delimiter_list() {
let (code, _, err) = paste(&["-d", "\\"], "", Path::new("/"));
assert_eq!(code, 1);
assert_eq!(err, "paste: delimiter list ends with an unescaped backslash: \\\n");
}
#[test]
fn resolves_relative_file_operands_against_shell_cwd() {
let dir = tempfile::tempdir().unwrap();
fs::write(dir.path().join("input"), "line").unwrap();
assert_eq!(
paste(&["input"], "", dir.path()),
(0, b"line\n".to_vec(), String::new())
);
}
}
crates/pi-builtins/src/pgrep.rs
+106
View File
@@ -0,0 +1,106 @@
//! The `pgrep` process-matching command, moved from `pi-shell`.
use brush_core::builtins;
use clap::Parser;
use crate::proc_match;
/// Finds processes matching the supplied selection criteria.
#[derive(Parser)]
#[command(disable_help_flag = true, disable_version_flag = true)]
pub(crate) struct PgrepCommand {
#[arg(num_args = 0.., trailing_var_arg = true, allow_hyphen_values = true)]
argv: Vec<String>,
}
impl builtins::Command for PgrepCommand {
type Error = brush_core::Error;
fn execute<SE: brush_core::ShellExtensions>(
&self,
context: brush_core::ExecutionContext<'_, SE>,
) -> impl Future<Output = Result<brush_core::ExecutionResult, Self::Error>> + Send {
proc_match::run(proc_match::ProcMatchMode::Grep, self.argv.clone(), context)
}
}
#[cfg(test)]
mod tests {
use std::io::Read as _;
use brush_core::{
ExecutionContext, Shell,
builtins::Command as _,
openfiles::{self, OpenFiles},
};
#[cfg(unix)]
fn matching_process() -> std::process::Child {
std::process::Command::new("/bin/sleep")
.arg("30")
.spawn()
.expect("spawn matching process")
}
use super::PgrepCommand;
async fn execute(argv: Vec<String>) -> (brush_core::ExecutionResult, String) {
let mut shell = Shell::builder().build().await.expect("build test shell");
let mut params = shell.default_exec_params();
let (mut output, writer) = std::io::pipe().expect("create output pipe");
params.set_fd(OpenFiles::STDIN_FD, openfiles::null().expect("open null stdin"));
params.set_fd(OpenFiles::STDOUT_FD, writer.into());
params.set_fd(OpenFiles::STDERR_FD, openfiles::null().expect("open null stderr"));
let command = PgrepCommand { argv };
let result = command
.execute(ExecutionContext { shell: &mut shell, command_name: "pgrep".into(), params })
.await
.expect("execute pgrep");
let mut stdout = String::new();
output.read_to_string(&mut stdout).expect("read pgrep output");
(result, stdout)
}
#[tokio::test]
async fn exits_one_when_no_process_matches() {
let (result, output) = execute(vec!["-p".into(), i32::MAX.to_string()]).await;
assert_eq!(u8::from(result.exit_code), 1);
assert!(output.is_empty());
}
#[cfg(unix)]
#[tokio::test]
async fn exits_zero_and_prints_matching_pid() {
let mut child = matching_process();
let pid = child.id();
let (result, output) = execute(vec!["-p".into(), pid.to_string()]).await;
child.kill().expect("kill matching process");
child.wait().expect("reap matching process");
assert_eq!(u8::from(result.exit_code), 0);
assert_eq!(output, format!("{pid}\n"));
}
#[cfg(unix)]
#[tokio::test]
async fn count_prints_number_of_matches() {
let mut child = matching_process();
let pid = child.id();
let (result, output) = execute(vec!["-c".into(), "-p".into(), pid.to_string()]).await;
child.kill().expect("kill matching process");
child.wait().expect("reap matching process");
assert_eq!(u8::from(result.exit_code), 0);
assert_eq!(output, "1\n");
}
#[cfg(unix)]
#[tokio::test]
async fn quiet_suppresses_matching_output() {
let mut child = matching_process();
let pid = child.id();
let (result, output) = execute(vec!["-q".into(), "-p".into(), pid.to_string()]).await;
child.kill().expect("kill matching process");
child.wait().expect("reap matching process");
assert_eq!(u8::from(result.exit_code), 0);
assert!(output.is_empty());
}
}
crates/pi-builtins/src/pidwait.rs
+76
View File
@@ -0,0 +1,76 @@
//! `pidwait` process-waiting builtin, moved from `pi-shell`.
use brush_core::builtins;
use clap::Parser;
use crate::proc_match;
/// Waits for processes selected by process attributes or a name pattern.
#[derive(Parser)]
#[command(disable_help_flag = true, disable_version_flag = true)]
pub(crate) struct PidwaitCommand {
#[arg(num_args = 0.., trailing_var_arg = true, allow_hyphen_values = true)]
argv: Vec<String>,
}
impl builtins::Command for PidwaitCommand {
type Error = brush_core::Error;
async fn execute<SE: brush_core::ShellExtensions>(
&self,
context: brush_core::ExecutionContext<'_, SE>,
) -> Result<brush_core::ExecutionResult, Self::Error> {
proc_match::run(proc_match::ProcMatchMode::Wait, self.argv.clone(), context).await
}
}
#[cfg(test)]
mod tests {
use std::{process::Command as ProcessCommand, time::Duration};
use brush_core::builtins::Command as _;
use super::PidwaitCommand;
async fn execute_bounded(argv: Vec<String>) -> anyhow::Result<brush_core::ExecutionResult> {
let mut shell = brush_core::Shell::builder().build().await?;
let params = shell.default_exec_params();
let command = PidwaitCommand { argv };
let context = brush_core::ExecutionContext {
shell: &mut shell,
command_name: "pidwait".to_string(),
params,
};
Ok(tokio::time::timeout(Duration::from_secs(2), command.execute(context))
.await
.expect("pidwait exceeded its two-second test bound")?)
}
#[tokio::test]
async fn exits_one_when_nothing_matches() -> anyhow::Result<()> {
let result = execute_bounded(vec!["-p".to_string(), i32::MAX.to_string()]).await?;
assert_eq!(u8::from(&result.exit_code), 1);
Ok(())
}
#[tokio::test]
async fn already_exited_pid_returns_promptly() -> anyhow::Result<()> {
#[cfg(unix)]
let mut child = ProcessCommand::new("sh").args(["-c", "exit 0"]).spawn()?;
#[cfg(windows)]
let mut child = ProcessCommand::new("cmd").args(["/C", "exit 0"]).spawn()?;
let pid = child.id();
// Leave the child unreaped so it remains visible in the process snapshot,
// while giving the trivial command enough time to reach its exited state.
tokio::time::sleep(Duration::from_millis(100)).await;
let outcome = execute_bounded(vec!["-p".to_string(), pid.to_string()]).await;
let _ = child.kill();
let _ = child.wait();
let result = outcome?;
assert!(result.is_success() || u8::from(&result.exit_code) == 1);
Ok(())
}
}
crates/pi-builtins/src/pkill.rs
+62
View File
@@ -0,0 +1,62 @@
//! `pkill` process-signalling builtin moved from `pi-shell`.
use brush_core::builtins;
use clap::Parser;
use crate::proc_match;
/// Selects processes by name or attributes and sends them a signal.
#[derive(Parser)]
#[command(disable_help_flag = true, disable_version_flag = true)]
pub(crate) struct PkillCommand {
/// Arguments interpreted by the shared process-matching engine.
#[arg(num_args = 0.., trailing_var_arg = true, allow_hyphen_values = true)]
argv: Vec<String>,
}
impl builtins::Command for PkillCommand {
type Error = brush_core::Error;
async fn execute<SE: brush_core::ShellExtensions>(
&self,
context: brush_core::ExecutionContext<'_, SE>,
) -> Result<brush_core::ExecutionResult, Self::Error> {
proc_match::run(proc_match::ProcMatchMode::Kill, self.argv.clone(), context).await
}
}
#[cfg(test)]
mod tests {
use brush_core::{Shell, builtins};
use super::PkillCommand;
const NO_MATCH: &str = "^__brush_pkill_test_no_such_process_6f239a1d__$";
async fn run(args: &str) -> brush_core::ExecutionResult {
let mut shell = Shell::builder()
.builtin("pkill", builtins::builtin::<PkillCommand, _>())
.build()
.await
.expect("test shell should build");
shell
.run_dash_c_command(format!("pkill {args} {NO_MATCH}"))
.await
.expect("pkill should execute")
}
#[tokio::test]
async fn exits_one_when_no_process_matches() {
assert_eq!(u8::from(run("").await.exit_code), 1);
}
#[tokio::test]
async fn accepts_a_signal_name() {
assert_eq!(u8::from(run("-TERM").await.exit_code), 1);
}
#[tokio::test]
async fn accepts_a_signal_number() {
assert_eq!(u8::from(run("-9").await.exit_code), 1);
}
}
crates/vendor/brush-builtins/src/popd.rs → crates/pi-builtins/src/popd.rs
View File
crates/pi-builtins/src/printenv.rs
+200
View File
@@ -0,0 +1,200 @@
//! `printenv` builtin: display values from the shell's exported environment.
//!
//! Ported from uutils coreutils 0.8.0.
use std::io::Write;
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use uucore::line_ending::LineEnding;
use crate::host::{Host, Utility, format_usage, matches_parser, util};
const OPT_NULL: &str = "null";
const ARG_VARIABLES: &str = "variables";
/// Parsed `printenv` invocation.
pub(crate) struct Printenv {
matches: ArgMatches,
}
matches_parser!(Printenv, app);
impl Utility for Printenv {
const NAME: &'static str = "printenv";
fn run(self, host: &mut Host) -> i32 {
let variables = self
.matches
.get_many::<String>(ARG_VARIABLES)
.into_iter()
.flatten()
.collect::<Vec<_>>();
let separator = LineEnding::from_zero_flag(self.matches.get_flag(OPT_NULL));
if variables.len() == 0 {
// Hash map iteration order is intentionally hidden from callers. Sort by
// name so repeated invocations produce the same environment dump.
let mut environment = host
.env()
.map(|(name, value)| (name.to_owned(), value.to_owned()))
.collect::<Vec<_>>();
environment.sort_unstable_by(|left, right| left.0.cmp(&right.0));
for (name, value) in environment {
if let Err(error) = write!(host.stdout, "{name}={value}{separator}") {
host.error(error, 1);
return 1;
}
}
if let Err(error) = host.stdout.flush() {
host.error(error, 1);
return 1;
}
return 0;
}
let mut error_found = false;
for env_var in variables {
// We silently ignore a=b as a variable, but still report failure in
// the exit status.
if env_var.contains('=') {
error_found = true;
continue;
}
if let Some(value) = host.var(env_var).map(str::to_owned) {
if let Err(error) = write!(host.stdout, "{value}{separator}") {
host.error(error, 1);
return 1;
}
if let Err(error) = host.stdout.flush() {
host.error(error, 1);
return 1;
}
} else {
error_found = true;
}
}
i32::from(error_found)
}
}
/// The `printenv` argument model.
fn app() -> Command {
Command::new(Printenv::NAME)
.version("0.8.0")
.about(
"Display the values of the specified environment VARIABLE(s), or (with no VARIABLE) \
display name and value pairs for them all.",
)
.override_usage(format_usage("printenv [OPTION]... [VARIABLE]..."))
.infer_long_args(true)
.arg(
Arg::new(OPT_NULL)
.short('0')
.long(OPT_NULL)
.help("end each output line with 0 byte rather than newline")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(ARG_VARIABLES)
.action(ArgAction::Append)
.num_args(1..),
)
}
/// Creates the `printenv` builtin registration.
pub(crate) fn printenv_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Printenv, SE>()
}
#[cfg(test)]
mod tests {
use std::ffi::OsString;
use clap::Parser;
use super::Printenv;
use crate::host::{Host, Utility, run_util};
const ENV: &[(&str, &str)] = &[("FOO", "bar"), ("BAZ", "qux")];
fn run_with_env(env: &[(&str, &str)], args: &[&str]) -> (i32, String, String) {
let (mut host, capture) = Host::for_test(Printenv::NAME, Vec::new(), ".");
for &(name, value) in env {
host.set_test_var(name, value);
}
let argv = std::iter::once(OsString::from(Printenv::NAME))
.chain(args.iter().map(OsString::from));
let parsed = Printenv::try_parse_from(argv).expect("test arguments must parse");
let code = parsed.run(&mut host);
(code, capture.out(), capture.err())
}
#[test]
fn named_variable_prints_scope_value() {
let (code, stdout, stderr) = run_with_env(ENV, &["FOO"]);
assert_eq!(code, 0);
assert_eq!(stdout, "bar\n");
assert_eq!(stderr, "");
}
#[test]
fn shell_export_absent_from_process_environment_is_printed() {
const NAME: &str = "__BRUSH_PRINTENV_TEST_ONLY_EXPORTED_VARIABLE__";
assert!(std::env::var_os(NAME).is_none());
let (code, stdout, stderr) = run_with_env(&[(NAME, "shell-value")], &[NAME]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (0, "shell-value\n", ""));
}
#[test]
fn unset_variable_is_silent_failure() {
let (code, stdout, stderr) = run_with_env(ENV, &["NOPE"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "", "unset variables fail without a message");
}
#[test]
fn mixed_set_and_unset_prints_set_ones_and_fails() {
let (code, stdout, stderr) = run_with_env(ENV, &["FOO", "NOPE", "BAZ"]);
assert_eq!(code, 1);
assert_eq!(stdout, "bar\nqux\n");
assert_eq!(stderr, "");
}
#[test]
fn no_args_dumps_scope_env_not_process_env_in_name_order() {
assert!(std::env::var_os("PATH").is_some());
let (code, stdout, stderr) = run_with_env(ENV, &[]);
assert_eq!(code, 0);
assert_eq!(stderr, "");
assert_eq!(stdout, "BAZ=qux\nFOO=bar\n");
}
#[test]
fn null_flag_terminates_with_nul() {
let (code, stdout, _) = run_with_env(ENV, &["-0", "FOO"]);
assert_eq!((code, stdout.as_str()), (0, "bar\0"));
let (code, stdout, _) = run_with_env(ENV, &["--null", "FOO", "BAZ"]);
assert_eq!((code, stdout.as_str()), (0, "bar\0qux\0"));
}
#[test]
fn name_containing_equals_is_ignored_but_fails() {
let (code, stdout, stderr) = run_with_env(ENV, &["FOO=bar", "BAZ"]);
assert_eq!(code, 1);
assert_eq!(stdout, "qux\n");
assert_eq!(stderr, "");
}
#[test]
fn help_renders_to_scope_stdout() {
let (code, capture) = run_util::<Printenv>(&["--help"], "", ".");
assert_eq!(code, 0);
assert!(capture.out().contains("Usage:"));
assert!(capture.out().contains("environment VARIABLE"));
assert_eq!(capture.err(), "");
}
}
crates/vendor/brush-builtins/src/printf.rs → crates/pi-builtins/src/printf.rs
View File
crates/pi-builtins/src/proc_match.rs
+937
View File
@@ -0,0 +1,937 @@
//! Shared process-matching engine behind `pgrep`, `pkill`, and `pidwait`.
//!
//! The three commands differ only in what they do with the processes they
//! select — print them, signal them, or wait for them — so selection, argument
//! parsing, and help rendering all live here, and each command is a thin front
//! end over [`run`].
//!
//! Ported from `pi-shell`, which previously defined all three inline.
// The three front ends are each feature-gated, so a build with only some of
// them enabled legitimately uses only part of this module.
#![allow(dead_code, reason = "consumed by the feature-gated pgrep/pkill/pidwait front ends")]
use std::{
collections::{HashMap, HashSet},
fs,
future::Future,
io::{self, BufRead, Write},
path::{Path, PathBuf},
time::Duration,
};
use brush_core::{
ExecutionContext, ExecutionExitCode, ExecutionResult, openfiles::OpenFiles,
};
use tokio_util::sync::CancellationToken;
use crate::{kill::signal_number, proc_snapshot};
/// What a process-matching command does with the processes it selects.
#[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) enum ProcMatchMode {
Grep,
Kill,
Wait,
}
#[derive(Default)]
struct ProcMatchOptions {
patterns: Vec<String>,
full: bool,
exact: bool,
ignore_case: bool,
invert: bool,
newest: bool,
oldest: bool,
parents: Vec<i32>,
groups: Vec<i32>,
sessions: Vec<i32>,
effective_users: Vec<u32>,
real_users: Vec<u32>,
real_groups: Vec<u32>,
terminals: Vec<Option<u64>>,
pids: Vec<i32>,
pid_files: Vec<String>,
explicit_pid: bool,
require_lock: bool,
older: Option<Duration>,
states: HashSet<char>,
ignore_ancestors: bool,
include_ancestors: bool,
count: bool,
list_name: bool,
list_full: bool,
quiet: bool,
delimiter: String,
signal: i32,
queue: Option<i32>,
echo: bool,
echo_command: bool,
interactive: bool,
}
/// Runs the process-matching body for `mode`.
///
/// `pgrep`, `pkill`, and `pidwait` each call this with their own mode; the
/// invoked name still comes from the execution context, so diagnostics and help
/// name the command the user actually typed.
pub(crate) fn run<SE: brush_core::ShellExtensions>(
mode: ProcMatchMode,
argv: Vec<String>,
context: ExecutionContext<'_, SE>,
) -> impl Future<Output = std::result::Result<ExecutionResult, brush_core::Error>> + Send {
{
let command_name = context.command_name.clone();
let cwd = context.shell.working_dir().to_path_buf();
async move {
#[cfg(unix)]
let stdin_watcher = context.try_fd(OpenFiles::STDIN_FD).and_then(|stdin| {
let fd = stdin.try_borrow_as_fd().ok()?.try_clone_to_owned().ok()?;
tokio::io::unix::AsyncFd::new(fd).ok()
});
let mut stdin = io::BufReader::new(context.stdin());
let mut options = match parse_proc_match_args(mode, &argv, &cwd, &mut stdin) {
Ok(ParseProcResult::Options(options)) => *options,
Ok(ParseProcResult::Help) => {
write_proc_match_help(context.stdout(), &command_name, mode)?;
return Ok(ExecutionResult::success());
},
Ok(ParseProcResult::Version) => {
writeln!(context.stdout(), "{command_name} {}", env!("CARGO_PKG_VERSION"))?;
return Ok(ExecutionResult::success());
},
Err((code, message)) => {
writeln!(context.stderr(), "{command_name}: {message}")?;
return Ok(ExecutionResult::new(code));
},
};
if context.is_cancelled() {
return Ok(ExecutionExitCode::Interrupted.into());
}
let processes = match select_processes(&mut options) {
Ok(processes) => processes,
Err(message) => {
writeln!(context.stderr(), "{command_name}: {message}")?;
return Ok(ExecutionResult::new(2));
},
};
if processes.is_empty() {
if options.count && !options.quiet {
writeln!(context.stdout(), "0")?;
}
return Ok(ExecutionResult::new(1));
}
match mode {
ProcMatchMode::Grep => {
if options.quiet {
return Ok(ExecutionResult::success());
}
if options.count {
writeln!(context.stdout(), "{}", processes.len())?;
} else {
let mut output = Vec::with_capacity(processes.len());
for process in &processes {
let line = if options.list_full
|| (cfg!(target_os = "macos") && options.list_name && options.full)
{
format!("{} {}", process.pid(), process.args().join(" "))
} else if options.list_name {
format!("{} {}", process.pid(), process.command_name())
} else {
process.pid().to_string()
};
output.push(line);
}
writeln!(context.stdout(), "{}", output.join(&options.delimiter))?;
}
},
ProcMatchMode::Kill => {
if options.count && !options.quiet {
writeln!(context.stdout(), "{}", processes.len())?;
}
let mut succeeded = false;
for process in &processes {
if context.is_cancelled() {
return Ok(ExecutionExitCode::Interrupted.into());
}
if options.interactive {
{
let mut stderr = context.stderr();
write!(stderr, "kill process {}? ", process.pid())?;
stderr.flush()?;
}
#[cfg(unix)]
let response = read_proc_confirmation(
&mut stdin,
context.cancel_token(),
stdin_watcher.as_ref(),
)
.await?;
#[cfg(not(unix))]
let response = read_proc_confirmation(&mut stdin, context.cancel_token()).await?;
let Some(response) = response else {
return Ok(ExecutionExitCode::Interrupted.into());
};
if !matches!(response.trim(), "y" | "Y" | "yes" | "YES") {
continue;
}
}
if context.is_cancelled() {
return Ok(ExecutionExitCode::Interrupted.into());
}
if !process.signal(options.signal, options.queue) {
if !options.quiet {
writeln!(
context.stderr(),
"{command_name}: signalling pid {} failed",
process.pid()
)?;
}
continue;
}
succeeded = true;
if options.echo_command && !options.quiet {
writeln!(context.stdout(), "kill -{} {}", options.signal, process.pid())?;
} else if options.echo && !options.quiet {
writeln!(
context.stdout(),
"{} killed (pid {})",
process.command_name(),
process.pid()
)?;
}
}
if !succeeded {
return Ok(ExecutionResult::new(1));
}
},
ProcMatchMode::Wait => {
if options.count && !options.quiet {
writeln!(context.stdout(), "{}", processes.len())?;
}
if options.echo && !options.quiet {
for process in &processes {
writeln!(
context.stdout(),
"waiting for {} (pid {})",
process.command_name(),
process.pid()
)?;
}
}
loop {
if processes
.iter()
.all(|process| process.status() == proc_snapshot::ProcessStatus::Exited)
{
break;
}
if context.is_cancelled() {
return Ok(ExecutionExitCode::Interrupted.into());
}
if let Some(cancel_token) = context.cancel_token() {
tokio::select! {
() = tokio::time::sleep(Duration::from_millis(50)) => {},
() = cancel_token.cancelled() => {
return Ok(ExecutionExitCode::Interrupted.into());
},
}
} else {
tokio::time::sleep(Duration::from_millis(50)).await;
}
}
},
}
Ok(ExecutionResult::success())
}
}
}
#[cfg(unix)]
async fn read_proc_confirmation<R: io::Read>(
stdin: &mut io::BufReader<R>,
cancel_token: Option<CancellationToken>,
watcher: Option<&tokio::io::unix::AsyncFd<std::os::fd::OwnedFd>>,
) -> io::Result<Option<String>> {
if cancel_token
.as_ref()
.is_some_and(CancellationToken::is_cancelled)
{
return Ok(None);
}
if let Some(watcher) = watcher {
if let Some(cancel_token) = cancel_token {
let ready = tokio::select! {
ready = watcher.readable() => ready,
() = cancel_token.cancelled() => return Ok(None),
};
drop(ready?);
} else {
drop(watcher.readable().await?);
}
}
let mut response = String::new();
stdin.read_line(&mut response)?;
Ok(Some(response))
}
#[cfg(not(unix))]
async fn read_proc_confirmation<R: io::Read>(
stdin: &mut io::BufReader<R>,
cancel_token: Option<CancellationToken>,
) -> io::Result<Option<String>> {
if cancel_token
.as_ref()
.is_some_and(CancellationToken::is_cancelled)
{
return Ok(None);
}
let mut response = String::new();
stdin.read_line(&mut response)?;
Ok(Some(response))
}
enum ParseProcResult {
Options(Box<ProcMatchOptions>),
Help,
Version,
}
fn parse_proc_match_args(
mode: ProcMatchMode,
argv: &[String],
cwd: &Path,
stdin: &mut impl BufRead,
) -> std::result::Result<ParseProcResult, (u8, String)> {
let mut options =
ProcMatchOptions { delimiter: "\n".to_string(), signal: 15, ..Default::default() };
let mut index = 0;
let mut options_done = false;
while index < argv.len() {
let arg = &argv[index];
if !options_done && arg == "--" {
options_done = true;
index += 1;
continue;
}
if !options_done && matches!(arg.as_str(), "--help" | "-h") {
return Ok(ParseProcResult::Help);
}
if !options_done && arg == "--version" {
return Ok(ParseProcResult::Version);
}
if mode == ProcMatchMode::Kill
&& !options_done
&& index == 0
&& arg.starts_with('-')
&& !arg.starts_with("--")
&& signal_number(&arg[1..]).is_some()
{
options.signal = signal_number(&arg[1..]).unwrap_or(15);
index += 1;
continue;
}
if !options_done && arg.starts_with("--") {
let (name, inline_value) = arg
.split_once('=')
.map_or((arg.as_str(), None), |(name, value)| (name, Some(value)));
let takes_value =
matches!(
name,
"--parent"
| "--pgroup" | "--session"
| "--euid" | "--uid"
| "--group" | "--terminal"
| "--pidfile"
| "--pid" | "--older"
| "--runstates"
| "--delimiter"
| "--signal" | "--queue"
);
let value = if takes_value {
if let Some(value) = inline_value {
Some(value)
} else {
index += 1;
argv.get(index).map(String::as_str)
}
} else {
None
};
if takes_value && value.is_none() {
return Err((2, format!("option '{name}' requires an argument")));
}
match name {
"--full" => options.full = true,
"--exact" => options.exact = true,
"--ignore-case" => options.ignore_case = true,
"--inverse" => options.invert = true,
"--newest" => options.newest = true,
"--oldest" => options.oldest = true,
"--parent" => parse_i32_list(value.unwrap_or_default(), &mut options.parents)?,
"--pgroup" => parse_i32_list(value.unwrap_or_default(), &mut options.groups)?,
"--session" => parse_i32_list(value.unwrap_or_default(), &mut options.sessions)?,
"--euid" => parse_user_list(value.unwrap_or_default(), &mut options.effective_users)?,
"--uid" => parse_user_list(value.unwrap_or_default(), &mut options.real_users)?,
"--group" => parse_group_list(value.unwrap_or_default(), &mut options.real_groups)?,
"--terminal" => parse_terminal_list(value.unwrap_or_default(), &mut options.terminals)?,
"--pidfile" => options
.pid_files
.push(value.unwrap_or_default().to_string()),
"--pid" => {
options.explicit_pid = true;
parse_i32_list(value.unwrap_or_default(), &mut options.pids)?;
},
"--older" => {
options.older = Some(Duration::from_secs(
value
.unwrap_or_default()
.parse()
.map_err(|_| (2, "invalid age".to_string()))?,
));
},
"--runstates" => parse_states(value.unwrap_or_default(), &mut options.states)?,
"--ignore-ancestors" => options.ignore_ancestors = true,
"--count" => options.count = true,
"--list-name" => options.list_name = true,
"--list-full" => options.list_full = true,
"--quiet" => options.quiet = true,
"--delimiter" => options.delimiter = value.unwrap_or_default().to_string(),
"--signal" if mode == ProcMatchMode::Kill => {
options.signal = signal_number(value.unwrap_or_default())
.ok_or_else(|| (2, "invalid signal".to_string()))?;
},
"--queue" if mode == ProcMatchMode::Kill && cfg!(target_os = "linux") => {
options.queue = Some(
value
.unwrap_or_default()
.parse()
.map_err(|_| (2, "invalid queue value".to_string()))?,
);
},
"--echo" if mode != ProcMatchMode::Grep => options.echo = true,
"--logpidfile" => options.require_lock = true,
"--lightweight" | "--ns" | "--nslist" | "--cgroup" | "--env" => {
return Err((2, format!("unsupported option '{name}'")));
},
_ => return Err((2, format!("unrecognized option '{name}'"))),
}
index += 1;
continue;
}
if !options_done && arg.starts_with('-') && arg != "-" {
let chars: Vec<char> = arg[1..].chars().collect();
let mut short_index = 0;
while short_index < chars.len() {
let option = chars[short_index];
let takes_value =
matches!(
option,
'P' | 'g' | 's' | 'u' | 'U' | 'G' | 't' | 'F' | 'p' | 'O' | 'r' | 'd'
) || (option == 'q' && mode == ProcMatchMode::Kill && cfg!(target_os = "linux"));
let owned_value;
let value = if takes_value {
if short_index + 1 < chars.len() {
owned_value = chars[short_index + 1..].iter().collect::<String>();
short_index = chars.len();
owned_value.as_str()
} else {
index += 1;
argv
.get(index)
.map(String::as_str)
.ok_or_else(|| (2, format!("option '-{option}' requires an argument")))?
}
} else {
""
};
match option {
'f' => options.full = true,
'x' => options.exact = true,
'i' => options.ignore_case = true,
'v' if mode == ProcMatchMode::Kill && !cfg!(target_os = "macos") => {
return Err((2, "unrecognized option '-v'".to_string()));
},
'v' => options.invert = true,
'n' => options.newest = true,
'o' => options.oldest = true,
'P' => parse_i32_list(value, &mut options.parents)?,
'g' => parse_i32_list(value, &mut options.groups)?,
's' => parse_i32_list(value, &mut options.sessions)?,
'u' => parse_user_list(value, &mut options.effective_users)?,
'U' => parse_user_list(value, &mut options.real_users)?,
'G' => parse_group_list(value, &mut options.real_groups)?,
't' => parse_terminal_list(value, &mut options.terminals)?,
'F' => options.pid_files.push(value.to_string()),
'L' => options.require_lock = true,
'p' => {
options.explicit_pid = true;
parse_i32_list(value, &mut options.pids)?;
},
'O' => {
options.older = Some(Duration::from_secs(
value.parse().map_err(|_| (2, "invalid age".to_string()))?,
));
},
'r' => parse_states(value, &mut options.states)?,
'a' => {
if cfg!(target_os = "macos") {
options.include_ancestors = true;
} else {
options.list_full = true;
}
},
'A' => options.ignore_ancestors = true,
'c' => options.count = true,
'l' if mode == ProcMatchMode::Kill && cfg!(target_os = "macos") => {
options.echo_command = true;
},
'l' => options.list_name = true,
'q' if mode == ProcMatchMode::Kill && cfg!(target_os = "linux") => {
options.queue = Some(
value
.parse()
.map_err(|_| (2, "invalid queue value".to_string()))?,
);
},
'q' if mode == ProcMatchMode::Grep && cfg!(target_os = "macos") => {
options.quiet = true;
},
'q' => return Err((2, "unrecognized option '-q'".to_string())),
'd' => options.delimiter = value.to_string(),
'e' if mode != ProcMatchMode::Grep => options.echo = true,
'I' if mode == ProcMatchMode::Kill && cfg!(target_os = "macos") => {
options.interactive = true;
},
'I' if mode == ProcMatchMode::Kill => {
return Err((2, "unrecognized option '-I'".to_string()));
},
'w' | 'H' => {
return Err((2, format!("unsupported option '-{option}'")));
},
_ => return Err((2, format!("unrecognized option '-{option}'"))),
}
short_index += 1;
}
index += 1;
continue;
}
options.patterns.push(arg.clone());
index += 1;
}
#[cfg(target_os = "windows")]
if !options.groups.is_empty()
|| !options.sessions.is_empty()
|| !options.effective_users.is_empty()
|| !options.real_users.is_empty()
|| !options.real_groups.is_empty()
|| !options.terminals.is_empty()
{
return Err((2, "selected process metadata is unavailable on Windows".to_string()));
}
if options.explicit_pid && !options.pid_files.is_empty() {
return Err((2, "-F and -p cannot be combined".to_string()));
}
if options.require_lock && options.pid_files.is_empty() {
return Err((2, "-L requires -F".to_string()));
}
for file in &options.pid_files {
let contents = if file == "-" {
if options.require_lock {
return Err((2, "-L cannot be used with '-F -'".to_string()));
}
let mut contents = String::new();
stdin
.read_to_string(&mut contents)
.map_err(|err| (3, format!("cannot read pidfile from standard input: {err}")))?;
contents
} else {
let path = resolve_shell_path(cwd, file);
let mut pidfile = fs::File::open(&path)
.map_err(|err| (3, format!("cannot read pidfile '{}': {err}", path.display())))?;
if options.require_lock
&& !pidfile_is_locked(&pidfile)
.map_err(|err| (3, format!("cannot inspect pidfile '{}': {err}", path.display())))?
{
return Err((3, format!("pidfile '{}' is not locked", path.display())));
}
let mut contents = String::new();
io::Read::read_to_string(&mut pidfile, &mut contents)
.map_err(|err| (3, format!("cannot read pidfile '{}': {err}", path.display())))?;
contents
};
let pid = contents
.split_whitespace()
.next()
.and_then(|value| value.parse::<i32>().ok())
.filter(|pid| *pid > 0)
.ok_or_else(|| (3, format!("invalid pidfile '{file}'")))?;
options.pids.push(pid);
}
if !cfg!(target_os = "macos") && options.patterns.len() > 1 {
return Err((2, "only one pattern can be provided".to_string()));
}
if options.patterns.is_empty() && !has_proc_selectors(&options) {
return Err((2, "no matching criteria specified".to_string()));
}
if options.invert && (options.newest || options.oldest) {
return Err((2, "-v cannot be combined with -n or -o".to_string()));
}
if options.newest && options.oldest {
return Err((2, "-n and -o are mutually exclusive".to_string()));
}
if mode != ProcMatchMode::Grep
&& (options.list_name || options.list_full || options.delimiter != "\n")
{
return Err((2, "unsupported output-format option for this command".to_string()));
}
Ok(ParseProcResult::Options(Box::new(options)))
}
fn has_proc_selectors(options: &ProcMatchOptions) -> bool {
!options.parents.is_empty()
|| !options.groups.is_empty()
|| !options.sessions.is_empty()
|| !options.effective_users.is_empty()
|| !options.real_users.is_empty()
|| !options.real_groups.is_empty()
|| !options.terminals.is_empty()
|| !options.pids.is_empty()
|| options.older.is_some()
|| !options.states.is_empty()
}
fn select_processes(
options: &mut ProcMatchOptions,
) -> std::result::Result<Vec<proc_snapshot::ProcInfo>, String> {
let all = proc_snapshot::ProcInfo::all();
let host_pid = std::process::id() as i32;
let host_group = all
.iter()
.find(|process| process.pid() == host_pid)
.and_then(proc_snapshot::ProcInfo::group_id);
let host_session = all
.iter()
.find(|process| process.pid() == host_pid)
.and_then(proc_snapshot::ProcInfo::session_id);
if let Some(host_group) = host_group {
for group in &mut options.groups {
if *group == 0 {
*group = host_group;
}
}
}
if let Some(host_session) = host_session {
for session in &mut options.sessions {
if *session == 0 {
*session = host_session;
}
}
}
let by_pid: HashMap<i32, Option<i32>> = all
.iter()
.map(|process| (process.pid(), process.ppid()))
.collect();
let exclude_ancestors = options.ignore_ancestors
|| (cfg!(target_os = "macos") && !options.include_ancestors && !options.invert);
let mut forbidden = HashSet::from([host_pid]);
if exclude_ancestors {
let mut current = by_pid.get(&host_pid).copied().flatten();
while let Some(pid) = current {
if !forbidden.insert(pid) {
break;
}
current = by_pid.get(&pid).copied().flatten();
}
}
let regex = if options.patterns.is_empty() {
None
} else {
let source = options
.patterns
.iter()
.map(|pattern| {
if options.exact {
format!("^(?:{pattern})$")
} else {
format!("(?:{pattern})")
}
})
.collect::<Vec<_>>()
.join("|");
Some(
regex::RegexBuilder::new(&source)
.case_insensitive(options.ignore_case)
.build()
.map_err(|err| format!("invalid regular expression: {err}"))?,
)
};
let mut selected = Vec::new();
for process in all {
if forbidden.contains(&process.pid()) {
continue;
}
let pattern_matches = regex.as_ref().is_none_or(|regex| {
let subject = if options.full {
process.args().join(" ")
} else {
process.match_name()
};
regex.is_match(&subject)
});
let selectors_match = (options.parents.is_empty()
|| process
.ppid()
.is_some_and(|value| options.parents.contains(&value)))
&& (options.groups.is_empty()
|| process
.group_id()
.is_some_and(|value| options.groups.contains(&value)))
&& (options.sessions.is_empty()
|| process
.session_id()
.is_some_and(|value| options.sessions.contains(&value)))
&& (options.effective_users.is_empty()
|| process
.effective_user_id()
.is_some_and(|value| options.effective_users.contains(&value)))
&& (options.real_users.is_empty()
|| process
.real_user_id()
.is_some_and(|value| options.real_users.contains(&value)))
&& (options.real_groups.is_empty()
|| process
.real_group_id()
.is_some_and(|value| options.real_groups.contains(&value)))
&& (options.terminals.is_empty() || options.terminals.contains(&process.terminal_id()))
&& (options.pids.is_empty() || options.pids.contains(&process.pid()))
&& options
.older
.is_none_or(|age| process.age().is_some_and(|process_age| process_age >= age))
&& (options.states.is_empty() || options.states.contains(&process.state()));
let matches = pattern_matches && selectors_match;
if matches != options.invert {
selected.push(process);
}
}
selected.sort_by_key(|process| (process.start_time(), process.pid()));
if options.newest {
selected = selected.into_iter().next_back().into_iter().collect();
} else if options.oldest {
selected.truncate(1);
}
Ok(selected)
}
fn parse_i32_list(value: &str, target: &mut Vec<i32>) -> std::result::Result<(), (u8, String)> {
for item in value.split(',') {
let parsed = item
.parse::<i32>()
.map_err(|_| (2, format!("invalid numeric selector '{item}'")))?;
target.push(parsed);
}
Ok(())
}
fn parse_user_list(value: &str, target: &mut Vec<u32>) -> std::result::Result<(), (u8, String)> {
for item in value.split(',') {
target.push(resolve_user(item).ok_or_else(|| (2, format!("unknown user '{item}'")))?);
}
Ok(())
}
fn parse_group_list(value: &str, target: &mut Vec<u32>) -> std::result::Result<(), (u8, String)> {
for item in value.split(',') {
target.push(resolve_group(item).ok_or_else(|| (2, format!("unknown group '{item}'")))?);
}
Ok(())
}
#[cfg(unix)]
fn resolve_user(value: &str) -> Option<u32> {
use std::ffi::CString;
if let Ok(id) = value.parse() {
return Some(id);
}
let name = CString::new(value).ok()?;
let mut record = std::mem::MaybeUninit::<libc::passwd>::zeroed();
let mut result = std::ptr::null_mut();
let mut buffer = vec![0u8; 16 * 1024];
// SAFETY: all pointers refer to live, writable storage for this call.
let status = unsafe {
libc::getpwnam_r(
name.as_ptr(),
record.as_mut_ptr(),
buffer.as_mut_ptr().cast(),
buffer.len(),
&raw mut result,
)
};
if status != 0 || result.is_null() {
return None;
}
// SAFETY: a successful getpwnam_r call initialized `record`.
Some(unsafe { record.assume_init() }.pw_uid)
}
#[cfg(not(unix))]
fn resolve_user(value: &str) -> Option<u32> {
value.parse().ok()
}
#[cfg(unix)]
fn resolve_group(value: &str) -> Option<u32> {
use std::ffi::CString;
if let Ok(id) = value.parse() {
return Some(id);
}
let name = CString::new(value).ok()?;
let mut record = std::mem::MaybeUninit::<libc::group>::zeroed();
let mut result = std::ptr::null_mut();
let mut buffer = vec![0u8; 16 * 1024];
// SAFETY: all pointers refer to live, writable storage for this call.
let status = unsafe {
libc::getgrnam_r(
name.as_ptr(),
record.as_mut_ptr(),
buffer.as_mut_ptr().cast(),
buffer.len(),
&raw mut result,
)
};
if status != 0 || result.is_null() {
return None;
}
// SAFETY: a successful getgrnam_r call initialized `record`.
Some(unsafe { record.assume_init() }.gr_gid)
}
#[cfg(not(unix))]
fn resolve_group(value: &str) -> Option<u32> {
value.parse().ok()
}
fn parse_terminal_list(
value: &str,
target: &mut Vec<Option<u64>>,
) -> std::result::Result<(), (u8, String)> {
for item in value.split(',') {
if matches!(item, "?" | "-") {
target.push(None);
} else if let Some(id) = resolve_terminal(item) {
target.push(Some(id));
} else if let Ok(id) = item.parse() {
target.push(Some(id));
} else {
return Err((2, format!("unknown terminal '{item}'")));
}
}
Ok(())
}
#[cfg(unix)]
fn resolve_terminal(value: &str) -> Option<u64> {
use std::os::unix::fs::MetadataExt;
let primary = if value.starts_with('/') {
PathBuf::from(value)
} else {
Path::new("/dev").join(value)
};
fs::metadata(&primary)
.or_else(|_| fs::metadata(Path::new("/dev").join(format!("tty{value}"))))
.ok()
.map(|metadata| metadata.rdev())
}
#[cfg(not(unix))]
fn resolve_terminal(_value: &str) -> Option<u64> {
None
}
fn parse_states(value: &str, target: &mut HashSet<char>) -> std::result::Result<(), (u8, String)> {
for state in value.split(',').flat_map(str::chars) {
if !state.is_ascii_alphabetic() {
return Err((2, format!("invalid process state '{state}'")));
}
target.insert(state.to_ascii_uppercase());
}
Ok(())
}
fn resolve_shell_path(cwd: &Path, value: &str) -> PathBuf {
let path = Path::new(value);
if path.is_absolute() {
path.to_path_buf()
} else {
cwd.join(path)
}
}
#[cfg(unix)]
fn pidfile_is_locked(file: &fs::File) -> io::Result<bool> {
use std::os::fd::AsRawFd;
let mut lock = libc::flock {
l_type: libc::F_WRLCK as libc::c_short,
l_whence: libc::SEEK_SET as libc::c_short,
l_start: 0,
l_len: 0,
l_pid: 0,
};
// SAFETY: `file` owns a valid fd and `lock` is writable for F_GETLK.
if unsafe { libc::fcntl(file.as_raw_fd(), libc::F_GETLK, &raw mut lock) } == -1 {
return Err(io::Error::last_os_error());
}
Ok(lock.l_type != libc::F_UNLCK as libc::c_short)
}
#[cfg(not(unix))]
fn pidfile_is_locked(_file: &fs::File) -> io::Result<bool> {
Err(io::Error::new(
io::ErrorKind::Unsupported,
"pidfile lock validation is unavailable on this platform",
))
}
fn write_proc_match_help(
mut output: impl Write,
name: &str,
mode: ProcMatchMode,
) -> io::Result<()> {
let action = match mode {
ProcMatchMode::Grep => "print matching process IDs",
ProcMatchMode::Kill => "signal matching processes",
ProcMatchMode::Wait => "wait for matching processes",
};
writeln!(output, "Usage: {name} [options] [pattern ...]")?;
writeln!(output, "{action}")?;
writeln!(
output,
" -f full command -x exact -i ignore case -v invert -n newest -o oldest"
)?;
#[cfg(not(target_os = "windows"))]
writeln!(
output,
" -P ppid -g pgrp -s sid -u euid -U uid -G gid -t tty -p pid -F pidfile"
)?;
#[cfg(target_os = "windows")]
writeln!(output, " -P ppid -p pid -F pidfile -O seconds -r states")?;
if mode == ProcMatchMode::Kill {
writeln!(output, " -SIGNAL, --signal SIGNAL choose signal (default TERM)")?;
}
#[cfg(target_os = "linux")]
if mode == ProcMatchMode::Kill {
writeln!(output, " -q value, --queue value send an integer with sigqueue")?;
}
#[cfg(target_os = "macos")]
if mode == ProcMatchMode::Grep {
writeln!(output, " -q suppress output")?;
}
Ok(())
}
crates/pi-builtins/src/proc_snapshot.rs
+1084
View File
File diff suppressed because it is too large Load Diff
crates/pi-builtins/src/ps.rs
+1486
View File
File diff suppressed because it is too large Load Diff
crates/vendor/brush-builtins/src/pushd.rs → crates/pi-builtins/src/pushd.rs
View File
crates/vendor/brush-builtins/src/pwd.rs → crates/pi-builtins/src/pwd.rs
View File
crates/vendor/brush-builtins/src/read.rs → crates/pi-builtins/src/read.rs
View File
crates/pi-builtins/src/readlink.rs
+346
View File
@@ -0,0 +1,346 @@
//! `readlink` builtin: print a symbolic link's value or a canonical file name.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
ffi::OsString,
fs,
io::Write,
path::{Path, PathBuf},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use uucore::{
display::Quotable,
fs::{MissingHandling, ResolveMode, canonicalize},
libc::EINVAL,
line_ending::LineEnding,
};
use crate::host::{Host, Utility, format_usage, matches_parser, os_bytes, util};
const OPT_CANONICALIZE: &str = "canonicalize";
const OPT_CANONICALIZE_MISSING: &str = "canonicalize-missing";
const OPT_CANONICALIZE_EXISTING: &str = "canonicalize-existing";
const OPT_NO_NEWLINE: &str = "no-newline";
const OPT_QUIET: &str = "quiet";
const OPT_SILENT: &str = "silent";
const OPT_VERBOSE: &str = "verbose";
const OPT_ZERO: &str = "zero";
const ARG_FILES: &str = "files";
/// Parsed `readlink` invocation.
pub(crate) struct Readlink {
matches: ArgMatches,
}
matches_parser!(Readlink, app);
impl Utility for Readlink {
const NAME: &'static str = "readlink";
fn run(self, host: &mut Host) -> i32 {
let mut no_trailing_delimiter = self.matches.get_flag(OPT_NO_NEWLINE);
let use_zero = self.matches.get_flag(OPT_ZERO);
let verbose = self.matches.get_flag(OPT_VERBOSE) || host.var("POSIXLY_CORRECT").is_some();
// GNU readlink -f/-e/-m follows symlinks first and then applies `..`
// (physical resolution). Logical mode would collapse `..` first.
let resolve_mode = if self.matches.get_flag(OPT_CANONICALIZE)
|| self.matches.get_flag(OPT_CANONICALIZE_EXISTING)
|| self.matches.get_flag(OPT_CANONICALIZE_MISSING)
{
ResolveMode::Physical
} else {
ResolveMode::None
};
let missing_handling = if self.matches.get_flag(OPT_CANONICALIZE_EXISTING) {
MissingHandling::Existing
} else if self.matches.get_flag(OPT_CANONICALIZE_MISSING) {
MissingHandling::Missing
} else {
MissingHandling::Normal
};
let files: Vec<PathBuf> = self
.matches
.get_many::<OsString>(ARG_FILES)
.map(|values| values.map(PathBuf::from).collect())
.unwrap_or_default();
if files.is_empty() {
host.error("missing operand", 1);
return 1;
}
if no_trailing_delimiter && files.len() > 1 {
let _ = writeln!(
host.stderr,
"readlink: ignoring --no-newline with multiple arguments"
);
no_trailing_delimiter = false;
}
let line_ending = if no_trailing_delimiter {
None
} else {
Some(LineEnding::from_zero_flag(use_zero))
};
for operand in &files {
let resolved = host.resolve(operand);
let path_result = if resolve_mode == ResolveMode::None {
fs::read_link(&resolved)
} else {
canonicalize(&resolved, missing_handling, resolve_mode)
};
match path_result {
Ok(path) => {
if show(&mut host.stdout, &path, line_ending).is_err() {
return 1;
}
},
Err(err) => {
if verbose {
let message = if err.raw_os_error() == Some(EINVAL) {
format!("{}: Invalid argument", operand.maybe_quote())
} else {
format!("{}: {err}", operand.maybe_quote())
};
let _ = writeln!(host.stderr, "readlink: {message}");
}
return 1;
},
}
}
0
}
}
/// The `readlink` argument model.
fn app() -> Command {
Command::new(Readlink::NAME)
.version("0.8.0")
.about("Print value of a symbolic link or canonical file name.")
.override_usage(format_usage("readlink [OPTION]... [FILE]..."))
.infer_long_args(true)
.arg(
Arg::new(OPT_CANONICALIZE)
.short('f')
.long(OPT_CANONICALIZE)
.help(
"canonicalize by following every symlink in every component of the given name \
recursively; all but the last component must exist",
)
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_CANONICALIZE_EXISTING)
.short('e')
.long(OPT_CANONICALIZE_EXISTING)
.help(
"canonicalize by following every symlink in every component of the given name \
recursively, all components must exist",
)
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_CANONICALIZE_MISSING)
.short('m')
.long(OPT_CANONICALIZE_MISSING)
.help(
"canonicalize by following every symlink in every component of the given name \
recursively, without requirements on components existence",
)
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_NO_NEWLINE)
.short('n')
.long(OPT_NO_NEWLINE)
.help("do not output the trailing delimiter")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_QUIET)
.short('q')
.long(OPT_QUIET)
.help("suppress most error messages")
.overrides_with_all([OPT_QUIET, OPT_SILENT, OPT_VERBOSE])
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_SILENT)
.short('s')
.long(OPT_SILENT)
.help("suppress most error messages")
.overrides_with_all([OPT_QUIET, OPT_SILENT, OPT_VERBOSE])
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_VERBOSE)
.short('v')
.long(OPT_VERBOSE)
.help("report error message")
.overrides_with_all([OPT_QUIET, OPT_SILENT, OPT_VERBOSE])
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_ZERO)
.short('z')
.long(OPT_ZERO)
.help("separate output with NUL rather than newline")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(ARG_FILES)
.action(ArgAction::Append)
.value_parser(clap::value_parser!(OsString))
.value_hint(clap::ValueHint::AnyPath),
)
}
/// Writes a resolved path verbatim, followed by the selected delimiter.
fn show(out: &mut impl Write, path: &Path, line_ending: Option<LineEnding>) -> std::io::Result<()> {
let bytes = os_bytes(path.as_os_str())
.ok_or_else(|| std::io::Error::new(std::io::ErrorKind::InvalidInput, "invalid path"))?;
out.write_all(bytes)?;
if let Some(line_ending) = line_ending {
write!(out, "{line_ending}")?;
}
out.flush()
}
/// Creates the `readlink` builtin registration.
pub(crate) fn readlink_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Readlink, SE>()
}
#[cfg(test)]
mod tests {
use std::{fs, path::PathBuf};
use super::Readlink;
use crate::host::run_util;
fn run_in(cwd: PathBuf, args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Readlink>(args, "", cwd);
(code, capture.out(), capture.err())
}
/// Canonicalized temp dir (macOS tempdirs live behind /var -> /private/var,
/// which -f/-e/-m resolution would otherwise expand mid-assertion).
fn canonical_tempdir() -> (tempfile::TempDir, PathBuf) {
let dir = tempfile::tempdir().unwrap();
let canon = fs::canonicalize(dir.path()).unwrap();
(dir, canon)
}
#[cfg(unix)]
#[test]
fn resolves_relative_operand_against_host_cwd() {
let (_dir, root) = canonical_tempdir();
std::os::unix::fs::symlink("target-file", root.join("link")).unwrap();
let (code, stdout, stderr) = run_in(root, &["link"]);
assert_eq!(code, 0);
assert_eq!(stdout, "target-file\n");
assert_eq!(stderr, "");
}
#[cfg(unix)]
#[test]
fn canonicalize_follows_symlink_to_absolute_path() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("target"), b"x").unwrap();
std::os::unix::fs::symlink("target", root.join("link")).unwrap();
let (code, stdout, stderr) = run_in(root.clone(), &["-f", "link"]);
assert_eq!(code, 0);
assert_eq!(stdout, format!("{}\n", root.join("target").display()));
assert_eq!(stderr, "");
}
#[test]
fn canonicalize_missing_builds_path_from_host_cwd() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root.clone(), &["-m", "missing/sub"]);
assert_eq!(code, 0);
assert_eq!(stdout, format!("{}\n", root.join("missing/sub").display()));
assert_eq!(stderr, "");
}
#[cfg(unix)]
#[test]
fn canonicalize_existing_fails_silently_on_missing_final_component() {
let (_dir, root) = canonical_tempdir();
let (code, stdout, stderr) = run_in(root, &["-e", "missing"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "");
}
#[cfg(unix)]
#[test]
fn non_symlink_is_silent_failure_by_default_and_einval_with_verbose() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("plain"), b"x").unwrap();
let (code, stdout, stderr) = run_in(root.clone(), &["plain"]);
assert_eq!((code, stdout.as_str(), stderr.as_str()), (1, "", ""));
let (code, stdout, stderr) = run_in(root, &["-v", "plain"]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "readlink: plain: Invalid argument\n");
}
#[cfg(unix)]
#[test]
fn no_newline_with_multiple_args_warns_and_keeps_delimiter() {
let (_dir, root) = canonical_tempdir();
std::os::unix::fs::symlink("a", root.join("l1")).unwrap();
std::os::unix::fs::symlink("b", root.join("l2")).unwrap();
let (code, stdout, stderr) = run_in(root, &["-n", "l1", "l2"]);
assert_eq!(code, 0);
assert_eq!(stdout, "a\nb\n");
assert_eq!(stderr, "readlink: ignoring --no-newline with multiple arguments\n");
}
#[cfg(unix)]
#[test]
fn zero_terminates_with_nul_and_no_newline_drops_delimiter() {
let (_dir, root) = canonical_tempdir();
std::os::unix::fs::symlink("a", root.join("l1")).unwrap();
let (code, stdout, _) = run_in(root.clone(), &["-z", "l1"]);
assert_eq!((code, stdout.as_str()), (0, "a\0"));
let (code, stdout, _) = run_in(root, &["-n", "l1"]);
assert_eq!((code, stdout.as_str()), (0, "a"));
}
#[test]
fn missing_operand_is_usage_error() {
let (code, stdout, stderr) = run_in(PathBuf::from("."), &[]);
assert_eq!(code, 1);
assert_eq!(stdout, "");
assert_eq!(stderr, "readlink: missing operand\n");
}
#[test]
fn help_renders_to_stdout() {
let (code, stdout, stderr) = run_in(PathBuf::from("."), &["--help"]);
assert_eq!(code, 0);
assert!(stdout.contains("Usage:"));
assert!(stdout.contains("canonical file name"));
assert_eq!(stderr, "");
}
}
crates/pi-builtins/src/realpath.rs
+466
View File
@@ -0,0 +1,466 @@
//! `realpath` builtin: print resolved absolute or relative paths.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
ffi::{OsStr, OsString},
io::{self, Write},
path::{Path, PathBuf},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{
Arg, ArgAction, ArgMatches, Command,
builder::{TypedValueParser, ValueParserFactory},
};
use uucore::{
display::Quotable,
fs::{MissingHandling, ResolveMode, canonicalize, make_path_relative_to},
line_ending::LineEnding,
};
use crate::host::{Host, Utility, format_usage, matches_parser, os_bytes, util};
const OPT_QUIET: &str = "quiet";
const OPT_STRIP: &str = "strip";
const OPT_ZERO: &str = "zero";
const OPT_PHYSICAL: &str = "physical";
const OPT_LOGICAL: &str = "logical";
const OPT_CANONICALIZE_MISSING: &str = "canonicalize-missing";
const OPT_CANONICALIZE: &str = "canonicalize";
const OPT_CANONICALIZE_EXISTING: &str = "canonicalize-existing";
const OPT_RELATIVE_TO: &str = "relative-to";
const OPT_RELATIVE_BASE: &str = "relative-base";
const ARG_FILES: &str = "files";
/// Custom parser that validates `OsString` is not empty.
#[derive(Clone, Debug)]
struct NonEmptyOsStringParser;
impl TypedValueParser for NonEmptyOsStringParser {
type Value = OsString;
fn parse_ref(
&self,
_cmd: &Command,
_arg: Option<&Arg>,
value: &OsStr,
) -> Result<Self::Value, clap::Error> {
if value.is_empty() {
let mut err = clap::Error::new(clap::error::ErrorKind::ValueValidation);
err.insert(
clap::error::ContextKind::Custom,
clap::error::ContextValue::String("invalid operand: empty string".to_string()),
);
return Err(err);
}
Ok(value.to_os_string())
}
}
impl ValueParserFactory for NonEmptyOsStringParser {
type Parser = Self;
fn value_parser() -> Self::Parser {
Self
}
}
/// Parsed `realpath` invocation.
pub(crate) struct Realpath {
matches: ArgMatches,
}
matches_parser!(Realpath, app);
impl Utility for Realpath {
const NAME: &'static str = "realpath";
fn run(self, host: &mut Host) -> i32 {
realpath_main(&self.matches, host)
}
}
fn realpath_main(matches: &ArgMatches, host: &mut Host) -> i32 {
let paths: Vec<PathBuf> = matches
.get_many::<OsString>(ARG_FILES)
.expect("required by clap")
.map(PathBuf::from)
.collect();
let strip = matches.get_flag(OPT_STRIP);
let line_ending = LineEnding::from_zero_flag(matches.get_flag(OPT_ZERO));
let quiet = matches.get_flag(OPT_QUIET);
let logical = matches.get_flag(OPT_LOGICAL);
let can_mode = if matches.get_flag(OPT_CANONICALIZE_MISSING) {
MissingHandling::Missing
} else if matches.get_flag(OPT_CANONICALIZE_EXISTING) {
// -e: all components must exist. Despite the name,
// MissingHandling::Existing requires all components to exist.
MissingHandling::Existing
} else {
// Default behavior (same as -E): all but the last component must exist.
// MissingHandling::Normal allows the final component to not exist.
MissingHandling::Normal
};
let resolve_mode = if strip {
ResolveMode::None
} else if logical {
ResolveMode::Logical
} else {
ResolveMode::Physical
};
let (relative_to, relative_base) =
match prepare_relative_options(matches, host, can_mode, resolve_mode) {
Ok(options) => options,
Err((path, err)) => {
host.error(format!("{}: {}", path.maybe_quote(), io_error_message(&err)), 1);
return host.exit_code();
},
};
for path in &paths {
if let Err(err) = resolve_path(
path,
line_ending,
resolve_mode,
can_mode,
relative_to.as_deref(),
relative_base.as_deref(),
host,
) {
if !quiet {
host.error(format!("{}: {}", path.maybe_quote(), io_error_message(&err)), 1);
} else {
host.fail(1);
}
}
}
host.exit_code()
}
fn app() -> Command {
Command::new("realpath")
.version("0.8.0")
.about("Print the resolved path")
.override_usage(format_usage("realpath [OPTION]... FILE..."))
.infer_long_args(true)
.arg(
Arg::new(OPT_QUIET)
.short('q')
.long(OPT_QUIET)
.help("Do not print warnings for invalid paths")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_STRIP)
.short('s')
.long(OPT_STRIP)
.visible_alias("no-symlinks")
.help("Only strip '.' and '..' components, but don't resolve symbolic links")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_ZERO)
.short('z')
.long(OPT_ZERO)
.help("Separate output filenames with \\0 rather than newline")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_LOGICAL)
.short('L')
.long(OPT_LOGICAL)
.help("resolve '..' components before symlinks")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_PHYSICAL)
.short('P')
.long(OPT_PHYSICAL)
.overrides_with_all([OPT_STRIP, OPT_LOGICAL])
.help("resolve symlinks as encountered (default)")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_CANONICALIZE)
.short('E')
.long(OPT_CANONICALIZE)
.overrides_with_all([OPT_CANONICALIZE_EXISTING, OPT_CANONICALIZE_MISSING])
.help("all but the last component must exist (default)")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_CANONICALIZE_EXISTING)
.short('e')
.long(OPT_CANONICALIZE_EXISTING)
.overrides_with_all([OPT_CANONICALIZE, OPT_CANONICALIZE_MISSING])
.help(
"canonicalize by following every symlink in every component of the given name \
recursively, all components must exist",
)
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_CANONICALIZE_MISSING)
.short('m')
.long(OPT_CANONICALIZE_MISSING)
.overrides_with_all([OPT_CANONICALIZE, OPT_CANONICALIZE_EXISTING])
.help(
"canonicalize by following every symlink in every component of the given name \
recursively, without requirements on components existence",
)
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_RELATIVE_TO)
.long(OPT_RELATIVE_TO)
.value_name("DIR")
.value_parser(NonEmptyOsStringParser)
.help("print the resolved path relative to DIR"),
)
.arg(
Arg::new(OPT_RELATIVE_BASE)
.long(OPT_RELATIVE_BASE)
.value_name("DIR")
.value_parser(NonEmptyOsStringParser)
.help("print absolute paths unless paths below DIR"),
)
.arg(
Arg::new(ARG_FILES)
.action(ArgAction::Append)
.required(true)
.value_parser(NonEmptyOsStringParser)
.value_hint(clap::ValueHint::AnyPath),
)
}
type PathIoError = (PathBuf, io::Error);
/// Prepare `--relative-to` and `--relative-base` options, converting them to
/// absolute paths. If `--relative-to` is not a descendant of `--relative-base`,
/// both values are discarded.
fn prepare_relative_options(
matches: &ArgMatches,
host: &Host,
can_mode: MissingHandling,
resolve_mode: ResolveMode,
) -> Result<(Option<PathBuf>, Option<PathBuf>), PathIoError> {
let relative_to = matches
.get_one::<OsString>(OPT_RELATIVE_TO)
.map(PathBuf::from);
let relative_base = matches
.get_one::<OsString>(OPT_RELATIVE_BASE)
.map(PathBuf::from);
let relative_to = canonicalize_relative_option(relative_to, host, can_mode, resolve_mode)?;
let relative_base = canonicalize_relative_option(relative_base, host, can_mode, resolve_mode)?;
if let (Some(base), Some(to)) = (relative_base.as_deref(), relative_to.as_deref())
&& !to.starts_with(base)
{
return Ok((None, None));
}
Ok((relative_to, relative_base))
}
/// Prepare one `relative-*` option.
fn canonicalize_relative_option(
relative: Option<PathBuf>,
host: &Host,
can_mode: MissingHandling,
resolve_mode: ResolveMode,
) -> Result<Option<PathBuf>, PathIoError> {
match relative {
None => Ok(None),
Some(path) => canonicalize_relative(&host.resolve(&path), can_mode, resolve_mode)
.map(Some)
.map_err(|err| (path, err)),
}
}
/// Make a `relative-to` or `relative-base` path value absolute.
fn canonicalize_relative(
path: &Path,
can_mode: MissingHandling,
resolve: ResolveMode,
) -> io::Result<PathBuf> {
let absolute = canonicalize(path, can_mode, resolve)?;
if can_mode == MissingHandling::Existing && !absolute.is_dir() {
absolute.read_dir()?; // Raise a not-a-directory error.
}
Ok(absolute)
}
/// Resolve a path to an absolute form and print it.
///
/// If `relative_to` and/or `relative_base` is given, the path is printed in a
/// relative form according to [`process_relative`].
fn resolve_path(
path: &Path,
line_ending: LineEnding,
resolve: ResolveMode,
can_mode: MissingHandling,
relative_to: Option<&Path>,
relative_base: Option<&Path>,
host: &mut Host,
) -> io::Result<()> {
let absolute = canonicalize(host.resolve(path), can_mode, resolve)?;
let output_path = process_relative(absolute, relative_base, relative_to);
let bytes = os_bytes(output_path.as_os_str())
.ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "path is not valid Unicode"))?;
host.stdout.write_all(bytes)?;
host.stdout.write_all(&[line_ending.into()])?;
host.stdout.flush()
}
/// Conditionally converts an absolute path to a relative form.
///
/// With only `relative_to`, the result is relative to it. With only
/// `relative_base`, descendants are relative to it. With both, descendants of
/// `relative_base` are relative to `relative_to`; other paths remain absolute.
fn process_relative(
path: PathBuf,
relative_base: Option<&Path>,
relative_to: Option<&Path>,
) -> PathBuf {
if let Some(base) = relative_base {
if path.starts_with(base) {
make_path_relative_to(path, relative_to.unwrap_or(base))
} else {
path
}
} else if let Some(to) = relative_to {
make_path_relative_to(path, to)
} else {
path
}
}
fn io_error_message(error: &io::Error) -> String {
let mut message = error.to_string();
if let Some(index) = message.find(" (os error ") {
message.truncate(index);
}
message
}
/// Creates the `realpath` builtin registration.
pub(crate) fn realpath_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Realpath, SE>()
}
#[cfg(test)]
mod tests {
use std::{fs, path::PathBuf};
use super::Realpath;
use crate::host::run_util;
/// Canonicalized temp dir (macOS tempdirs live behind /var -> /private/var,
/// which canonicalization would otherwise expand mid-assertion).
fn canonical_tempdir() -> (tempfile::TempDir, PathBuf) {
let dir = tempfile::tempdir().unwrap();
let canonical = fs::canonicalize(dir.path()).unwrap();
(dir, canonical)
}
#[cfg(unix)]
#[test]
fn resolves_relative_operand_against_host_cwd() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("target"), b"x").unwrap();
std::os::unix::fs::symlink("target", root.join("link")).unwrap();
let (code, capture) = run_util::<Realpath>(&["link"], "", root.clone());
assert_eq!(code, 0);
assert_eq!(capture.out(), format!("{}\n", root.join("target").display()));
assert_eq!(capture.err(), "");
}
#[test]
fn canonicalize_missing_builds_path_from_host_cwd() {
let (_dir, root) = canonical_tempdir();
let (code, capture) = run_util::<Realpath>(&["-m", "missing/sub"], "", root.clone());
assert_eq!(code, 0);
assert_eq!(capture.out(), format!("{}\n", root.join("missing/sub").display()));
assert_eq!(capture.err(), "");
}
#[test]
fn relative_to_option_resolves_against_host_cwd_and_relativizes_output() {
let (_dir, root) = canonical_tempdir();
fs::create_dir(root.join("sub")).unwrap();
fs::write(root.join("sub/file"), b"x").unwrap();
let (code, capture) =
run_util::<Realpath>(&["--relative-to", "sub", "sub/file"], "", root);
assert_eq!(code, 0);
assert_eq!(capture.out(), "file\n");
assert_eq!(capture.err(), "");
}
#[test]
fn zero_flag_terminates_with_nul() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("f"), b"x").unwrap();
let (code, capture) = run_util::<Realpath>(&["-z", "f"], "", root.clone());
assert_eq!(code, 0);
assert_eq!(capture.stdout(), format!("{}\0", root.join("f").display()).as_bytes());
assert_eq!(capture.err(), "");
}
#[test]
fn nonexistent_operand_errors_but_later_operands_still_process() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("f"), b"x").unwrap();
let (code, capture) = run_util::<Realpath>(&["missing/x", "f"], "", root.clone());
assert_eq!(code, 1);
assert_eq!(capture.out(), format!("{}\n", root.join("f").display()));
assert!(capture.err().contains("realpath: missing/x"), "stderr: {}", capture.err());
assert!(capture.err().contains("No such file"), "stderr: {}", capture.err());
}
#[test]
fn quiet_suppresses_error_messages_but_preserves_failure_status() {
let (_dir, root) = canonical_tempdir();
let (code, capture) = run_util::<Realpath>(&["-q", "missing/x"], "", root);
assert_eq!(code, 1);
assert_eq!(capture.out(), "");
assert_eq!(capture.err(), "");
}
#[cfg(unix)]
#[test]
fn strip_keeps_symlinks_unresolved() {
let (_dir, root) = canonical_tempdir();
fs::write(root.join("target"), b"x").unwrap();
std::os::unix::fs::symlink("target", root.join("link")).unwrap();
let (code, capture) = run_util::<Realpath>(&["-s", "link"], "", root.clone());
assert_eq!(code, 0);
assert_eq!(capture.out(), format!("{}\n", root.join("link").display()));
assert_eq!(capture.err(), "");
}
#[test]
fn empty_operand_is_rejected() {
let (code, capture) = run_util::<Realpath>(&[""], "", ".");
assert_eq!(code, 1);
assert_eq!(capture.out(), "");
assert!(capture.err().contains("invalid value"), "stderr: {}", capture.err());
}
#[test]
fn help_renders_to_host_stdout() {
let (code, capture) = run_util::<Realpath>(&["--help"], "", ".");
assert_eq!(code, 0);
assert!(capture.out().contains("Usage:"));
assert!(capture.out().contains("Print the resolved path"));
assert_eq!(capture.err(), "");
}
}
crates/vendor/brush-builtins/src/return_.rs → crates/pi-builtins/src/return_.rs
View File
crates/pi-uu-grep/src/rg.rs → crates/pi-builtins/src/rg.rs
+199 -329
View File
@@ -1,3 +1,10 @@
//! `rg` builtin: ripgrep-compatible search, with ripgrep defaults — recursive
//! directory search, ignore/hidden filtering, and binary-file suppression.
//!
//! Shares the PCRE2 JIT probe with the `grep` builtin (`crate::grep`); the two
//! commands otherwise have separate argument models and output formats, which is
//! why they are separate modules rather than one with a mode flag.
//! `rg` implemented as an in-process shell builtin on top of the ripgrep
//! libraries, with ripgrep defaults: recursive directory search, ignore/hidden
//! filtering, and binary-file suppression.
@@ -9,6 +16,7 @@ use std::{
path::{Path, PathBuf},
};
use brush_core::openfiles::OpenFile;
use clap::{ArgAction, Parser, ValueEnum};
use grep_cli::DecompressionReaderBuilder;
use grep_matcher::{Captures, LineTerminator, Matcher};
@@ -18,6 +26,8 @@ use grep_regex::{RegexMatcher, RegexMatcherBuilder};
use grep_searcher::{
BinaryDetection, Encoding, Searcher, SearcherBuilder, Sink, SinkContext, SinkFinish, SinkMatch,
};
use crate::host::{Host, Utility};
use ignore::{
Match,
gitignore::{Gitignore, GitignoreBuilder},
@@ -33,7 +43,7 @@ use ignore::{
about = "ripgrep recursively searches the current directory for lines matching a regex pattern.",
args_override_self = true
)]
struct RgCli {
pub(crate) struct Rg {
/// A pattern to search for. May be repeated.
#[arg(short = 'e', long = "regexp", value_name = "PATTERN")]
patterns: Vec<String>,
@@ -472,8 +482,8 @@ enum CompiledMatcher {
}
enum RgOutput {
Buffered(BufWriter<pi_uutils_ctx::CtxStdout>),
Direct(pi_uutils_ctx::CtxStdout),
Buffered(BufWriter<OpenFile>),
Direct(OpenFile),
}
impl Write for RgOutput {
@@ -829,7 +839,7 @@ fn count_matches<M: Matcher>(matcher: &M, line: &[u8]) -> io::Result<u64> {
Ok(count)
}
fn build_rust_matcher(patterns: &[String], cli: &RgCli) -> Result<RegexMatcher, grep_regex::Error> {
fn build_rust_matcher(patterns: &[String], cli: &Rg) -> Result<RegexMatcher, grep_regex::Error> {
let crlf = cli.crlf && !cli.no_crlf && !cli.null_data;
let mut builder = RegexMatcherBuilder::new();
builder
@@ -850,7 +860,7 @@ fn build_rust_matcher(patterns: &[String], cli: &RgCli) -> Result<RegexMatcher,
builder.build_many(patterns)
}
fn build_pcre_matcher(patterns: &[String], cli: &RgCli) -> Result<PcreMatcher, String> {
fn build_pcre_matcher(host: &Host, patterns: &[String], cli: &Rg) -> Result<PcreMatcher, String> {
let unicode = !cli.no_unicode;
let mut builder = PcreMatcherBuilder::new();
builder
@@ -864,13 +874,13 @@ fn build_pcre_matcher(patterns: &[String], cli: &RgCli) -> Result<PcreMatcher, S
.crlf(cli.crlf && !cli.no_crlf && !cli.null_data)
.utf(unicode)
.ucp(unicode)
.jit_if_available(*crate::PCRE2_JIT_ENABLED);
.jit_if_available(crate::grep::pcre2_jit_enabled(host));
builder
.build_many(patterns)
.map_err(|error| error.to_string())
}
fn build_matcher(patterns: &[String], cli: &RgCli) -> Result<CompiledMatcher, String> {
fn build_matcher(host: &Host, patterns: &[String], cli: &Rg) -> Result<CompiledMatcher, String> {
let engine = cli.engine.unwrap_or(if cli.pcre2 {
RegexEngine::Pcre2
} else {
@@ -880,10 +890,10 @@ fn build_matcher(patterns: &[String], cli: &RgCli) -> Result<CompiledMatcher, St
RegexEngine::Default => build_rust_matcher(patterns, cli)
.map(CompiledMatcher::Rust)
.map_err(|error| error.to_string()),
RegexEngine::Pcre2 => build_pcre_matcher(patterns, cli).map(CompiledMatcher::Pcre),
RegexEngine::Pcre2 => build_pcre_matcher(host, patterns, cli).map(CompiledMatcher::Pcre),
RegexEngine::Auto => match build_rust_matcher(patterns, cli) {
Ok(matcher) => Ok(CompiledMatcher::Rust(matcher)),
Err(_) => build_pcre_matcher(patterns, cli).map(CompiledMatcher::Pcre),
Err(_) => build_pcre_matcher(host, patterns, cli).map(CompiledMatcher::Pcre),
},
}
}
@@ -894,7 +904,7 @@ enum BinaryMode {
Explicit,
}
fn binary_detection(cli: &RgCli, mode: BinaryMode) -> BinaryDetection {
fn binary_detection(cli: &Rg, mode: BinaryMode) -> BinaryDetection {
if cli.text || cli.null_data {
return BinaryDetection::none();
}
@@ -905,7 +915,7 @@ fn binary_detection(cli: &RgCli, mode: BinaryMode) -> BinaryDetection {
}
}
fn build_searcher(cli: &RgCli, opts: &SearchOptions, mode: BinaryMode) -> Result<Searcher, String> {
fn build_searcher(cli: &Rg, opts: &SearchOptions, mode: BinaryMode) -> Result<Searcher, String> {
let (encoding, bom_sniffing) = match cli.encoding.as_deref() {
None | Some("auto") => (None, true),
Some("none") => (None, false),
@@ -931,14 +941,14 @@ fn build_searcher(cli: &RgCli, opts: &SearchOptions, mode: BinaryMode) -> Result
Ok(builder.build())
}
fn read_pattern_file(path: &OsStr) -> Result<Vec<String>, String> {
fn read_pattern_file(host: &mut Host, path: &OsStr) -> Result<Vec<String>, String> {
let mut text = String::new();
if path == OsStr::new("-") {
pi_uutils_ctx::stdin()
host.stdin
.read_to_string(&mut text)
.map_err(|err| format!("rg: -: {err}"))?;
} else {
let resolved = pi_uutils_ctx::resolve(path);
let resolved = host.resolve(path);
File::open(&resolved)
.and_then(|mut file| file.read_to_string(&mut text))
.map_err(|err| format!("rg: {}: {err}", path.to_string_lossy()))?;
@@ -949,10 +959,10 @@ fn read_pattern_file(path: &OsStr) -> Result<Vec<String>, String> {
.collect())
}
fn resolve_patterns(cli: &RgCli) -> Result<(Vec<String>, Vec<OsString>), String> {
fn resolve_patterns(host: &mut Host, cli: &Rg) -> Result<(Vec<String>, Vec<OsString>), String> {
let mut patterns = cli.patterns.clone();
for pattern_file in &cli.pattern_files {
patterns.extend(read_pattern_file(pattern_file.as_os_str())?);
patterns.extend(read_pattern_file(host, pattern_file.as_os_str())?);
}
let mut paths = Vec::new();
if cli.files || cli.type_list || !cli.patterns.is_empty() || !cli.pattern_files.is_empty() {
@@ -968,7 +978,7 @@ fn resolve_patterns(cli: &RgCli) -> Result<(Vec<String>, Vec<OsString>), String>
Ok((patterns, paths))
}
fn search_options(cli: &RgCli) -> SearchOptions {
fn search_options(cli: &Rg) -> SearchOptions {
let context = cli.context.unwrap_or(0);
let count_matches = cli.count_matches || (cli.count && cli.only_matching);
let line_number = (cli.line_number || cli.column || cli.vimgrep) && !cli.no_line_number;
@@ -1016,7 +1026,7 @@ fn parse_size(input: &str) -> Result<u64, String> {
Ok(value.saturating_mul(multiplier))
}
fn type_builder(cli: &RgCli) -> Result<TypesBuilder, String> {
fn type_builder(cli: &Rg) -> Result<TypesBuilder, String> {
let mut builder = TypesBuilder::new();
builder.add_defaults();
for name in &cli.type_clears {
@@ -1036,7 +1046,7 @@ fn type_builder(cli: &RgCli) -> Result<TypesBuilder, String> {
Ok(builder)
}
fn print_type_list<W: Write>(cli: &RgCli, out: &mut W) -> Result<(), String> {
fn print_type_list<W: Write>(cli: &Rg, out: &mut W) -> Result<(), String> {
let builder = type_builder(cli)?;
for def in builder.definitions() {
write!(out, "{}: ", def.name()).map_err(|err| err.to_string())?;
@@ -1109,8 +1119,8 @@ impl PathFilters {
}
}
fn build_path_filters(cli: &RgCli) -> Result<PathFilters, String> {
let cwd = pi_uutils_ctx::cwd();
fn build_path_filters(host: &mut Host, cli: &Rg) -> Result<PathFilters, String> {
let cwd = host.cwd().to_path_buf();
let max_filesize = cli
.max_filesize
.as_ref()
@@ -1147,7 +1157,7 @@ fn build_path_filters(cli: &RgCli) -> Result<PathFilters, String> {
} else {
let mut builder = GitignoreBuilder::new(&cwd);
for path in &cli.ignore_files {
let resolved = pi_uutils_ctx::resolve(path);
let resolved = host.resolve(path);
if let Some(error) = builder.add(&resolved) {
return Err(format!("rg: {}: {error}", path.to_string_lossy()));
}
@@ -1166,8 +1176,8 @@ fn build_path_filters(cli: &RgCli) -> Result<PathFilters, String> {
Ok(PathFilters { overrides, explicit, types, max_filesize })
}
fn build_walk(cli: &RgCli, root: &Path) -> Result<RgWalk, String> {
let filters = build_path_filters(cli)?;
fn build_walk(host: &mut Host, cli: &Rg, root: &Path) -> Result<RgWalk, String> {
let filters = build_path_filters(host, cli)?;
let unrestricted_no_ignore = cli.unrestricted >= 1;
let include_hidden = (cli.hidden || cli.unrestricted >= 2) && !cli.no_hidden;
let no_ignore = (cli.no_ignore || unrestricted_no_ignore) && !cli.ignore;
@@ -1260,7 +1270,8 @@ fn process_reader<M: Matcher, R: Read, W: Write>(
reason = "file processing needs the matcher, searcher, output state, and path metadata"
)]
fn process_file<M: Matcher, W: Write>(
cli: &RgCli,
host: &mut Host,
cli: &Rg,
matcher: &M,
searcher: &mut Searcher,
path: &Path,
@@ -1288,12 +1299,13 @@ fn process_file<M: Matcher, W: Write>(
Ok(any_match) => SearchOutcome { any_match, had_error: false },
Err(error) => SearchOutcome {
any_match: false,
had_error: report_path_error(display, path, error, opts),
had_error: report_path_error(host, display, path, error, opts),
},
}
}
fn report_path_error(
host: &mut Host,
display: Option<&[u8]>,
fallback: &Path,
err: io::Error,
@@ -1303,7 +1315,7 @@ fn report_path_error(
let name = display
.map(|bytes| String::from_utf8_lossy(bytes).into_owned())
.unwrap_or_else(|| fallback.display().to_string());
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: {name}: {err}");
let _ = writeln!(host.stderr, "rg: {name}: {err}");
}
true
}
@@ -1313,7 +1325,8 @@ fn report_path_error(
reason = "required by standard walk/configure interfaces and search parameters"
)]
fn search_collected_files<M: Matcher, W: Write>(
cli: &RgCli,
host: &mut Host,
cli: &Rg,
matcher: &M,
searcher: &mut Searcher,
operand: &OsStr,
@@ -1323,14 +1336,14 @@ fn search_collected_files<M: Matcher, W: Write>(
stats: &mut Stats,
out: &mut W,
) -> SearchOutcome {
let mut files = match collect_filtered_files(cli, root) {
let mut files = match collect_filtered_files(host, cli, root) {
Ok(files) => files,
Err(_) if pi_uutils_ctx::is_cancelled() => {
Err(_) if host.is_cancelled() => {
return SearchOutcome { any_match: false, had_error: true };
},
Err(err) => {
if !opts.no_messages {
let _ = writeln!(pi_uutils_ctx::stderr(), "{err}");
let _ = writeln!(host.stderr, "{err}");
}
return SearchOutcome { any_match: false, had_error: true };
},
@@ -1343,7 +1356,7 @@ fn search_collected_files<M: Matcher, W: Write>(
if opts.quiet && any_match {
break;
}
if processed_file && pi_uutils_ctx::is_cancelled() {
if processed_file && host.is_cancelled() {
had_error = true;
break;
}
@@ -1351,10 +1364,10 @@ fn search_collected_files<M: Matcher, W: Write>(
let display_path = display_path(operand, root, &path);
let display_bytes = display_path.as_os_str().as_encoded_bytes().to_vec();
let display = (show_names || opts.json).then_some(display_bytes.as_slice());
let outcome = process_file(cli, matcher, searcher, &path, display, opts, stats, out);
let outcome = process_file(host, cli, matcher, searcher, &path, display, opts, stats, out);
any_match |= outcome.any_match;
had_error |= outcome.had_error;
if pi_uutils_ctx::is_cancelled() {
if host.is_cancelled() {
had_error = true;
break;
}
@@ -1367,7 +1380,8 @@ fn search_collected_files<M: Matcher, W: Write>(
reason = "required by standard walk/configure interfaces and search parameters"
)]
fn search_dir<M: Matcher, W: Write>(
cli: &RgCli,
host: &mut Host,
cli: &Rg,
matcher: &M,
searcher: &mut Searcher,
operand: &OsStr,
@@ -1379,23 +1393,25 @@ fn search_dir<M: Matcher, W: Write>(
) -> SearchOutcome {
if cli.sortr.as_deref() == Some("path") {
return search_collected_files(
cli, matcher, searcher, operand, root, show_names, opts, stats, out,
host, cli, matcher, searcher, operand, root, show_names, opts, stats, out,
);
}
let walk = match build_walk(cli, root) {
let walk = match build_walk(host, cli, root) {
Ok(walk) => walk,
Err(err) => {
if !opts.no_messages {
let _ = writeln!(pi_uutils_ctx::stderr(), "{err}");
let _ = writeln!(host.stderr, "{err}");
}
return SearchOutcome { any_match: false, had_error: true };
},
};
let any_match = std::cell::Cell::new(false);
let had_error = std::cell::Cell::new(false);
let cancel = host.cancel_flag();
let mut walk_err = host.stderr_clone();
let streamed = match walk.request.for_each_entry_with_heartbeat(
|| {
if pi_uutils_ctx::is_cancelled() {
if cancel.load(std::sync::atomic::Ordering::Relaxed) {
Err(io::Error::from(io::ErrorKind::Interrupted))
} else {
Ok::<(), io::Error>(())
@@ -1419,7 +1435,7 @@ fn search_dir<M: Matcher, W: Write>(
let display_path = display_path(operand, root, path);
let display_bytes = display_path.as_os_str().as_encoded_bytes().to_vec();
let display = (show_names || opts.json).then_some(display_bytes.as_slice());
let outcome = process_file(cli, matcher, searcher, path, display, opts, stats, out);
let outcome = process_file(host, cli, matcher, searcher, path, display, opts, stats, out);
any_match.set(any_match.get() || outcome.any_match);
had_error.set(had_error.get() || outcome.had_error);
Ok(if opts.quiet && any_match.get() {
@@ -1432,7 +1448,7 @@ fn search_dir<M: Matcher, W: Write>(
had_error.set(true);
if !opts.no_messages {
let _ =
writeln!(pi_uutils_ctx::stderr(), "rg: {}: {}", error.path.display(), error.error);
writeln!(walk_err, "rg: {}: {}", error.path.display(), error.error);
}
Ok(pi_walker::WalkDecision::Include)
},
@@ -1440,7 +1456,7 @@ fn search_dir<M: Matcher, W: Write>(
Ok(pi_walker::WalkStatus::Complete | pi_walker::WalkStatus::Stopped) => {
Some(SearchOutcome { any_match: any_match.get(), had_error: had_error.get() })
},
Err(pi_walker::WalkError::Interrupted(_)) if pi_uutils_ctx::is_cancelled() => {
Err(pi_walker::WalkError::Interrupted(_)) if host.is_cancelled() => {
// Harness cancellation; the shell wrapper overrides the exit code
// and stay-silent on stderr — no spurious "interrupted" diagnostic.
had_error.set(true);
@@ -1449,27 +1465,28 @@ fn search_dir<M: Matcher, W: Write>(
Err(err) => {
had_error.set(true);
if !opts.no_messages {
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: {err}");
let _ = writeln!(host.stderr, "rg: {err}");
}
Some(SearchOutcome { any_match: any_match.get(), had_error: had_error.get() })
},
};
streamed.unwrap_or_else(|| {
search_collected_files(cli, matcher, searcher, operand, root, show_names, opts, stats, out)
search_collected_files(host, cli, matcher, searcher, operand, root, show_names, opts, stats, out)
})
}
fn collect_filtered_files(cli: &RgCli, root: &Path) -> Result<Vec<PathBuf>, String> {
let walk = build_walk(cli, root)?;
fn collect_filtered_files(host: &mut Host, cli: &Rg, root: &Path) -> Result<Vec<PathBuf>, String> {
let walk = build_walk(host, cli, root)?;
let cancel = host.cancel_flag();
let outcome = match walk.request.collect_with_heartbeat(|| {
if pi_uutils_ctx::is_cancelled() {
if cancel.load(std::sync::atomic::Ordering::Relaxed) {
Err(io::Error::from(io::ErrorKind::Interrupted))
} else {
Ok::<(), io::Error>(())
}
}) {
Ok(outcome) => outcome,
Err(pi_walker::WalkError::Interrupted(_)) if pi_uutils_ctx::is_cancelled() => {
Err(pi_walker::WalkError::Interrupted(_)) if host.is_cancelled() => {
return Err(String::from("rg: cancelled"));
},
Err(err) => return Err(format!("rg: {err}")),
@@ -1487,27 +1504,27 @@ fn collect_filtered_files(cli: &RgCli, root: &Path) -> Result<Vec<PathBuf>, Stri
Ok(files)
}
fn list_files<W: Write>(cli: &RgCli, paths: &[OsString], out: &mut W) -> SearchOutcome {
fn list_files<W: Write>(host: &mut Host, cli: &Rg, paths: &[OsString], out: &mut W) -> SearchOutcome {
let mut any = false;
let mut had_error = false;
let mut processed_operand = false;
for operand in paths {
if processed_operand && pi_uutils_ctx::is_cancelled() {
if processed_operand && host.is_cancelled() {
had_error = true;
break;
}
processed_operand = true;
let resolved = pi_uutils_ctx::resolve(operand);
let resolved = host.resolve(operand);
match std::fs::metadata(&resolved) {
Ok(meta) if meta.is_dir() => {
let mut files = match collect_filtered_files(cli, &resolved) {
let mut files = match collect_filtered_files(host, cli, &resolved) {
Ok(files) => files,
Err(_) if pi_uutils_ctx::is_cancelled() => {
Err(_) if host.is_cancelled() => {
had_error = true;
break;
},
Err(err) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "{err}");
let _ = writeln!(host.stderr, "{err}");
had_error = true;
continue;
},
@@ -1529,11 +1546,11 @@ fn list_files<W: Write>(cli: &RgCli, paths: &[OsString], out: &mut W) -> SearchO
},
Ok(_) => {},
Err(err) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: {}: {err}", operand.to_string_lossy());
let _ = writeln!(host.stderr, "rg: {}: {err}", operand.to_string_lossy());
had_error = true;
},
}
if pi_uutils_ctx::is_cancelled() {
if host.is_cancelled() {
had_error = true;
break;
}
@@ -1552,7 +1569,7 @@ fn default_paths(paths: &mut Vec<OsString>, use_implicit_stdin: bool) {
}
}
fn show_names_for(paths: &[OsString], recursive: bool, cli: &RgCli, opts: &SearchOptions) -> bool {
fn show_names_for(paths: &[OsString], recursive: bool, cli: &Rg, opts: &SearchOptions) -> bool {
if cli.no_filename {
false
} else if cli.with_filename || opts.files_with_matches || opts.files_without_match || cli.vimgrep
@@ -1588,7 +1605,8 @@ fn write_json_summary<W: Write>(out: &mut W, stats: &Stats) -> io::Result<()> {
}
fn execute_search<M: Matcher, W: Write>(
cli: &RgCli,
host: &mut Host,
cli: &Rg,
matcher: &M,
paths: &[OsString],
opts: &SearchOptions,
@@ -1597,20 +1615,20 @@ fn execute_search<M: Matcher, W: Write>(
let mut auto_searcher = match build_searcher(cli, opts, BinaryMode::Automatic) {
Ok(searcher) => searcher,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: {error}");
let _ = writeln!(host.stderr, "rg: {error}");
return 2;
},
};
let mut explicit_searcher = match build_searcher(cli, opts, BinaryMode::Explicit) {
Ok(searcher) => searcher,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: {error}");
let _ = writeln!(host.stderr, "rg: {error}");
return 2;
},
};
let recursive = paths.iter().any(|path| {
path.as_os_str() != OsStr::new("-")
&& std::fs::metadata(pi_uutils_ctx::resolve(path)).is_ok_and(|meta| meta.is_dir())
&& std::fs::metadata(host.resolve(path)).is_ok_and(|meta| meta.is_dir())
});
let show_names = show_names_for(paths, recursive, cli, opts);
let mut stats = Stats::new();
@@ -1621,7 +1639,7 @@ fn execute_search<M: Matcher, W: Write>(
if opts.quiet && any_match {
break;
}
if processed_operand && pi_uutils_ctx::is_cancelled() {
if processed_operand && host.is_cancelled() {
had_error = true;
break;
}
@@ -1631,7 +1649,7 @@ fn execute_search<M: Matcher, W: Write>(
match process_reader(
matcher,
&mut explicit_searcher,
pi_uutils_ctx::stdin(),
&mut host.stdin,
display,
opts,
&mut stats,
@@ -1641,20 +1659,21 @@ fn execute_search<M: Matcher, W: Write>(
Err(error) => {
had_error = true;
if !opts.no_messages {
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: <stdin>: {error}");
let _ = writeln!(host.stderr, "rg: <stdin>: {error}");
}
},
}
if pi_uutils_ctx::is_cancelled() {
if host.is_cancelled() {
had_error = true;
break;
}
continue;
}
let resolved = pi_uutils_ctx::resolve(operand);
let resolved = host.resolve(operand);
match std::fs::metadata(&resolved) {
Ok(meta) if meta.is_dir() => {
let outcome = search_dir(
host,
cli,
matcher,
&mut auto_searcher,
@@ -1672,6 +1691,7 @@ fn execute_search<M: Matcher, W: Write>(
let display =
(show_names || opts.json).then_some(operand.as_os_str().as_encoded_bytes());
let outcome = process_file(
host,
cli,
matcher,
&mut explicit_searcher,
@@ -1689,11 +1709,11 @@ fn execute_search<M: Matcher, W: Write>(
had_error = true;
if !opts.no_messages {
let _ =
writeln!(pi_uutils_ctx::stderr(), "rg: {}: {error}", operand.to_string_lossy());
writeln!(host.stderr, "rg: {}: {error}", operand.to_string_lossy());
}
},
}
if pi_uutils_ctx::is_cancelled() {
if host.is_cancelled() {
had_error = true;
break;
}
@@ -1719,21 +1739,12 @@ fn execute_search<M: Matcher, W: Write>(
}
}
/// Runs the ripgrep-compatible builtin and returns a process-style exit code.
pub fn run(argv: Vec<OsString>) -> i32 {
let cli = match RgCli::try_parse_from(argv) {
Ok(cli) => cli,
Err(error) => {
let rendered = error.to_string();
if error.use_stderr() {
let _ = write!(pi_uutils_ctx::stderr(), "{rendered}");
return 2;
}
let _ = write!(pi_uutils_ctx::stdout(), "{rendered}");
return 0;
},
};
impl Utility for Rg {
const NAME: &'static str = "rg";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
let cli = self;
let opts = search_options(&cli);
if opts.json
&& (cli.files
@@ -1746,18 +1757,18 @@ pub fn run(argv: Vec<OsString>) -> i32 {
|| cli.only_matching
|| cli.vimgrep)
{
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: --json cannot be combined with summary modes");
let _ = writeln!(host.stderr, "rg: --json cannot be combined with summary modes");
return 2;
}
let mut out = if cli.line_buffered && !cli.no_line_buffered {
RgOutput::Direct(pi_uutils_ctx::stdout())
RgOutput::Direct(host.stdout_clone())
} else {
RgOutput::Buffered(BufWriter::new(pi_uutils_ctx::stdout()))
RgOutput::Buffered(BufWriter::new(host.stdout_clone()))
};
let (patterns, mut paths) = match resolve_patterns(&cli) {
let (patterns, mut paths) = match resolve_patterns(host, &cli) {
Ok(resolved) => resolved,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "{error}");
let _ = writeln!(host.stderr, "{error}");
return 2;
},
};
@@ -1765,7 +1776,7 @@ pub fn run(argv: Vec<OsString>) -> i32 {
return match print_type_list(&cli, &mut out) {
Ok(()) => 0,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: {error}");
let _ = writeln!(host.stderr, "rg: {error}");
2
},
};
@@ -1773,10 +1784,10 @@ pub fn run(argv: Vec<OsString>) -> i32 {
let pattern_stdin_consumed = cli.pattern_files.iter().any(|file| file == OsStr::new("-"));
default_paths(
&mut paths,
!cli.files && !pattern_stdin_consumed && pi_uutils_ctx::stdin_is_search_input(),
!cli.files && !pattern_stdin_consumed && host.stdin_is_search_input(),
);
if cli.files {
let outcome = list_files(&cli, &paths, &mut out);
let outcome = list_files(host, &cli, &paths, &mut out);
let _ = out.flush();
return if outcome.had_error {
2
@@ -1789,284 +1800,143 @@ pub fn run(argv: Vec<OsString>) -> i32 {
if patterns.is_empty() {
return 1;
}
let matcher = match build_matcher(&patterns, &cli) {
let matcher = match build_matcher(host, &patterns, &cli) {
Ok(matcher) => matcher,
Err(error) => {
let _ = writeln!(pi_uutils_ctx::stderr(), "rg: {error}");
let _ = writeln!(host.stderr, "rg: {error}");
return 2;
},
};
match &matcher {
CompiledMatcher::Rust(matcher) => execute_search(&cli, matcher, &paths, &opts, &mut out),
CompiledMatcher::Pcre(matcher) => execute_search(&cli, matcher, &paths, &opts, &mut out),
CompiledMatcher::Rust(matcher) => execute_search(host, &cli, matcher, &paths, &opts, &mut out),
CompiledMatcher::Pcre(matcher) => execute_search(host, &cli, matcher, &paths, &opts, &mut out),
}
}
}
#[cfg(test)]
mod tests {
use std::{
collections::HashMap,
sync::{Arc, atomic::AtomicBool},
};
use parking_lot::Mutex;
use pi_uutils_ctx::{ScopeIo, scope};
use super::*;
use crate::host::{Host, run_util};
/// Sink that collects writes into a shared buffer for assertions.
struct SharedBuf(Arc<Mutex<Vec<u8>>>);
impl Write for SharedBuf {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.0.lock().extend_from_slice(buf);
Ok(buf.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
fn run_rg(args: &[&str], stdin: &str) -> (i32, String, String) {
run_rg_in(args, stdin, &std::env::temp_dir())
}
fn run_rg_in(args: &[&str], stdin: &str, cwd: &Path) -> (i32, String, String) {
let out = Arc::new(Mutex::new(Vec::new()));
let err = Arc::new(Mutex::new(Vec::new()));
let io = ScopeIo {
stdin: Box::new(io::Cursor::new(stdin.as_bytes().to_vec())),
stdin_fd: None,
stdin_is_search_input: true,
stdout: Box::new(SharedBuf(Arc::clone(&out))),
stderr: Box::new(SharedBuf(Arc::clone(&err))),
cwd: cwd.to_path_buf(),
env: HashMap::new(),
cancel: Arc::new(AtomicBool::new(false)),
};
let argv: Vec<OsString> = std::iter::once("rg")
.chain(args.iter().copied())
.map(OsString::from)
.collect();
let code = scope(io, || run(argv));
let stdout = String::from_utf8(out.lock().clone()).expect("utf8 stdout");
let stderr = String::from_utf8(err.lock().clone()).expect("utf8 stderr");
(code, stdout, stderr)
fn run(args: &[&str], stdin: &str) -> (i32, String, String) {
let (code, capture) = run_util::<Rg>(args, stdin, "/");
(code, capture.out(), capture.err())
}
#[test]
fn max_count_accepts_an_attached_value() {
let (code, stdout, stderr) = run_rg(&["-m1", "hit"], "hit\nmiss\nhit\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "hit\n");
fn max_count_stops_after_one_match() {
let (code, out, err) = run(&["-m1", "hit", "-"], "hit\nmiss\nhit\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "hit\n");
}
#[test]
fn pcre2_matches_lookbehind_patterns() {
let (code, stdout, stderr) = run_rg(&["--pcre2", "(?<=foo)bar"], "foobar\nbar\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "foobar\n");
fn pcre2_supports_lookbehind() {
let (code, out, err) = run(&["--pcre2", "(?<=foo)bar", "-"], "foobar\nbar\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "foobar\n");
}
#[test]
fn pcre2_compiles_the_multiline_native_grep_crash_pattern() {
let pattern = r"^(\s+)- (not_null|unique|accepted_values|relationships|expression_is_true|[a-z_]+):\s*$\n(?!\1 (arguments|config|description|name):)";
let (code, stdout, stderr) =
run_rg(&["--pcre2", "--multiline", pattern], " - not_null:\n severity: warn\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, " - not_null:\n");
fn replacement_expands_named_capture() {
let (code, out, err) = run(&["-o", "--replace=${word}-x", "(?P<word>foo)", "-"], "foo bar\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "foo-x\n");
}
#[test]
fn auto_engine_promotes_patterns_that_need_pcre2() {
let (code, stdout, stderr) = run_rg(&["--engine=auto", "(?<=foo)bar"], "foobar\nbar\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "foobar\n");
}
#[test]
fn replacement_expands_capture_groups() {
let (code, stdout, stderr) =
run_rg(&["-o", "--replace=${word}-x", "(?P<word>foo)"], "foo bar\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "foo-x\n");
}
#[test]
fn byte_offset_reports_the_absolute_match_position() {
let (code, stdout, stderr) = run_rg(&["--byte-offset", "hit"], "zero\nhit\n");
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "5:hit\n");
}
#[test]
fn json_emits_structured_search_events() {
let (code, stdout, stderr) = run_rg(&["--json", "hit"], "miss\nhit\n");
assert_eq!(code, 0, "{stderr}");
let events: Vec<serde_json::Value> = stdout
.lines()
.map(|line| serde_json::from_str(line).expect("each output line should be JSON"))
.collect();
let kinds: Vec<&str> = events
.iter()
.map(|event| event["type"].as_str().expect("event type"))
.collect();
fn json_emits_search_events_and_summary() {
let (code, out, err) = run(&["--json", "hit", "-"], "miss\nhit\n");
assert_eq!(code, 0, "{err}");
let events: Vec<serde_json::Value> = out.lines().map(|line| serde_json::from_str(line).unwrap()).collect();
let kinds: Vec<_> = events.iter().map(|event| event["type"].as_str().unwrap()).collect();
assert_eq!(kinds, ["begin", "match", "end", "summary"]);
assert_eq!(events[1]["data"]["lines"]["text"], "hit\n");
assert_eq!(events[3]["data"]["stats"]["searches"], 1);
assert_eq!(events[3]["data"]["stats"]["searches_with_match"], 1);
assert_eq!(events[3]["data"]["stats"]["matched_lines"], 1);
assert_eq!(events[3]["data"]["stats"]["matches"], 1);
}
#[test]
fn explicit_encoding_transcodes_input_before_matching() {
let tree = unique_tree("encoding");
std::fs::write(tree.join("utf16.txt"), b"h\0i\0t\0\n\0")
.expect("UTF-16 fixture should be written");
let (code, stdout, stderr) =
run_rg_in(&["--encoding=utf-16le", "hit", "utf16.txt"], "", &tree);
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "hit\n");
let _ = std::fs::remove_dir_all(tree);
fn ignore_file_is_resolved_against_shell_cwd() {
let tree = tempfile::tempdir().unwrap();
std::fs::write(tree.path().join("keep.txt"), "hit\n").unwrap();
std::fs::write(tree.path().join("skip.txt"), "hit\n").unwrap();
std::fs::write(tree.path().join("rules.ignore"), "skip.txt\n").unwrap();
let (code, capture) = run_util::<Rg>(&["--ignore-file=rules.ignore", "hit", "."], "", tree.path());
assert_eq!(code, 0, "{}", capture.err());
assert!(capture.out().contains("keep.txt:hit\n"));
assert!(!capture.out().contains("skip.txt"));
}
#[test]
fn explicit_ignore_file_filters_recursive_search() {
let tree = unique_tree("ignore-file");
std::fs::write(tree.join("keep.txt"), "hit\n").expect("included fixture should be written");
std::fs::write(tree.join("skip.txt"), "hit\n").expect("ignored fixture should be written");
std::fs::write(tree.join("rules.ignore"), "skip.txt\n")
.expect("ignore rules should be written");
let (code, stdout, stderr) =
run_rg_in(&["--ignore-file=rules.ignore", "hit", "."], "", &tree);
assert_eq!(code, 0, "{stderr}");
assert!(stdout.contains("keep.txt:hit\n"), "{stdout:?}");
assert!(!stdout.contains("skip.txt"), "{stdout:?}");
let _ = std::fs::remove_dir_all(tree);
fn files_mode_preserves_relative_paths() {
let tree = tempfile::tempdir().unwrap();
std::fs::create_dir(tree.path().join("sub")).unwrap();
std::fs::write(tree.path().join("sub/file.txt"), "x\n").unwrap();
let (code, capture) = run_util::<Rg>(&["--files", "sub"], "", tree.path());
assert_eq!(code, 0, "{}", capture.err());
assert_eq!(capture.out(), "sub/file.txt\n");
}
#[test]
fn glob_case_insensitive_applies_to_regular_globs() {
let tree = unique_tree("glob-case");
std::fs::write(tree.join("UPPER.TXT"), "hit\n").expect("fixture should be written");
let (code, stdout, stderr) =
run_rg_in(&["--glob-case-insensitive", "--glob=*.txt", "hit", "."], "", &tree);
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "UPPER.TXT:hit\n");
let _ = std::fs::remove_dir_all(tree);
fn recursive_walk_observes_cancellation() {
let tree = tempfile::tempdir().unwrap();
std::fs::create_dir(tree.path().join("root")).unwrap();
std::fs::write(tree.path().join("root/file"), "hit\n").unwrap();
let parsed = Rg::try_parse_from(["rg", "hit", "root"]).unwrap();
let (mut host, capture) = Host::for_test("rg", Vec::new(), tree.path());
host.cancel_for_test();
assert_eq!(parsed.run(&mut host), 2);
assert!(capture.out().is_empty());
assert!(capture.err().is_empty());
}
#[test]
fn search_zip_decompresses_supported_files() {
let tree = unique_tree("search-zip");
fn auto_engine_and_byte_offsets_match_ripgrep() {
let (code, out, err) = run(&["--engine=auto", "(?<=foo)bar", "-"], "foobar\nbar\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "foobar\n");
let (code, out, err) = run(&["--byte-offset", "hit", "-"], "zero\nhit\n");
assert_eq!(code, 0, "{err}");
assert_eq!(out, "5:hit\n");
}
#[test]
fn encoding_and_case_insensitive_globs_apply_to_files() {
let tree = tempfile::tempdir().unwrap();
std::fs::write(tree.path().join("utf16.txt"), b"h\0i\0t\0\n\0").unwrap();
let (code, capture) =
run_util::<Rg>(&["--encoding=utf-16le", "hit", "utf16.txt"], "", tree.path());
assert_eq!(code, 0, "{}", capture.err());
assert_eq!(capture.out(), "hit\n");
std::fs::write(tree.path().join("UPPER.TXT"), "hit\n").unwrap();
let (code, capture) = run_util::<Rg>(
&["--glob-case-insensitive", "--glob=*.txt", "hit", "."],
"",
tree.path(),
);
assert_eq!(code, 0, "{}", capture.err());
assert!(capture.out().contains("UPPER.TXT:hit\n"));
}
#[test]
fn search_zip_decompresses_gzip() {
let tree = tempfile::tempdir().unwrap();
let gzip = [
31, 139, 8, 0, 0, 0, 0, 0, 2, 255, 203, 205, 44, 46, 230, 202, 200, 44, 225, 2, 0, 26, 30,
21, 140, 9, 0, 0, 0,
31, 139, 8, 0, 0, 0, 0, 0, 2, 255, 203, 205, 44, 46, 230, 202, 200, 44, 225, 2,
0, 26, 30, 21, 140, 9, 0, 0, 0,
];
std::fs::write(tree.join("sample.gz"), gzip).expect("gzip fixture should be written");
let (code, stdout, stderr) = run_rg_in(&["--search-zip", "hit", "sample.gz"], "", &tree);
assert_eq!(code, 0, "{stderr}");
assert_eq!(stdout, "hit\n");
let _ = std::fs::remove_dir_all(tree);
}
/// Run `rg` with the cancel flag pre-set, mirroring the shell wrapper's
/// behavior when `abort`/`timeout` fires mid-walk.
fn run_rg_cancelled(args: &[&str], cwd: &Path) -> (i32, String, String) {
let out = Arc::new(Mutex::new(Vec::new()));
let err = Arc::new(Mutex::new(Vec::new()));
let io = ScopeIo {
stdin: Box::new(io::empty()),
stdin_fd: None,
stdin_is_search_input: false,
stdout: Box::new(SharedBuf(Arc::clone(&out))),
stderr: Box::new(SharedBuf(Arc::clone(&err))),
cwd: cwd.to_path_buf(),
env: HashMap::new(),
cancel: Arc::new(AtomicBool::new(true)),
};
let argv: Vec<OsString> = std::iter::once("rg")
.chain(args.iter().copied())
.map(OsString::from)
.collect();
let code = scope(io, || run(argv));
let stdout = String::from_utf8(out.lock().clone()).expect("utf8 stdout");
let stderr = String::from_utf8(err.lock().clone()).expect("utf8 stderr");
(code, stdout, stderr)
}
fn unique_tree(label: &str) -> PathBuf {
let root = std::env::temp_dir().join(format!(
"pi-uu-rg-{label}-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
std::fs::create_dir_all(&root).expect("temp tree should be created");
root
}
#[test]
fn recursive_search_observes_scope_cancellation() {
// Regression for #3933: rg's recursive walker used to pass a no-op
// heartbeat to pi_walker, so cancellation was not observed during
// directory traversal even after the uutils ctx cancel flag was set.
let tree = unique_tree("search");
let walk_root = tree.join("walk-root");
std::fs::create_dir_all(&walk_root).expect("walk root should be created");
std::fs::write(walk_root.join("haystack.txt"), "match-me\n").expect("walked file written");
let later_file = tree.join("later.txt");
std::fs::write(&later_file, "match-me\n").expect("later file written");
let (code, stdout, stderr) = run_rg_cancelled(
&[
"match-me",
walk_root.to_str().expect("utf8 path"),
later_file.to_str().expect("utf8 path"),
],
&tree,
);
assert!(stdout.is_empty(), "cancelled walk should not output matches: {stdout:?}");
assert!(
stderr.is_empty(),
"cancelled walk should stay silent — diagnostic is the shell's job: {stderr:?}"
);
assert_eq!(code, 2, "interrupted directory walk should report had_error (exit 2)");
let _ = std::fs::remove_dir_all(&tree);
}
#[test]
fn files_mode_observes_scope_cancellation() {
// Regression for #3933: `rg --files <dir>` routes through
// `collect_filtered_files`, whose heartbeat was likewise a no-op.
let tree = unique_tree("files");
let walk_root = tree.join("walk-root");
std::fs::create_dir_all(&walk_root).expect("walk root should be created");
std::fs::write(walk_root.join("alpha.txt"), "alpha\n").expect("walked file written");
let later_file = tree.join("later.txt");
std::fs::write(&later_file, "later\n").expect("later file written");
let (code, stdout, stderr) = run_rg_cancelled(
&[
"--files",
walk_root.to_str().expect("utf8 path"),
later_file.to_str().expect("utf8 path"),
],
&tree,
);
assert!(stdout.is_empty(), "cancelled --files walk should not enumerate paths: {stdout:?}");
assert!(stderr.is_empty(), "cancelled --files walk should stay silent: {stderr:?}");
// Cancellation is an error for standalone utility status; the shell
// wrapper rewrites it to the user-visible cancelled status (130).
assert_eq!(code, 2, "cancelled --files walk should stop before later operands");
let _ = std::fs::remove_dir_all(&tree);
std::fs::write(tree.path().join("sample.gz"), gzip).unwrap();
let (code, capture) =
run_util::<Rg>(&["--search-zip", "hit", "sample.gz"], "", tree.path());
assert_eq!(code, 0, "{}", capture.err());
assert_eq!(capture.out(), "hit\n");
}
}
use brush_core::{ShellExtensions, builtins::Registration};
use crate::host::util;
/// Creates the ripgrep-compatible `rg` builtin registration.
pub(crate) fn rg_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Rg, SE>()
}
crates/pi-builtins/src/rm.rs
+1676
View File
File diff suppressed because it is too large Load Diff
crates/pi-builtins/src/sed.rs
+9533
View File
File diff suppressed because it is too large Load Diff
crates/pi-builtins/src/seq.rs
+930
View File
@@ -0,0 +1,930 @@
//! `seq` builtin: display numbers from FIRST to LAST in steps of INCREMENT.
//!
//! Ported from uutils coreutils 0.8.0.
use std::{
error::Error,
ffi::{OsStr, OsString},
io::{BufWriter, Write},
};
use brush_core::{ShellExtensions, builtins::Registration};
use clap::{Arg, ArgAction, ArgMatches, Command};
use num_bigint::BigUint;
use num_traits::{ToPrimitive, Zero};
use uucore::{
extendedbigdecimal::ExtendedBigDecimal,
fast_inc::fast_inc,
format::{Format, num_format, num_format::FloatVariant},
};
use crate::host::{Host, Utility, format_usage, matches_parser, util};
mod number {
use num_traits::Zero;
use uucore::extendedbigdecimal::ExtendedBigDecimal;
/// A number with a specified number of integer and fractional digits.
///
/// This struct can be used to represent a number along with information
/// on how many significant digits to use when displaying the number.
/// The [`PreciseNumber::num_integral_digits`] field also includes the width
/// needed to display the "-" character for a negative number.
/// [`PreciseNumber::num_fractional_digits`] provides the number of decimal
/// digits after the decimal point (a.k.a. precision), or None if that number
/// cannot intuitively be obtained (i.e. hexadecimal floats).
/// Note: Those 2 fields should not necessarily be interpreted literally, but as
/// matching GNU `seq` behavior: the exact way of guessing desired precision
/// from user input is a matter of interpretation.
///
/// You can get an instance of this struct by calling [`str::parse`].
#[derive(Debug)]
pub struct PreciseNumber {
pub number: ExtendedBigDecimal,
pub num_integral_digits: usize,
pub num_fractional_digits: Option<usize>,
}
impl PreciseNumber {
pub fn one() -> Self {
// We would like to implement `num_traits::One`, but it requires
// a multiplication implementation, and we don't want to
// implement that here.
Self {
number: ExtendedBigDecimal::one(),
num_integral_digits: 1,
num_fractional_digits: Some(0),
}
}
/// Decide whether this number is zero (either positive or negative).
pub fn is_zero(&self) -> bool {
// We would like to implement `num_traits::Zero`, but it
// requires an addition implementation, and we don't want to
// implement that here.
self.number.is_zero()
}
}
}
mod numberparse {
//! Parsing numbers for use in `seq`.
//!
//! This module provides an implementation of [`FromStr`] for the
//! [`PreciseNumber`] struct.
use std::str::FromStr;
use uucore::{
extendedbigdecimal::ExtendedBigDecimal,
parser::num_parser::{ExtendedParser, ExtendedParserError},
};
use super::number::PreciseNumber;
/// An error returned when parsing a number fails.
#[derive(Debug, PartialEq, Eq)]
pub enum ParseNumberError {
Float,
Nan,
}
/// Compute the number of integral and fractional digits in input string,
/// and wrap the result in a PreciseNumber.
/// We know that the string has already been parsed correctly, so we don't
/// need to be too careful.
fn compute_num_digits(input: &str, ebd: ExtendedBigDecimal) -> PreciseNumber {
let input = input.to_lowercase();
let input = input.trim_start();
// Leading + is ignored for this.
let input = input.strip_prefix('+').unwrap_or(input);
// Integral digits for any hex number is ill-defined (0 is fine as an output)
// Fractional digits for an floating hex number is ill-defined, return None
// as we'll totally ignore that number for precision computations.
// Still return 0 for hex integers though.
if input.starts_with("0x") || input.starts_with("-0x") {
return PreciseNumber {
number: ebd,
num_integral_digits: 0,
num_fractional_digits: if input.contains('.') || input.contains('p') {
None
} else {
Some(0)
},
};
}
// Split the exponent part, if any
let parts: Vec<&str> = input.split('e').collect();
debug_assert!(parts.len() <= 2);
// Count all the digits up to `.`, `-` sign is included.
let (mut int_digits, mut frac_digits) = match parts[0].find('.') {
Some(i) => {
// Cover special case .X and -.X where we behave as if there was a leading 0:
// 0.X, -0.X.
let int_digits = match i {
0 => 1,
1 if parts[0].starts_with('-') => 2,
_ => i,
};
(int_digits, parts[0].len() - i - 1)
},
None => (parts[0].len(), 0),
};
// If there is an exponent, reparse that (yes this is not optimal,
// but we can't necessarily exactly recover that from the parsed number).
if parts.len() == 2 {
let exp = parts[1].parse::<i64>().unwrap_or(0);
// For positive exponents, effectively expand the number. Ignore negative
// exponents. Also ignore overflowed exponents (unwrap_or(0)).
if exp > 0 {
int_digits += exp.try_into().unwrap_or(0);
}
frac_digits = if exp < frac_digits as i64 {
// Subtract from i128 to avoid any overflow
(frac_digits as i128 - exp as i128).try_into().unwrap_or(0)
} else {
0
}
}
PreciseNumber {
number: ebd,
num_integral_digits: int_digits,
num_fractional_digits: Some(frac_digits),
}
}
// Note: We could also have provided an `ExtendedParser` implementation for
// PreciseNumber, but we want a simpler custom error.
impl FromStr for PreciseNumber {
type Err = ParseNumberError;
fn from_str(input: &str) -> Result<Self, Self::Err> {
let ebd = match ExtendedBigDecimal::extended_parse(input) {
Ok(ebd) => match ebd {
// Handle special values
ExtendedBigDecimal::BigDecimal(_) | ExtendedBigDecimal::MinusZero => {
// TODO: GNU `seq` treats small numbers < 1e-4950 as 0, we could do the same
// to avoid printing senselessly small numbers.
ebd
},
ExtendedBigDecimal::Infinity | ExtendedBigDecimal::MinusInfinity => {
return Ok(Self {
number: ebd,
num_integral_digits: 0,
num_fractional_digits: Some(0),
});
},
ExtendedBigDecimal::Nan | ExtendedBigDecimal::MinusNan => {
return Err(ParseNumberError::Nan);
},
},
Err(ExtendedParserError::Underflow(ebd)) => ebd, // Treat underflow as 0
Err(_) => return Err(ParseNumberError::Float),
};
Ok(compute_num_digits(input, ebd))
}
}
#[cfg(test)]
mod tests {
use bigdecimal::BigDecimal;
use uucore::extendedbigdecimal::ExtendedBigDecimal;
use super::{ParseNumberError, super::number::PreciseNumber};
/// Convenience function for parsing a [`Number`] and unwrapping.
fn parse(s: &str) -> ExtendedBigDecimal {
s.parse::<PreciseNumber>().unwrap().number
}
/// Convenience function for getting the number of integral digits.
fn num_integral_digits(s: &str) -> usize {
s.parse::<PreciseNumber>().unwrap().num_integral_digits
}
/// Convenience function for getting the number of fractional digits.
fn num_fractional_digits(s: &str) -> usize {
s.parse::<PreciseNumber>()
.unwrap()
.num_fractional_digits
.unwrap()
}
/// Convenience function for making sure the number of fractional digits is
/// "None"
fn num_fractional_digits_is_none(s: &str) -> bool {
s.parse::<PreciseNumber>()
.unwrap()
.num_fractional_digits
.is_none()
}
#[test]
fn test_parse_minus_zero_int() {
assert_eq!(parse("-0e0"), ExtendedBigDecimal::MinusZero);
assert_eq!(parse("-0e-0"), ExtendedBigDecimal::MinusZero);
assert_eq!(parse("-0e1"), ExtendedBigDecimal::MinusZero);
assert_eq!(parse("-0e+1"), ExtendedBigDecimal::MinusZero);
assert_eq!(parse("-0.0e1"), ExtendedBigDecimal::MinusZero);
assert_eq!(parse("-0x0"), ExtendedBigDecimal::MinusZero);
}
#[test]
fn test_parse_minus_zero_float() {
assert_eq!(parse("-0.0"), ExtendedBigDecimal::MinusZero);
assert_eq!(parse("-0e-1"), ExtendedBigDecimal::MinusZero);
assert_eq!(parse("-0.0e-1"), ExtendedBigDecimal::MinusZero);
}
#[test]
fn test_parse_big_int() {
assert_eq!(parse("0"), ExtendedBigDecimal::zero());
assert_eq!(parse("0.1e1"), ExtendedBigDecimal::one());
assert_eq!(parse("0.1E1"), ExtendedBigDecimal::one());
assert_eq!(
parse("1.0e1"),
ExtendedBigDecimal::BigDecimal("10".parse::<BigDecimal>().unwrap())
);
}
#[test]
fn test_parse_hexadecimal_big_int() {
assert_eq!(parse("0x0"), ExtendedBigDecimal::zero());
assert_eq!(
parse("0x10"),
ExtendedBigDecimal::BigDecimal("16".parse::<BigDecimal>().unwrap())
);
}
#[test]
fn test_parse_big_decimal() {
assert_eq!(
parse("0.0"),
ExtendedBigDecimal::BigDecimal("0.0".parse::<BigDecimal>().unwrap())
);
assert_eq!(parse(".0"), ExtendedBigDecimal::BigDecimal("0.0".parse::<BigDecimal>().unwrap()));
assert_eq!(
parse("1.0"),
ExtendedBigDecimal::BigDecimal("1.0".parse::<BigDecimal>().unwrap())
);
assert_eq!(
parse("10e-1"),
ExtendedBigDecimal::BigDecimal("1.0".parse::<BigDecimal>().unwrap())
);
assert_eq!(
parse("-1e-3"),
ExtendedBigDecimal::BigDecimal("-0.001".parse::<BigDecimal>().unwrap())
);
}
#[test]
fn test_parse_inf() {
assert_eq!(parse("inf"), ExtendedBigDecimal::Infinity);
assert_eq!(parse("infinity"), ExtendedBigDecimal::Infinity);
assert_eq!(parse("+inf"), ExtendedBigDecimal::Infinity);
assert_eq!(parse("+infinity"), ExtendedBigDecimal::Infinity);
assert_eq!(parse("-inf"), ExtendedBigDecimal::MinusInfinity);
assert_eq!(parse("-infinity"), ExtendedBigDecimal::MinusInfinity);
}
#[test]
fn test_parse_invalid_float() {
assert_eq!("1.2.3".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Float);
assert_eq!("1e2e3".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Float);
assert_eq!("1e2.3".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Float);
assert_eq!("-+-1".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Float);
}
#[test]
fn test_parse_invalid_hex() {
assert_eq!("0xg".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Float);
}
#[test]
fn test_parse_invalid_nan() {
assert_eq!("nan".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Nan);
assert_eq!("NAN".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Nan);
assert_eq!("NaN".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Nan);
assert_eq!("nAn".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Nan);
assert_eq!("-nan".parse::<PreciseNumber>().unwrap_err(), ParseNumberError::Nan);
}
#[test]
fn test_num_integral_digits() {
// no decimal, no exponent
assert_eq!(num_integral_digits("123"), 3);
// decimal, no exponent
assert_eq!(num_integral_digits("123.45"), 3);
assert_eq!(num_integral_digits("-0.1"), 2);
assert_eq!(num_integral_digits("-.1"), 2);
// exponent, no decimal
assert_eq!(num_integral_digits("123e4"), 3 + 4);
assert_eq!(num_integral_digits("123e-4"), 3);
assert_eq!(num_integral_digits("-1e-3"), 2);
// decimal and exponent
assert_eq!(num_integral_digits("123.45e6"), 3 + 6);
assert_eq!(num_integral_digits("123.45e-6"), 3);
assert_eq!(num_integral_digits("123.45e-1"), 3);
assert_eq!(num_integral_digits("-0.1e0"), 2);
assert_eq!(num_integral_digits("-0.1e2"), 4);
assert_eq!(num_integral_digits("-.1e0"), 2);
assert_eq!(num_integral_digits("-.1e2"), 4);
assert_eq!(num_integral_digits("-1.e-3"), 2);
assert_eq!(num_integral_digits("-1.0e-4"), 2);
// minus zero int
assert_eq!(num_integral_digits("-0e0"), 2);
assert_eq!(num_integral_digits("-0e-0"), 2);
assert_eq!(num_integral_digits("-0e1"), 3);
assert_eq!(num_integral_digits("-0e+1"), 3);
assert_eq!(num_integral_digits("-0.0e1"), 3);
// minus zero float
assert_eq!(num_integral_digits("-0.0"), 2);
assert_eq!(num_integral_digits("-0e-1"), 2);
assert_eq!(num_integral_digits("-0.0e-1"), 2);
// TODO In GNU `seq`, the `-w` option does not seem to work with
// hexadecimal arguments. In order to match that behavior, we
// report the number of integral digits as zero for hexadecimal
// inputs.
assert_eq!(num_integral_digits("0xff"), 0);
}
#[test]
fn test_num_fractional_digits() {
// no decimal, no exponent
assert_eq!(num_fractional_digits("123"), 0);
assert_eq!(num_fractional_digits("0xff"), 0);
// decimal, no exponent
assert_eq!(num_fractional_digits("123.45"), 2);
assert_eq!(num_fractional_digits("-0.1"), 1);
assert_eq!(num_fractional_digits("-.1"), 1);
// exponent, no decimal
assert_eq!(num_fractional_digits("123e4"), 0);
assert_eq!(num_fractional_digits("123e-4"), 4);
assert_eq!(num_fractional_digits("123e-1"), 1);
assert_eq!(num_fractional_digits("-1e-3"), 3);
// decimal and exponent
assert_eq!(num_fractional_digits("123.45e6"), 0);
assert_eq!(num_fractional_digits("123.45e1"), 1);
assert_eq!(num_fractional_digits("123.45e-6"), 8);
assert_eq!(num_fractional_digits("123.45e-1"), 3);
assert_eq!(num_fractional_digits("-0.1e0"), 1);
assert_eq!(num_fractional_digits("-0.1e2"), 0);
assert_eq!(num_fractional_digits("-.1e0"), 1);
assert_eq!(num_fractional_digits("-.1e2"), 0);
assert_eq!(num_fractional_digits("-1.e-3"), 3);
assert_eq!(num_fractional_digits("-1.0e-4"), 5);
// minus zero int
assert_eq!(num_fractional_digits("-0e0"), 0);
assert_eq!(num_fractional_digits("-0e-0"), 0);
assert_eq!(num_fractional_digits("-0e1"), 0);
assert_eq!(num_fractional_digits("-0e+1"), 0);
assert_eq!(num_fractional_digits("-0.0e1"), 0);
// minus zero float
assert_eq!(num_fractional_digits("-0.0"), 1);
assert_eq!(num_fractional_digits("-0e-1"), 1);
assert_eq!(num_fractional_digits("-0.0e-1"), 2);
// Hexadecimal numbers
assert_eq!(num_fractional_digits("0xff"), 0);
assert!(num_fractional_digits_is_none("0xff.1"));
}
#[test]
fn test_parse_min_exponents() {
// Make sure exponents < i64::MIN do not cause errors
assert!("1e-9223372036854775807".parse::<PreciseNumber>().is_ok());
assert!("1e-9223372036854775808".parse::<PreciseNumber>().is_ok());
assert!("1e-92233720368547758080".parse::<PreciseNumber>().is_ok());
}
#[test]
fn test_parse_max_exponents() {
// Make sure exponents much bigger than i64::MAX cause errors
assert!("1e9223372036854775807".parse::<PreciseNumber>().is_ok());
assert!("1e92233720368547758070".parse::<PreciseNumber>().is_err());
}
}
}
mod error {
//! Errors returned by seq.
// pi-uutils: `translate!` message lookups are literalized with the en-US
// strings from upstream's locales/en-US.ftl.
use thiserror::Error;
use uucore::display::Quotable;
use super::numberparse::ParseNumberError;
#[derive(Debug, Error)]
pub enum SeqError {
/// An error parsing the input arguments.
///
/// The parameters are the [`String`] argument as read from the
/// command line and the underlying parsing error itself.
#[error("invalid {} argument: {}", parse_error_type(.1), .0.quote())]
ParseError(String, ParseNumberError),
/// The increment argument was zero, which is not allowed.
///
/// The parameter is the increment argument as a [`String`] as read
/// from the command line.
#[error("invalid Zero increment value: {}", .0.quote())]
ZeroIncrement(String),
/// No arguments were passed to this function, 1 or more is required
#[error("missing operand")]
NoArguments,
/// Both a format and equal width where passed to seq
#[error("format string may not be specified when printing equal width strings")]
FormatAndEqualWidth,
}
fn parse_error_type(e: &ParseNumberError) -> &'static str {
match e {
ParseNumberError::Float => "floating point",
ParseNumberError::Nan => "'not-a-number'",
}
}
}
use self::{error::SeqError, number::PreciseNumber};
const OPT_SEPARATOR: &str = "separator";
const OPT_TERMINATOR: &str = "terminator";
const OPT_EQUAL_WIDTH: &str = "equal-width";
const OPT_FORMAT: &str = "format";
const ARG_NUMBERS: &str = "numbers";
/// How many emitted numbers to print between cancellation polls.
const CANCEL_POLL_INTERVAL: u64 = 4096;
#[derive(Clone)]
struct SeqOptions<'a> {
separator: OsString,
terminator: OsString,
equal_width: bool,
format: Option<&'a str>,
}
/// A range of floats.
///
/// The elements are (first, increment, last).
type RangeFloat = (ExtendedBigDecimal, ExtendedBigDecimal, ExtendedBigDecimal);
/// Turn short args with attached value, for example "-s,", into two args "-s"
/// and "," to make them work with clap.
fn split_short_args_with_value(args: Vec<OsString>) -> Vec<OsString> {
let mut v: Vec<OsString> = Vec::new();
for arg in args {
let bytes = arg.as_encoded_bytes();
if bytes.len() > 2
&& (bytes.starts_with(b"-f") || bytes.starts_with(b"-s") || bytes.starts_with(b"-t"))
{
let (short_arg, value) = bytes.split_at(2);
// SAFETY:
// Both `short_arg` and `value` only contain content that originated from
// `OsStr::as_encoded_bytes`
v.push(unsafe { OsString::from_encoded_bytes_unchecked(short_arg.to_vec()) });
v.push(unsafe { OsString::from_encoded_bytes_unchecked(value.to_vec()) });
} else {
v.push(arg);
}
}
v
}
fn select_precision(
first: &PreciseNumber,
increment: &PreciseNumber,
last: &PreciseNumber,
) -> Option<usize> {
match (first.num_fractional_digits, increment.num_fractional_digits, last.num_fractional_digits)
{
(Some(0), Some(0), Some(0)) => Some(0),
(Some(f), Some(i), Some(_)) => Some(f.max(i)),
_ => None,
}
}
/// Parsed `seq` invocation.
pub(crate) struct Seq {
matches: ArgMatches,
}
matches_parser!(Seq, uu_app);
impl Utility for Seq {
const NAME: &'static str = "seq";
fn rewrite_argv(argv: Vec<OsString>) -> Result<Vec<OsString>, String> {
Ok(split_short_args_with_value(argv))
}
fn run(self, host: &mut Host) -> i32 {
match seq_main(&self.matches, host) {
Ok(()) => host.exit_code(),
Err(err) => {
host.error(err, 1);
1
},
}
}
}
fn seq_main(matches: &ArgMatches, host: &mut Host) -> Result<(), Box<dyn Error>> {
let numbers_option = matches.get_many::<String>(ARG_NUMBERS);
if numbers_option.is_none() {
return Err(SeqError::NoArguments.into());
}
let numbers = numbers_option.unwrap().collect::<Vec<_>>();
let options = SeqOptions {
separator: matches
.get_one::<OsString>(OPT_SEPARATOR)
.cloned()
.unwrap_or_else(|| OsString::from("\n")),
terminator: matches
.get_one::<OsString>(OPT_TERMINATOR)
.cloned()
.unwrap_or_else(|| OsString::from("\n")),
equal_width: matches.get_flag(OPT_EQUAL_WIDTH),
format: matches.get_one::<String>(OPT_FORMAT).map(String::as_str),
};
if options.equal_width && options.format.is_some() {
return Err(SeqError::FormatAndEqualWidth.into());
}
let first = if numbers.len() > 1 {
match numbers[0].parse() {
Ok(num) => num,
Err(e) => return Err(SeqError::ParseError(numbers[0].to_owned(), e).into()),
}
} else {
PreciseNumber::one()
};
let increment = if numbers.len() > 2 {
match numbers[1].parse() {
Ok(num) => num,
Err(e) => return Err(SeqError::ParseError(numbers[1].to_owned(), e).into()),
}
} else {
PreciseNumber::one()
};
if increment.is_zero() {
return Err(SeqError::ZeroIncrement(numbers[1].to_owned()).into());
}
let last: PreciseNumber = {
// We are guaranteed that `numbers.len()` is greater than zero
// and at most three because of the argument specification in
// `uu_app()`.
let n: usize = numbers.len();
match numbers[n - 1].parse() {
Ok(num) => num,
Err(e) => return Err(SeqError::ParseError(numbers[n - 1].to_owned(), e).into()),
}
};
// If a format was passed on the command line, use that.
// If not, use some default format based on parameters precision.
let (format, padding, fast_allowed) = if let Some(str) = options.format {
(Format::<num_format::Float, &ExtendedBigDecimal>::parse(str)?, 0, false)
} else {
let precision = select_precision(&first, &increment, &last);
let padding = if options.equal_width {
let precision_value = precision.unwrap_or(0);
first
.num_integral_digits
.max(increment.num_integral_digits)
.max(last.num_integral_digits)
+ if precision_value > 0 {
precision_value + 1
} else {
0
}
} else {
0
};
let formatter = match precision {
// format with precision: decimal floats and integers
Some(precision) => num_format::Float {
variant: FloatVariant::Decimal,
width: padding,
alignment: num_format::NumberAlignment::RightZero,
precision: Some(precision),
..Default::default()
},
// format without precision: hexadecimal floats
None => num_format::Float { variant: FloatVariant::Shortest, ..Default::default() },
};
// Allow fast printing if precision is 0 (integer inputs), `print_seq` will do
// further checks.
(Format::from_formatter(formatter), padding, precision == Some(0))
};
let result = print_seq(
host,
(first.number, increment.number, last.number),
&options.separator,
&options.terminator,
&format,
fast_allowed,
padding,
);
match result {
Ok(()) => Ok(()),
Err(err) if err.kind() == std::io::ErrorKind::BrokenPipe => {
// GNU seq prints the Broken pipe message but still exits with status 0.
let _ = writeln!(host.stderr, "seq: write error: {err}");
Ok(())
},
Err(err) => Err(format!("write error: {err}").into()),
}
}
fn uu_app() -> Command {
Command::new(Seq::NAME)
.trailing_var_arg(true)
.infer_long_args(true)
.version("0.8.0")
.about("Display numbers from FIRST to LAST, in steps of INCREMENT.")
.override_usage(format_usage(
"seq [OPTION]... LAST\nseq [OPTION]... FIRST LAST\nseq [OPTION]... FIRST INCREMENT LAST",
))
.arg(
Arg::new(OPT_SEPARATOR)
.short('s')
.long("separator")
.help("Separator character (defaults to \\n)")
.value_parser(clap::value_parser!(OsString)),
)
.arg(
Arg::new(OPT_TERMINATOR)
.short('t')
.long("terminator")
.help("Terminator character (defaults to \\n)")
.value_parser(clap::value_parser!(OsString)),
)
.arg(
Arg::new(OPT_EQUAL_WIDTH)
.short('w')
.long("equal-width")
.help("Equalize widths of all numbers by padding with zeros")
.action(ArgAction::SetTrue),
)
.arg(
Arg::new(OPT_FORMAT)
.short('f')
.long(OPT_FORMAT)
.help("use printf style floating-point FORMAT"),
)
.arg(
// we use allow_hyphen_values instead of allow_negative_numbers because clap removed
// the support for "exotic" negative numbers like -.1 (see https://github.com/clap-rs/clap/discussions/5837)
Arg::new(ARG_NUMBERS)
.allow_hyphen_values(true)
.action(ArgAction::Append)
.num_args(1..=3),
)
}
/// Integer print, default format, positive increment: fast code path
/// that avoids reformatting digit at all iterations.
fn fast_print_seq(
host: &Host,
mut stdout: impl Write,
first: &BigUint,
increment: u64,
last: &BigUint,
separator: &OsStr,
terminator: &OsStr,
padding: usize,
) -> std::io::Result<()> {
// Nothing to do, just return.
if last < first {
return Ok(());
}
// Do at most u64::MAX loops. We can print in the order of 1e8 digits per
// second, u64::MAX is 1e19, so it'd take hundreds of years for this to
// complete anyway. TODO: we can move this test to `print_seq` if we care about
// this case.
let loop_cnt = ((last - first) / increment).to_u64().unwrap_or(u64::MAX);
// Format the first number.
let first_str = first.to_string();
// Makeshift log10.ceil
let last_length = last.to_string().len();
// Allocate a large u8 buffer, that contains a preformatted string
// of the number followed by the `separator`.
//
// | ... head space ... | number | separator |
// ^0 ^ start ^ num_end ^ size (==buf.len())
//
// We keep track of start in this buffer, as the number grows.
// When printing, we take a slice between start and end.
let size = last_length.max(padding) + separator.len();
// Fill with '0', this is needed for equal_width, and harmless otherwise.
let mut buf = vec![b'0'; size];
let buf = buf.as_mut_slice();
let num_end = buf.len() - separator.len();
let mut start = num_end - first_str.len();
// Initialize buf with first and separator.
buf[start..num_end].copy_from_slice(first_str.as_bytes());
buf[num_end..].copy_from_slice(separator.as_encoded_bytes());
// Normally, if padding is > 0, it should be equal to last_length,
// so start would be == 0, but there are corner cases.
start = start.min(num_end - padding);
// Prepare the number to increment with as a string
let inc_str = increment.to_string();
let inc_str = inc_str.as_bytes();
for i in 0..loop_cnt {
// Poll periodically so shell abort/timeout is observed.
if i % CANCEL_POLL_INTERVAL == 0 && host.is_cancelled() {
return Ok(());
}
stdout.write_all(&buf[start..])?;
fast_inc(buf, &mut start, num_end, inc_str);
}
// Write the last number without separator, but with terminator.
stdout.write_all(&buf[start..num_end])?;
stdout.write_all(terminator.as_encoded_bytes())?;
stdout.flush()?;
Ok(())
}
fn done_printing<T: Zero + PartialOrd>(next: &T, increment: &T, last: &T) -> bool {
if increment >= &T::zero() {
next > last
} else {
next < last
}
}
/// Arbitrary precision decimal number code path ("slow" path)
fn print_seq(
host: &Host,
range: RangeFloat,
separator: &OsStr,
terminator: &OsStr,
format: &Format<num_format::Float, &ExtendedBigDecimal>,
fast_allowed: bool,
padding: usize, // Used by fast path only
) -> std::io::Result<()> {
let mut stdout = BufWriter::new(host.stdout_clone());
let (first, increment, last) = range;
if fast_allowed {
// Test if we can use fast code path.
// First try to convert the range to BigUint (u64 for the increment).
let (first_bui, increment_u64, last_bui) =
(first.to_biguint(), increment.to_biguint().and_then(|x| x.to_u64()), last.to_biguint());
if let (Some(first_bui), Some(increment_u64), Some(last_bui)) =
(first_bui, increment_u64, last_bui)
{
return fast_print_seq(
host,
stdout,
&first_bui,
increment_u64,
&last_bui,
separator,
terminator,
padding,
);
}
}
let mut value = first;
let mut is_first_iteration = true;
let mut iterations: u64 = 0;
while !done_printing(&value, &increment, &last) {
// Poll periodically so shell abort/timeout is observed.
if iterations.is_multiple_of(CANCEL_POLL_INTERVAL) && host.is_cancelled() {
return Ok(());
}
iterations += 1;
if !is_first_iteration {
stdout.write_all(separator.as_encoded_bytes())?;
}
format.fmt(&mut stdout, &value)?;
// TODO Implement augmenting addition.
value = value + increment.clone();
is_first_iteration = false;
}
if !is_first_iteration {
stdout.write_all(terminator.as_encoded_bytes())?;
}
stdout.flush()?;
Ok(())
}
/// Creates the `seq` builtin registration.
pub(crate) fn seq_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Seq, SE>()
}
#[cfg(test)]
mod tests {
use clap::Parser;
use super::Seq;
use crate::host::{Host, Utility, run_util};
fn run(args: &[&str]) -> (i32, String, String) {
let (code, capture) = run_util::<Seq>(args, "", "/");
(code, capture.out(), capture.err())
}
#[test]
fn single_operand_counts_from_one() {
assert_eq!(run(&["3"]), (0, "1\n2\n3\n".into(), String::new()));
}
#[test]
fn first_increment_last_arithmetic() {
assert_eq!(run(&["2", "2", "10"]), (0, "2\n4\n6\n8\n10\n".into(), String::new()));
}
#[test]
fn separator_joins_values_terminator_ends_them() {
assert_eq!(run(&["-s", ",", "1", "3"]), (0, "1,2,3\n".into(), String::new()));
assert_eq!(run(&["-s,", "1", "3"]), (0, "1,2,3\n".into(), String::new()));
}
#[test]
fn equal_width_pads_with_zeros() {
assert_eq!(run(&["-w", "8", "10"]), (0, "08\n09\n10\n".into(), String::new()));
}
#[test]
fn float_increment_selects_widest_precision() {
assert_eq!(run(&["1", "0.5", "2"]), (0, "1.0\n1.5\n2.0\n".into(), String::new()));
}
#[test]
fn invalid_operand_reports_error_and_fails() {
assert_eq!(run(&["foo"]), (1, String::new(), "seq: invalid floating point argument: 'foo'\n".into()));
}
#[test]
fn zero_increment_is_rejected() {
assert_eq!(run(&["1", "0", "5"]), (1, String::new(), "seq: invalid Zero increment value: '0'\n".into()));
}
#[test]
fn custom_format_is_preserved() {
assert_eq!(run(&["-f", "%04.1f", "1", "2"]), (0, "01.0\n02.0\n".into(), String::new()));
}
#[test]
fn hexadecimal_float_parsing_is_preserved() {
assert_eq!(run(&["0x1p0", "0x1p0", "0x3p0"]), (0, "1\n2\n3\n".into(), String::new()));
}
#[test]
fn cancelled_host_stops_emission() {
let seq = Seq::try_parse_from(["seq", "1", "1000000"]).unwrap();
let (mut host, capture) = Host::for_test("seq", Vec::new(), "/");
host.cancel_for_test();
assert_eq!(seq.run(&mut host), 0);
assert_eq!(capture.out(), "");
assert_eq!(capture.err(), "");
}
#[test]
fn help_renders_to_stdout() {
let (code, capture) = run_util::<Seq>(&["--help"], "", "/");
assert_eq!(code, 0);
assert!(capture.out().contains("Usage:"));
assert!(capture.out().contains("steps of INCREMENT"));
assert_eq!(capture.err(), "");
}
}
crates/vendor/brush-builtins/src/set.rs → crates/pi-builtins/src/set.rs
View File
crates/pi-builtins/src/sha1sum.rs
+37
View File
@@ -0,0 +1,37 @@
//! `sha1sum` builtin: compute and check SHA-1 digests.
//!
//! Ported from uutils coreutils 0.8.0.
use brush_core::{ShellExtensions, builtins::Registration};
use clap::ArgMatches;
use uucore::checksum::AlgoKind;
use crate::{
cksum,
host::{Host, Utility, matches_parser, util},
};
/// Parsed `sha1sum` invocation.
pub(crate) struct Sha1sum {
matches: ArgMatches,
}
matches_parser!(Sha1sum, app);
impl Utility for Sha1sum {
const NAME: &'static str = "sha1sum";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
cksum::run(host, AlgoKind::Sha1, self.matches, None)
}
}
fn app() -> clap::Command {
cksum::command(Sha1sum::NAME, false)
}
/// Creates the `sha1sum` builtin registration.
pub(crate) fn sha1sum_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Sha1sum, SE>()
}
crates/pi-builtins/src/sha224sum.rs
+37
View File
@@ -0,0 +1,37 @@
//! `sha224sum` builtin: compute and check SHA-224 digests.
//!
//! Ported from uutils coreutils 0.8.0.
use brush_core::{ShellExtensions, builtins::Registration};
use clap::ArgMatches;
use uucore::checksum::AlgoKind;
use crate::{
cksum,
host::{Host, Utility, matches_parser, util},
};
/// Parsed `sha224sum` invocation.
pub(crate) struct Sha224sum {
matches: ArgMatches,
}
matches_parser!(Sha224sum, app);
impl Utility for Sha224sum {
const NAME: &'static str = "sha224sum";
const USAGE_ERROR: u8 = 2;
fn run(self, host: &mut Host) -> i32 {
cksum::run(host, AlgoKind::Sha224, self.matches, None)
}
}
fn app() -> clap::Command {
cksum::command(Sha224sum::NAME, false)
}
/// Creates the `sha224sum` builtin registration.
pub(crate) fn sha224sum_builtin<SE: ShellExtensions>() -> Registration<SE> {
util::<Sha224sum, SE>()
}

Some files were not shown because too many files have changed in this diff Show More