Standard plugin · import "fs"

Filesystem

List directory entries as an ordinary table, then use Ibex filters and row-wise map to build repeatable file jobs.

Function

`fs::list`

fs::list(dir: String, pattern: String = "*", recursive: Bool = false) -> DataFrame

Returns one row for each matching entry under dir, ordered by its path. The directory must exist. A path that is missing or is not a directory raises a runtime error.

Column	Type	Meaning
`path`	String	Path to the entry, with forward slashes in the returned representation.
`name`	String	Entry filename, including its final extension.
`stem`	String	Filename without its final extension. For `prices.daily.csv`, the stem is `prices.daily`.
`ext`	String	Final extension without the dot; empty when the filename has no extension.
`size_bytes`	Int64	File size in bytes. Directories report zero.
`is_dir`	Bool	Whether the entry is a directory.

Arguments

Choose which entries to return

`dir`

Required directory to enumerate. The listing is non-recursive unless the third argument is true.

`pattern`

Optional shell-style glob matched against each entry’s filename only. Defaults to *, which matches all names at that level.

`recursive`

Optional Boolean. When true, include entries in descendant directories. Matching still applies to each entry’s name.

Supported glob syntax

* matches any run of characters, ? matches one character, and bracket classes such as [abc] or [a-z] match one character in a set or range.

import "fs";

let files = fs::list("data", "*.csv", true);
files[filter !is_dir, select { path, stem, ext, size_bytes }];

Batch processing

Use the listing as a file job table

Each row of fs::list can feed a row-wise map. Within the block, path, stem, and the other fields are scalar values for that entry. The fields returned by the block form a result row, so the output table can record both the source and the work performed.

import "csv";
import "fs";
import "parquet";

fs::list("data/csv", "*.csv")[map {
    source = path,
    target = `data/parquet/${stem}.parquet`,
    rows = parquet::write(
        csv::read(path),
        `data/parquet/${stem}.parquet`
    )
}];

For an input named sales.csv, stem is sales, so the target becomes data/parquet/sales.parquet. parquet::write returns the row count, which becomes the rows value in that job’s result.

Related docs

Continue

See the I/O guide for the full CSV-to-Parquet example, and the language guide for table clauses and map.