read_resource_batches

Warning

🚧 Sprout is still in active development and evolving quickly, so the documentation and functionality may not work as described and could undergo substantial changes 🚧

read_resource_batches(
    resource_properties: ResourceProperties,
    paths: list[Path] | None = None,
)

Read all the batch resource file(s) into a list of (Polars) DataFrames.

Use this function to read the Parquet file(s) specified in paths into a list of Polars DataFrames, and perform checks on each of the DataFrames against the resource_properties. The resource_properties object is used to check the data and ensure it is correct. While Sprout generally assumes that the files stored in the resources/<name>/batch/ folder are already correctly structured and tidy, this function still runs checks to ensure the data are correct by comparing to the properties.

Parameters

resource_properties : ResourceProperties

The ResourceProperties object that contains the properties of the resource you want to check the data against.

paths : list[Path] | None = None

A list of paths for all the files in the resource’s batch/ folder. Use path_resource_batch_files() to help provide the correct paths to the batch files. Defaults to the batch files of the given resource.

Returns

list[pl.DataFrame]

A list of DataFrame objects from all the batch files.

Raises

FileNotFoundError

If a file in the list of paths doesn’t exist.

ValueError

If the batch file name is not in the expected pattern.

ValueError

If the timestamp column name matches an existing column in the DataFrame.

Examples

import seedcase_sprout as sp

with sp.ExamplePackage():
    resource_properties = sp.example_resource_properties()
    sp.write_resource_batch(sp.example_data(), resource_properties)

    sp.read_resource_batches(resource_properties)

PosixPath('/tmp/tmpaske_8qi/example-package/resources/example-resource/batch/2025-06-26T013034Z-b960286f-be01-4845-9992-34b656f411f2.parquet')

[shape: (3, 4)
 ┌─────┬──────────┬──────────┬────────────────────────┐
 │ id  ┆ name     ┆ value    ┆ _batch_file_timestamp_ │
 │ --- ┆ ---      ┆ ---      ┆ ---                    │
 │ i64 ┆ str      ┆ f64      ┆ str                    │
 ╞═════╪══════════╪══════════╪════════════════════════╡
 │ 34  ┆ Helly R  ┆ 123.123  ┆ 2025-06-26T013034Z     │
 │ 99  ┆ Mark S   ┆ 9988.0   ┆ 2025-06-26T013034Z     │
 │ 100 ┆ Ms Casey ┆ -76.0009 ┆ 2025-06-26T013034Z     │
 └─────┴──────────┴──────────┴────────────────────────┘]