ParquetStore#
- class activitysim.core.workflow.checkpoint.ParquetStore(directory: Path, mode: str = 'a', gitignore: bool = True)#
Storage interface for parquet-based table storage.
This store will store each saved table in a parquet-format archive, resulting in a hierarchy of separate files in a defined structure, as opposed to a single monolithic repository files containing all the data.
This interface will fall back to storing tables in a gzipped pickle if the parquet format fails (as might happen if datatypes for some columns are not homogenous and values are stored as “object”).
Methods
__init__
(directory[, mode, gitignore])Initialize a storage interface for parquet-based table storage.
close
()Close this store.
from_hdf
(source_filename, dest_filename[, mode])Create a new checkpoint store from an existing HdfStore.
get_dataframe
(table_name[, checkpoint_name])Load table from store as a pandas DataFrame.
Get a list of all checkpoint names in this store.
make_zip_archive
(output_filename)Compress this pipeline into a zip archive.
put
(table_name, df[, complib, checkpoint_name])Store a table.
wipe
()Remove this store, including all subdirectories.
Attributes
The directory location of this ParquetStore.
This store is open.
This store is read-only.