Using Skim Dataset#

ActivitySim 1.2 offers two internal frameworks for managing skim data.

SkimDict, the legacy framework, that stores all skim data in one large omnibus array, with various offset lookups and tools to access values.
SkimDataset, an xarray.Dataset based framework, which mimics the SkimDict interface, and adds a number of features optimized specifically for use with sharrow. This framework is automatically used when sharrow is enabled, and there is no user configuration to enable it separately.

Skims in Shared Memory#

These two internal frameworks manage shared memory differently when running ActivitySim in multiprocessing mode.

For SkimDict, shared memory is used only when running with multiprocessing active, and is allocated via the allocate_shared_skim_buffers function, which in turn invokes Network_LOS.allocate_shared_skim_buffers.

For SkimDataset, shared memory is used regardless of whether multiprocessing is active or not. The shared memory allocation is done via the Dataset.shm.to_shared_memory method called at the end of the load_skim_dataset_to_shared_memory function.

Skim Dataset API#

class activitysim.core.skim_dataset.DatasetWrapper(dataset, orig_key, dest_key, time_key=None, *, time_map=None)#

Mimics the SkimWrapper interface to allow legacy code to access data.

Parameters

dataset: Dataset
orig_key, dest_key: str: name of columns in target dataframe to use as origin and destination lookups, respectively
time_keystr, optional
time_mapMapping, optional: A mapping from time period index numbers to (more aggregate) time period names.

lookup(key, reverse=False)#

Generally not called by the user - use __getitem__ instead

Parameters

keyhashable: The key (identifier) for this skim object
odbool (optional): od=True means lookup standard origin-destination skim value od=False means lookup destination-origin skim value

Returns

impedances: pd.Series: A Series of impedances which are elements of the Skim object and with the same index as df

max(key)#: return max skim value in either o-d or d-o direction

reverse(key)#: return skim value in reverse (d-o) direction

set_df(df)#

Set the dataframe

Parameters

dfDataFrame: The dataframe which contains the origin and destination ids

Returns

self (to facilitate chaining)

class activitysim.core.skim_dataset.SkimDataset(dataset)#

A wrapper around xarray.Dataset containing skim data, with time period management.

get_skim_usage()#

return set of keys of skims looked up. e.g. {‘DIST’, ‘SOV’}

Returns

set:

lookup(orig, dest, key)#

Return list of skim values of skims(s) at orig/dest in skim with the specified key (e.g. ‘DIST’)

Parameters

orig: list of orig zone_ids
dest: list of dest zone_ids
key: str

Returns

Numpy.ndarray: list of skim values for od pairs

wrap(orig_key, dest_key)#

Get a wrapper for the given keys.

Parameters

orig_key, dest_keystr

Returns

DatasetWrapper

wrap_3d(orig_key, dest_key, dim3_key)#

Get a 3d wrapper for the given keys.

Parameters

orig_key, dest_keystr

Returns

DatasetWrapper

activitysim.core.skim_dataset.load_skim_dataset_to_shared_memory(skim_tag='taz')#

Load skims from disk into shared memory.

Parameters

skim_tagstr, default “taz”

Returns

xarray.Dataset

activitysim.core.skim_dataset.load_sparse_maz_skims(dataset, land_use_index, remapper, zone_system, maz2taz_file_name, maz_to_maz_tables=(), max_blend_distance=None, data_file_resolver=None)#

Load sparse MAZ data on top of TAZ skim data.

Parameters

datasetxarray.Dataset: The existing dataset at TAZ resolution only.
land_use_indexpandas.Index: The index of the land use table. For two and three zone systems, these index values should be MAZ identifiers.
remapperdict, optional: A dictionary mapping where the keys are the original (nominal) zone id’s, and the values are the recoded (typically zero-based contiguous) zone id’s. Recoding improves runtime efficiency.
zone_systemint: Currently 1, 2 and 3 are supported.
maz2taz_file_namestr
maz_to_maz_tablesCollection[]
max_blend_distanceoptional
data_file_resolverfunction

Returns

xarray.Dataset

Using Sharrow

Workflows