Shadow Pricing#
The shadow pricing calculator used by work and school location choice.
Structure#
Configuration File:
shadow_pricing.yaml
Turning on and saving shadow prices#
Shadow pricing is activated by setting the use_shadow_pricing
to True in the settings.yaml file.
Once this setting has been activated, ActivitySim will search for shadow pricing configuration in
the shadow_pricing.yaml file. When shadow pricing is activated, the shadow pricing outputs will be
exported by the tracing engine. As a result, the shadow pricing output files will be prepended with
trace
followed by the iteration number the results represent. For example, the shadow pricing
outputs for iteration 3 of the school location model will be called
trace.shadow_price_school_shadow_prices_3.csv
.
In total, ActivitySim generates three types of output files for each model with shadow pricing:
trace.shadow_price_<model>_desired_size.csv
The size terms by zone that the ctramp and daysim methods are attempting to target. These equal the size term columns in the land use data multiplied by size term coefficients.trace.shadow_price_<model>_modeled_size_<iteration>.csv
These are the modeled size terms after the iteration of shadow pricing identified by thenumber. In other words, these are the predicted choices by zone and segment for the model after the iteration completes. (Not applicable for simulation
option.)trace.shadow_price_<model>_shadow_prices_<iteration>.csv
The actual shadow price for each zone and segment after theof shadow pricing. This is the file that can be used to warm start the shadow pricing mechanism in ActivitySim. (Not applicable for simulation
option.)
There are three shadow pricing methods in activitysim: ctramp
, daysim
, and simulation
.
The first two methods try to match model output with workplace/school location model size terms,
while the last method matches model output with actual employment/enrollmment data.
The simulation approach operates the following steps. First, every worker / student will be assigned without shadow prices applied. The modeled share and the target share for each zone are compared. If the zone is overassigned, a sample of people from the over-assigned zones will be selected for re-simulation. Shadow prices are set to -999 for the next iteration for overassigned zones which removes the zone from the set of alternatives in the next iteration. The sampled people will then be forced to choose from one of the under-assigned zones that still have the initial shadow price of 0. (In this approach, the shadow price variable is really just a switch turning that zone on or off for selection in the subsequent iterations. For this reason, warm-start functionality for this approach is not applicable.) This process repeats until the overall convergence criteria is met or the maximum number of allowed iterations is reached.
Because the simulation approach only re-simulates workers / students who were over-assigned in the previous iteration, run time is significantly less (~90%) than the CTRAMP or DaySim approaches which re-simulate all workers and students at each iteration.
Configuration#
- settings activitysim.abm.tables.shadow_pricing.ShadowPriceSettings#
Bases:
PydanticReadable
Settings used for shadow pricing.
- Config:
extra: str = forbid
- Fields:
SHADOW_PRICE_METHOD (Literal['ctramp', 'daysim', 'simulation'])
source_file_paths ()
- field LOAD_SAVED_SHADOW_PRICES: bool = True#
Global switch to enable/disable loading of saved shadow prices.
This is ignored if global use_shadow_pricing switch is False
- field MAX_ITERATIONS_SAVED: int = 1#
Number of shadow price iterations for warm start.
A warm start means saved shadow_prices were found in a file and loaded.
- field PERCENT_TOLERANCE: float = 5#
zone passes if modeled is within percent_tolerance of predicted_size
- field SEGMENT_TO_NAME: dict[str, str] = {'school': 'school_segment', 'workplace': 'income_segment'}#
Mapping from model_selector to persons_segment_name.
- field SHADOW_PRICE_METHOD: Literal['ctramp', 'daysim', 'simulation'] = 'ctramp'#
- field TARGET_THRESHOLD: float = 20#
ignore criteria for zones smaller than target_threshold (total employmnet or enrollment)
Examples#
Implementation#
- activitysim.abm.tables.shadow_pricing.ShadowPriceCalculator(state: State, model_settings: TourLocationComponentSettings, num_processes, shared_data=None, shared_data_lock=None, shared_data_choice=None, shared_data_choice_lock=None, shared_sp_choice_df=None)#
- activitysim.abm.tables.shadow_pricing.buffers_for_shadow_pricing(shadow_pricing_info)#
Allocate shared_data buffers for multiprocess shadow pricing
Allocates one buffer per model_selector. Buffer datatype and shape specified by shadow_pricing_info
buffers are multiprocessing.Array (RawArray protected by a multiprocessing.Lock wrapper) We don’t actually use the wrapped version as it slows access down and doesn’t provide protection for numpy-wrapped arrays, but it does provide a convenient way to bundle RawArray and an associated lock. (ShadowPriceCalculator uses the lock to coordinate access to the numpy-wrapped RawArray.)
- Parameters:
- shadow_pricing_infodict
- Returns:
- data_buffersdict {<model_selector><shared_data_buffer>}
- dict of multiprocessing.Array keyed by model_selector
- activitysim.abm.tables.shadow_pricing.buffers_for_shadow_pricing_choice(state, shadow_pricing_choice_info)#
Same as above buffers_for_shadow_price function except now we need to store the actual choices for the simulation based shadow pricing method
This allocates a multiprocessing.Array that can store the choice for each person and then wraps a dataframe around it. That means the dataframe can be shared and accessed across all threads. Parameters ———- shadow_pricing_info : dict Returns ——-
data_buffers : dict {<model_selector> : <shared_data_buffer>} dict of multiprocessing.Array keyed by model_selector
and wrapped in a pandas dataframe
- activitysim.abm.tables.shadow_pricing.shadow_price_data_from_buffers_choice(data_buffers, shadow_pricing_info, model_selector)#
- Parameters:
- data_buffersdict of {<model_selector><multiprocessing.Array>}
multiprocessing.Array is simply a convenient way to bundle Array and Lock we extract the lock and wrap the RawArray in a numpy array for convenience in indexing The shared data buffer has shape (<num_zones, <num_segments> + 1) extra column is for reverse semaphores with TALLY_CHECKIN and TALLY_CHECKOUT
- shadow_pricing_infodict
- dict of useful info
dtype: sp_dtype, block_shapes : OrderedDict({<model_selector>: <shape tuple>}) dict mapping model_selector to block shape (including extra column for semaphores) e.g. {‘school’: (num_zones, num_segments + 1)
- model_selectorstr
location type model_selector (e.g. school or workplace)
- Returns:
- shared_data, shared_data_lock
shared_data : multiprocessing.Array or None (if single process) shared_data_lock : numpy array wrapping multiprocessing.RawArray or None (if single process)
- activitysim.abm.tables.shadow_pricing.shadow_price_data_from_buffers(data_buffers, shadow_pricing_info, model_selector)#
- Parameters:
- data_buffersdict of {<model_selector><multiprocessing.Array>}
multiprocessing.Array is simply a convenient way to bundle Array and Lock we extract the lock and wrap the RawArray in a numpy array for convenience in indexing The shared data buffer has shape (<num_zones, <num_segments> + 1) extra column is for reverse semaphores with TALLY_CHECKIN and TALLY_CHECKOUT
- shadow_pricing_infodict
- dict of useful info
dtype: sp_dtype, block_shapes : OrderedDict({<model_selector>: <shape tuple>}) dict mapping model_selector to block shape (including extra column for semaphores) e.g. {‘school’: (num_zones, num_segments + 1)
- model_selectorstr
location type model_selector (e.g. school or workplace)
- Returns:
- shared_data, shared_data_lock
shared_data : multiprocessing.Array or None (if single process) shared_data_lock : numpy array wrapping multiprocessing.RawArray or None (if single process)
- activitysim.abm.tables.shadow_pricing.load_shadow_price_calculator(state: State, model_settings: TourLocationComponentSettings)#
Initialize ShadowPriceCalculator for model_selector (e.g. school or workplace)
If multiprocessing, get the shared_data buffer to coordinate global_desired_size calculation across sub-processes
- Parameters:
- stateworkflow.State
- model_settingsTourLocationComponentSettings
- Returns:
- spcShadowPriceCalculator
- activitysim.abm.tables.shadow_pricing.add_size_tables(state: State, disaggregate_suffixes: dict[str, Any], scale: bool = True) None #
inject tour_destination_size_terms tables for each model_selector (e.g. school, workplace)
Size tables are pandas dataframes with locations counts for model_selector by zone and segment tour_destination_size_terms
if using shadow pricing, we scale size_table counts to sample population (in which case, they have to be created while single-process)
Scaling is problematic as it breaks household result replicability across sample sizes It also changes the magnitude of the size terms so if they are used as utilities in expression files, their importance will diminish relative to other utilities as the sample size decreases.
Scaling makes most sense for a full sample in conjunction with shadow pricing, where shadow prices can be adjusted iteratively to bring modelled counts into line with desired (size table) counts.
- activitysim.abm.tables.shadow_pricing.get_shadow_pricing_info(state)#
return dict with info about dtype and shapes of desired and modeled size tables
block shape is (num_zones, num_segments + 1)
- Returns:
- shadow_pricing_info: dict
dtype: <sp_dtype>, block_shapes: dict {<model_selector>: <block_shape>}
- activitysim.abm.tables.shadow_pricing.get_shadow_pricing_choice_info(state)#
return dict with info about dtype and shapes of desired and modeled size tables
block shape is (num_zones, num_segments + 1)
- Returns:
- shadow_pricing_info: dict
dtype: <sp_dtype>, block_shapes: dict {<model_selector>: <block_shape>}