Merge pull request #1252 from cal-itp/hqta-explore-2024

2024 HQTA Methodology Revisions

high_quality_transit_areas/README.md

@@ -11,32 +11,59 @@ These datasets are updated **monthly**. We usually use a Wednesday in the middle
 
 ## Understanding the Open Data
 
-Per statute, High Quality Transit Areas are defined as the _areas surrounding_ high quality transit corridors and major transit stops. Therefore, if you're looking for our most authoratative data on where HQTAs are within California, please start with the HQTA Areas (polygon) dataset.
+### Frequency Standards have Diverged
 
-Major transit stop is a more restrictive definition that high quality transit corridor. A high quality corridor only requires that there be service in that corridor meeting the frequency standard. It need not be one route alone meeting the frequency standard – therefore we cut corridors into segments and make the high quality corridor determination based on the stop with the most trips in that segment (even if those trips are on different routes). It is possible and valid for a high quality corridor to contain no major transit stops at all.
+Assembly Bill 2553(2024) is changing the definition of a bus major transit stop to be the intersection of two corridors with a service interval (headway) of every 20 minutes (3 trips/hr), instead of every 15 minutes (4 trips/hr). Although the bill formally takes effect January 1, 2025, our dataset now incorporates this change.
 
-On the other hand, major transit stops exist at the intersection of two high-quality transit corridors as described above, and also at any rail station, most ferry terminals, and select BRT stations.
+This means that there are now two frequency standards captured by this dataset:
 
-Using the HQTA Areas dataset, it's possible to determine if an area qualifies because it is a high quality corridor, major transit stop, or both. This is useful for certain kinds of analyses, since some statutes and programs only reference major transit stops and don't include high quality corridors.
+* 4 trips/hr (15min headway) for high quality transit corridors, as well as bus rapid transit major stops (along with other requirements for BRT)
+* 3 trips/hr (20min headway) for bus corridors intersecting to form a bus major stop
 
-We provide the HQTA Stops dataset as a convienience for certain kinds of analysis where it would be helpful to know actual stop locations. Note that the `hq_corridor_bus` type includes all stops in a high quality corridor. Since we make the high quality corridor determination at the corridor segment level and only then find the stops within that corridor, not every `hq_corridor_stop` is guaranteed to have frequent service (though they could all have frequent service, and at least one stop every 1,250 meters will have frequent service).
+With this change, bus major stops are no longer a subset of high quality transit corridors, since the intersection of two corridors with frequencies of 3 trips/hr now creates a major stop despite neither being a high quality transit corridor.
 
-### Additional Details
-* The AM peak period is defined as 12:00 AM - 11:59 AM. The PM peak period is defined as 12:00 PM - 11:59 PM.
-    * Frequencies are calculated for each stop per hour. The highest frequency in the period before noon is counted for AM peak frequency; the highest frequency in the period after noon is counted for PM peak frequency.
+For more details, see [links](#High-Quality-Transit-Areas-Relevant-Statutes) to each statute below.
+
+### How We Count Frequencies
+
+* The AM peak period is defined as 6:00 AM - 8:59 AM. The PM peak period is defined as 3:00 PM - 6:59 PM.
+    * Average hourly frequencies are calculated for each stop per peak period.
 * Corridors are cut for each operator-route-direction every 1,250 meters.
    * Stops have a 50 meter buffer drawn around them and are spatially joined to corridors.
-   * For each 1,250 meter corridor, the highest frequency is counted (if there are multiple stops).
-*  `peak_trips`: the number of trips per hour are calculated separately for the AM peak and PM peak. This column should be interpreted as: *at least x trips per hour in the peak period*. 
-    * If these values differ, the *lesser* value is reported. 
-    * Ex: 5 AM peak trips, 4 PM peak trips would show a value of `peak_trips = 4`.
+   * For each 1,250 meter corridor segment, the highest frequency stop is counted (if there are multiple stops).
+* We count _all_ trips at the highest frequency stop in each corridor segment, even if they're marketed as different "routes".
+    * For example, if Route A serves the stop twice an hour, and Route B twice an hour, we consider the stop frequency to be 4 trips/hour, qualifying it as a high quality transit corridor and potentially a major bus stop (if an intersecting corridor is present)
+
+### Why Not one "Route" at a Time?
+
+![Long Beach Transit map as of October 2024, focusing on the area south of Willow St and east from Magnolia Ave to Lakewood Bl](img/lbt_map_2024.png)
+
+There is not a consistent practice as to what constitutes one "route" in the transit industry. A single "route" can have some trips that deviate from the usual routing without changing the route name or number, or have some trips only serve part of the route (shortlining). Multiple "routes" can combine to serve the same corridor and stops for many miles. The latter practice, often called interlining, is especially common, as is having local and rapid routes that take the same path but with different stop spacing.
+
+In the above example, Atlantic Ave is served by only Route 61, while Pacific Coast Highway is served by Routes 171, 172, and 173. Those routes combine to equal a higher frequency for riders travelling along PCH than along Atlantic, yet if we only considered one "route" at a time we would categorize Atlantic as a high quality corridor and not PCH (nor Anaheim and 7th, which function the same). We believe this would inaccurately reflect the transit rider experience.
+
+By considering each corridor segment seperately, our methodology is sensitive to where "routes" combine to offer high frequencies, as well as to where they diverge (or where a single route shortlines) and no longer meet the relevant frequency standard.
+
+### HQTA Areas is the Primary Dataset
+
+Per statute, High Quality Transit Areas are defined as the _half-mile surrounding_ high quality transit corridors and major transit stops. Our HQTA Areas (polygon) dataset already incorporates this half-mile buffer.
+
+Using the HQTA Areas dataset, it's possible to determine if an area qualifies because it is a high quality transit corridor, major transit stop, or both. This is useful for certain kinds of analyses, since some statutes and programs only reference major transit stops and don't include high quality transit corridors.
+
+We provide the HQTA Stops dataset as a convienience for certain kinds of analysis where it would be helpful to know actual stop locations. Note that the `hq_corridor_bus` type includes all stops in a high quality corridor. Since we make the high quality corridor determination at the corridor segment level and only then find the stops within that corridor, not every `hq_corridor_stop` is guaranteed to have frequent service (though they could all have frequent service, and at least one stop every 1,250 meters will have frequent service).
+
+### Additional Details
+
 * `hqta_type = hq_corridor_bus`
     * `hqta_details` is `corridor_frequent_stop` if the stop has at least 4 peak trips, `corridor_other_stop` otherwise.
 * `hqta_type = major_stop_bus`
-    * Major stop bus is designation of stops appearing within 50 meters of the intersection of 2 frequent corridors. In our analysis, corridors have an operator and route associated to track it.
+    * Major stop bus is designation of stops appearing within 500 feet of the intersection of 2 "major stop precursor corridors" with frequencies of at least 3 trips/hr. In our analysis, corridors have an operator and route associated to track it. All stops within 500 feet of the intersection are collectively considered the "major transit stop", and the half-mile major transit stop buffer extends from each stop and is dissolved into a single polygon.
     * If the intersection was between 2 different operators (different agency names), `hqta_details = intersection_2_bus_routes_different_operators`.
     * If the intersection was between routes of the same operator, `hqta_details = intersection_2_bus_routes_same_operator`
 * A half-mile buffered dataset is provided for high quality transit areas. We recommend using the polygon dataset. To get the stops (points) that create these half mile polygons, we do also provide high quality transit stops.
+*  `peak_trips`: This column should be interpreted as: *at least x trips per hour in the peak period*. 
+    * If AM and PM values differ, the *lesser* value is reported. 
+    * Ex: 5 AM peak trips, 4 PM peak trips would show a value of `peak_trips = 4`.
 
 [![hqta_methodology_mermaid](https://mermaid.ink/img/pako:eNqFkk1rwzAMhv-K8Ai5NNBLGWQw6FduY7CWnQxFjZXVkNhBVthK6X9fmqxry0h3sYX0SNjvq4PKvSGVqiRJtBMrJaXwWpODBQpCAvMprBldsAIr8XUAdAbefCMUtOuaouigHYB1VlLoQoBYdlRRnEK8xUDx6Dr7jmxxW1KIf_G2VLOtkPdzX3o-9T0sJ9kkm55bL8SavuRCjcfjv8jMsyEegkrraKgWKPfO3L4jyx6XsytGiMXeIEVRxH35eLra4xhF2mlXlP4z3yELrGdPPSFsWxGT5Bly3Eiv7IY7PX-IsMOaBpA7M8LJnfOIzql_gI3YigYoNVIVcYXWtKvRmaRVZ55WaRsaKrApRav2py2KjfjV3uUqFW5opJraoNDC4gdjdU6SseL5pd-2bumO3-XAyjk?type=png)](https://mermaid.live/edit#pako:eNqFkk1rwzAMhv-K8Ai5NNBLGWQw6FduY7CWnQxFjZXVkNhBVthK6X9fmqxry0h3sYX0SNjvq4PKvSGVqiRJtBMrJaXwWpODBQpCAvMprBldsAIr8XUAdAbefCMUtOuaouigHYB1VlLoQoBYdlRRnEK8xUDx6Dr7jmxxW1KIf_G2VLOtkPdzX3o-9T0sJ9kkm55bL8SavuRCjcfjv8jMsyEegkrraKgWKPfO3L4jyx6XsytGiMXeIEVRxH35eLra4xhF2mlXlP4z3yELrGdPPSFsWxGT5Bly3Eiv7IY7PX-IsMOaBpA7M8LJnfOIzql_gI3YigYoNVIVcYXWtKvRmaRVZ55WaRsaKrApRav2py2KjfjV3uUqFW5opJraoNDC4gdjdU6SseL5pd-2bumO3-XAyjk)
 
@@ -45,13 +72,13 @@ We provide the HQTA Stops dataset as a convienience for certain kinds of analysi
 [PRC 21155](https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?sectionNum=21155.&lawCode=PRC)
 * Major transit stop definition: _A major transit stop is as defined in Section 21064.3, except that, for purposes of this section, it also includes major transit stops that are included in the applicable regional transportation plan_
 * High-quality transit corridor definition: _For purposes of this section, a high-quality transit corridor means a corridor with fixed route bus service with service intervals no longer than 15 minutes during peak commute hours._
-    * Statute does not define "peak commute hours", given that peaks may vary locally we look for any hour in the morning plus any hour in the afternoon/evening.
+    * Statute does not define "peak commute hours", however we've started using fixed peak periods to align our methodology with other stateholders.
 
 [PRC 21064.3](https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?sectionNum=21064.3.&lawCode=PRC)
 * _Major transit stop means a site containing any of the following:
 (a) An existing rail or bus rapid transit station.
 (b) A ferry terminal served by either a bus or rail transit service.
-(c) The intersection of two or more major bus routes with a frequency of service interval of 15 minutes or less during the morning and afternoon peak commute periods._
+(c) The intersection of two or more major bus routes with a frequency of service interval of ~15~ 20 minutes or less during the morning and afternoon peak commute periods._
     * "Intersection" may not be sufficiently well-defined for this analysis
 
 [PRC 21060.2](https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=PRC&sectionNum=21060.2.&highlight=true&keyword=bus%20rapid%20transit)

high_quality_transit_areas/assemble_hqta_points.py

@@ -36,7 +36,7 @@ def combine_stops_by_hq_types(crs: str) -> gpd.GeoDataFrame:
     major_stop_bus = catalog.major_stop_bus.read().to_crs(crs)
     stops_in_corridor = catalog.stops_in_hq_corr.read().to_crs(crs)
 
-    trip_count_cols = ["am_max_trips", "pm_max_trips"]
+    trip_count_cols = ["am_max_trips_hr", "pm_max_trips_hr"]
 
     max_arrivals = pd.read_parquet(
         f"{GCS_FILE_PATH}max_arrivals_by_stop.parquet", 

high_quality_transit_areas/create_bus_hqta_types.py

@@ -21,7 +21,8 @@
 from calitp_data_analysis import utils
 from segment_speed_utils import helpers
 from update_vars import (GCS_FILE_PATH, analysis_date, 
-                         PROJECT_CRS, SEGMENT_BUFFER_METERS
+                         PROJECT_CRS, SEGMENT_BUFFER_METERS,
+                         INTERSECTION_BUFFER_METERS
                         )
 
 def buffer_around_intersections(buffer_size: int) -> gpd.GeoDataFrame: 
@@ -132,7 +133,7 @@ def create_stops_along_corridors(all_stops: gpd.GeoDataFrame) -> gpd.GeoDataFram
 
     # Start with the gdf of all the hqta_segments
     # that have a sjoin with an orthogonal route
-    bus_intersections = buffer_around_intersections(SEGMENT_BUFFER_METERS)
+    bus_intersections = buffer_around_intersections(INTERSECTION_BUFFER_METERS)
 
     # Grab point geom with all stops
     gtfs_keys = helpers.import_scheduled_trips(

high_quality_transit_areas/logs/hqta_processing.log

@@ -37,3 +37,15 @@
 2024-09-25 10:25:48.480 | INFO     | __main__:<module>:175 - C3_create_bus_hqta_types 2024-09-18 execution time: 0:00:26.100650
 2024-09-25 10:27:51.226 | INFO     | __main__:<module>:219 - D1_assemble_hqta_points 2024-09-18 execution time: 0:00:18.606053
 2024-09-25 10:30:59.443 | INFO     | __main__:<module>:168 - D2_assemble_hqta_polygons 2024-09-18 execution time: 0:00:30.215506
+2024-10-09 10:58:41.588 | INFO     | __main__:<module>:275 - A1_rail_ferry_brt_stops 2024-09-18 execution time: 0:00:19.051895
+2024-10-09 11:03:32.388 | INFO     | __main__:<module>:248 - B1_create_hqta_segments execution time: 0:04:32.849858
+2024-10-09 11:04:45.205 | INFO     | __main__:<module>:301 - B2_sjoin_stops_to_segments 2024-09-18 execution time: 0:00:53.314014
+2024-10-09 11:05:12.682 | INFO     | __main__:<module>:140 - C1_prep_pairwise_intersections 2024-09-18 execution time: 0:00:08.100040
+2024-10-10 09:43:19.078 | INFO     | __main__:<module>:275 - A1_rail_ferry_brt_stops 2024-09-18 execution time: 0:00:28.597295
+2024-10-10 09:48:33.530 | INFO     | __main__:<module>:248 - B1_create_hqta_segments execution time: 0:04:48.617534
+2024-10-10 09:49:52.721 | INFO     | __main__:<module>:301 - B2_sjoin_stops_to_segments 2024-09-18 execution time: 0:00:59.880352
+2024-10-10 09:50:22.959 | INFO     | __main__:<module>:140 - C1_prep_pairwise_intersections 2024-09-18 execution time: 0:00:09.107672
+2024-10-10 09:51:12.460 | INFO     | __main__:<module>:124 - C2_find_intersections 2024-09-18 execution time: 0:00:29.617306
+2024-10-10 10:02:25.556 | INFO     | __main__:<module>:175 - C3_create_bus_hqta_types 2024-09-18 execution time: 0:00:30.074431
+2024-10-10 10:03:17.217 | INFO     | __main__:<module>:219 - D1_assemble_hqta_points 2024-09-18 execution time: 0:00:21.338133
+2024-10-10 10:04:22.212 | INFO     | __main__:<module>:168 - D2_assemble_hqta_polygons 2024-09-18 execution time: 0:00:27.841877

high_quality_transit_areas/prep_pairwise_intersections.py

@@ -17,22 +17,29 @@
 from calitp_data_analysis import utils
 from update_vars import GCS_FILE_PATH, analysis_date
 
-def prep_bus_corridors(is_hq_corr: bool) -> gpd.GeoDataFrame:
+def prep_bus_corridors(is_ms_precursor: bool = False, is_hq_corr: bool = False) -> gpd.GeoDataFrame:
     """
-    Import all hqta segments with hq_transit_corr tagged.
-    
-    Only keep if hq_transit_corr == True
+    Import all hqta segments with ms_precursor or is_hq_corr tagged.
+    Since these definitions have diverged, may be called to return either.
+    Only keep if ms_precursor or hq_transit_corr == True
     """
-    bus_hqtc = gpd.read_parquet(
+    assert is_ms_precursor or is_hq_corr, 'must select exactly one category of segment'
+    if is_ms_precursor:
+        hqtc_or_ms_pre = gpd.read_parquet(
+            f"{GCS_FILE_PATH}all_bus.parquet",
+            filters = [[("ms_precursor", "==", is_ms_precursor)]]
+        ).reset_index(drop=True)
+    elif is_hq_corr:
+        hqtc_or_ms_pre = gpd.read_parquet(
         f"{GCS_FILE_PATH}all_bus.parquet",
         filters = [[("hq_transit_corr", "==", is_hq_corr)]]
     ).reset_index(drop=True)
-
-    bus_hqtc = bus_hqtc.assign(
+        
+    hqtc_or_ms_pre = hqtc_or_ms_pre.assign(
         route_type = "3"
     )
 
-    return bus_hqtc
+    return hqtc_or_ms_pre
 
 
 def sjoin_against_other_operators(
@@ -132,7 +139,7 @@ def pairwise_intersections(
 
     start = datetime.datetime.now()
 
-    corridors = prep_bus_corridors(is_hq_corr=True)   
+    corridors = prep_bus_corridors(is_ms_precursor=True)   
 
     pairwise_intersections(corridors)    
 

high_quality_transit_areas/rail_ferry_brt_stops.py

@@ -147,8 +147,10 @@ def grab_ferry_stops(
     """
     Grab all the ferry stops.
     """    
-    # only stops without bus service
-    angel_and_alcatraz = ['2483552', '2483550', '43002']         
+    # only stops without bus or rail service
+    angel_and_alcatraz = ['2483552', '2483550', '43002', 'AIF']
+    havasu = ['havasuLanding', 'londonBridge'] #  also one of these is in AZ
+    no_bus_or_rail = angel_and_alcatraz + havasu
 
     return gdf[
         (gdf.route_type.isin(route_types)) &