Create OSM All Street Network

http://twiki.psrc.local/Data/CreateOSMAll-StreetsNetwork

aysim uses a fine-grained "all-streets" network to calculate accessibility for activity modeling. This network is different than the model network and includes all local streets and small arterials, which are generally not used in the travel model due to size limitations in assignment. The all-streets network is also used to build a separate Daysim input, the node-to-node parcel distance file. This node-to-node file is built from the all-streets network. The process below describes the process of downloading and formatting the all-streets Open Street Maps (OSM) network, then building a node-to-node-distance input from that OSM network. The final results of this process will produce the following files: node_index_2014.txt node_to_node_distance_2014.h5 parcel_nodes_2014.txt

1. Download OSM data and build a shapefile: Go to https://www.openstreetmap.org/export#map=10/47.6534/-121.9812 Click on Export which will open up a panel where you can enter a bounding box. The coordinates for our region are: (46.73, -123.03, 48.3, -120.95) Download the data using the overpass API Use this arcgis editing tool to import the OSM data as a shapefile: https://www.arcgis.com/home/item.html?id=75716d933f1c40a784243198e0dc11a1

2. Create a Network from the OSM data: Use select by attributes to select the “types” you are interested in keeping. For 2014, I used the following query:

highway = 'trunk' OR highway = 'trunk_link' OR highway = 'motorway' OR highway = 'motorway_link' OR highway = 'primary' OR highway = 'primary_link' OR highway = 'residential' OR highway = 'road' OR highway = 'secondary' OR highway = 'service' OR highway = 'tertiary' OR highway = 'tertiary_link' OR highway = 'secondary_link' OR highway = 'cycleway' OR highway = 'footway' OR highway = 'pedestrian' OR highway = 'living_street' OR highway = 'unclassified' OR highway = 'steps'

OSM data does not come as a routeable network. To create one you must follow these steps:

• Split the shapefile into three separate layers – one that has the attribute bridge = 1, one that has tunnel = 1 and one that includes everything else (tunnel and bridge = 0). This is so that when you planarize the lines later, roads will only split where there’s an at-grade intersection. Bridge attributes can be found in the field man_made , e.g., man_made ==
• Planarize (advanced editing toolbar->Planarize lines) all of the links in each separate shapefile, and then merge each layer back into one layer.
• In ArcGIS , create a network dataset using the the layer from the previous step. This will create nodes at every intersection.
• Use Network Analyst Service Area tool to locate disconnected links. You can do this by turning polygon generation off and making sure line generation is turned on. Set the Default Breaks value to a very large number so that the solver will find links that are very far away. Use the “create network location” tool to create a location on the network and then run the solver. Lines will be generated from the point to every connected part of the network. You may have do this separately for each island (e.g. Vashon). At the end, export lines and merge them if necessary. Use this line featureclass to select feature in the all streets network by using SelectByLocation tool with the ‘share line segement’ as the spatial operation. The resulting selection is the connected network. Switch the selection to get the disconnected network. Delete/review these links. This is your final network. Recreate a network dataset using this final street network to create junctions. Create a node_id column in the nodes file and make it unique. One way is to use the field calculator and set the field equal to ObjectID .
• The next step is to add from_node_id and to_node_id fields to the street network file and update them. A non programmatic way to do this that worked well is described here: http://gis.stackexchange.com/questions/125090/assign-point-ids-to-respective-start-and-end-attributes-of-a-polyline You should now have a complete node file, with a node_id field, and an edges file with from_node_id and to_node_id fields. For 2014, these can be found here: R:\SoundCast\all_streets_2014\final_network.gdb\connected_network

3. Prep files or DTA lite process: Ensure that input_link.csv and input_node.csv match format example shown below. Note that the full example files can be found in R:\SoundCast\all_streets_2014\DaysimDataTools\2_Parcel_Buffering\2_DTALite. All of these attributes are populated with the same values. Just create the fields and populate them with these values. input_link.csv name from_node_id to_node_id direction speed_limit_in_mph link_type lane_capacity_in_vhc_per_hour link_id number_of_lanes length_in_mile Twin Firs Trail 254488 254491 0 25 2 2000 5 2 0.005573895 Twin Firs Trail 254491 254491 0 25 2 2000 6 2 0.405481889 input_node.csv node_id x y 1 1106751 197237 2 1107442 198973 The first step in the DTAlite process is a preprocessing step that creates a file indicating the node_ids that are within 3 miles of each node. This is a data reduction step because we don’t care about parcels/nodes that are beyond 3 miles from any given parcel/node. Furthermore, we don’t care about nodes that are not associated with a parcel. So in GIS, find the node closest to each parcel (use spatial join) and export this file in the same format as the samp_input_node.csv in this directory: R:\SoundCast\all_streets_2014\DaysimDataTools\2_Parcel_Buffering\1_NetworkDataPrep

4. Run Network_DataPrepv2.exe Read the readme.txt file in this location: R:\SoundCast\all_streets_2014\DaysimDataTools\2_Parcel_Buffering\1_NetworkDataPrep and make sure you have prepped the files correctly. Run Network_DataPrepv2.exe in R:\SoundCast\all_streets_2014\DaysimDataTools\2_Parcel_Buffering\1_NetworkDataPrep This will create an output file called input_od_pairs.csv, which are all the node combinations required to get the distance from a given parcel/node to every parcel/node within a 3 mile linear distance. Copy input_od_pairs.csv to R:\SoundCast\all_streets_2014\DaysimDataTools\2_Parcel_Buffering\2_DTALite

5. Run DTALite64.exe: Export your full node and edges file to match the input_node.csv and input_link.csv found in the dir listed in the alst step of 4. Make sure you include ALL nodes to fully represent the network. Read the readme file, make sure all of your inputs are there and in the correct format and Run DTALite64.exe The output is called output_shortest_path.txt and contains the network distance from each node to every other node (specified by the input_od_pairs file) within 3 miles.

6. Create files used for Soundcast: Parcel_nodes_2014.txt: Id column is the parcel_id and node_id is the node_id. Create this using the parcel to node correspondence produced in an earlier step. Node_index_2014.txt: This file is created by the script Soundcast/scripts/utils/create_node_node_index.py Node_to_node_distance_2014.txt This file is created by the script Soundcast/scripts/utils/create_node_node_index.py

7. Update PugetSoundNetwork.h5 used by accessibility.py: Using Python, Pandas and Pandana, create a Pandana network object using your edges and nodes files (read into memory as a pandas dataframe): http://udst.github.io/pandana/network.html Then write this to disk using the Network.save_hdf5 method: http://udst.github.io/pandana/loaders.html