There are inherent trade-offs when you want to model a large study area such as an entire region or an entire country: 1) model precision vs 2) area size vs 3) memory/speed. You need to trade off one of these three.
For the first, you can model a coarser-grained network, such as only major roads in the region/country, rather than millions of fine-grained residential streets and paths. For the second, you can study a smaller area. For the third, you can provision a machine with lots of memory and then let the script run for a while to complete the process. What you trade off will be up to your own needs for this analysis.
In the example code below, I chose to trade off #1: I've modeled this region (West Midlands) by its motorways and trunk roads. Given a different analytical goal, you may trade off other things instead. After creating the model, I randomly sample 1000 origin and destination lat-long points, snap them to the nearest nodes in the graph, and solve the shortest paths by travel time (accounting for speed limits) with multiprocessing.
import osmnx as ox
# get boundaries of West Midlands region by its OSM ID
gdf = ox.geocode_to_gdf('R151283', by_osmid=True)
polygon = gdf.iloc[0]['geometry']
# get network of motorways and trunk roads, with speed and travel time
cf = '["highway"~"motorway|motorway_link|trunk|trunk_link"]'
G = ox.graph_from_polygon(polygon, network_type='drive', custom_filter=cf)
G = ox.add_edge_speeds(G)
G = ox.add_edge_travel_times(G)
# randomly sample lat-lng points across the graph
origin_points = ox.utils_geo.sample_points(ox.get_undirected(G), 1000)
origin_nodes = ox.nearest_nodes(G, origin_points.x, origin_points.y)
dest_points = ox.utils_geo.sample_points(ox.get_undirected(G), 1000)
dest_nodes = ox.nearest_nodes(G, dest_points.x, dest_points.y)
%%time
# solve 1000 shortest paths between origins and destinations
# minimizing travel time, using all available CPUs
paths = ox.shortest_path(G, origin_nodes, dest_nodes, weight='travel_time', cpus=None)
# elapsed time: 9.8 seconds
For faster modeling, you can load the network data from a .osm XML file instead of having to make numerous calls to the Overpass API. OSMnx by default divides your query area into 50km x 50km pieces, then queries Overpass for each piece one a time to not exceed the server's per-query memory limits. You can configure this max_query_area_size
parameter, as well as the server memory allocation, if you prefer to use OSMnx's API querying functions rather than its from-file functionality.