Uncertain why ModelRunner's sequencer chokes halfway through scenario

[carbz]

Sequencing a larger scenario seems to get most of the way there with when we encounter a 'Population' Key error. Unsure how to proceed with completed a modelrunner scenario.

Log from http://23.253.225.19:8888/worker_data/16c0d241-6cc5-4ea7-8d81-4cafa381986b/job_log.txt is below

reading input from /home/mr/model_runner/worker_data/16c0d241-6cc5-4ea7-8d81-4cafa381986b/input
discarding /home/mr/miniconda/envs/model_runner/bin from PATH
prepending /home/mr/miniconda/envs/sequencer/bin to PATH
2015-04-13 14:20:52,968 : NetworkPlan [INFO] : Asserting Input Projections Match
2015-04-13 14:20:52,972 : NetworkPlan [INFO] : Aligning Network Nodes With Input Metrics
/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/sequencer/Utils.py:61: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
fake_nodes['m_coords'] = fake_nodes['m_coords'].apply(lambda x: ())
2015-04-13 14:21:00,352 : NetworkPlan [INFO] : Computing Pairwise Distances
2015-04-13 14:21:00,353 : NetworkPlan [INFO] : Using haversine Distance
2015-04-13 14:21:04,386 : NetworkPlan [ERROR] : Dataset Contains Edges, Less Than 0.5 Meters!
This can result in incorrect alignment of metrics and network,
where fake nodes are incorrectly assigned metrics.
This error is resolved by buffering your input data.
2015-04-13 14:21:04,410 : NetworkPlan [INFO] : Directing Network Away From Roots
2015-04-13 14:21:04,955 : NetworkPlan [INFO] : Directing SUBGRAPH 1 / 1460
2015-04-13 14:21:04,963 : NetworkPlan [INFO] : DONE!
2015-04-13 14:21:04,963 : NetworkPlan [INFO] : Directing SUBGRAPH 2 / 1460
...
...
...
2015-04-13 14:21:09,400 : NetworkPlan [INFO] : Directing SUBGRAPH 327 / 1460
2015-04-13 14:21:09,407 : NetworkPlan [INFO] : DONE!
2015-04-13 14:21:09,407 : NetworkPlan [INFO] : Directing SUBGRAPH 328 / 1460
Traceback (most recent call last):
File "/home/mr/model_runner/scripts/mvmax_sequencer.py", line 19, in
nwp = NetworkPlan(shp_file, csv_file, prioritize='Population')
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/sequencer/NetworkPlan.py", line 71, in init
self.direct_network()
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/sequencer/NetworkPlan.py", line 198, in direct_network
graphs = [self._depth_first_directed(g) for g in self.get_subgraphs()]
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/sequencer/NetworkPlan.py", line 152, in _depth_first_directed
self._graph_priority(graph.nodes())))
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/sequencer/NetworkPlan.py", line 186, in _graph_priority
return self.metrics[self.priority_metric].ix[nodes].idxmax()
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/pandas/core/frame.py", line 1780, in getitem
return self._getitem_column(key)
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/pandas/core/frame.py", line 1787, in _getitem_column
return self._get_item_cache(key)
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/pandas/core/generic.py", line 1068, in _get_item_cache
values = self._data.get(item)
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/pandas/core/internals.py", line 2849, in get
loc = self.items.get_loc(item)
File "/home/mr/miniconda/envs/sequencer/lib/python2.7/site-packages/pandas/core/index.py", line 1402, in get_loc
return self._engine.get_loc(_values_from_object(key))
File "pandas/index.pyx", line 134, in pandas.index.IndexEngine.get_loc (pandas/index.c:3807)
File "pandas/index.pyx", line 154, in pandas.index.IndexEngine.get_loc (pandas/index.c:3687)
File "pandas/hashtable.pyx", line 696, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12310)
File "pandas/hashtable.pyx", line 704, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12261)
KeyError: 'Population'

[carbz]

Still getting the same issue but isolated it down to a smaller region. Suspect it stems from a (single?) value in the metrics-local.csv file but there are no obvious inconsistencies in the inputs.

http://23.253.225.19:8888/worker_data/60bb5fd9-fedf-440b-a524-980b9c6d10ec/job_log.txt

[carbz]

@Naigege, Per Natali's insight, the problem seems to stem from how Sequencer deals with sub-graphs of the networks-proposed.shp files that do not contain any fake nodes. In this case, the nodes with the higher 'Population' value becomes the starting point of that sub-graph. This issue seems related to the fact that the use cases being reported do not have a 'Population' field included and have sub graphs without fake nodes. An initial test of this by adding in a 'Population' field seems to remove the original error message but introduces a new one below...

http://23.253.225.19:8888/worker_data/65db50c1-39ce-4e33-894a-a1e8dabd082a/job_log.txt

reading input from /home/mr/model_runner/worker_data/65db50c1-39ce-4e33-894a-a1e8dabd082a/input
...
MemoryError

[Naigege]

Tested the same scenario with Set network =100 on network planner, model runner sequenced it successfully. So our deduction is right on fake nodes.

[chrisnatali]

Closing this in favor of handling these hard coded settings more flexibly via this issue:
SEL-Columbia/sequencer#44