Issue running code

[jwbrown]

I have managed to install and get the first stage to run....

When running script_train_fixed_params i am getting an error you may be able to help me with.... it looks like /tft_outputs/data/traffic/hourly_data.csv is not being created from the downloaded data set, i am not sure when it should be created as the only errors prior to this one are about deprecated methods as i am running on tensorflow 2.7.0

The error trace is:

*** Training from defined parameters for traffic ***
Loading & splitting data...
Traceback (most recent call last):
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/jbrown/tft/script_train_fixed_params.py", line 247, in
main(
File "/home/jbrown/tft/script_train_fixed_params.py", line 89, in main
raw_data = pd.read_csv(data_csv_path)#, index_col=0)
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 575, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 933, in init
self._engine = self._make_engine(f, self.engine)
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1217, in _make_engine
self.handles = get_handle( # type: ignore[call-overload]
File "/home/jbrown/miniforge3/envs/tf/lib/python3.9/site-packages/pandas/io/common.py", line 789, in get_handle
handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/home/jbrown/tft_outputs/data/traffic/hourly_data.csv'

[jwbrown]

I looked deeper and the dowload job is being killed after the loop where all the frame.append's are done.... looking to see if i can figure out why

Edit: it was being killed due to a lack of memory... i ran the download and split on a different machine and it completed and am now processing the files on the main machine... i probably just need to run the whole job set elsewhere...