HPC: read_base_lci_database = true & update_base_database_with_future_information = true fails updating with future info

[schunschun]

This happened when running image in Kestral HPC shared account.

When I set both read_base_lci_database and update_base_database_with_future_information true, only read_base_lci_database is completed and updating with future fails. So I had to rerun with read_base_lci_database with false and update_base_database_with_future_information with true.

**debug_output
Tue May 7 18:45:54 MDT 2024
ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity
ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug
ecoinvent_2100_SSP2-Base_image
/kfs2/shared-projects/liaison/Soomin/Dynamic-LCA-with-LiAISON/results/resultecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug
0 minutes and 7 seconds elapsed for copying.
'Starting the Code'
'Importing brightway module.....'
Using environment variable BRIGHTWAY2_DIR for data directory:
/kfs2/shared-projects/liaison/Soomin/env/liaisonecoinvent_2100_SSP2-Base_image/
'All input data parameters read'
'Running db reader'
Project does not exist
Creating base project base_project_ecoinvent38
Biosphere database already present!!! No setup is needed
Reading ecoinvent3.8
/kfs2/shared-projects/liaison/liaison/ecoinvent/ecoinvent_3.8_cutoff_ecoSpold02/datasets
Warning: No valid output stream.
Title: Extracting ecospold2 files:
Started: 05/07/2024 18:46:03
Finished: 05/07/2024 18:47:53
Total time elapsed: 00:01:50
CPU %: 95.70
Memory %: 1.31
Extracted 19565 datasets in 110.27 seconds
Applying strategy: normalize_units
Applying strategy: update_ecoinvent_locations
Applying strategy: remove_zero_amount_coproducts
Applying strategy: remove_zero_amount_inputs_with_no_activity
Applying strategy: remove_unnamed_parameters
Applying strategy: es2_assign_only_product_with_amount_as_reference_product
Applying strategy: assign_single_product_as_activity
Applying strategy: create_composite_code
Applying strategy: drop_unspecified_subcategories
Applying strategy: fix_ecoinvent_flows_pre35
Applying strategy: drop_temporary_outdated_biosphere_flows
Applying strategy: link_biosphere_by_flow_uuid
Applying strategy: link_internal_technosphere_by_composite_code
Applying strategy: delete_exchanges_missing_activity
Applying strategy: delete_ghost_exchanges
Applying strategy: remove_uncertainty_from_negative_loss_exchanges
Applying strategy: fix_unreasonably_high_lognormal_uncertainties
Applying strategy: set_lognormal_loc_value
Applying strategy: convert_activity_parameters_to_list
Applying strategy: add_cpc_classification_from_single_reference_product
Applying strategy: delete_none_synonyms
Applied 21 strategies in 4.76 seconds
19565 datasets
629959 exchanges
0 unlinked exchanges

Vacuuming database
Vacuuming database
Tue May 7 18:54:20 MDT 2024
copying result

debug_error****
mkdir: cannot create directory ‘/kfs2/shared-projects/liaison/Soomin/env/liaisonecoinvent_2100_SSP2-Base_image/’: File exists
rm: cannot remove 'log*': No such file or directory
mkdir: cannot create directory ‘/scratch/schun/liaison/’: File exists
rm: cannot remove '/scratch/schun/liaison/jobdirsubmit_ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug': No such file or directory
rm: cannot remove '/shared-projects/liaison/Soomin/env/liaison/logs/': No such file or directory
rm: cannot remove '/kfs2/shared-projects/liaison/Soomin/Dynamic-LCA-with-LiAISON/results/resultecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug': No such file or directory
mkdir: cannot create directory ‘/scratch/schun/liaison/jobdirsubmit_ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug/data/output/’: File exists
Traceback (most recent call last):
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/peewee.py", line 3252, in execute_sql
cursor.execute(sql, params or ())
sqlite3.OperationalError: cannot VACUUM from within a transaction

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/kfs3/scratch/schun/liaison/jobdirsubmit_ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug/code/main.py", line 215, in
main()
File "/kfs3/scratch/schun/liaison/jobdirsubmit_ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug/code/main.py", line 183, in main
run_database_reader(ecoinvent_file=ecoinvent_file, base_database=base_database, base_project=base_project, bw=bw)
File "/kfs3/scratch/schun/liaison/jobdirsubmit_ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug/code/main.py", line 89, in run_database_reader
reader(ecoinvent_file=ecoinvent_file, base_database=base_database, base_project=base_project, bw=bw)
File "/kfs3/scratch/schun/liaison/jobdirsubmit_ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug/code/image_ecoinvent_updater/main_database_reader.py", line 57, in reader
read_ecoinvent_database(base_database, ecoinvent_file, bw)
File "/kfs3/scratch/schun/liaison/jobdirsubmit_ecoinvent_2100_SSP2-Base_image_LDAC_07_UseElectricity_liaison_lca_debug/code/image_ecoinvent_updater/main_database_reader.py", line 40, in read_ecoinvent_database
ei_38.write_database()
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2io/importers/base_lci.py", line 269, in write_database
db.write(existing)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2data/project.py", line 358, in writable_project
return wrapped(*args, **kwargs)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2data/backends/peewee/database.py", line 260, in write
self._efficient_write_many_data(data)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2data/backends/peewee/database.py", line 193, in _efficient_write_many_data
self.delete(keep_params=True, warn=False)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2data/project.py", line 358, in writable_project
return wrapped(*args, **kwargs)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2data/backends/peewee/database.py", line 349, in delete
sqlite3_lci_db.vacuum()
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2data/sqlite.py", line 57, in vacuum
self.execute_sql('VACUUM;')
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/bw2data/sqlite.py", line 50, in execute_sql
return self.db.execute_sql(*args, **kwargs)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/peewee.py", line 3250, in execute_sql
with exception_wrapper:
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/peewee.py", line 3020, in exit
reraise(new_type, new_type(exc_value, exc_args), traceback)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/peewee.py", line 192, in reraise
raise value.with_traceback(tb)
File "/home/schun/.conda-envs/liaison/lib/python3.10/site-packages/peewee.py", line 3252, in execute_sql
cursor.execute(sql, params or ())
peewee.OperationalError: cannot VACUUM from within a transaction
cp: cannot stat '.csv': No such file or directory

[tjlca]

Resolved and yaml file updated