Clover hpc #26
Open
Clover hpc #26
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…oblems with some compilers. Clean up anomylous naming and spacing. This is in order to allow automatic editing of scripts matching given patterns in later changes. Used the following commands to achieve this in shell terminal in CLOVER directory: # Clean up inconsistency in type of inverted commas. This has caused problems with some compilers. SRC="‘" DST="'" sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py SRC="’" DST="'" sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py # Clean up anomylous naming and spacing. This is in order to allow automatic editing of scripts matching given patterns in later changes. sed -i "" -e "s/location_input_data/location_inputs/g" Scripts/*/*py sed -i "" -e "s/_inputs = pd.read_csv/_inputs = pd.read_csv/g" Scripts/*/*py sed -i "" -e "s/_inputs = pd.read_csv/_inputs = pd.read_csv/g" Scripts/*/*py
Reconfigure to take '.' as filepath - necessary for HPC where CLOVER is copied to the node before processing. (May wish to amend if CLOVER dir gets very big)
Commands used:
SRC="\/\*\*\*YOUR LOCAL FILE PATH\*\*\*\/CLOVER 4.0"
DST="."
sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py
SRC="\/\*\*\*YOUR LOCAL FILE PATH\*\*\*\/CLOVER"
DST="."
sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py
# Reconfigure to take kwargs for location
sed -i "" -e 's/__init__(self)/__init__(self,**kwargs)/g' Scripts/*/*py
SRC="self.location = 'Bahraich'"
DST="self.location = kwargs.get('location')"
sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py
SRC="self.location = ‘Bahraich’" # This is required too as there is some inconsistency in type of inverted commas used in original CLOVER scripts.
DST="self.location = kwargs.get('location')"
sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py
This reverts commit 599cc85.
# Reconfigure to take '.' as filepath - necessary for HPC where CLOVER is copied to the node before processing. (May wish to amend if CLOVER dir gets very big) Bash commands: SRC="\/\*\*\*YOUR LOCAL FILE PATH\*\*\*\/CLOVER 4.0" DST="." sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py SRC="\/\*\*\*YOUR LOCAL FILE PATH\*\*\*\/CLOVER" DST="." sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py
# Reconfigure to take kwargs for location
sed -i "" -e 's/__init__(self)/__init__(self,**kwargs)/g' Scripts/*/*py
SRC="self.location = 'Bahraich'"
DST="self.location = kwargs.get('location')"
sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py
SRC="self.location = ‘Bahraich’" # This was required too as there was some inconsistency in type of inverted commas used in original CLOVER scripts - may have been fixed by earlier commands
DST="self.location = kwargs.get('location')"
sed -i "" -e "s/$SRC/$DST/g" Scripts/*/*py
# Set up dependencies in taking kwargs for location
sed -i "" -e "s/Diesel()\./Diesel(kwargs)./g" Scripts/*/*py
sed -i "" -e "s/Grid()\./Grid(kwargs)./g" Scripts/*/*py
sed -i "" -e "s/Solar()\./Solar(kwargs)./g" Scripts/*/*py
sed -i "" -e "s/Finance()\./Finance(kwargs)./g" Scripts/*/*py
sed -i "" -e "s/GHGs()\./GHGs(kwargs)./g" Scripts/*/*py
sed -i "" -e "s/Load()\./Load(kwargs)./g" Scripts/*/*py
sed -i "" -e "s/Energy_System()\./Energy_System(kwargs)./g" Scripts/*/*py
Replace spaces with underscores in filenames because spaces cause problems in file handling This stops github from recognising that files are the same, which is a pity! This will need to be resolved to merge hpc CLOVER with new features in main branch Bash commands: rename -s ' scripts' '_scripts' Scripts/*/ for file in Scripts/*/*py; do sed "s| scripts|_scripts|" $file > temp; mv temp $file; done
…s those specified in location)
# Set up scripts to allow specification of inputs with kwargs (overrides those specified in location)
bash commands used:
for name in grid location finance GHG device optimisation energy_system scenario diesel
do
cat > ${name}_line.py << EOF
# Replace input values with keywords if specified
if kwargs.get('${name}_inputs'):
for i in kwargs.get('${name}_inputs'):
if not i[0] in self.${name}_inputs.index:
print("Couldn't find entry",i[0],"in ${name}_inputs. Perhaps it's misspelt in kwargs? Printing list of possible variables and exiting.")
print(self.${name}_inputs.index)
exit(1)
self.${name}_inputs[i[0]] = i[1]
EOF
for file in Scripts/*/*py
do
sed "/ self.${name}_inputs = pd.read_csv/r ${name}_line.py" $file > tmp
mv tmp $file
done
rm ${name}_line.py
done
Relating to specific projects - I think unlikely to be harmful, but may not be useful
Bhinjpur
Accomodate revised file structuring (keep Jobs in separate dir and repository from main CLOVER files
Reduced number of devices for example run
New hpc users don't seem to have a work directory of the same format
Directory structure changed in 2018
Changed file format to remove \r newlines which were causing problems
- Moved launch file & added it's new home to the path in .bash_profile (readme instructions, could also add to setup file) - Corrected time between renewables ninja requests in Load_Solar_Bhinjpur.py
Hoping this will encourage these necessary folders to be recognised in upload & download of repo
| @@ -18,15 +18,23 @@ | |||
| import numpy as np | |||
|
|
|||
| class Diesel(): | |||
| def __init__(self): | |||
| def __init__(self,**kwargs): | |||
There was a problem hiding this comment.
def __init__(self, *, location, diesel_inputs=None):
self.location = location
| self.location_filepath = self.CLOVER_filepath + '/Locations/' + self.location | ||
| self.generation_filepath = self.location_filepath + '/Generation/' | ||
| self.diesel_filepath = self.generation_filepath + 'Diesel/Diesel inputs.csv' | ||
| self.diesel_inputs = pd.read_csv(self.diesel_filepath,header=None,index_col=0).round(decimals=3) | ||
|
|
||
| # Replace input values with keywords if specified | ||
| if kwargs.get('diesel_inputs'): |
There was a problem hiding this comment.
if diesel_input is not None
There was a problem hiding this comment.
If the kwarg is specified as above, then this would default to None if the kwarg was not specified.
| self.location = 'Bahraich' | ||
| self.CLOVER_filepath = '/***YOUR LOCAL FILE PATH***/CLOVER 4.0' | ||
| self.location = kwargs.get('location') | ||
| self.CLOVER_filepath = '.' |
There was a problem hiding this comment.
Python's os module contains built-in functionality for getting the working directory.
Added indication of which kwargs can/must be specified per script (some way towards Ben's suggested safer way of specifying arguments) Removed some of my obsolete options (quick battery recharge, efficient diesel - introduced in the earlier "miscellaneous" commit): Left a few other additions in (which were also introduced in the earlier "miscellaneous" commit) : - In Optimisation.py: Additional metrics around storage performance reported: LCSE, Cumulative storage cost - In Load.py: optional arguments for different approaches to load generation - In Optimisation.py: optional argument to dump spare battery capacity at the end of a simulation period
… redundant script
Clover hpc to merge
BenWinchester
added
duplicate
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.