Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Direct Linking to COM from DATA for wavepostsbs Job #3303

Open
wants to merge 34 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
c0ec095
removed white space
AntonMFernando-NOAA Feb 3, 2025
e8ddc88
Merge branch 'NOAA-EMC:develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 3, 2025
f2669df
Merge branch 'NOAA-EMC:develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 4, 2025
b06793b
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
a417365
Merge branch 'bug/NCO-COM' of https://github.com/AntonMFernando-NOAA/…
AntonMFernando-NOAA Feb 4, 2025
66269bd
modified ush\wave_grid_interp_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
5f59736
update ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
13dca27
Merge branch 'NOAA-EMC:develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 4, 2025
f7d3bfc
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
971cafb
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
53f7378
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
a674b8b
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
173d413
modifed ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
514fd35
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
2b2c234
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 4, 2025
007e1ac
Merge branch 'NOAA-EMC:develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 5, 2025
849feb7
added missing fi to ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 5, 2025
ae24fbd
Merge branch 'bug/NCO-COM' of https://github.com/AntonMFernando-NOAA/…
AntonMFernando-NOAA Feb 5, 2025
dc52172
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 5, 2025
cf52ba5
Merge branch 'NOAA-EMC:develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 11, 2025
20e2808
Update ush/wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 11, 2025
519767e
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 11, 2025
ec3dca7
Merge branch 'NOAA-EMC:develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 13, 2025
79b2926
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 13, 2025
901860f
Merge branch 'bug/NCO-COM' of https://github.com/AntonMFernando-NOAA/…
AntonMFernando-NOAA Feb 13, 2025
89f6112
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 13, 2025
b9978c3
Update ush/wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 13, 2025
b5eaebb
Update ush/wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 13, 2025
f53e11c
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 13, 2025
f3d9244
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 13, 2025
92f4ea5
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 14, 2025
2b73e50
modified ush\wave_grib2_sbs.sh
AntonMFernando-NOAA Feb 14, 2025
eb3f5f1
Merge branch 'develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 14, 2025
3a03f57
Merge branch 'NOAA-EMC:develop' into bug/NCO-COM
AntonMFernando-NOAA Feb 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 32 additions & 11 deletions ush/wave_grib2_sbs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -169,11 +169,11 @@ if [[ ! -s "${com_dir}/${outfile}.idx" ]]; then
fi

if (( fhr > 0 )); then
${WGRIB2} gribfile -set_date "${PDY}${cyc}" -set_ftime "${fhr} hour fcst" -grib "${com_dir}/${outfile}"
${WGRIB2} gribfile -set_date "${PDY}${cyc}" -set_ftime "${fhr} hour fcst" -grib "${outfile}"
err=$?
else
${WGRIB2} gribfile -set_date "${PDY}${cyc}" -set_ftime "${fhr} hour fcst" \
-set table_1.4 1 -set table_1.2 1 -grib "${com_dir}/${outfile}"
-set table_1.4 1 -set table_1.2 1 -grib "${outfile}"
err=$?
fi

Expand All @@ -183,27 +183,50 @@ if [[ ! -s "${com_dir}/${outfile}.idx" ]]; then
echo '********************************************* '
echo "*** FATAL ERROR : ERROR IN ${pgm} *** " # FIXME: This is not an error in $pgm, but in WGRIB2
echo '********************************************* '
echo ' '
echo " "
Fixed Show fixed Hide fixed
set_trace
exit 3
fi

# Create index
${WGRIB2} -s "${com_dir}/${outfile}" > "${com_dir}/${outfile}.idx"
${WGRIB2} -s "${outfile}" > "${outfile}.idx"

# Create grib2 subgrid is this is the source grid
# Create grib2 subgrid if this is the source grid
if [[ "${grdID}" = "${WAV_SUBGRBSRC}" ]]; then
for subgrb in ${WAV_SUBGRB}; do
subgrbref=$(echo ${!subgrb} | cut -d " " -f 1-20)
subgrbnam=$(echo ${!subgrb} | cut -d " " -f 21)
subgrbres=$(echo ${!subgrb} | cut -d " " -f 22)
subfnam="${WAV_MOD_TAG}.${cycle}${ENSTAG}.${subgrbnam}.${subgrbres}.f${FH3}.grib2"
${COPYGB2} -g "${subgrbref}" -i0 -x "${com_dir}/${outfile}" "${com_dir}/${subfnam}"
${WGRIB2} -s "${com_dir}/${subfnam}" > "${com_dir}/${subfnam}.idx"
${COPYGB2} -g "${subgrbref}" -i0 -x "${outfile}" "${subfnam}"
${WGRIB2} -s "${subfnam}" > "${subfnam}.idx"
done
fi

# 1.e Save in /com
# Move grib files to COM directory
if [[ -s "${outfile}" ]] && [[ -s "${outfile}.idx" ]]; then
cpfs "${outfile}" "${com_dir}/${outfile}"
cpfs "${outfile}.idx" "${com_dir}/${outfile}.idx"
echo "Copied ${outfile} and ${outfile}.idx from ${GRIBDATA} to COM"
else
echo "${outfile} and ${outfile}.idx not found in ${GRIBDATA} to copy to COM"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be an error?

Copy link
Contributor Author

@AntonMFernando-NOAA AntonMFernando-NOAA Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JessicaMeixner-NOAA What this does is copying the ${outfile} to the COM directory, after the GRIB packing program is run. We are trying to first run the GRIB packing program in the working directory and then move the grib and .idx files to the COM dir. Is this what you are concerned about?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's great! Thanks for these updates. What I'm concerned with is if the grib and .idx files do not exist, then perhaps that should be an error message and not just simply a " echo "${outfile} and ${outfile}.idx not found" ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JessicaMeixner-NOAA "${outfile} and ${outfile}.idx not found in ${GRIBDATA} to copy to COM Currently, this is the error message. What changes would you recommend for this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it say something like "ERROR: explanation" and then exit instead of continuing on?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JessicaMeixner-NOAA I will revise the script based on your suggestion. Thank you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JessicaMeixner-NOAA I reviewed the script again, and it turns out the issue is addressed in the if blocks (L233-256). However, it only checks the COM files. So, I think your suggestion is correct.

fi

if [[ "${grdID}" = "${WAV_SUBGRBSRC}" ]]; then
for subgrb in ${WAV_SUBGRB}; do
subgrbnam=$(echo "${!subgrb}" | cut -d " " -f 21)
subgrbres=$(echo "${!subgrb}" | cut -d " " -f 22)
subfnam="${WAV_MOD_TAG}.${cycle}${ENSTAG}.${subgrbnam}.${subgrbres}.f${FH3}.grib2"
AntonMFernando-NOAA marked this conversation as resolved.
Show resolved Hide resolved
if [[ -s "${subfnam}" ]] && [[ -s "${subfnam}.idx" ]]; then
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed
cpfs "${subfnam}" "${com_dir}/${subfnam}"
cpfs "${subfnam}.idx" "${com_dir}/${subfnam}.idx"
echo "Copied ${subfnam} and ${subfnam}.idx from ${GRIBDATA} to COM"
else
echo "${subfnam} and ${subfnam}.idx not found in ${GRIBDATA} to copy to COM"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be an error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same above.

fi
done
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed
fi

if [[ ! -s "${com_dir}/${outfile}" ]]; then
set +x
Expand Down Expand Up @@ -232,16 +255,14 @@ if [[ ! -s "${com_dir}/${outfile}.idx" ]]; then

if [[ "${SENDDBN}" = 'YES' ]] && [[ ${outfile} != *global.0p50* ]]; then
set +x
echo " Alerting GRIB file as ${com_dir}/${outfile}"
echo " Alerting GRIB index file as ${com_dir}/${outfile}.idx"
echo " Alerting GRIB file as ${outfile}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the com_dir be removed from this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JessicaMeixner-NOAA After the L255 the grib and the .idx files should be in both the COM directory and the working directory, since the working directory is not removed yet. So, this should run without an error if those files exist in both dirs. If @aerorahul thinks this is not correct I can change it. But I think this should be correct because we need an alert to send for the files available in COM dir, while checking the files in the working directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is just an echo. its fine to be either. the crucial line is the dbn_alert that alerts the correct path of this file from com_dir.

echo " Alerting GRIB index file as ${outfile}.idx"
set_trace
"${DBNROOT}/bin/dbn_alert" MODEL "${alertName}_WAVE_GB2" "${job}" "${com_dir}/${outfile}"
"${DBNROOT}/bin/dbn_alert" MODEL "${alertName}_WAVE_GB2_WIDX" "${job}" "${com_dir}/${outfile}.idx"
else
echo "${outfile} is global.0p50 or SENDDBN is NO, no alert sent"
fi


# --------------------------------------------------------------------------- #
# 3. Clean up the directory

Expand Down
2 changes: 1 addition & 1 deletion ush/wave_grid_interp_sbs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ source "${USHgfs}/preamble.sh"
for ID in ${waveGRD} ${grdID}; do
${NLN} "${DATA}/mod_def.${ID}" "mod_def.${ID}"
done


# --------------------------------------------------------------------------- #
# 1. Generate GRID file with all data
Expand Down