Skip to content

Commit

Permalink
review and improve error handling when invoking makedirs (#1365)
Browse files Browse the repository at this point in the history
* review and improve error handling when invoking makedirs

If (usual case) there are multiple files in a directory,
then there is a race condition to create the directory, multiple
instances will see the directory is not there, and will run makedirs
to create it. the "loser" just won't create the directory... but with
unlink... the unlink fails (because directory exists.) and
then the operation fails. (not retried? FIXME?)

So I think it's better to leave that failure mode to humans.
  • Loading branch information
petersilva authored Jan 10, 2025
1 parent a0e9c19 commit 86bc42c
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions sarracenia/flow/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1659,6 +1659,7 @@ def mkdir(self, msg) -> bool:
except Exception as ex:
logger.warning("making %s: %s" % (msg['new_dir'], ex))
logger.debug('Exception details:', exc_info=True)
return False

if os.path.isdir(path):
logger.debug( f"no need to mkdir {path} as it exists" )
Expand Down Expand Up @@ -1709,6 +1710,7 @@ def link1file(self, msg, symbolic=True) -> bool:
except Exception as ex:
logger.warning("making %s: %s" % (msg['new_dir'], ex))
logger.debug('Exception details:', exc_info=True)
return False

ok = True
try:
Expand Down Expand Up @@ -1775,6 +1777,8 @@ def do_download(self) -> None:
except Exception as ex:
logger.warning("making %s: %s" % (msg['new_dir'], ex))
logger.debug('Exception details:', exc_info=True)
self.reject(msg, 422, f"cannot create directory {msg['new_dir']} to put file in it." )
continue

os.chdir(msg['new_dir'])
logger.debug( f"chdir {msg['new_dir']}")
Expand Down

0 comments on commit 86bc42c

Please sign in to comment.