diff --git a/docs/source/Explanation/FileCompletion.rst b/docs/source/Explanation/FileCompletion.rst index 6b08e4c62..f6e880b8d 100644 --- a/docs/source/Explanation/FileCompletion.rst +++ b/docs/source/Explanation/FileCompletion.rst @@ -23,16 +23,16 @@ Inflight Table | NONE |Send `sr3_post(7) `_ |post only when file is complete | | |by AMQP after file is complete. | | | | |(Best when available) | -| | - fewer round trips (no renames) | - Default on sr_sarra. | -| | - least overhead / highest speed | - Default on sr_subscribe and sender | -| | | when post_broker is set. | +| | - fewer round trips (no renames) | | +| | - least overhead / highest speed | DEFAULT | +| | | | +-------------+---------------------------------------+--------------------------------------+ | |Files transferred with a *.tmp* suffix.|sending to most other systems | | .tmp |When complete, renamed without suffix. |(.tmp support built-in) | | (Suffix) |Actual suffix is settable. |Use to send to Sundew | | | | | -| | - requires extra round trips for |(usually a good choice) | -| | rename (a little slower) | - default when no post broker set | +| | - requires extra round trips for | | +| | rename (a little slower) | | +-------------+---------------------------------------+--------------------------------------+ | |Files transferred to a subdir or dir |sending to some other systems | | tmp/ |When complete, renamed to parent dir. | | diff --git a/docs/source/Reference/sr3_options.7.rst b/docs/source/Reference/sr3_options.7.rst index ada4d6c5e..db00f09f3 100644 --- a/docs/source/Reference/sr3_options.7.rst +++ b/docs/source/Reference/sr3_options.7.rst @@ -953,18 +953,18 @@ include config include another configuration within this configuration. -inflight (default: .tmp or NONE if post_broker set) ------------------------------------------------------------- +inflight (default: None) +--------------------------------- -The **inflight** option sets how to ignore files when they are being transferred +The **inflight** option sets how to ignore files when they are being transferred or (in mid-flight betweeen two systems). Incorrect setting of this option causes unreliable transfers, and care must be taken. See `Delivery Completion <../Explanation/FileCompletion.html>`_ for more details. The value can be a file name suffix, which is appended to create a temporary name during -the transfer. If **inflight** is set to **.**, then it is a prefix, to conform with +the transfer. If **inflight** is set to **.**, then it is a prefix, to conform with the standard for "hidden" files on unix/linux. -If **inflight** ends in / (example: *tmp/* ), then it is a prefix, and specifies a +If **inflight** ends in / (example: *tmp/* ), then it is a prefix, and specifies a sub-directory of the destination into which the file should be written while in flight. Whether a prefix or suffix is specified, when the transfer is @@ -979,7 +979,7 @@ This is the same as setting the **fileAgeMin** setting. Lastly, **inflight** can be set to *NONE*, which case the file is written directly with the final name, where the recipient will wait to receive a post notifying it -of the file's arrival. This is the fastest, lowest overhead option when it is available. +of the file's arrival. This is the fastest, lowest overhead option when it is available. It is also the default when a *post_broker* is given, indicating that some other process is to be notified after delivery. diff --git a/docs/source/fr/Explication/AssurerLaLivraison.rst b/docs/source/fr/Explication/AssurerLaLivraison.rst index 122ce8502..15b0612a1 100644 --- a/docs/source/fr/Explication/AssurerLaLivraison.rst +++ b/docs/source/fr/Explication/AssurerLaLivraison.rst @@ -24,18 +24,18 @@ Tableau de Inflight | |Fichier envoyé avec le bon nom |Envoyer à Sarracenia, et | | NONE |message`sr3_post(7) `_ |publié quand le fichier est complet | | |AMQP après que le transfert. | | -| | | (Meilleur quand disponible) | -| | - moins d´aller-retours | défaut pour sr_sarra. | -| | - plus efficace / vite | défaut sur sr_subscribe et sender | -| | | quand post_broker est spécifié. | +| | | Valeur de défaut | +| | - moins d´aller-retours | | +| | - plus efficace / vite | | +| | | | +-------------+---------------------------------------+--------------------------------------+ | |avec un suffixe *.tmp*. |Envoi à la plupart des autres systèmes| | .tmp |Lorsqu'il est complet, renommé au fin |(.tmp intégré) | | (Suffixe) |Le suffixe réel est réglable. |Utiliser pour envoyer à Sundew. | | | | | -| | -voyages aller-retour supplémentaires |(généralement un bon choix) | -| | pour renommer (un peu plus lent) | - défaut quand il n´y a pas de | -| | | post_broker | +| | -voyages aller-retour supplémentaires | | +| | pour renommer (un peu plus lent) | | +| | | | +-------------+---------------------------------------+--------------------------------------+ | |Fichier placés dans un sous-répertoire |Envoi à des systèmes qui n´acceptent | | tmp/ |ou répertoire. Déplacé au fin de |les suffixes | diff --git a/docs/source/fr/Reference/sr3_options.7.rst b/docs/source/fr/Reference/sr3_options.7.rst index 2233ca2db..ae84e4eb5 100644 --- a/docs/source/fr/Reference/sr3_options.7.rst +++ b/docs/source/fr/Reference/sr3_options.7.rst @@ -946,8 +946,8 @@ include config inclure une autre configuration dans cette configuration. -inflight (défaut: .tmp ou NONE si post_broker est définit) -------------------------------------------------------------------- +inflight (défaut: None) +-------------------------------- L’option **inflight** définit comment ignorer les fichiers lorsqu’ils sont transférés ou (en plein vol entre deux systèmes). Un réglage incorrect de cette option provoque des diff --git a/sarracenia/flow/__init__.py b/sarracenia/flow/__init__.py index 30c00d8af..f332dc4ec 100644 --- a/sarracenia/flow/__init__.py +++ b/sarracenia/flow/__init__.py @@ -2439,8 +2439,7 @@ def send(self, msg, options) -> int: # if renaming used, check that the protocol supports it ... #================================= - if not hasattr(self.proto[self.scheme], - 'rename') and options.inflight.startswith('.'): + if not hasattr(self.proto[self.scheme], 'rename') and options.inflight: logger.warning("%s, rename not supported" % self.scheme) inflight = None diff --git a/sarracenia/sr.py b/sarracenia/sr.py index 85f5fe3d8..41f29d632 100755 --- a/sarracenia/sr.py +++ b/sarracenia/sr.py @@ -2784,12 +2784,14 @@ def convert1(self,cfg): synonyms = sarracenia.config.Config.synonyms accept_all_seen=False + post_broker_seen=False + inflight_seen=False acceptUnmatched_explicit=False pos_args_present=False with open(v3_config_path, 'w') as v3_cfg: v3_cfg.write( f'# created by: sr3 convert {cfg}\n') if component in [ 'shovel', 'winnow' ]: - v3_cfg.write('# topicCopy on is only there for bug-for-bug compat with v2. turn it off if you can.\n') + v3_cfg.write('# topicCopy on is only there for bug-for-bug compat with v2. Leave it off if you can.\n') v3_cfg.write('#topicCopy on\n') if component in [ 'sarra', 'sender', 'subscribe' ]: @@ -2809,9 +2811,17 @@ def convert1(self,cfg): k = line[0] if k in synonyms: k = synonyms[k] + if k in [ 'post_broker' ]: + post_broker_seen=True + if k in [ 'inflight' ]: + inflight_seen=True if k in [ 'queueName' ]: queueName=line[1] + if not inflight_seen and post_broker_seen and component in [ 'sarra', 'sender', 'subscribe' ]: + v3_cfg.write('#sr3 inflight defaults to None, v2 defaulted to .tmp when post_broker set.\n') + v3_cfg.write('inflight .tmp\n') + #2nd re-write pass. subtopicFound=False with open(v2_config_path, 'r') as v2_cfg: