Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sweep:integration] allow to disable DM transfer as an FTS failover #7975

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ exclude: |

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
Expand Down
1 change: 1 addition & 0 deletions dirac.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -467,6 +467,7 @@ Systems
{
Location = DIRAC/DataManagementSystem/Agent/RequestOperations/ReplicateAndRegister
FTSMode = True # If True, will use FTS to transfer files
DMMode = True # If false, DataManager will not be used as a failover of FTS transfers
FTSBannedGroups = lhcb_user # list of groups for which not to use FTS
}
SetFileStatus
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ Details: :py:mod:`~DIRAC.DataManagementSystem.Agent.RequestOperations.ReplicateA
Extra configuration options:

* `FTSMode`: If True, will use FTS to transfer files
* `DMMode`: if False, will not use DataManager transfer as FTS failover
* `FTSBannedGroups` : list of groups for which not to use FTS

------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ The timeout for the operation is then calculated from this value and the number

The `ReplicateAndRegister` section accepts extra attributes, specific to FTSTransfers:
* FTSMode (default False): if True, delegate transfers to FTS
* DMMode (default True): if False, will not use DataManager as a failover transfer for FTS
* FTSBannedGroups: list of DIRAC group whose transfers should not go through FTS.

This of course does not cover all possible needs for a specific VO, hence all developers are encouraged to create and keep
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -518,7 +518,12 @@ def dmTransfer(self, fromFTS=False):
return S_OK()
# # loop over files
if fromFTS:
self.log.info("Trying transfer using replica manager as FTS failed")
if getattr(self, "DMMode", True):
self.log.info("Trying transfer using DataManager as FTS failed")
else:
self.log.info("DataManager replication disabled, skipping")
return S_OK()

else:
self.log.info("Transferring files using Data manager...")
errors = defaultdict(int)
Expand Down
1 change: 1 addition & 0 deletions src/DIRAC/RequestManagementSystem/ConfigTemplate.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ Agents
{
Location = DIRAC/DataManagementSystem/Agent/RequestOperations/ReplicateAndRegister
FTSMode = False
DMMode = True
FTSBannedGroups = dirac_user, lhcb_user
LogLevel = INFO
MaxAttempts = 256
Expand Down
Loading