Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark records as skipped if we do not see them during an import run #922

Merged
merged 3 commits into from
Feb 10, 2024
Merged

Mark records as skipped if we do not see them during an import run #922

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7964466
Found another place we need to include the monad lib. This time so th…
orangewolf Feb 8, 2024
c488439
Merge branch 'main' of github.com:samvera/bulkrax
orangewolf Feb 9, 2024
3c40642
mark entries as skipped if they have not been seen during an importer…
orangewolf Feb 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions app/models/bulkrax/importer.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,10 +195,19 @@ def import_objects(types_array = nil)
end
end
parser.create_objects(types)
mark_unseen_as_skipped
rescue StandardError => e
set_status_info(e)
end

# After an import any entries we did not touch are skipped.
# They are not really pending, complete for the last run, or failed
def mark_unseen_as_skipped
entries.where.not(id. seen).find_each do |entry|
entry.set_status_info('Skipped')
end
end

# Prepend the base_url to ensure unique set identifiers
# @todo - move to parser, as this is OAI specific
def unique_collection_identifier(id)
Expand Down
5 changes: 5 additions & 0 deletions app/models/concerns/bulkrax/status_info.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ module StatusInfo
scope :failed, -> { where(status_message: 'Failed') }
scope :complete, -> { where(status_message: 'Complete') }
scope :pending, -> { where(status_message: 'Pending') }
scope :skipped, -> { where(status_message: 'Skipped') }
end

def current_status
Expand All @@ -28,6 +29,10 @@ def succeeded?
current_status&.status_message&.match(/^Complete$/)
end

def skipped?
current_status&.status_message&.match('Skipped')
end

def status
current_status&.status_message || 'Pending'
end
Expand Down
Loading