Batch sample import service

[JeffreyThiessen]

What does this PR do and why?

Describe in detail what your merge request does and why.

• Added BatchFileImportService for importing a spreadsheet of samples

Additional fixes/changes in this pr

• Fixed bug where tsv files were not being read as csv files sometimes
• refactored sample metadata import service to be a child of new parent class BaseSpreadsheetImportService
• simplified spreadsheet parsing

Note: I decided to split this PR into 2 parts, this part which handles the sample file creation and refactor, and a separate PR for doing the metadata parsing.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other pull requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. seed your db
2. Start irida next with command line enabled
3. Sign into User3
4. navigate to http://localhost:3000/borrelia/borrelia-burgdorferi/outbreak-2024/-/samples
5. run the following in the terminal to import new samples.

u = User.all[3];n = Project.all[0].namespace;

# replace this path with the path to your irida next git directory
filepath = '/home/CSCScience.ca/jthiessen/git/irida-next-core/test/fixtures/files/batch_sample_import_valid.csv'

blob = ActiveStorage::Blob.create_and_upload!(io: File.open(filepath), filename: fname)

s = Samples::BatchFileImportService.new(n,u, blob.id, {sample_name_column: 'sample_name', project_puid_column: 'project_puid', sample_description_column: "description"} )

s.execute

6. You should see 2 new samples added to the project you are viewing

PR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the PR acceptance checklist for this PR.

[github-actions]

Code Metrics Report

Coverage	Test Execution Time
92.1%	10m25s

Code coverage of files in pull request scope (56.3%)

Files	Coverage
app/jobs/samples/batch_sample_import_job.rb	0.0%
app/services/base_spreadsheet_import_service.rb	96.0%
app/services/samples/batch_file_import_service.rb	0.0%
app/services/samples/metadata/file_import_service.rb	100.0%

---

Reported by octocov

[deepsidhu85]

Base of this looks great, just have one comment for now

[deepsidhu85]

I'm thinking this can be done in the process_sample_row once the sample is persisted

[JeffreyThiessen]

That makes sense. The metadata headers should probably be processed here with the other headers though.