Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

does not respond to the touch API #266

Closed
2 tasks done
ll4strw opened this issue Mar 20, 2024 · 7 comments
Closed
2 tasks done

does not respond to the touch API #266

ll4strw opened this issue Mar 20, 2024 · 7 comments
Assignees
Labels
Milestone

Comments

@ll4strw
Copy link

ll4strw commented Mar 20, 2024

  • main
  • 4-3-stable

I have two resources with the following settings and a rule that executes periodically the tiering plugin

$ imeta ls -R laptopResc  
AVUs defined for resource laptopResc:
attribute: irods::storage_tiering::group
value: groupie
units: 0
----
attribute: irods::storage_tiering::time
value: 15
units:

$ imeta ls -R laptopResc1
AVUs defined for resource laptopResc1:
attribute: irods::storage_tiering::group
value: groupie
units: 1

Upon itouching an empty data object into laptopResc, this file stays there forever and does not get moved to laptopResc1. On the other hand, any other data object, which contains data, honors the tiering rules

$ itouch test
$ itouch test33
$ echo "hello" >> test1 && iput test1
$ echo "hello" >> test2 && iput test2
$ ils -l
/laptopZone/home/rods:
  rods              0 laptopResc            0 2024-03-20.09:29 & test
  rods              1 laptopResc1           11 2024-03-20.10:00 & test1
  rods              1 laptopResc1           11 2024-03-20.10:09 & test2
  rods              0 laptopResc            0 2024-03-20.10:10 & test33

However, if I have the following local directory made of an empty file (test33) and a data file (file1)

$ ls -l TEST33
total 4
-rw-rw-r-- 1 irods irods 5 Mar 20 10:15 file1
-rw-rw-r-- 1 irods irods 0 Mar 20 10:15 test333

and I copy it over to iRODS

irsync -R laptopResc -r TEST33 i:TEST33

then the empty file DOES get moved to the second storage tier

$ ils -l /laptopZone/home/rods/TEST33
/laptopZone/home/rods/TEST33:
  rods              1 laptopResc1            5 2024-03-20.10:18 & file1
  rods              1 laptopResc1            0 2024-03-20.10:18 & test333

Finally, by adding again an empty data object to the TEST33 collection results in a missed transfer to the socond storage tier again

$ itouch TEST33/empty_file
$ ils -l /laptopZone/home/rods/TEST33
/laptopZone/home/rods/TEST33:
  rods              0 laptopResc            0 2024-03-20.10:22 & empty_file
  rods              1 laptopResc1            5 2024-03-20.10:18 & file1
  rods              1 laptopResc1            0 2024-03-20.10:18 & test333

Is this the expected behavior? If so, why to treat differently an empty file generated using itouch and one generated via irsync? Thanks in advance, L.

@trel
Copy link
Member

trel commented Mar 20, 2024

Definitely not expected behavior.

Which version of iRODS is this?

@trel trel added the bug label Mar 20, 2024
@ll4strw
Copy link
Author

ll4strw commented Mar 20, 2024

Ubuntu 22.04.3 LTS

irods-server/jammy,now 4.3.1-0~jammy amd64
irods-icommands/jammy,now 4.3.1-0~jammy amd64
irods-rule-engine-plugin-unified-storage-tiering/jammy,now 4.3.1.1-0~jammy amd64

@trel
Copy link
Member

trel commented Mar 20, 2024

Got it - we'll work to reproduce what you're seeing.

@korydraughn
Copy link
Collaborator

The plugin does not respond to itouch because the plugin doesn't know about the PEPs for the touch API.

We'll get that resolved.

@korydraughn korydraughn added this to the 4.3.1.2 milestone Mar 20, 2024
@alanking
Copy link
Contributor

Subset of #200, I believe (as seen here: #200 (comment))

@trel
Copy link
Member

trel commented Mar 20, 2024

ah, indeed.

@korydraughn korydraughn removed this from the 4.3.1.2 milestone Apr 26, 2024
@alanking alanking added this to the 4.3.3.1 milestone Dec 5, 2024
@alanking alanking self-assigned this Jan 14, 2025
@trel trel changed the title Empty data object transfer between storage tiers does not respond to the touch API Jan 23, 2025
@alanking
Copy link
Contributor

alanking commented Jan 30, 2025

Created #326 to address restaging. This will not be implemented for 4.3.3.1. For 4.3.3.1, data objects created by touch will have access_time annotated so that they tier out just like any other means of creation. touch will also update access_time on existing data objects.

alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 30, 2025
This commit adds tests for various storage tiering
behaviors as it relates to the touch API:

- touch-created object on lowest tier tiers out
- touched existing object updates access_time
- touched existing object triggers restage (skip... for now)
- touch-created object on higher tier does NOT restage
- touched existing collection does NOT trigger restage
alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 30, 2025
alanking added a commit that referenced this issue Jan 30, 2025
This commit adds tests for various storage tiering
behaviors as it relates to the touch API:

- touch-created object on lowest tier tiers out
- touched existing object updates access_time
- touched existing object triggers restage (skip... for now)
- touch-created object on higher tier does NOT restage
- touched existing collection does NOT trigger restage
alanking added a commit that referenced this issue Jan 30, 2025
alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 30, 2025
This commit adds tests for various storage tiering
behaviors as it relates to the touch API:

- touch-created object on lowest tier tiers out
- touched existing object updates access_time
- touched existing object triggers restage (skip... for now)
- touch-created object on higher tier does NOT restage
- touched existing collection does NOT trigger restage
alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 30, 2025
alanking added a commit that referenced this issue Jan 30, 2025
This commit adds tests for various storage tiering
behaviors as it relates to the touch API:

- touch-created object on lowest tier tiers out
- touched existing object updates access_time
- touched existing object triggers restage (skip... for now)
- touch-created object on higher tier does NOT restage
- touched existing collection does NOT trigger restage
alanking added a commit that referenced this issue Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

4 participants