Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

queries fall over when searching for data objects with 'select' in the filename #281

Closed
2 tasks done
trel opened this issue Dec 4, 2024 · 11 comments
Closed
2 tasks done

queries fall over when searching for data objects with 'select' in the filename #281

trel opened this issue Dec 4, 2024 · 11 comments
Assignees
@alanking
Labels
bug
Milestone
4.3.3.1

Comments

@trel
Copy link
Member

trel commented Dec 4, 2024 •
edited by alanking
Loading

  • main
  • 4-3-stable

From irod-chat ... https://groups.google.com/g/irod-chat/c/C7Rs0gijExw

Similar to irods/irods#4697 ...

We need to lowercase the queries used in storage tiering as a workaround until iRODS has a proper parser everywhere.

Currently, this is the behavior with storage_tiering installed...

$ itouch "select"
Level 0: NO_COLUMN_NAME_FOUND: query fill failed for [SELECT DATA_ID, DATA_COLL_ID, DATA_NAME, DATA_REPL_NUM, DATA_VERSION,DATA_TYPE_NAME, DATA_SIZE, DATA_RESC_NAME, DATA_PATH, DATA_OWNER_NAME, DATA_OWNER_ZONE, DATA_REPL_STATUS, DATA_STATUS, DATA_CHECKSUM, DATA_EXPIRY, DATA_MAP_ID, DATA_COMMENTS, DATA_CREATE_TIME, DATA_MODIFY_TIME, DATA_MODE, DATA_RESC_HIER, DATA_RESC_ID, COLL_NAME WHERE DATA_NAME = 'select' AND COLL_NAME = '/tempZone/home/rods' AND DATA_REPL_NUM = '0']
@trel trel added the bug label Dec 4, 2024
@trel
Copy link
Member Author

trel commented Dec 4, 2024

follow up post on the chat list...

No it are the special SQL chars....

I excluded the files with the following chars in my specific query & ' _ in the filenames and now the tiering works again

and not ( R_DATA_MAIN.data_name like '%:%'
or R_DATA_MAIN.data_name like '%''%'
or R_DATA_MAIN.data_name like '%_%'
)
But good to hear a fix is pending.

@alanking alanking added this to the 4.3.3.1 milestone Dec 4, 2024
@trel
Copy link
Member Author

trel commented Jan 23, 2025

can we just use GenQuery2?

@alanking
Copy link
Contributor

can we just use GenQuery2?

Sure, but I would hesitate to introduce it in 4.3.3.1. That said, it's not necessarily my decision. :)

@trel
Copy link
Member Author

trel commented Jan 23, 2025

If we have a GenQuery1 solution that works - sure, let's do that here.

But if it's not obvious how to do that... I think we can use GenQuery2 safely enough...

@alanking
Copy link
Contributor

So we need a test that creates a data object with "select" and/or characters in the set &'_: and make sure that it tiers out and restages properly. Plus, itouch "select", because easy enough.

As for a solution, we are looking to fix the GenQuery1 implementation of the internal queries (because we cannot fundamentally change the violating queries in a minor release) for 4.3.3.1. If that doesn't work, this will get booted from the milestone and the queries re-implemented using GenQuery2.

@alanking alanking mentioned this issue Jan 30, 2025
Closed
@alanking
Copy link
Contributor

I was able to reproduce the itouch select issue without storage tiering installed, so I created a separate issue for that in the main repo: irods/irods#8148

We can still add a test to see if data objects with "select" in the path are tiered out. Just can't use itouch to create them, evidently.

@korydraughn
Copy link
Collaborator

The itouch issue was previously reported at irods/irods#7174 and closed as part of the 5.0.0 milestone.

@alanking
Copy link
Contributor

alanking commented Jan 30, 2025 •
edited
Loading

Ah, I was looking for that and didn't find it. We can close irods/irods#8148 as a duplicate. Thanks

@alanking alanking mentioned this issue Jan 31, 2025
Open
@alanking
Copy link
Contributor

Now that I'm digging into this, I see the issue from the Google Group is directly related to an apostrophe in the object name. I didn't see that before, so, apologies for all the flailing. That would fall squarely under the irods/irods#3902 umbrella. We cannot fix that issue without changing our queries to GenQuery2 in 4.3 (#290).

Given that 4.3.3.1 is supposed to be a stable release, I think we should delay this until later. I wouldn't rule it out as a possibility for 4.3, but just not for this release.

We can fix the "select" in the object name issue for this issue and circle back for the apostrophe issue in a later release. I've created #328 for the apostrophe situation.

@korydraughn
Copy link
Collaborator

I agree we should delay this until later.

@alanking alanking mentioned this issue Jan 31, 2025
Merged
alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 31, 2025
@alanking
[irods#281] Add tier-out test for object with 'select' in name
ce31f16
alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 31, 2025
@alanking
[irods#281] Fix tiering out objects with 'select' in name
c2af09b 
Using lowercase 'select' rather than 'SELECT' in GenQuery1
is a known workaround for an issue with the historical
parser for dealing with queries that deal with the word
'select'. All internal queries in this plugin have been
converted to use lowercase letters for all GenQuery syntax.

Also, any instance which was touched and was using boost::format
has been changed to use fmt::format.

And updated README to use lowercase 'select' in example
queries, as well.
alanking added a commit that referenced this issue Jan 31, 2025
@alanking
[#281] Add tier-out test for object with 'select' in name
814acdb
alanking added a commit that referenced this issue Jan 31, 2025
@alanking
[#281] Fix tiering out objects with 'select' in name
da343c0 
Using lowercase 'select' rather than 'SELECT' in GenQuery1
is a known workaround for an issue with the historical
parser for dealing with queries that deal with the word
'select'. All internal queries in this plugin have been
converted to use lowercase letters for all GenQuery syntax.

Also, any instance which was touched and was using boost::format
has been changed to use fmt::format.

And updated README to use lowercase 'select' in example
queries, as well.
@alanking alanking mentioned this issue Jan 31, 2025
Merged
alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 31, 2025
@alanking
[irods#281] Add tier-out test for object with 'select' in name
e0a485b
alanking added a commit to alanking/irods_capability_storage_tiering that referenced this issue Jan 31, 2025
@alanking
[irods#281] Fix tiering out objects with 'select' in name
2d6b76f 
Using lowercase 'select' rather than 'SELECT' in GenQuery1
is a known workaround for an issue with the historical
parser for dealing with queries that deal with the word
'select'. All internal queries in this plugin have been
converted to use lowercase letters for all GenQuery syntax.

Also, any instance which was touched and was using boost::format
has been changed to use fmt::format.

And updated README to use lowercase 'select' in example
queries, as well.
alanking added a commit that referenced this issue Jan 31, 2025
@alanking
[#281] Add tier-out test for object with 'select' in name
9fe63fc
alanking added a commit that referenced this issue Jan 31, 2025
@alanking
[#281] Fix tiering out objects with 'select' in name
8229813 
Using lowercase 'select' rather than 'SELECT' in GenQuery1
is a known workaround for an issue with the historical
parser for dealing with queries that deal with the word
'select'. All internal queries in this plugin have been
converted to use lowercase letters for all GenQuery syntax.

Also, any instance which was touched and was using boost::format
has been changed to use fmt::format.

And updated README to use lowercase 'select' in example
queries, as well.
@alanking
Copy link
Contributor

Objects with "select" in the name should now tier out appropriately. Objects with apostrophes in the name will be addressed in #328. Closing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alanking alanking

Labels
bug
Milestone
4.3.3.1
Development

No branches or pull requests
3 participants
@trel @korydraughn @alanking