SNOW-1774037: OCSP check fails when execption is not of type RevocationCheckError #2094
Assignees
Labels
bug
status-fixed_awaiting_release
The issue has been fixed, its PR merged, and now awaiting the next release cycle of the connector.
status-triage_done
Initial triage done, will be further handled by the driver team
Comments
github-actions
bot
changed the title
sfc-gh-dszmolka
added
status-triage
|
hi - thanks for raising this issue with us. taking a look. do you think it would be possible to post a full(er) stack of the error thrown? we supposed to raise a edit: re: to diagnose why you're having OCSP errors, this could be helpful:
|
sfc-gh-dszmolka
added
the
status-information_needed
|
Hi @sfc-gh-dszmolka , thank you for the follow up. Yes, we are currently in the process of diagnosing the OCSP issue within our environment, suspect its a timeout due to some missing whitelisting. Errors started appearing with no code changes on our side so assuming something in either our SF tenant or GCP infra has changed. Will report back findings. Regardless, I suspect the error routine there is incorrect, which is making pin pointing this issue harder. Thanks again. |
sfc-gh-dszmolka
added
status-triage_done
|
Here are the results from the OCSP check, noting the failure on OCSP_RESPONDER: ocsp.pki.goog:80: URL Check: Failed: HTTP/1.1 404 Not Found |
|
this issue seems to be coming from SnowCD itself, and is unrelated to the problem. (it looks to be trying to check
Is it possible to share it, from the original issue you saw? the stack of exceptions from the the python application, not just a single error line? |
|
Hello, everyone. On our project there was exactly the same problem described above knowhoper. The problem appeared also unexpectedly and without any changes in code/infrastructure. The error in the logs fully corresponds to the error attached by knowhoper. So far it turned out that everything works fine with insecure_mode=True. I would be very grateful for any help and information related to this issue. All the logs we were able to get: Some important dependencies: |
|
hi @flekkk thank you, this response was very useful! looks like, at least in your case, something causes the public key verification to somehow get 5 arguments: despite only four should be passed: public_key.verify(
signature, digest, padding.PKCS1v15(), utils.Prehashed(chosen_hash)
)and we don't have error handling for this situation as it should never happen. Anyways; if you're up for a little more debugging and gathering more info, I created a patched version of 0.1 make sure you're running Snowflake PythonConnector 3.12.3, as the patch was made for this version. Should work with other 3.12.x too. I see you already have it.
or you can just use this very simple one to connect to Snowflake and import snowflake.connector
import logging
import os
for logger_name in ['snowflake','botocore']:
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
ch = logging.FileHandler('python_connector.log')
ch.setLevel(logging.DEBUG)
ch.setFormatter(logging.Formatter('%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
logger.addHandler(ch)
conn = snowflake.connector.connect(
user='REPLACE_USER',
password='REPLACE_PASSWORD',
account='REPLACE-ACCOUNT',
database='REPLACE_DB',
schema='REPLACE_SCHEMA'
)
conn.cursor().execute("PUT file:///PATH/TO/test.txt @REPLACE_STAGE OVERWRITE=TRUE")Main thing is, that either way, the DEBUG level logs need to be turned on. Thanks for your help here ! |
|
Hi, this seems to still happen for me even when upgraded to 3.12.3, started happening 11/8/24 any thoughts on what's wrong? it only happens for a very specific query that i do, only fetches 400 rows and I'm not sure why it happens using insecure_mode=True solves the issue |
|
@opqpop & folks, this looks to be some issue in GCP US CENTRAL, when verifying the certificate during Snowflake/GCP Bucket connection. So far, could not reproduce with an account in GCP US CENTRAL, so if any of you has the possibility to try with the method described in this comment in the environment where the issue reproduces for you, and shares the debug logs, that could potentially massively help. Thanks in advance ! |
|
Hi, @sfc-gh-dszmolka please see attached logs. You will see the error and debug information in there. Noting the logs were flooded with the line Any help would be much appreciated. |
|
this is super helpful @knowhoper , thank you ! This is unexpected. The Python Connector seems to be unprepared for this type of certificate, we expect them to be signed with a RSA key. During handling the certificate, it looks as if it is parsed into several parts I don't think this works well with I'm not entirely sure how you end up with connecting to the regional GCS endpoint ( While the bug is being investigated then later fixed, if you're up for a bit more experimenting, here's another patched version of It has no other change just to add a generic way of handling any kind of exception which might come from edit: also, because no matter how I try to connect to Just in case this is caused by a bad / old cache, can you please try removing contents of |
|
Hi @sfc-gh-dszmolka, I've added the patched file in and don't receive any error with this. See attached logs. |
|
great to hear that @knowhoper and thank you for testing it so quickly + sharing the logs ! We'll fix it in the driver so a patch would not be needed. I'll keep this thread updated with the progress. |
sfc-gh-dszmolka
added
the
status-in_progress
7 tasks
|
@sfc-gh-dszmolka your change works, I see it in the logs posted by @knowhoper but it causes the connector to go into fail-open mode and stop checking OCSP. I'd like to test another solution before we advise people to do that though. |
|
thanks a ton @sfc-gh-mkeller indeed my patch was only intended no more than a quick band-aid; so if any of the folks here who experience the issue could test it with installing the driver with the proposed fix from the dev branch, that would be massive help @knowhoper @flekkk @opqpop if you get a chance 🙇 |
|
Sure thing! FYI I used the following poetry declaration, we are seeing no errors now. |
|
really appreciate for testing so quick @knowhoper , looks good! |
sfc-gh-dszmolka
added
status-pr_pending_merge
|
Let's get the code merging! If anyone encounters an issue feel free to reach out the team |
|
the fix has been merged! however it auto-closed this issue, so i'm reopening it to be able to properly track with the release |
sfc-gh-dszmolka
added
status-fixed_awaiting_release
|
Hey guys, when will the fix be released, indicatively? I am having lots of trouble fetching company data from Snowflake |
|
there's now Thanksgiving in the US so i believe this might also imply some change freeze this week, so earliest next week I think. But even now, you can fully unblock yourself by
conn = snowflake.connector.connect(
account='myaccount',
..
insecure_mode=True,i'll keep this thread posted about the release. |
|
Thanks, it works for now. |
|
released in PythonConnector v3.12.4 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Python version
3.10.3
Operating system and processor architecture
macOS-14.7-arm64-arm-64bit
Installed packages
What did you do?
The text was updated successfully, but these errors were encountered: