Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SNOW-1366543: Log Error coming from line CArrowTableIterator.cpp#L230 making a lot noise #1939

Closed
rajathpatel23 opened this issue May 4, 2024 · 8 comments
Closed

SNOW-1366543: Log Error coming from line CArrowTableIterator.cpp#L230 making a lot noise #1939

rajathpatel23 opened this issue May 4, 2024 · 8 comments
Assignees
@sfc-gh-jrose
Labels
bug status-fixed_awaiting_release The issue has been fixed, its PR merged, and now awaiting the next release cycle of the connector. status-triage_done Initial triage done, will be further handled by the driver team

Comments

@rajathpatel23
Copy link

rajathpatel23 commented May 4, 2024 •
edited
Loading

We are getting this log error while processing data from Snowflake. It seems to be making a lot of noise; I would like to understand - what was the change made in the snowflake ticket? I feel the logger there is more of debug logger than actual log error.

The changes introduced in this PR seem to be start to give us the log error

Could you help me with this? Also, can you provide more info on the support for pandas that have come with this change?
@github-actions github-actions bot changed the title The log error seems to causing a lot noise SNOW-1366543: The log error seems to causing a lot noise May 4, 2024
@rajathpatel23 rajathpatel23 changed the title SNOW-1366543: The log error seems to causing a lot noise Log Error in coming from line CArrowTableIterator.cpp#L230 making a lot noise May 4, 2024
@rajathpatel23 rajathpatel23 changed the title Log Error in coming from line CArrowTableIterator.cpp#L230 making a lot noise Log Error coming from line CArrowTableIterator.cpp#L230 making a lot noise May 4, 2024
@rajathpatel23
Copy link
Author

I tried raising a support ticket - however I could not create one, hence having this issue as a placeholder for now.

@sfc-gh-dszmolka sfc-gh-dszmolka mentioned this issue May 6, 2024
Closed
@sfc-gh-jrose
Copy link
Contributor

Hey Rajat,
That error log is indeed very noisy. It is left over from a debug session that I had while working on the PR you mentioned and does not have a lot of value outside of that. I've created a PR here to remove it: #1942

That PR was for adding support for structured types to the pandas_fetch_all function. Currently most nested data returned by the connector comes as json encoded strings, but in the future it will support returning structured data instead. This was one part of the overall change to add that support.

@sfc-gh-dszmolka
Copy link
Contributor

hi - thank you for submitting this issue with us.

Can you please elaborate what kind of log messages exactly you mean? Do you have a snippet from a log where the unexpected loglines would be visible ? Please make sure to sanitize them from any account etc. details.

We have some more information about the structured data types mentioned in the fetch_pandas_all changelog entry, please take a peek here if you're interested in this Public Preview feature.

@sfc-gh-dszmolka
Copy link
Contributor

ah, late to the party :)

@sfc-gh-dszmolka sfc-gh-dszmolka added bug status-pr_pending_merge A PR is made and is under review status-triage_done Initial triage done, will be further handled by the driver team labels May 6, 2024
@sfc-gh-dszmolka sfc-gh-dszmolka changed the title Log Error coming from line CArrowTableIterator.cpp#L230 making a lot noise SNOW-1366543: Log Error coming from line CArrowTableIterator.cpp#L230 making a lot noise May 6, 2024
@rajathpatel23
Copy link
Author

@sfc-gh-jrose Thank you so much for taking a look at this quickly.

@parinpshah94
Copy link

@sfc-gh-jrose Thanks for making this fix, do you know when a release with this fix will be published to PyPi?

@sfc-gh-jrose
Copy link
Contributor

The version for this fix should be published later this week.

@sfc-gh-dszmolka
Copy link
Contributor

version 3.10.1 released with the fix

@sfc-gh-dszmolka sfc-gh-dszmolka added status-fixed_awaiting_release The issue has been fixed, its PR merged, and now awaiting the next release cycle of the connector. and removed status-pr_pending_merge A PR is made and is under review labels May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sfc-gh-jrose sfc-gh-jrose

Labels
bug status-fixed_awaiting_release The issue has been fixed, its PR merged, and now awaiting the next release cycle of the connector. status-triage_done Initial triage done, will be further handled by the driver team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@parinpshah94 @rajathpatel23 @sfc-gh-dszmolka @sfc-gh-jrose