Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLOB type is always decoded to string #461

Open
robinvandernoord opened this issue Jun 20, 2023 · 2 comments
Open

BLOB type is always decoded to string #461

robinvandernoord opened this issue Jun 20, 2023 · 2 comments

Comments

@robinvandernoord
Copy link

robinvandernoord commented Jun 20, 2023

I'm having an issue with blobs in the database. According to the documentation, this field type is mapped to Python str.
When looking at the code, this is done with .decode('utf-8'). However, in our database we have some raw images, which of course can't be decoded as utf-8.
image
When looking through the adapter code (emmett/orm/adapters.py 229: def parse), I saw an option blob_decode.
However, I can't seem to find where to set that option. Is this an argument to app.config.db or adapter_args?
I have monkey patched the emmett.orm.adapters.parse method to forcefully set blob_decode to False, but that's very hacky and I've already seen some selects where my patch is not applied.

Summary: how do I get raw bytes from my bytea/blob database fields?

@gi0baro
Copy link
Member

gi0baro commented Jun 27, 2023

BLOB encoding/decoding is a pyDAL feature kept in place for retro-compatibility reasons.

In order to use raw BLOB values with postgres, you should define your own parser/representer, eg:

from emmett.orm.engines import adapters
from emmett.orm.engines.postgres import (
    PostgresAdapter, 
    JSONBPostgreParser,
    JSONBPostgreRepresenter,
    parse_type,
    repr_type
)

class PlainBlobParser(JSONBPostgreParser):
    @parse_type('blob')
    def _blob(self, value):
        return value

class PlainBlobRepresenter(JSONBPostgreRepresenter):
    @repr_type('blob')
    def _blob(self, value):
        return value

@adapters.register('postgres')
class PlainBlobPostgresAdapter(PostgresAdapter):
    def _load_dependencies(self):
        super()._load_dependencies()
        self.parser = PlainBlobParser(self)
        self.representer = PlainBlobRepresenter(self)

I know it's not super fancy, eventually for 2.6 I came up with a setting in Database config to do this.

@robinvandernoord
Copy link
Author

Thanks for the solution, I'll try this soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants