Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi Query will cause create two duckdb driver #7581

Open
wlf061 opened this issue Dec 27, 2023 · 2 comments
Open

Multi Query will cause create two duckdb driver #7581

wlf061 opened this issue Dec 27, 2023 · 2 comments
Assignees
Labels
question The issue is a question. Please use Stack Overflow for questions.

Comments

@wlf061
Copy link

wlf061 commented Dec 27, 2023

Describe the bug
I build a new image from cubejs/cube:v0.33.61, and fork duckdb driver:https://github.com/cube-js/cube/tree/v0.34.40/packages/cubejs-duckdb-driver, add log on the init function.
When i send post query, the querytype is multi, There are two records about init,i think the dirver's constructor has called twice,and created to duckdb instance.

To Reproduce
Steps to reproduce the behavior:

  1. Frok duckdb' project to repo and name it "customize-duckdb-cubejs-driver" , add log in constructor and init function
    image

  2. Build New Image:

  • Dockfile
    `
    FROM cubejs/cube:v0.34.40
    RUN apt-get update && apt-get install -y git
    RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts
    RUN mkdir -p node_modules
    COPY package.json .
    COPY cube.js .
    COPY schema ./schema
    RUN --mount=type=ssh npm install

`

  • the package.json that dependencies the repo of step1
    { ... "dependencies": { "customize-duckdb-cubejs-driver": "bitbucket:test/customize-duckdb-cubejs-driver" } ... }
  1. start cube service
  • docker-compose.file
    image
  1. Send Multi Query
    { "query": { "measures": [ "test.currency", ], "dimensions": [ "test.local_book_date" ], "timeDimensions": [{ "dimension": "test.local_book_date", "compareDateRange": [["2023-06-01","2023-07-31"], ["2023-08-01","2043-12-11"]] } ] }, "queryType":"multi" }

The driver and db instance are init twice, the related log
`
taihu-cube-pc-cube_api-1 | create duckdb driver
taihu-cube-pc-cube_api-1 | create duckdb driver
.....
taihu-cube-pc-cube_api-1 | init duckdb
taihu-cube-pc-cube_api-1 | init duckdb

`
Expected behavior
The Driver and db are init once.

Version:
v0.33.61 and v.34.40

@igorlukanin igorlukanin added the question The issue is a question. Please use Stack Overflow for questions. label Jan 2, 2024
@igorlukanin
Copy link
Member

Hi @wlf061 👋 Let me add @ovr so he might provide his opinion as well. I think there are no strict guarantees on how many times a driver would be initialized. It might happen multiple times while Cube is running.

Do you think this is something undesirable?

@wlf061
Copy link
Author

wlf061 commented Jan 3, 2024

@igorlukanin Thanks for your response. I think there has no problem,if init and release are performed in pairs.
But from the code:

  1. init code
    `
    public async getOrchestratorApi(context: RequestContext): Promise {
    ....
    this.orchestratorStorage.set(orchestratorId, orchestratorApi);

`
we use default datasource and there has no configure of orchestrator, from the log, the orchestratorId are same for the multi query, Although created twice, only one instance is stored here.

  1. release execution chain
    `
    servers.ts:releaseConnections ->OrchestratorStorage.ts:releaseConnections->OrchestratorApi.ts:release->driver.release
    public async releaseConnections() {
    const result = [];

    for (const orchestratorApi of this.storage.values()) {
    result.push(orchestratorApi.release());
    }

    await Promise.all(result);

    this.storage.clear();
    }

`
you can see, there has just release one driver.I think there has a leak.
If there is something wrong with the analysis here, please let me know

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question The issue is a question. Please use Stack Overflow for questions.
Projects
None yet
Development

No branches or pull requests

3 participants