Error Loading Extensions with COI

[amiller-gh]

What happens?

I switched my codebase to the experimental COI implementation, and all extension loading began to fail with:

Error: IO Error: Extension "https://extensions.duckdb.org/v1.1.2/wasm_threads/parquet.duckdb_extension.wasm" could not be loaded: Could not load dynamic lib: parquet
LinkError: WebAssembly.Instance(): Import #3541 "env" "memory": mismatch in shared state of memory, declared = 0, imported = 1

My instantiation is as follows:

import * as duckdb from '@duckdb/duckdb-wasm';
import duckdb_pthread_worker from "@duckdb/duckdb-wasm/dist/duckdb-browser-coi.pthread.worker.js?url";
import coi_worker from '@duckdb/duckdb-wasm/dist/duckdb-browser-coi.worker.js?url';
import eh_worker from '@duckdb/duckdb-wasm/dist/duckdb-browser-eh.worker.js?url';
import mvp_worker from '@duckdb/duckdb-wasm/dist/duckdb-browser-mvp.worker.js?url';
import duckdb_wasm_coi from '@duckdb/duckdb-wasm/dist/duckdb-coi.wasm?url';
import duckdb_wasm_eh from '@duckdb/duckdb-wasm/dist/duckdb-eh.wasm?url';
import duckdb_wasm from '@duckdb/duckdb-wasm/dist/duckdb-mvp.wasm?url';

const MANUAL_BUNDLES: duckdb.DuckDBBundles = {
  mvp: { mainModule: duckdb_wasm, mainWorker: mvp_worker },
  eh: { mainModule: duckdb_wasm_eh, mainWorker: eh_worker },
  coi: { mainModule: duckdb_wasm_coi, mainWorker: coi_worker, pthreadWorker: duckdb_pthread_worker },
};

const logger = new duckdb.ConsoleLogger();
const bundle = await duckdb.selectBundle(MANUAL_BUNDLES);
const worker = new Worker(bundle?.mainWorker as string);
const db = new duckdb.AsyncDuckDB(logger, worker);
await db.instantiate(bundle?.mainModule as string, bundle?.pthreadWorker as string);

// Confirmed that this logs the COI modules
console.log('Using DuckDB bundle: ', bundle); 

(Interestingly, I get a different error when I try to load the spatial extension: the fetch call errors with a 403 forbidden, this may be a different error entirely)

To Reproduce

See above instantiation code. I've tested this on 1.92.0, 1.29.1-dev13.0, 1.28.1-dev297.0, and 1.28.1.

Browser/Environment:

Chrome 130

Device:

Macbook Pro

DuckDB-Wasm Version:

1.29.0, 1.29.1-dev13.0, 1.28.1-dev297.0, and 1.28.1

DuckDB-Wasm Deployment:

Loaded locally, installed via NPM, extensions from extensions.duckdb.org

Full Name:

Adam Miller

Affiliation:

Universe

[amiller-gh]

If it helps, this is the error thrown when DuckDB tries to automatically load the extension instead of when I explicitly install it:

[amiller-gh]

Still fails on the latest 1.29.1-dev17.0 build, new URL this time: https://extensions.duckdb.org/v1.1.3/wasm_threads/parquet.duckdb_extension.wasm.

Fails with an AccessDenied error.

[amiller-gh]

Possibly related: #1916

@carlopi, any secret to loading these extensions?

It seems like https://extensions.duckdb.org/v1.1.1 and https://extensions.duckdb.org/v1.1.2 exist, and can be downloaded, but are missing an expected symbol for COI builds? https://extensions.duckdb.org/v1.1.3 extensions appear to not be present at all.

[ajwerner]

I'm also very interested in getting this to work. The COI/threads can be critical for latency of page loads due to the blocking waterfall described in #381 and friends. I'm fine with building my own versions of these extensions (in my case, json), if there's a way to get it all working.