embedding demension change bug

[huangzhuo1949]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

v0.15.1

Cloud or Self Hosted

Self Hosted (Source), Self Hosted (Docker)

Steps to reproduce

ERROR:root:consume document failed
Traceback (most recent call last):
  File "/app/api/core/indexing_runner.py", line 205, in run
    raise e
  File "/app/api/core/indexing_runner.py", line 195, in run
    self._load(
  File "/app/api/core/indexing_runner.py", line 831, in _load
    tokens += future.result()
              ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/api/core/indexing_runner.py", line 888, in _process_chunk
    index_processor.load(dataset, chunk_documents, with_keywords=False)
  File "/app/api/core/rag/index_processor/processor/paragraph_index_processor.py", line 57, in load
    vector.create(documents)
  File "/app/api/core/rag/datasource/vdb/vector_factory.py", line 151, in create
    self._vector_processor.create(texts=texts, embeddings=embeddings, **kwargs)
  File "/app/api/core/rag/datasource/vdb/pgvector/pgvector.py", line 95, in create
    return self.add_texts(texts, embeddings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/api/core/rag/datasource/vdb/pgvector/pgvector.py", line 112, in add_texts
    psycopg2.extras.execute_values(
  File "/app/api/.venv/lib/python3.12/site-packages/psycopg2/extras.py", line 1299, in execute_values
    cur.execute(b''.join(parts))
psycopg2.errors.DataException: expected 1536 dimensions, not 1024

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot]

To resolve the "expected 1536 dimensions, not 1024" error in your self-hosted Dify v0.15.1 setup, you should ensure that the embedding model is set to 'text-embedding-ada-002', which uses 1536 dimensions. Additionally, check the settings of each document in your dataset to confirm they are using the same embedding model. If there are any documents with inconsistent dimensions, reindex them to align all documents to the expected 1536 dimensions ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[huangzhuo1949]

fix in #12914