[Bug]: Server crash during quick match #2650

mikiher · 2024-02-23T06:01:19Z

Describe the issue

This was reported in an audiobookshelf-windows issue.

Combined audiobookshelf-windows log (app + server): logfile.txt

Library scan logs from just before the crash:

Some log lines before the crash and the crash message itself (from logfile.txt):

[Server] [2024-02-20 21:25:52.992] INFO: [LibraryScanner] Starting library scan d1a72115-2bb0-4819-a30e-816bb864d43e for Informative PDFs
[Server] [2024-02-20 21:25:53.346] INFO: [LibraryScanner] Library scan d1a72115-2bb0-4819-a30e-816bb864d43e completed in 0:00.0 | 1 Added | 0 Updated | 0 Missing
[Server] [2024-02-20 21:25:53.444] INFO: [LibraryController] Scan complete
[Server] [2024-02-20 21:25:53.445] INFO: [LibraryScan] Scan log saved "C:\Users\D-Admin\AppData\Local\Audiobookshelf\metadata\logs\scans\2024-02-20_d1a72115-2bb0-4819-a30e-816bb864d43e.txt"
[Server] [2024-02-20 21:25:56.445] INFO: [SocketAuthority] Socket Connected oaITvLcPlAuKQm8YAABR
[Server] [2024-02-20 21:26:00.515] INFO: [SocketAuthority] Socket oaITvLcPlAuKQm8YAABR disconnected from client "root" after 4070ms (Reason: transport close)
[Server] [2024-02-20 21:26:34.985] INFO: [LibraryScanner] Starting library scan 536c2c57-6366-4037-83dd-8b557c8ccd50 for Informative PDFs
[Server] [2024-02-20 21:26:35.129] INFO: [LibraryScanner] Library scan 536c2c57-6366-4037-83dd-8b557c8ccd50 completed in 0:00.0 | 0 Added | 0 Updated | 1 Missing
[Server] [2024-02-20 21:26:35.326] INFO: [LibraryController] Scan complete
[Server] [2024-02-20 21:26:35.328] INFO: [LibraryScan] Scan log saved "C:\Users\D-Admin\AppData\Local\Audiobookshelf\metadata\logs\scans\2024-02-20_536c2c57-6366-4037-83dd-8b557c8ccd50.txt"
[Server] [2024-02-20 21:26:35.359] INFO: [LibraryScanner] Starting library scan 9bd64de4-abdc-4d2b-a0e9-4192b2781d8c for eBooks
[Server] [2024-02-20 21:26:35.681] INFO: [LibraryScanner] Library scan 9bd64de4-abdc-4d2b-a0e9-4192b2781d8c completed in 0:00.0 | 1 Added | 0 Updated | 0 Missing
[Server] [2024-02-20 21:26:35.799] INFO: [LibraryScan] Scan log saved "C:\Users\D-Admin\AppData\Local\Audiobookshelf\metadata\logs\scans\2024-02-20_9bd64de4-abdc-4d2b-a0e9-4192b2781d8c.txt"
[Server] [2024-02-20 21:26:35.821] INFO: [LibraryController] Scan complete
[Server] [2024-02-20 21:27:05.618] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=bragg%20apple%20cider%20vinegar%20miracle%20health%20system"
[Server] [2024-02-20 21:27:05.623] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=with%20the"
[Server] [2024-02-20 21:27:05.627] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=bragg%2C%20paul%20c."
[Server] [2024-02-20 21:27:05.630] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=%2C%201895-1976%2C%20author%3B%20bragg%2C"
[Server] [2024-02-20 21:27:05.633] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=2020"
[Server] [2024-02-20 21:27:05.640] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=santa%20barbara%2C"
[Server] [2024-02-20 21:27:05.643] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=9780877900795"
[Server] [2024-02-20 21:27:05.645] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=f40857a023175425f44b23b3c055b243"
[Server] [2024-02-20 21:27:05.646] INFO: [Audnexus] Searching for author "https://api.audnex.us/authors?name=anna%E2%80%99s%20archive"
[Server] [2024-02-20 21:27:06.923] INFO: [CoverManager] Downloaded libraryItem cover "C:/Users/D-Admin/AppData/Local/Audiobookshelf/metadata/items/d671b140-627b-4328-afbf-0c1bdbd1f33c/cover.jpg" from url "https://books.google.com/books/content?id=w85IJeGVcVkC&printsec=frontcover&img=1&zoom=1&edge=curl&source=gbs_api" for "Bragg apple cider vinegar miracle health system_ with the -- Bragg, Paul C. (Paul Chappuis), 1895-1976, author; Bragg, -- 2020 -- Santa Barbara, -- 9780877900795 -- f40857a023175425f44b23b3c055b243 -- Annaâ€™s Archive"
[Server] node:internal/process/promises:279
[Server]             triggerUncaughtException(err, true /* fromPromise */);
[Server]             ^
[Server] Error
[Server]     at Database.<anonymous> (C:\snapshot\audiobookshelf\node_modules\sequelize\lib\dialects\sqlite\query.js:185:27)
[Server]     at C:\snapshot\audiobookshelf\node_modules\sequelize\lib\dialects\sqlite\query.js:183:50
[Server]     at new Promise (<anonymous>)
[Server]     at Query.run (C:\snapshot\audiobookshelf\node_modules\sequelize\lib\dialects\sqlite\query.js:183:12)
[Server]     at C:\snapshot\audiobookshelf\node_modules\sequelize\lib\sequelize.js:315:28
[Server]     at async SQLiteQueryInterface.insert (C:\snapshot\audiobookshelf\node_modules\sequelize\lib\dialects\abstract\query-interface.js:308:21)
[Server]     at async BookAuthor.save (C:\snapshot\audiobookshelf\node_modules\sequelize\lib\model.js:2490:35)
[Server]     at async Function.create (C:\snapshot\audiobookshelf\node_modules\sequelize\lib\model.js:1362:12)
[Server]     at async Function.fullUpdateFromOld (C:\snapshot\audiobookshelf\server\models\LibraryItem.js)
[Server]     at async Scanner.quickMatchLibraryItem (C:\snapshot\audiobookshelf\server\scanner\Scanner.js) {
[Server]   name: 'SequelizeUniqueConstraintError',
[Server]   errors: [
[Server]     ValidationErrorItem {
[Server]       message: 'bookId must be unique',
[Server]       type: 'unique violation',
[Server]       path: 'bookId',
[Server]       value: 'e1381b45-cefb-4640-a90a-8313a9734d64',
[Server]       origin: 'DB',
[Server]       instance: bookAuthor {
[Server]         dataValues: {
[Server]           id: 'ed7ef9da-2732-4148-a4ed-ed278165ab17',
[Server]           authorId: '1d6b2e72-b072-4ff0-bd46-d483367fb540',
[Server]           bookId: 'e1381b45-cefb-4640-a90a-8313a9734d64',
[Server]           createdAt: 2024-02-21T04:27:07.849Z
[Server]         },
[Server]         _previousDataValues: { authorId: undefined, bookId: undefined },
[Server]         uniqno: 1,
[Server]         _changed: Set(2) { 'authorId', 'bookId' },
[Server]         _options: {
[Server]           isNewRecord: true,
[Server]           _schema: null,
[Server]           _schemaDelimiter: '',
[Server]           attributes: undefined,
[Server]           include: undefined,
[Server]           raw: undefined,
[Server]           silent: undefined
[Server]         },
[Server]         isNewRecord: true
[Server]       },
[Server]       validatorKey: 'not_unique',
[Server]       validatorName: null,
[Server]       validatorArgs: []
[Server]     },
[Server]     ValidationErrorItem {
[Server]       message: 'authorId must be unique',
[Server]       type: 'unique violation',
[Server]       path: 'authorId',
[Server]       value: '1d6b2e72-b072-4ff0-bd46-d483367fb540',
[Server]       origin: 'DB',
[Server]       instance: bookAuthor {
[Server]         dataValues: {
[Server]           id: 'ed7ef9da-2732-4148-a4ed-ed278165ab17',
[Server]           authorId: '1d6b2e72-b072-4ff0-bd46-d483367fb540',
[Server]           bookId: 'e1381b45-cefb-4640-a90a-8313a9734d64',
[Server]           createdAt: 2024-02-21T04:27:07.849Z
[Server]         },
[Server]         _previousDataValues: { authorId: undefined, bookId: undefined },
[Server]         uniqno: 1,
[Server]         _changed: Set(2) { 'authorId', 'bookId' },
[Server]         _options: {
[Server]           isNewRecord: true,
[Server]           _schema: null,
[Server]           _schemaDelimiter: '',
[Server]           attributes: undefined,
[Server]           include: undefined,
[Server]           raw: undefined,
[Server]           silent: undefined
[Server]         },
[Server]         isNewRecord: true
[Server]       },
[Server]       validatorKey: 'not_unique',
[Server]       validatorName: null,
[Server]       validatorArgs: []
[Server]     }
[Server]   ],
[Server]   parent: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: bookAuthors.bookId, bookAuthors.authorId] {
[Server]     errno: 19,
[Server]     code: 'SQLITE_CONSTRAINT',
[Server]     sql: 'INSERT INTO `bookAuthors` (`id`,`createdAt`,`bookId`,`authorId`) VALUES ($1,$2,$3,$4);'
[Server]   },
[Server]   original: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: bookAuthors.bookId, bookAuthors.authorId] {
[Server]     errno: 19,
[Server]     code: 'SQLITE_CONSTRAINT',
[Server]     sql: 'INSERT INTO `bookAuthors` (`id`,`createdAt`,`bookId`,`authorId`) VALUES ($1,$2,$3,$4);'
[Server]   },
[Server]   fields: [ 'bookId', 'authorId' ],
[Server]   sql: 'INSERT INTO `bookAuthors` (`id`,`createdAt`,`bookId`,`authorId`) VALUES ($1,$2,$3,$4);'
[Server] }

Steps to reproduce the issue

I don't fully understand what happened here yet, so I'm not sure how to reproduce.

Audiobookshelf version

v2.7.2

How are you running audiobookshelf?

Windows Tray App

The text was updated successfully, but these errors were encountered:

advplyr · 2024-02-23T09:51:00Z

Usually issues like this are related to the sqlite db being stored on a network drive. The sqlite db has to be on a local drive.

mikiher · 2024-02-23T17:13:18Z

The database is at "C:\Users\D-Admin\AppData\Local\Audiobookshelf\config\absdatabase.sqlite".
It's highly unlikely that this is a network drive.

mikiher · 2024-02-23T17:16:47Z

I have not thouroughly checked the scan logs yet, but I have an intuition that this has to do with some addition->deletion->re-addition sequence.

advplyr · 2024-02-23T21:51:10Z

These unique constraint errors are usually from race conditions

mikiher · 2024-02-24T16:46:41Z

OK, I was able to reproduce it, and this one has nothing to do with race conditions, just a standard logic bug.

TLDR: The bug is caused by duplicate authors extracted from the quick match result.

I ran a quick match on a test book, using the same original title that was used in the original bug report for the quick match search on Google Books. The quick match result had the following author:
"Patricia Bragg, N.D., Ph.D., Paul C Bragg, N.D., PH.D."
This is further split by Scanner.quickMatchBookBuildUpdatePayload into a few authors:

Patricia Bragg
N.D.
Ph.D.
Paul C Bragg
N.D.
Ph.D.

Then the following happens:

[2024-02-24 17:48:08.633] DEBUG: [BookMetadata] Key updated authors [
  {
    id: '9fce60b4-6880-4664-8506-6551bd997b53',
    name: 'Patricia Bragg'
  },
  { id: '8ab84a75-31ad-4da1-bb4e-9e9e6d1a7311', name: 'N.D.' },
  { id: 'ff5d5c9c-828d-4a5c-80ed-b497dc303ec3', name: 'Ph.D.' },
  { id: 'b2b276c1-45fe-4b07-ac33-ac0e5032ae56', name: 'Paul C Bragg' },
  { id: '8ab84a75-31ad-4da1-bb4e-9e9e6d1a7311', name: 'N.D.' },
  { id: 'ff5d5c9c-828d-4a5c-80ed-b497dc303ec3', name: 'Ph.D.' }
] (BookMetadata.js:176)
[2024-02-24 17:48:08.634] DEBUG: [BookMetadata] Key updated publisher Health Science Publications, Inc. (BookMetadata.js:176)
[2024-02-24 17:48:08.634] DEBUG: [BookMetadata] Key updated isbn 9780877905011 (BookMetadata.js:176)
[2024-02-24 17:48:08.635] DEBUG: [LibraryItem] Success saving abmetadata to "F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\metadata\items\78d38635-5f39-45d6-bf99-fb9c75825554\metadata.json" (LibraryItem.js:310)
[2024-02-24 17:48:08.645] DEBUG: [LibraryItem] "Bragg Apple Cider Vinegar Miracle Health System" author "Patricia Bragg" was added (LibraryItem.js:328)
[2024-02-24 17:48:08.651] DEBUG: [ApiCacheManager] bookAuthor.afterCreate: Clearing cache (ApiCacheManager.js:21)
[2024-02-24 17:48:08.651] DEBUG: [LibraryItem] "Bragg Apple Cider Vinegar Miracle Health System" author "N.D." was added (LibraryItem.js:328)
[2024-02-24 17:48:08.655] DEBUG: [ApiCacheManager] bookAuthor.afterCreate: Clearing cache (ApiCacheManager.js:21)
[2024-02-24 17:48:08.656] DEBUG: [LibraryItem] "Bragg Apple Cider Vinegar Miracle Health System" author "Ph.D." was added (LibraryItem.js:328)
[2024-02-24 17:48:08.660] DEBUG: [ApiCacheManager] bookAuthor.afterCreate: Clearing cache (ApiCacheManager.js:21)
[2024-02-24 17:48:08.661] DEBUG: [LibraryItem] "Bragg Apple Cider Vinegar Miracle Health System" author "Paul C Bragg" was added (LibraryItem.js:328)
[2024-02-24 17:48:08.666] DEBUG: [ApiCacheManager] bookAuthor.afterCreate: Clearing cache (ApiCacheManager.js:21)
[2024-02-24 17:48:08.666] DEBUG: [LibraryItem] "Bragg Apple Cider Vinegar Miracle Health System" author "N.D." was added (LibraryItem.js:328)
[2024-02-24 17:48:08.668] FATAL: [Server] Unhandled rejection: SequelizeUniqueConstraintError: Validation error, promise: Promise {
  <rejected> Error
      at Database.<anonymous> (F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\node_modules\sequelize\lib\dialects\sqlite\query.js:185:27)
      at F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\node_modules\sequelize\lib\dialects\sqlite\query.js:183:50
      at new Promise (<anonymous>)
      at Query.run (F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\node_modules\sequelize\lib\dialects\sqlite\query.js:183:12)
      at F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\node_modules\sequelize\lib\sequelize.js:315:28
      at async SQLiteQueryInterface.insert (F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\node_modules\sequelize\lib\dialects\abstract\query-interface.js:308:21)
      at async BookAuthor.save (F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\node_modules\sequelize\lib\model.js:2490:35)
      at async bookAuthor.create (F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\node_modules\sequelize\lib\model.js:1362:12)
      at async libraryItem.fullUpdateFromOld (F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\server\models\LibraryItem.js:329:13)
      at async Database.updateLibraryItem (F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\server\Database.js:431:21) {
    name: 'SequelizeUniqueConstraintError',
    errors: [ [ValidationErrorItem], [ValidationErrorItem] ],
    parent: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: bookAuthors.bookId, bookAuthors.authorId] {
      errno: 19,
      code: 'SQLITE_CONSTRAINT',
      sql: 'INSERT INTO `bookAuthors` (`id`,`createdAt`,`bookId`,`authorId`) VALUES ($1,$2,$3,$4);'
    },
    original: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: bookAuthors.bookId, bookAuthors.authorId] {
      errno: 19,
      code: 'SQLITE_CONSTRAINT',
      sql: 'INSERT INTO `bookAuthors` (`id`,`createdAt`,`bookId`,`authorId`) VALUES ($1,$2,$3,$4);'
    },
    fields: [ 'bookId', 'authorId' ],
    sql: 'INSERT INTO `bookAuthors` (`id`,`createdAt`,`bookId`,`authorId`) VALUES ($1,$2,$3,$4);'
  }
} (Server.js:166)

As you can see, the unique contraint error happens when we try to insert the BookAuthors table entry with the second instance of author 'N.D.' (and with same book), which we already added before.

I'll take a stab at fixing this (by not adding duplicate entries).

There's also the separate issue of having Ph.D and other author titles separated from the author name by the simple comma-separation logic. I'll address that in a separate bug.

advplyr · 2024-03-16T23:05:50Z

Fixed in v2.8.1

bwalocha · 2024-12-01T14:00:15Z

The same issue appeares during Library Scan whet metadata.nfo contains duplicated Author entry (Author: Author_001, Author_001), the latest docker image (v2.17.3):

`General Information

ID: 123
Title: BookTitle
Author: Author_001, Author_001
Director:
Read By:
Series Name: Series
Position in Series: 1
Genre:
Tags:
Release Date: 2000
Publisher:
Language: English
Asin:
ISBN:
ISBN Pdf:
ISBN ePub:
ISBN Mobi:
Audiobook: No
Explicit: No
Abridged: No

Media Information

Source:
Media Types: pdf (1)
Number of Files: 1
Audio Length:
Cover Url:

Book Description

`

mikiher · 2024-12-01T15:13:05Z

Thanks for reporting. I'll handle this.

advplyr · 2024-12-01T15:15:14Z

I was testing that and when deleting the book the authors were not removed. I opened an issue for that and am working on it #3668 (PR #3670)

mikiher · 2024-12-01T18:18:08Z

Are you sure #3670 fixes the complaint above?

The issue seems to be a SequelizeUniqueConstraintError when metadata is read from an nfo file containing duplicate authors.
Have you checked this?

advplyr · 2024-12-01T18:46:28Z

No it doesn't fix this issue. I meant to say that is the PR I opened to fix the issue in #3668. I thought it was relevant since there may be overlap in what you are working on but I wasn't clear in what I meant.
I deleted that and updated my original comment.

mikiher · 2024-12-01T18:56:28Z

Got it, thanks. I'll use your changes when working on this.

advplyr · 2024-12-15T18:28:09Z

@mikiher Are you still working on this?

mikiher · 2024-12-15T18:38:43Z

Not recently. I think this is still an issue. Can you reopen and assign this to me?

mikiher added the bug Something isn't working label Feb 23, 2024

This was referenced Feb 25, 2024

Fix dup author addition logic #2659

Merged

[Bug]: Incorrect authors created from match results due to comma-separation #2661

Open

advplyr closed this as completed in #2659 Feb 25, 2024

mikiher mentioned this issue Feb 25, 2024

Audiobookshelf crashes randomly? mikiher/audiobookshelf-windows#8

Closed

mikiher mentioned this issue Jul 11, 2024

[Bug]: FATAL: [Server] Unhandled rejection: SequelizeUniqueConstraintError: Validation error #3123

Closed

advplyr reopened this Dec 15, 2024

advplyr assigned mikiher Dec 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Server crash during quick match #2650

[Bug]: Server crash during quick match #2650

mikiher commented Feb 23, 2024

advplyr commented Feb 23, 2024

mikiher commented Feb 23, 2024

mikiher commented Feb 23, 2024

advplyr commented Feb 23, 2024

mikiher commented Feb 24, 2024

advplyr commented Mar 16, 2024

bwalocha commented Dec 1, 2024 •

edited

Loading

mikiher commented Dec 1, 2024

advplyr commented Dec 1, 2024 •

edited

Loading

mikiher commented Dec 1, 2024

advplyr commented Dec 1, 2024 •

edited

Loading

mikiher commented Dec 1, 2024

advplyr commented Dec 15, 2024

mikiher commented Dec 15, 2024 •

edited

Loading

[Bug]: Server crash during quick match #2650

[Bug]: Server crash during quick match #2650

Comments

mikiher commented Feb 23, 2024

Describe the issue

Steps to reproduce the issue

Audiobookshelf version

How are you running audiobookshelf?

advplyr commented Feb 23, 2024

mikiher commented Feb 23, 2024

mikiher commented Feb 23, 2024

advplyr commented Feb 23, 2024

mikiher commented Feb 24, 2024

advplyr commented Mar 16, 2024

bwalocha commented Dec 1, 2024 • edited Loading

`General Information

Media Information

Book Description

mikiher commented Dec 1, 2024

advplyr commented Dec 1, 2024 • edited Loading

mikiher commented Dec 1, 2024

advplyr commented Dec 1, 2024 • edited Loading

mikiher commented Dec 1, 2024

advplyr commented Dec 15, 2024

mikiher commented Dec 15, 2024 • edited Loading

bwalocha commented Dec 1, 2024 •

edited

Loading

advplyr commented Dec 1, 2024 •

edited

Loading

advplyr commented Dec 1, 2024 •

edited

Loading

mikiher commented Dec 15, 2024 •

edited

Loading