Replies: 11 comments 34 replies
-
|
The only place where file metadata is used is in the downloader when creating I think you point out something important, the scanner should definitely be checking all the metadata fields to populate the audiobook details (the scanner does use the track # metadata tag already to try to get the correct order of the tracks). As someone who has a lot of experience with the other media servers, I would really appreciate your input on this. If you could outline your ideal scenario that would be a great place to start. |
Beta Was this translation helpful? Give feedback.
-
|
I will put something together on an example use case and try to spread the word to some other folks who might have input too. I think the right way to do it is likely to store the metadata in an external nfo. I have been burnt in the past where another application has overwritten my previously curated ID3 tags. Being able to import and export to a nfo makes life easier and allows one to share a library of media between different applications without compatibility issues. Is/will scraping be within the scope of this project? |
Beta Was this translation helpful? Give feedback.
-
|
I spent a bit of time looking over my library and thinking about my ideal situation for metadata management. I realize that there is no single correct way to do things but this is what I currently have. About 90% of my library is in m4b format. I still have a few books in multitrack mp3 format that I will eventually combine into a single m4b when time permits. The majority of my books originate from Audible but I am not the one personally ripping them so existing tags are not always complete. Artist and Album Artist: Author I haven't been using it but possible Series and Series Part could be useful to some people. I also have a desc.txt for each book containing a summary of the book. I find the description tag is almost never populated when I grab a new audiobook and must be added manually. In addition to booksonic using the desc.txt, I believe some m4b formatting tools like m4b-tool can use the desc.txt to bake in the description automatically when combing multiple mp3's into a m4b. My preference would be to scan the tags during import and output a single nfo to be stored in the book directory with all the relevant metadata needed in Audiobookshelf. This way the original files are not modified. |
Beta Was this translation helpful? Give feedback.
-
|
I need to put some thought into this, but question first: Is desc.txt a normal naming convention for other software? Or is booksonic just picking it up because it is a text file? |
Beta Was this translation helpful? Give feedback.
-
|
I've only come across it it in booksonic |
Beta Was this translation helpful? Give feedback.
-
|
So I wrote a little script to generate an
I think the important part of handling metadata is going to be pulling it from the audio files and using it in AudioBookshelf. Then users can optionally have an |
Beta Was this translation helpful? Give feedback.
-
|
The challenge with sourcing metadata from the audio files is going to be matching metadata fields with what they actually are. I'm looking through my audiobooks and there isn't a standard being followed. Sometimes the author, not the narrator, is stored in the It looks like the description is sometimes stored in the We may need to set the standard if there is none already. |
Beta Was this translation helpful? Give feedback.
-
|
Honestly that is half the problem and also why this project has so much potential, there is no standard and no leader in the audiobook space to set a standard. If you ask 10 people how they organize their books you will get 5 different answers. I'm guessing that Plex probably has the biggest market share for self hosted audiobook software right now. Making it easy for those users to easily migrate to Audiobookshelf will likely be a major win. It may just take setting a standard, sticking too it, and becoming the leader in this space. |
Beta Was this translation helpful? Give feedback.
-
|
There is some good information in this document |
Beta Was this translation helpful? Give feedback.
-
|
Just scanned my library and it picked up all my desc.txt perfectly. I don't think it is bringing in the tags correctly though. I looked at a few books that have the date, composer, and artist tags populated but the details fields in Audiobookshelf remain blank after a scan. The log file doesn't show much. Is there a way to increase the verbose level for debugging purposes? |
Beta Was this translation helpful? Give feedback.
-
|
It only scrapes the metadata for new audiobooks. Have you tried with removing the audiobook and scanning again? |
Beta Was this translation helpful? Give feedback.
-
|
That did the trick. |
Beta Was this translation helpful? Give feedback.
-
|
One thing to note, removing and rescanning will cause you to lose your playback progress. My environment is for testing but any users who have been using the software for a while now might not be too keen on losing that history. |
Beta Was this translation helpful? Give feedback.
-
|
Agree, I did 2 things to try to remedy this for existing users.
The re-scan button is going to check the embedded cover art and the metadata on audio files again I updated the website docs with the metadata mapping info, but it is pretty weak. At some point these docs are going to need to be organized better. |
Beta Was this translation helpful? Give feedback.
-
|
Another update related to metadata in v1.3.4 You can now opt to store cover art in your audiobook directory. And cover art from URLs are downloaded and saved, so http:// urls won't be set on audiobooks going forward. This is important because those URLs can change or get taken down, and it is also better to not be making all those requests outside your server to get cover art. |
Beta Was this translation helpful? Give feedback.
-
|
I don't know if you are still using the OpenLibrary API to scrape for info but what are your thoughts on grabbing the GoodReads ID if the API returns it? If it can't be automated with a scan even having a field in the details sections would be helpful for now. I see it being presented something like this where the ID is a link to the relevant GoodReads page. |
Beta Was this translation helpful? Give feedback.
-
|
With movies and series there is 1 id for IMDB and TMDB. With ISBN a book can have dozens of ISBN's of 13 or 10 digits that come up with unpredictable results in my testing. The original Book class had an ISBN field. I'm open to adding the ISBN field back, but where do you suggest we fetch metadata from with that ISBN? |
Beta Was this translation helpful? Give feedback.
-
|
My thought was to continue using openlibrary with the hope that searching by ISBN would help improve the results but if the book description coming out of openlibrary are a mess it isn't really a proper solution. I don't really have an alternative suggestion. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
I've used google books API it is the best one available right now I think, but I thought all of the requests required an API key. I had completely dismissed google books api because I thought it was pay only, nice find |
Beta Was this translation helpful? Give feedback.
-
|
Google books turned out to be a success so far. I released book matching as experimental in It is much faster then open library and so far I'm getting better results, especially for description. Let me know what you think and if you are getting accurate results. |
Beta Was this translation helpful? Give feedback.
-
|
I posted a discussion about this, but research indicates that this thread might be more appropriate. The NFO creation seems to have been removed in the latest build. Is there any plan for a replacement? Audiobookself is a wonderful metadata editor, especially with its "Match" support. But without some way to export my edits, I can't feed them back to the rest of my scripts, and since the mobile app doesn't work for me, I have to export the metadata to use it. |
Beta Was this translation helpful? Give feedback.
-
|
I don't plan on adding it back in, the new abmetadata file is a much better solution. If you explain your use-case in more detail maybe we can find another way for you to do what you need to do. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you! I like the abmetadata file, but it's missing one thing: a button to CHOOSE to save it to the book folder just for that one book. That button was removed when NFO format was removed. I don't miss NFO, but I do miss that button. The use case is that most books are automatically added to the database based on extracted metadata and folder names, so they're generally "good enough." But sometimes I comb through books, for example by looking at everything a given author wrote, and notice a book's out of its series (super common, the providers almost never get that right). So I fix it, probably by clicking "Match", but also by editing directly. Anyhow, I want my scripts to know about the change ... so I used to click the "Save Metadata to Folder" button. That means the created file is golden. This isn't as good if the abmetadata is ALWAYS in the book's folder (as with the option). That means I have to decide whether I edited the metadata or not, and it turns out it's REALLY hard to get all the little corner cases inside a script. OTOH, it would be (for my use case) just as good if the abmetadata file had a flag that meant something like "manually inspected and approved." I'd be fine with that. |
Beta Was this translation helpful? Give feedback.
-
|
What about using the ASIN or ISBN fields to determine whether it was correctly matched? |
Beta Was this translation helpful? Give feedback.
-
|
Not bad, and if I want to edit by hand I can just make something up. (That
will happen with podcast books.)
…On Wed, Apr 27, 2022 at 7:01 PM advplyr ***@***.***> wrote:
What about using the ASIN or ISBN fields to determine whether it was
correctly matched?
—
Reply to this email directly, view it on GitHub
<#59 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ7H6NDF4IV7WGXFZK3B3TVHHWN5ANCNFSM5EZJHD2A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
It looks like you're using an extension of the FFMPEG "metadata" output as the meta data format. There are a few gotchas with that as a format. First, it isn't simple to parse, as the chapters require special handing (so you can't simply generate key=value pairs for everything) like a normal property file. Secondly, properly escaping the characters is a little funky if you want to use ffmpeg's standard. But except for the rare case of leading or trailing spaces, you can conform to the ffmpeg standard by just escaping =;#\n and \ characters with a leading . But I didn't see any escaping going on in a first glance at your code... so not sure. Lastly, the key=value format gets a little ugly when you're dealing with lots of data and possibly multi-line data. I personally json is a better format.. its pretty easy to read as a human and most of the time a computer is going to read it and render it. |
Beta Was this translation helpful? Give feedback.
-
|
Agree JSON is easier to work with in code. The reason I started with a file similar to ffmpegs is because I thought it would be easier for the user to read it and update it as necessary. I am not using the exact specifications of ffmetadata, I was just basing a similar metadata format off of that, so I'm not sure a lot of that applies. I did have to work with ffmetadata when embedding metadata in with ffmpeg and passing in a text file with a list of audio files for transcoding. https://github.com/advplyr/audiobookshelf/blob/master/server/utils/ffmpegHelpers.js#L7 I am calling the metadata format abmetadata and the generator and parser is here: https://github.com/advplyr/audiobookshelf/blob/master/server/utils/abmetadataGenerator.js Here is an example abmetadata file: This is harder to parse in scripts, but easier to read and update for users. I'm not set on using this, I could be convinced to just use JSON or YAML. |
Beta Was this translation helpful? Give feedback.
-
|
Question: right now I'm pregenerating OPF when I move folders into
Audiobookshelf's folder, it seems to be pretty reliable. What would happen
if I instead created one of these metadata files instead? Would my
generated file be looked at first, and loaded without further
modifications? That would actually simplify my scripts.
…-Wm
On Sat, Apr 30, 2022 at 7:52 AM advplyr ***@***.***> wrote:
Agree JSON is easier to work with in code. The reason I started with a
file similar to ffmpegs is because I thought it would be easier for the
user to read it and update it as necessary. I am not using the exact
specifications of ffmetadata, I was just basing a similar metadata format
off of that, so I'm not sure a lot of that applies.
I did have to work with ffmetadata when embedding metadata in with ffmpeg
and passing in a text file with a list of audio files for transcoding.
https://github.com/advplyr/audiobookshelf/blob/master/server/utils/ffmpegHelpers.js#L7
I am calling the metadata format abmetadata and the generator and parser
is here:
https://github.com/advplyr/audiobookshelf/blob/master/server/utils/abmetadataGenerator.js
This is not meant to be used in ffmpeg.
Any metadata that is multiple lines is put in its own section like how
chapters are done in ffmetadata. Currently only description could have
multiple lines.
Here is an example abmetadata file:
;ABMETADATA2
#audiobookshelf v2.0.7
media=book
title=Wizards First Rule
subtitle=
authors=Terry Goodkind
narrators=Sam Tsoutsouvas
publishedYear=2008
publisher=Brilliance Audio
isbn=
asin=B002V0QK4C
language=
genres=Audiobook Fantasy
series=Sword of Truth
explicit=N
[DESCRIPTION]
The masterpiece that started Terry Goodkind's New York Times bestselling epic Sword of Truth In the aftermath of the brutal murder of his father, a mysterious woman, Kahlan Amnell, appears in Richard Cypher's forest sanctuary seeking help...and more.
[CHAPTER]
start=0
end=6004.6675
title=Terry Goodkind - SOT Bk01 - Wizards First Rule 01
[CHAPTER]
start=6004.6675
end=12000.946
title=Terry Goodkind - SOT Bk01 - Wizards First Rule 02
[CHAPTER]
start=12000.946
end=17997.250625
title=Terry Goodkind - SOT Bk01 - Wizards First Rule 03
This is harder to parse in scripts, but easier to read and update for
users. I'm not set on using this, I could be convinced to just use JSON or
YAML.
—
Reply to this email directly, view it on GitHub
<#59 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ7H6IRDW6KIBM3YLE233DVHVCKLANCNFSM5EZJHD2A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
If the abmetadata file is saved as I wouldn't say that file is finalized yet, it is possible we move to a different format of metadata. Maybe JSON is better, maybe YAML, not sure. There isn't really a standard to go off of so we must create one. |
Beta Was this translation helpful? Give feedback.
-
|
I'll start doing that, then ... I suspect it'll make it easier to have just
a single source of truth.
Yeah, I hear you about the "no standard format." It's amazing to me that
it's taken this long to come up with an audiobook server. The various past
efforts have been just other products rebadged (although Booksonic-Air
deserves a salute for doing as well as it does). I'll keep an eye here for
format changes, certainly.
If you asked me, and you shouldn't, I'd pick yaml for a file describing a
single book, and JSON for a file describing a lot of books. I abhor XML.
…-Wm
On Sat, Apr 30, 2022 at 2:39 PM advplyr ***@***.***> wrote:
If the abmetadata file is saved as metadata.abs inside your library item
folder then it will be used as the source of truth for metadata.
I wouldn't say that file is finalized yet, it is possible we move to a
different format of metadata. Maybe JSON is better, maybe YAML, not sure.
There isn't really a standard to go off of so we must create one.
—
Reply to this email directly, view it on GitHub
<#59 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ7H6M5LDQAD5B4DJE6X43VHWR7ZANCNFSM5EZJHD2A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
I have bounced between a few different audiobook server products (Plex, Emby, Booksonic, audioserve) and they all seem to treat metadata differently. Will Audiobookshelf use ID3v2 tags to store metadata moving forward? I only ask as it can be quite a chore to move from one method to the next.
My library is currently setup for booksonic which includes a cover.jpg, desc.txt, and reader.txt in the root directly which is a bit limiting so I welcome the option for storing more information for each book.
Can you speak to how the currently available fields correspond to the ID3v2 tags?
For example
Author=Artist
Narrarator=Composer
Beta Was this translation helpful? Give feedback.
All reactions