Merge branch 'python3-dev'

IMP: (DDL) Allow additonal downloading from mega, mediafire and pixeldrain links when using DDL GC.
IMP: (DDL -- config.ini) ddl_priority_order can be customized to set the priority order in which the DDL links will be attempted
IMP: (DDL) Will cycle thru ddl_priority_order in sequence attempting each link as it gets to it, until link is able to be downloaded or link exhaustion occurs
IMP: Don't post-process or scan extensions if provided in ignore_search_words
IMP: listAnnualSeries api endpoint added
IMP: (DDL) Additional add-on server entry for DDL
IMP: (DDL) Improved Pack parsing / detection
IMP: Dupechecker will now prioritize (F#)/(f#) in filenames as 'Fixed' over normally named files or lower numbered (f#)
IMP: Pack issue # tracking will track issues that belong to a particular pack so individually queueing issues within pack before it's completed, will not redownload issues belonging to the pack.
IMP: Added Link column to DDL Manage page (display which type of link was/is being used)
IMP: (DDL --> Manage) Adjusted column widths & spacing to correct some double-lines occurring
IMP: (DDL --> Manage) Show pack issue range for packs as opposed to showing the individual item that was being searched for
IMP: Notification of packs being downloaded will now show which DDL link was used to download as well as complete name of pack
IMP: Update current default user_agent string to a more current string
IMP: CT will now use user_agent string defined by Mylar as opposed to a random generic one on each request (if not updated, will force-update it accordingly)
IMP: Removed individual provider entries for nzbsu and dognzb from GUI, now treated as newznabs. If enabled, will be converted to newznab entry provided doesn't already exist as a newznab.
IMP: If CV indicates series year does not exist (ie. 2099) instead of using 2099, will use year of first issue or if not invalid, current year
IMP: When refreshing/adding a series that has a directory created with the incorrect year (2099), rename existing directory using correct year
IMP: Allow for future removal of outdated config.ini options with BAD_DEFINITIONS code
IMP: CV Api Key entry field on configuration page proper placeholder of None so will auto-remove value prior to typing in field
IMP: Add option to remove SABnzbd completed downloads from SABnzbd history
IMP: ASCII art logo added to startup sequence (cause why not, right?)
IMP: Test python module requirements prior to actually starting the program - failure will result in application exit with appropriate log messages
IMP: Basic configuration check started - comic location failure on existence/creation/permissions will throw a popup error until resolved
IMP: Filter input box on Watchlist page & searchresults page will accept exlusion filters preceded by (:-<term>)
IMP: (mylar3#1473) Weeklypull will honor ignore_publishers when displaying the weekly pullist
IMP: Wildcards (*) are acceptable in the ignore_publishers value in the config.ini (ie. panini*)
IMP: scheduled searches will have status checked prior to actually searching (cannot be in a Downloaded or Snatched status) unless it was initiated manually (ie. magnifying glass search)
IMP: Added Remove option to DDL table options so items that are stuck in a Start/Incomplete status can be removed
IMP: Detect and handle CV reusing existing ComicIDs via GUI Dialog (Keep / Delete) (see mylar3#1506)
IMP: Status color-coded legend to the bottom of the main watchlist/index page
IMP: format_booktype now set to True by default (allows use of $Type in folder format)
IMP: Post-Processing option is now enabled by default
IMP: default usenet_retention set to 3500 days.
IMP: default enforce_permissions set to False
IMP: default metatagging option cmtag_start_year_as_volume set to True
IMP: requirements.txt (added): pycryptodome, tenacity
FIX: Annuals were written to incorrect table sometimes resulting in status/filename being locked in db and unable to remove
FIX: Annuals that had the year present in the annual name and no issue number would not be recognized during series scans
FIX: (carepackage) when creating carepackage, if requirements were set to == would cause error if it was higher/lower versioning
FIX: Volume of series on cv > 10 would not be recognized as such when adding/refreshing series
FIX: Parsing of files that had part /volume would fail when part was part of the series title
FIX: One-shots with no issue number (but year is present) in filename will now be properly accounted for in series rechecks if booktype is set correctly
FIX: Removed some incorrect logger.warn references so that will log regardless of language
FIX: Error when removing directory that doesn't physically exist where it was recorded as being (delete series option)
FIX: (searchresults --> Edit) would not update comic location path when booktype was changed via dropdown
FIX: (searchresults --> Edit) GN was incorrectly labelled as GC within the booktype dropdown
FIX: Incorrect status colours for annuals when integration is enabled (Archived, Ignored)
FIX: Improper parsing of sabnzbd version in with newer versions would cause errors when checking / sending to sabnzbd
FIX: Remove depreciated imghdr module in lieu of using pillow (api.py)
FIX: Replace depreciated SafeConfigParser call from configparser when generating carepackage
FIX: Depreciation of pkg_resources - use packaging now (addition to requirements)
FIX: Ensure index page will load if provider_order is in error for some reason
FIX: Index page would display Unknown variations for published date when invalid instead of N/A
FIX: Ensure post-processing would be allowed for CV series that have been removed from CV but retained as is within Mylar
FIX: Mass publishers variable would sometimes be in an incorrect type (str) as opposed to type (list) which would cause the mass publishers job to not run
FIX: Selecting All checkbox on seriesdetail page will now select only relevant table (issues / annuals) as opposed to both
FIX: Discord notifications would always throw a could not send notification log message, but notif would be sent ok
FIX: When parsing results of DDL, if pack was detected and an invalid issue number was parsed, would traceback
FIX: DDL --> mediafire would try to get filesize of file that could possibly not exist due to failure resulting in traceback error
FIX: DDL --> traceback when count of links discovered would be more than indicated, not using any links discovered prior to the invalid count check
FIX: DDL --> traceback due to invalid variable reference (comicinfo[[0]['pack'])
FIX: DDL --> prefer upscaled option would incorrectly assume SD-Digital instead of just a normal/undefined labelling of said issue
FIX: Removal of () in folder structure
FIX: Link to CV for items within collected editions
FIX: Searchresults filter would filter everything if used
FIX: Casing when invoking updater.dbUpdate (@cwar)
FIX: (mylar3#1510) Adding by comicid would result in error in GUI, but series would still add successfully.
FIX: (mylar3#1508) Typo in logger when viewing a Paused series that contained annuals
FIX: (mylar3#1507) .Black and .White numbering exceptions added
FIX: When popup appeared, would open all hidden/minimized options in config page if tab was open
FIX: Infinity numbering vs Infinity in series name fix
FIX: Would fail to set selection when previous failures resulted in main link being used
FIX: Remove some additional information from cleaned ini due to previous config additions
FIX:(mylar3#1502) sabnzbd version detection problem on startup re: versioning format
FIX: Numeric sort for issue number column on Watchlist and Wanted pages instead of alpha-based
FIX: Booktype will default to Print instead of None
FIX: Error when searching for items being returned no issue numbers
FIX: Ensure Minimum & Maximum filesize restrictions contain only numerics
FIX: StoryArcs
--> Search for Missing would fail during post-processing due to invalid variable reference in db table
--> Search for Missing would use invalid issueid when attempting to post-process when downloading resulting in traceback
--> Search for Missing will now mark issues belonging to watchlisted series as Wanted if not already in that status
--> Search for Missing option now indicates what it does on mouseover
FIX: Feedparser would blow up (traceback error) when parsing non-digital dates in some cases
FIX: StoryArcs -> Search For Missing button would traceback when initiating the search
FIX: series.json creation as per schema (update to v1.0.2), FIX: proper reload of issues table on regeneration of json
FIX: Subset DDL site(s) to better handle failures/failed marking
FIX: Folder monitor would die on lock, FIX: spamming pack exclusions
FIX: html_cache in cache_dir for GC html files so won't redownload html file until item is successfully downloaded.
FIX: html_cache location added to cleanup_cache section
FIX: Better handling of corrupt images being retrieved from CV (will retry alternate image size if available)
FIX: dbupdater would error when attempting to merge previous_failed_id info into a non-existent dictionary
FIX: Wanted page may not display if IssueNumber field is blank or some other field is empty. Will attempt to default to ComicName.
FIX: DDL ReQueue / Restart would throw a red popup warning due to invalid reference (One-Shot) on Manage DDL page
FIX: DDL  -> If it was the first link used, would cause a traceback when clearing the queue of the id
FIX: DDL --> If it was a retry/requeue, would error on attempting to preload a variable that's not really needed (one-shot) / site name
FIX: DDL --> When initiating a ReQueue/Restart, the resume option will only be attempted if it a main server GC link

Merge branch 'python3-dev'

FIX:(mylar3#1463) Truncated image retrieval check - if failure, will retrieve alternate image quality
FIX:(mylar3#1469) incorrect logger reference causing traceback on SAB snatches that are completed quickly
FIX:(mylar3#1471) pillow check so it's case-insensitive (went from Pillow to pillow)
FIX: Move gzip header to fix tempermental ddl (@scooterpsu)
FIX: requirements -> bump versions for cheroot (>8.2.1) and pillow (>10.1 due to python3.12 not working on less than this verison)

Merge branch 'python3-dev'

IMP: Prefer HD-Upscale/HD-Digital added as an option under the DDL banner in the config page.
IMP: Permissions redid to make sure that owner / group values entered are present prior to attempting to enforce. If not available will default to the user running mylar or the group user belong to depending on what is not present.
IMP: Manage DDL page -> will now show the full filename of the pack instead of series #issuenumber (ie. Superhero # 1)
FIX: (mylar3#1461) MAC OS metadata files would be scanned in mistakingly during series page loads / rechecks
FIX: (mylar3#1444) Annuals not included during group metatagging operation
FIX: (mylar3#1446) Occasional db conflict during issue updates in previous PR
FIX: Fixed some table overlapping on the Manage DDL page resulting in overflows
FIX: Increased last issue table on main page as was cutting off/overlapping on large issue numbers
FIX: When updating series info with new issue data, if a series was paused it would un-pause and mark issues as wanted if the options were enabled (mark all as wanted, mark upcoming as wanted)
FIX: Provider order would not be honored if DDL was an available option due to the way the search thru number padding was. Will now search all possible queries in a provider before moving onto the next provider regardless of the issue number
FIX: Searching would take the cover date as the actual search date, not the store date which caused problems around the end of the year
FIX: New issues set to Skipped even when series is not paused
FIX: CDH would fail to post-process items immediately if additional items were in the sabnzbd queue
FIX: When searching for issues published in late 2023, wouldn't match to 2024 filenames due to invalid date comparison
FIX: Mark all issues as wanted config option would mark every issue of a series as Wanted after each recheck/refresh that was not in a Downloaded status, regardless of previous state (ie. manually marking an issue as Skipped)
FIX: Pack downloads were broken due to previous PR variable change

Merge branch 'python3-dev'

IMP:(mylar3#1421) (mass-add): Do not add series if in ignored publisher list (@morsok)
IMP: webserve: introduce /prometheus_metrics with queue metrics (@OddBloke)
IMP: helpers: convert queue_info to a generator (@OddBloke)
IMP: helpers: convert queue_info to return a namedtuple (@OddBloke)
IMP: webserve: refactor queue information generation to helpers (@OddBloke)
FIX:(mylar3#1432) story arc banner upload field could accept null values in error
FIX: Working around accidentally invalid API key (starting with None) (@Nebelherr23)
FIX: weeklypull: handle invalid annuallink values gracefully Emit a WARNING log message instead of a traceback. (@OddBloke)
FIX: Correct wording about worker repair (@OddBloke - mylar3#1417)
FIX: Downloading annuals would work intermittently (usually not working thru GC)
FIX: Post-processing would not match annuals properly and ignore them during a run
FIX: DDL failing to search/download due to CF/FS problems
FIX: getcomics: release the DDL_LOCK at the end of downloadit (@OddBloke)
FIX: getcomics: add fallthrough return value to downloadit (@OddBloke)

Merge branch 'python3-dev'

IMP: Add Queues section to Activity/Jobs tab (@OddBloke)
IMP: Support fetching all search results using pagination when using DDL (@OddBloke)
IMP: Handle literal-HTML descriptions which occasionally occur - DDL (@OddBloke)
IMP: Clean up dead code, unused variables/imports, large commented blocks of code (@OddBloke)
IMP: Remove unused ALLOW_PACKS configuration options (@OddBloke)
IMP: Update README to include domain name for documentation instead of wiki
FIX:(mylar3#1403) Fix call of sfs.checker when using DDL RSS search  (@OddBloke)
FIX:(mylar3#1396)(mylar3#1398) Align custom non-English logger functions with stdlib (@OddBloke)
FIX: Stop searching when first matching issue is found with DDL (@OddBloke)

Merge branch 'python3-dev'

IMP: Add SOCKS proxy support (@NickSica)
IMP: getcomics: support fetching all search results using pagination (@OddBloke)
IMP: helpers: handle absent filename in ddzstat (@OddBloke)
FIX: Update configuration screen to allow for requests[socks] in requirements.txt
FIX: queues were not firing off due to a logging error that aborted the startup queue process

Merge branch 'python3-dev'

IMP:Attempt to parse issue ID from webpage, if notes field doesn't exist (newer ComicTagger versions) (@bbtufty)
IMP: Added kubernetes detection (@lavahot)
IMP: queue_schedule: introduce helper functions for common start and shutdown code (@OddBloke)
FIX:(mylar3#1336)(mylar3#1232)(mylar3#873) Import would fail when importing a filename with no issue number
FIX:(mylar3#1231) Incomplete series would cause post-processing to fail if Latest Date and Publication Run were not generated
FIX:(mylar3#1361) Use shutil.move to support cross-device renames (@beardypig)
FIX:If annual gathering failed during generation, would break the add series queue
FIX:Import would fail if a series on the watchlist was being refreshed during, or if it hadn't completed for w/e reason
FIX:Ignore 0-byte files/placeholders during filescans
FIX: getcomics correct logging call (@OddBloke)

Merge branch 'python3-dev'

FIX: urllib3 breaking changes
FIX:(mylar3#1344) Attempted fix for CF errors being returned when using GC
FIX: When Alternate Filename is blank, it writes a blank space
FIX: Imprint mappings (thanks @qubidt)
FIX: When determining match to download, TPB series spanning multiple years would fail to match
FIX: During import, if title contained One-Shot - would not find the match on CV
FIX: Storyarcs failing to properly generate file path locations in OPDS

Merge branch 'python3-dev'

IMP: Update look4untaggedcbzs.py (@DieselTech)
FIX: Tweaked Issue ID parsing to handle any subsequent numbers in Notes field (@falo2k)
FIX: Rechecking Story-Arc directory fails on missing issue numbers (@qubidt)
FIX:(mylar3#1179) Allow for POST method on pages where large amounts of data is being sent (ie.config/manage)
FIX:(mylar3#1321) Imprints would take json naming convention over CV (ini option added)

Merge branch 'python3-dev'

IMP:(mylar3#1285) Change StoryArc base folder location via config.ini (storyarc_location)
IMP: Set default docker paths for Comic Location (/comics) and SABnzbd Download directory (/downloads) (if not set)
IMP: (API getWanted results empty  mylar3#1266) getWanted API endpoint will now return issues and annuals (and story_arcs when queried for)
FIX:(mylar3#1277) invalid issue numbering with Alpha & Omega during rename
FIX:(mylar3#1295) Make sure to reset the sort counter if no series are present after removing from watchlist
FIX:(mylar3#1284) trailing slash in url if not present
FIX:(mylar3#1297) Fypo correction
FIX: Adding series with legacy numbering (issue number in brackets)
FIX: imprint field being used within Folder Format string causing extra spaces
FIX: DBUpdater would traceback and kill scheduled job on CV entries with no comicid
FIX: (DC 2022 Annual not downloading from GetComics mylar3#1059) 2022 Annuals not being searched / detected properly
FIX: Resuming DDL would cause progress bars to display incorrect % and elongated bar
FIX: DDL Manage page could have duplicates listed (one with a Volume, one without)
FIX: Changing series directory location and then changing booktype would reset the directory location to the default instead of previously set one
FIX: Storyarc would not rename existing files within arc directory with updated reading order number
FIX: Post-processing an issue that belongs to multiple arcs would only post-process against one arc and then cause an error
FIX: Manual post-processing of story arc issues that are not on watchlist would result in a volume-related error in some cases
FIX: (Missing issue count in Have column on main page mylar3#1270) issue count missing on main page (default theme)
FIX: Storyarc issue count not showing on detail page
FIX: Unable to properly load resizeimage function on storyarc detail page when image present
FIX: Manage DDL showing duplicate buttons
FIX: Secondary watermark on series detail page was not aligned within column
FIX:(mylar3#1278) Editing series location path via Edit Settings would not take initial value (had to repeat steps)
FIX: Variable error on startup (docker)
FIX: CDH category removal if not present when querying SABnzbd history
FIX: Remove eta variable from sabnzbd response as no longer supported (and wasn't really used)
FIX: Manage page would not show full annual titles but just the series it was attached to (ie. Series as opposed to Series Annual)