next »

[iconoclasthero]

This thread is for linux tools for downloading and manipulating audiobooks. 

[iconoclasthero]

Tools:  ffmpeg, mediainfo

To find out if the file you have has encoded chapter breaks (e.g., OverDrive and Audibe), use mediainfo:

Code:

$ mediainfo Superforecasting_\ The\ Art\ and\ Science\ of\ Prediction.m4b 
General
Complete name                            : Superforecasting_ The Art and Science of Prediction.m4b
Format                                   : MPEG-4
Codec ID                                 : M4V  (isom/iso2/avc1)
File size                                : 269 MiB
Duration                                 : 9 h 45 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 64.2 kb/s
Album                                    : Superforecasting: The Art and Science of Prediction
Album/Sorted by                          : Superforecasting: The Art and Science of Prediction
Album/Performer                          : Philip Tetlock, Dan Gardner
Album/Performer/Sort                     : Philip Tetlock, Dan Gardner
Part/Position                            : 1
Part/Total                               : 1
Track name                               : Superforecasting: The Art and Science of Prediction
Performer                                : Philip Tetlock, Dan Gardner
Performer/Sorted by                      : Philip Tetlock, Dan Gardner
Composer                                 : Joel Richards
Composer/Sorted by                       : Joel Richards
Genre                                    : Business
Recorded date                            : 2015
Tagged date                              : UTC 2020-09-01 09:03:27
Copyright                                : ©2015 Philip Tetlock Consulting, Inc., and Connaught Street, Inc. (P)2015 Audible, Inc.
Cover                                    : Yes
Title/Sort                               : Superforecasting: The Art and Science of Prediction

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 9 h 45 min
Duration_LastFrame                       : -10 ms
Bit rate mode                            : Variable
Bit rate                                 : 62.8 kb/s
Maximum bit rate                         : 69.0 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 22.05 kHz
Frame rate                               : 21.533 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 263 MiB (98%)
Language                                 : English
Default                                  : Yes
Alternate group                          : 1
Tagged date                              : UTC 2020-09-01 09:03:27
Menus                                    : 3
tagc                                     : public.main-program-content

Menu
ID                                       : 3
Codec ID                                 : text
Duration                                 : 9 h 45 min
Encoded date                             : UTC 2020-05-24 13:33:29
Tagged date                              : UTC 2020-05-24 13:33:29
Menu For                                 : 2
00:00:00.000                             : Introduction
00:00:32.000                             : Chapter 1
00:50:59.136                             : Chapter 2
01:37:41.367                             : Chapter 3
02:49:34.102                             : Chapter 4
03:39:18.423                             : Chapter 5
04:25:05.994                             : Chapter 6
05:18:55.007                             : Chapter 7
06:03:44.482                             : Chapter 8
06:42:41.570                             : Chapter 9
07:21:58.860                             : Chapter 10
08:01:09.741                             : Chapter 11
08:41:55.592                             : Chapter 12
09:24:17.387                             : Epilogue
09:28:34.711                             : Appendix
09:44:44.750                             : End

As you can see, there are chapter markers encoded in the .m4b file...if they're not there, you can stop here because the necessary info is lacking.  (NB: Getting the chapter info from e.g., a .json or .xml is outside of the scope of this discussion.)

Using a script similar to this one I've modified:

Code:

#!/bin/bash
# Author: http://crunchbang.org/forums/viewtopic.php?id=38748#p414992
# m4bronto

#     Chapter #0:0: start 0.000000, end 1290.013333
#       first   _     _     start    _     end

title="Enter title here"

while [ $# -gt 0 ]; do

ffmpeg -i "$1" 2> tmp.txt

while read -r first _ _ start _ end; do
  if [[ $first = Chapter ]]; then
    read  # discard line with Metadata:
    read _ _ chapter
    ffmpeg -activation_bytes 12345678 -vsync 2 -i "$1" -ss "${start%?}" -to "$end" -vn -b:a 17k -acodec libopus "$title -- $chapter.opus" </dev/null
  fi
done <tmp.txt

rm tmp.txt

shift
done

To go through what I understand of the ffmpeg line:  
ffmpeg -activation_bytes 12345678 -vsync 2 -i "$1" -ss "${start%?}" -to "$end" -vn -b:a 17k -acodec libopus "$title -- $chapter.opus" </dev/null

-activation_bytes 12345678: Replace 12345678 with your Audible activation bytes and you can directly convert .aax files into split chapters.  If you're not decoding an Audible .aax, this parameter is ignored and can be left out.
-i "$1": The -i flag specifies the input file and "$1" is the first command-line parameter.
-ss & -to: Specify the start and end times for a chapter.
-vn: Disable video
-b:a 17k:Sets the bitrate of the audio stream to 17 kbps
-acodec libopus: Sets the audio codec to libopus
"$title -- $chapter.opus" is the output file name using the title specified in the script and the chapter number.  (Depending on the format of the original file, this may need to be modified.)
-vsync 2: No idea; probably irrelevant with an audiobook since there is no video component.

Obviously, you can modify the bitrate, codec, etc., to suit your preferences, but this will spit out:

Code:

$ l -tr1
'Superforecasting_ The Art and Science of Prediction.m4b'
'Superforecasting -- Introduction.opus'
'Superforecasting -- Chapter 1.opus'
'Superforecasting -- Chapter 2.opus'
'Superforecasting -- Chapter 3.opus'
'Superforecasting -- Chapter 4.opus'
'Superforecasting -- Chapter 5.opus'
'Superforecasting -- Chapter 6.opus'
'Superforecasting -- Chapter 7.opus'
'Superforecasting -- Chapter 8.opus'
'Superforecasting -- Chapter 9.opus'
'Superforecasting -- Chapter 10.opus'
'Superforecasting -- Chapter 11.opus'
'Superforecasting -- Chapter 12.opus'
'Superforecasting -- Epilogue.opus'
'Superforecasting -- Appendix.opus'
'Superforecasting -- End.opus'

So, depending on the book, you can number the files to your liking.  Generally, I'd move the intro to Chapter 00 and then number the last 3 as Chapters 13-15.

You can quickly renumber the single-digit chapter numbers with:

Code:

$ for i in Superforecasting\ --\ Chapter\ ?.opus; do mv "$i" "${i/Chapter /Chapter 0}"; done
$ ls -tr1
'Superforecasting_ The Art and Science of Prediction.m4b'
'Superforecasting -- Introduction.opus'
'Superforecasting -- Chapter 01.opus'
'Superforecasting -- Chapter 02.opus'
'Superforecasting -- Chapter 03.opus'
'Superforecasting -- Chapter 04.opus'
'Superforecasting -- Chapter 05.opus'
'Superforecasting -- Chapter 06.opus'
'Superforecasting -- Chapter 07.opus'
'Superforecasting -- Chapter 08.opus'
'Superforecasting -- Chapter 09.opus'
'Superforecasting -- Chapter 10.opus'
'Superforecasting -- Chapter 11.opus'
'Superforecasting -- Chapter 12.opus'
'Superforecasting -- Epilogue.opus'
'Superforecasting -- Appendix.opus'
'Superforecasting -- End.opus'

[iconoclasthero]

Tools: audible, audible-cl

To download and decode an audiobook you've purchased from Audible, you can use the audible/audible-cli tools obviating the need to pay for OpenAudible.  (Seriously, I need to pay you to defeat someone else's DRM??)

Get audible from https://github.com/mkb79/Audible
Get audible-cli from https://github.com/mkb79/audible-cli

You will need to make sure you have the dependencies listed on the respective github pages, most substantially >= python 3.6 and pip.

Once you have everything installed, ideally in a pathed directory (e.g., in ~/bin/ or /opt/), you should run

Code:

audible-quickstart

first and follow the prompts.  After that, you can retrieve your activation-bytes (required for decoding with ffmpeg) with

Code:

$ audible activation-bytes

To download e.g., Savage Continent: Europe in the Aftermath of WWII, you can simply put the title in the command (as opposed to figuring out the ASIN—which you do with the -a ASIN-NUMBER flag):

Code:

$ audible download --aax --cover --cover-size 1215 -o ./ -t "SAvage contiNent"

Found the following matches for 'SAvage contiNent'
+---------+-----------------------------------------------------------+------------+
| % match | title                                                     |    asin    |
+---------+-----------------------------------------------------------+------------+
|  100.0  | Savage Continent: Europe in the Aftermath of World War II | B009CYJ4C0 |
+---------+-----------------------------------------------------------+------------+
Proceed with this audiobook(s) [y/N]: y
File /dev/shm/cache/Savage_Continent_Europe_in_the_Aftermath_of_World_War_II_(1215).jpg downloaded to /dev/shm/cache in 0:00:00.267727.
/dev/shm/cache/Savage_Continent_Europe_in_the_Aftermath_of_World_War_II-LC_64_22050_stereo.aax:   4%|▏  | 18.7M/425M [00:06<02:30, 2.84MB/s]

As you can see, you will be prompted to confirm that the title you entered is correct, and then a readout on how long it will take to download, speed, size, etc.

The options are pretty straightforward: 
--aax is to download the audiobook in -aax format (I do not know what -aaxc format is)
--cover is to download the cover with the book
--cover-size 1215 is the biggest size listed in $ audible download -h
-o ./ is to download in the directory the script was called from.  Adjust ./ to whatever directory suits your needs.
-t "SAvage contiNent" is to search by the cAsE insEnsiTive "title" you specify (yes, quote the title).

Once you have the .aax downloaded you can decode the file with the following command if audible is in your path:

Code:

$ ffmpeg -activation_bytes `audible activation-bytes` -vn -i "Savage_Continent_Europe_in_the_Aftermath_of_World_War_II-LC_64_22050_stereo.aax" -acodec copy "Savage_Continent_Europe_in_the_Aftermath_of_World_War_II-LC_64_22050_stereo.mp4 

If not, you will need to replace `audible activation-bytes` with your actual activation bytes minus the backticks:

Code:

$ ffmpeg -activation_bytes 12345678 -vn -i "Savage_Continent_Europe_in_the_Aftermath_of_World_War_II-LC_64_22050_stereo.aax" -acodec copy "Savage_Continent_Europe_in_the_Aftermath_of_World_War_II-LC_64_22050_stereo.mp4 

where 12345678 is replaced with your activation bytes as found earlier.

The ffmpeg options are generally straightforward except:
-acodec copy: Unless you convert to your terminal file format here (e.g., I might convert to opus by using -acodec libopus -b:a 17k), using the -acodec copy flag will prevent generation loss resulting from reencoding the native m4b format as mp4 as opposed to simply decoding it (which does not cause generational loss going from lossy > lossy).  NB:  Failure to specify '-acodec copy' here will result in reencoding of the original and thus generational loss.

[Dallis24F]

This is a work in progress.
Please do not add comments. They will be removed.
Thanks

[iconoclasthero]

Tools:  powershell, hoopla-dl script, ffmpeg, mplayer

There's a couple different ways to install powershell.  I installed it via apt (sudo apt install powershell) but there's also a snap (https://snapcraft.io/powershell) so whatever suits your fancy.

Invoke powershell via the command line ($ pwsh).  (NB: I tried to add a shebang (#!/usr/bin/pwsh) to the powershell script to get it to run through the interpreter from the native shell but that wasn't successful so you will need to actually enter the shell.)

The hoopla-dl script isn't particularly helpful in terms of command-line documentation (or documentation in general) so I usually refer back to what I used in the past.  I am going to walk through the download of https://www.hoopladigital.com/title/11872742, The Death of the USS Thresher:

Code:

$ echo $SHELL
/bin/bash

$ pwsh
PowerShell 7.1.4
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

PS /dev/shm/cache> which ffmpeg                                                                                                             /home/user/bin/ffmpeg
PS /dev/shm/cache>  ./hoopla2.5.ps1 -Outputfolder /dev/shm/cache -FfmpegBin /home/user/bin/ffmpeg -KeepDecryptedData -TitleID 11872742
WARNING: Epubzip binary not found (/dev/shm/cache/epubzip.exe). If you are downloading ebooks (rather than comics or audiobooks), you may wish to download the binary from https://github.com/dino-/epub-tools/releases, specify a different path with -EpubZipBin, or specify -KeepDecryptedData so that you can manually pack afterward.
                                                                                                                                                Directory: /dev/shm/cache                                                                                                                                                                                                                                                           Mode                 LastWriteTime         Length Name                                                                                      ----                 -------------         ------ ----                                                                                      -----           9/20/2021  7:59 AM      297520325 The Death of the USS Thresher.m4a                                                         Decrypted data for 11872742 (The Death of the USS Thresher) stored in /tmp/dec-183294214-20210920075715                                                                                                                                                                                 

PS /dev/shm/cache> ls -l './The Death of the USS Thresher.m4a'; mediainfo './The Death of the USS Thresher.m4a'
-rw-rw-r-- 1 bvargo bvargo 297520325 Sep 20 07:59 './The Death of the USS Thresher.m4a'
General
Complete name                            : ./The Death of the USS Thresher.m4a
Format                                   : MPEG-4
Format profile                           : Apple audio with iTunes info
Codec ID                                 : M4A  (isom/iso2)
File size                                : 284 MiB
Duration                                 : 5 h 6 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 129 kb/s
Writing application                      : Lavf58.29.100

Audio
ID                                       : 1
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 5 h 6 min
Bit rate mode                            : Variable
Bit rate                                 : 128 kb/s
Maximum bit rate                         : 130 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 281 MiB (99%)
Default                                  : Yes
Alternate group                          : 1

PS /dev/shm/cache> ls /tmp/dec-183294214-20210920075715/
filesequence0000000.ts	filesequence0000062.ts	filesequence0000124.ts	filesequence0000186.ts	filesequence0000248.ts
filesequence0000001.ts	filesequence0000063.ts	filesequence0000125.ts	filesequence0000187.ts	filesequence0000249.ts
filesequence0000002.ts	filesequence0000064.ts	filesequence0000126.ts	filesequence0000188.ts	filesequence0000250.ts
.
.
.
filesequence0000055.ts	filesequence0000117.ts	filesequence0000179.ts	filesequence0000241.ts	filesequence0000303.ts
filesequence0000059.ts	filesequence0000121.ts	filesequence0000183.ts	filesequence0000245.ts	ttm_9781541423053.m3u8
filesequence0000060.ts	filesequence0000122.ts	filesequence0000184.ts	filesequence0000246.ts
filesequence0000061.ts	filesequence0000123.ts	filesequence0000185.ts	filesequence0000247.ts

Key points with the hoopla-dl script parameters: 
-TitleID comes from the title in the URL for the audiobook.
-KeepDecryptedData saves the individual one minute audio files.  These can be stitched back together using ffmpeg concatenate but most likely they're just going to be deleted.  They're left in /tmp, e.g., in /tmp/dec-183294214-20210920075715, in this case.
-Ffmpegbin is the location of ffmpeg.  If you installed it via apt, it should be /usr/bin/ffmpeg.  If you compiled it yourself, it may be e.g., ~/bin/ffmpeg.  To find what version is called via the path, use `which ffmpeg`
-Outputfolder I use /dev/shm/cache because it is an easy ram disk and I don't commit anything to disk until finished manipulating.  Modify to suit your needs...I'm even though I call the full path, I'm still just using ./
At least for me, I can call ls -l, mediainfo, and ffmpeg from within Powershell to get an output of the finished file(s).  You can also see from the MediaInfo output that this is not chaptered, so aside from breaking it in half just for smaller files, this is done, dress it up in a .torrent, spank it on the bottom, and send it on it's way. 

To break it into smaller files:
First, figure out a midpoint.  If you just go by MediaInfo, you have a length of 05:06 hh:mm.  With mplayer
you can get a precise length (down to hundredths of a second in this case). From there, you can call two ffmpeg processes.  In the following two examples, the file will be split at the midpoint without re-encoding (and, hence, without generational loss).
Mplayer to look at all file info:

Code:

$ mplayer -vo dummy -ao dummy -identify The\ Death\ of\ the\ USS\ Thresher.m4a
MPlayer 1.3.0 (Debian), built with gcc-9 (C) 2000-2016 MPlayer Team
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing The Death of the USS Thresher.m4a.
libavformat version 58.29.100 (external)
libavformat file format detected.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f4b5eb4e600]Protocol name not provided, cannot determine if input is local or a network protocol, buffers and access patterns cannot be configured optimally without knowing the protocol
ID_AUDIO_ID=0
[lavf] stream 0: audio (aac), -aid 0, -alang und
Clip info:
 major_brand: M4A 
ID_CLIP_INFO_NAME0=major_brand
ID_CLIP_INFO_VALUE0=M4A 
 minor_version: 512
ID_CLIP_INFO_NAME1=minor_version
ID_CLIP_INFO_VALUE1=512
 compatible_brands: isomiso2
ID_CLIP_INFO_NAME2=compatible_brands
ID_CLIP_INFO_VALUE2=isomiso2
 encoder: Lavf58.29.100
ID_CLIP_INFO_NAME3=encoder
ID_CLIP_INFO_VALUE3=Lavf58.29.100
ID_CLIP_INFO_N=4
Load subtitles in ./
ID_FILENAME=The Death of the USS Thresher.m4a
ID_DEMUXER=lavfpref
ID_AUDIO_FORMAT=MP4A
ID_AUDIO_BITRATE=128000
ID_AUDIO_RATE=44100
ID_AUDIO_NCH=2
ID_START_TIME=0.00
ID_LENGTH=18396.65
ID_SEEKABLE=1
ID_CHAPTERS=0
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
libavcodec version 58.54.100 (external)
AUDIO: 44100 Hz, 2 ch, floatle, 128.0 kbit/4.54% (ratio: 16000->352800)
ID_AUDIO_BITRATE=128000
ID_AUDIO_RATE=44100
ID_AUDIO_NCH=2
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
No such audio driver 'dummy'
Could not open/initialize audio device -> no sound.
Audio: no sound
Video: no video


Exiting... (End of file)
ID_EXIT=EOF

A lot of info, but all we need is the length of the file (in seconds), so pipe to grep for "ID_LENGTH":

Code:

$ mplayer -vo dummy -ao dummy -identify The\ Death\ of\ the\ USS\ Thresher.m4a |grep ID_LENGTH
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.
ID_LENGTH=18396.65
No such audio driver 'dummy'
Could not open/initialize audio device -> no sound.

Divide that number in half and put it into an ffmpeg command:

Code:

ffmpeg -loglevel error -stats -i The\ Death\ of\ the\ USS\ Thresher.m4a -acodec copy -to 9198.325s The\ Death\ of\ the\ USS\ Thresher\ --\ Part\ 01.m4a 
size=  145274kB time=02:33:18.33 bitrate= 129.4kbits/s speed=7.46e+03x    

$ ffmpeg -loglevel error -stats -i The\ Death\ of\ the\ USS\ Thresher.m4a -acodec copy -ss 9198.325s The\ Death\ of\ the\ USS\ Thresher\ --\ Part\ 02.m4a 
size=  145274kB time=02:33:18.31 bitrate= 129.4kbits/s speed=4.79e+03x

-to is the stop time of the first half, i.e., 9198.325 seconds into the whole.
-ss is the start time of the second half i.e., 9198.325 seconds into the whole.

If you want to convert formats, e.g., to 17 kb/s opus, you can do that directly by specifying the codec and bitrate instead of using the -acodec copy flag:

Code:

$ ffmpeg -loglevel error -stats -i The\ Death\ of\ the\ USS\ Thresher.m4a -acodec libopus -b:a 17k -to 9198.325s The\ Death\ of\ the\ USS\ Thresher\ --\ Part\ 01.opus
size=   19754kB time=02:33:18.33 bitrate=  17.6kbits/s speed=49.4x    
video:0kB audio:19062kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.629999%

$ ffmpeg -loglevel error -stats -i The\ Death\ of\ the\ USS\ Thresher.m4a -acodec libopus -b:a 17k -ss 9198.325s The\ Death\ of\ the\ USS\ Thresher\ --\ Part\ 02.opus
size=   20058kB time=02:33:18.32 bitrate=  17.9kbits/s speed=45.4x