Release 2021.01.29

Changelog.md

@@ -8,7 +8,8 @@ # Instuctions for creating release
 * Change "Merged with youtube-dl" version in Readme.md if needed
 * Commit to master as `Release <version>`
 * Push to origin/release - build task will now run
-* Update version.py and run `make issuetemplates`
+* Update version.py using devscripts\update-version.py
+* Run `make issuetemplates`
 * Commit to master as `[version] update :ci skip all`
 * Push to origin/master
 * Update changelog in /releases
@@ -16,6 +17,39 @@ # Instuctions for creating release
 -->
 
 
+### 2021.01.29
+* **Features from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl)**: Co-authored by @animelover1984 and @bbepis
+    * Add `--get-comments`
+    * [youtube] Extract comments
+    * [billibilli] Added BiliBiliSearchIE, BilibiliChannelIE
+    * [billibilli] Extract comments
+    * [billibilli] Better video extraction
+    * Write playlist data to infojson
+    * [FFmpegMetadata] Embed infojson inside the video
+    * [EmbedThumbnail] Try embedding in mp4 using ffprobe and `-disposition`
+    * [EmbedThumbnail] Treat mka like mkv and mov like mp4
+    * [EmbedThumbnail] Embed in ogg/opus
+    * [VideoRemuxer] Conditionally remux video
+    * [VideoRemuxer] Add `-movflags +faststart` when remuxing to mp4
+    * [ffmpeg] Print entire stderr in verbose when there is error
+    * [EmbedSubtitle] Warn when embedding ass in mp4
+    * [anvato] Use NFLTokenGenerator if possible
+* **Parse additional metadata**: New option `--parse-metadata` to extract additional metadata from existing fields
+    * The extracted fields can be used in `--output`
+    * Deprecated `--metadata-from-title`
+* [Audius] Add extractor
+* [youtube] Extract playlist description and write it to `.description` file
+* Detect existing files even when using `recode`/`remux` (`extract-audio` is partially fixed)
+* Fix wrong user config from v2021.01.24
+* [youtube] Report error message from youtube as error instead of warning
+* [FormatSort] Fix some fields not sorting from v2021.01.24
+* [postprocessor] Deprecate `avconv`/`avprobe`.  All current functionality is left untouched. But don't expect any new features to work with avconv
+* [postprocessor] fix `write_debug` to not throw error when there is no `_downloader`
+* [movefiles] Don't give "cant find" warning when move is unnecessary
+* Refactor `update-version`, `pyinst.py` and related files
+* [ffmpeg] Document more formats that are supported for remux/recode
+
+
 ### 2021.01.24
 * **Merge youtube-dl:** Upto [2021.01.24](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.01.16)
 * Plugin support ([documentation](https://github.com/pukkandan/yt-dlp#plugins))

README.md

@@ -56,12 +56,14 @@ # NEW FEATURES
 
 * **Merged with youtube-dl v2021.01.24.1**: You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
 
+* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--get-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, Playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/pukkandan/yt-dlp/pull/31) for details.
+
 * **Youtube improvements**:
     * All Youtube Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) works correctly and support downloading multiple pages of content
     * Youtube search works correctly (`ytsearch:`, `ytsearchdate:`) along with Search URLs
     * Redirect channel's home URL automatically to `/video` to preserve the old behaviour
 
-* **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom
+* **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom, audius
 
 * **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, tiktok, akamai, ina
 
@@ -73,10 +75,11 @@ # NEW FEATURES
 
 * **Portable Configuration**: Configuration files are automatically loaded from the home and root directories. See [configuration](#configuration) for details
 
-* **Other new options**: `--list-formats-as-table`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc
+* **Other new options**: `--parse-metadata`, `--list-formats-as-table`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc
 
 * **Improvements**: Multiple `--postprocessor-args` and `--external-downloader-args`, `%(duration_string)s` in `-o`, faster archive checking, more [format selection options](#format-selection) etc
 
+
 See [changelog](Changelog.md) or [commits](https://github.com/pukkandan/yt-dlp/commits) for the full list of changes
 
 
@@ -320,8 +323,8 @@ ## Download Options:
     --downloader-args NAME:ARGS      Give these arguments to the external
                                      downloader. Specify the downloader name and
                                      the arguments separated by a colon ":". You
-                                     can use this option multiple times (Alias:
+                                     can use this option multiple times
-                                     --external-downloader-args)
+                                     (Alias: --external-downloader-args)
 
 ## Filesystem Options:
     -a, --batch-file FILE            File containing URLs to download ('-' for
@@ -511,17 +514,17 @@ ## Video Format Options:
     --list-formats-old               Present the output of -F in the old form
                                      (Alias: --no-list-formats-as-table)
     --youtube-include-dash-manifest  Download the DASH manifests and related
-                                     data on YouTube videos (default) (Alias:
+                                     data on YouTube videos (default)
-                                     --no-youtube-skip-dash-manifest)
+                                     (Alias: --no-youtube-skip-dash-manifest)
     --youtube-skip-dash-manifest     Do not download the DASH manifests and
-                                     related data on YouTube videos (Alias:
+                                     related data on YouTube videos
-                                     --no-youtube-include-dash-manifest)
+                                     (Alias: --no-youtube-include-dash-manifest)
     --youtube-include-hls-manifest   Download the HLS manifests and related data
-                                     on YouTube videos (default) (Alias:
+                                     on YouTube videos (default)
-                                    --no-youtube-skip-hls-manifest)
+                                     (Alias: --no-youtube-skip-hls-manifest)
     --youtube-skip-hls-manifest      Do not download the HLS manifests and
-                                     related data on YouTube videos (Alias:
+                                     related data on YouTube videos
-                                     --no-youtube-include-hls-manifest)
+                                     (Alias: --no-youtube-include-hls-manifest)
     --merge-output-format FORMAT     If a merge is required (e.g.
                                      bestvideo+bestaudio), output to given
                                      container format. One of mkv, mp4, ogg,
@@ -575,15 +578,16 @@ ## Post-Processing Options:
                                      VBR or a specific bitrate like 128K
                                      (default 5)
     --remux-video FORMAT             Remux the video into another container if
-                                     necessary (currently supported: mp4|mkv).
+                                     necessary (currently supported: mp4|mkv|flv
-                                     If target container does not support the
+                                     |webm|mov|avi|mp3|mka|m4a|ogg|opus). If
+                                     target container does not support the
                                      video/audio codec, remuxing will fail. You
                                      can specify multiple rules; eg.
                                      "aac>m4a/mov>mp4/mkv" will remux aac to
                                      m4a, mov to mp4 and anything else to mkv.
     --recode-video FORMAT            Re-encode the video into another format if
-                                     re-encoding is necessary (currently
+                                     re-encoding is necessary. The supported
-                                     supported: mp4|flv|ogg|webm|mkv|avi)
+                                     formats are the same as --remux-video
     --postprocessor-args NAME:ARGS   Give these arguments to the postprocessors.
                                      Specify the postprocessor/executable name
                                      and the arguments separated by a colon ":"
@@ -619,12 +623,13 @@ ## Post-Processing Options:
     --parse-metadata FIELD:FORMAT    Parse additional metadata like title/artist
                                      from other fields. Give field name to
                                      extract data from, and format of the field
-                                     seperated by a ":". The format syntax is
+                                     seperated by a ":". Either regular
-                                     the same as --output. Regular expression
+                                     expression with named capture groups or a
-                                     with named capture groups may also be used.
+                                     similar syntax to the output template can
-                                     The parsed parameters replace existing
+                                     also be used. The parsed parameters replace
-                                     values. This option can be used multiple
+                                     any existing values and can be use in
-                                     times. Example: --parse-metadata
+                                     output templateThis option can be used
+                                     multiple times. Example: --parse-metadata
                                      "title:%(artist)s - %(title)s" matches a
                                      title like "Coldplay - Paradise". Example
                                      (regex): --parse-metadata
@@ -771,7 +776,7 @@ # OUTPUT TEMPLATE
  - `is_live` (boolean): Whether this video is a live stream or a fixed-length video
  - `start_time` (numeric): Time in seconds where the reproduction should start, as specified in the URL
  - `end_time` (numeric): Time in seconds where the reproduction should end, as specified in the URL
- - `format` (string): A human-readable description of the format 
+ - `format` (string): A human-readable description of the format
  - `format_id` (string): Format code specified by `--format`
  - `format_note` (string): Additional info about the format
  - `width` (numeric): Width of the video

docs/supportedsites.md

@@ -80,6 +80,8 @@ # Supported sites
  - **AudioBoom**
  - **audiomack**
  - **audiomack:album**
+ - **Audius**
+ - **audius:playlist**
  - **AWAAN**
  - **awaan:live**
  - **awaan:season**
@@ -112,7 +114,9 @@ # Supported sites
  - **BiliBili**
  - **BilibiliAudio**
  - **BilibiliAudioAlbum**
+ - **BilibiliChannel**
  - **BiliBiliPlayer**
+ - **BiliBiliSearch**: Bilibili video search, "bilisearch" keyword
  - **BioBioChileTV**
  - **Biography**
  - **BIQLE**

youtube_dlc/options.py

@@ -1096,9 +1096,9 @@ def _dict_from_multiple_values_options_callback(
         help=(
             'Parse additional metadata like title/artist from other fields. '
             'Give field name to extract data from, and format of the field seperated by a ":". '
-            'The format syntax is the same as --output. '
+            'Either regular expression with named capture groups or a '
-            'Regular expression with named capture groups may also be used. '
+            'similar syntax to the output template can also be used. '
-            'The parsed parameters replace existing values. '
+            'The parsed parameters replace any existing values and can be use in output template'
             'This option can be used multiple times. '
             'Example: --parse-metadata "title:%(artist)s - %(title)s" matches a title like '
             '"Coldplay - Paradise". '