Commit Graph

63 Commits

Author SHA1 Message Date
Simon Sawicki 1c51c520f7
[fd/fragment] Improve progress calculation (#8241)
This uses the download speed from all threads and also adds smoothing to speed and eta

Authored by: Grub4K
2023-10-08 02:01:01 +02:00
coletdjnz 3d2623a898
[compat, networking] Deprecate old functions (#2861)
Authored by: coletdjnz, pukkandan
2023-07-15 16:18:35 +05:30
pukkandan 337734d4a8
[cleanup] Misc 2023-07-06 21:39:55 +05:30
pukkandan 424f3bf033
[downloader/fragment] Do not sleep between fragments
Closes #6599
2023-06-21 06:10:39 +05:30
Simon Sawicki edbe5b589d
Bugfixes for 4823ec9f46
Hotfix for fragmented downloads

Authored by: bashonly
2023-05-25 22:52:44 +02:00
pukkandan 4823ec9f46
Update to ytdl-commit-d1c6c5
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6

Except:
    * 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
        * There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
    * e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
        * We plan to implement this differently
2023-05-24 23:30:43 +05:30
pukkandan 46f1370e9a
[devscripts/cli_to_api] Add script 2023-05-24 23:29:30 +05:30
pukkandan 5b28cef72d
[cleanup] Misc 2023-02-28 23:51:06 +05:30
pukkandan f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
2023-02-28 23:34:43 +05:30
Simon Sawicki 6839ae1f6d
[utils] `traverse_obj`: Fix more bugs
and cleanup uses of `default=[]`

Continued from b1bde57bef
2023-02-10 19:36:55 +05:30
bashonly 7e68567e50
[downloader/hls] Allow extractors to provide AES key (#6158)
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-02-08 11:09:32 +05:30
pukkandan 814bba3933
[downloader/fragment] HLS download can continue without first fragment
Closes #5274
2022-10-18 19:20:51 +05:30
pukkandan da4db748fa
[utils] Add `deprecation_warning`
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
2022-08-30 21:03:07 +05:30
pukkandan be5c1ae862
Standardize retry mechanism (#1649)
* [utils] Create `RetryManager`
* Migrate all retries to use the manager
* [extractor] Add wrapper methods for convenience
* Standardize console messages for retries
* Add `--retry-sleep` for extractors
2022-08-02 01:43:18 +05:30
pukkandan ac66811112
[compat] Remove more functions
Removing any more will require changes to a large number of extractors
2022-06-25 00:08:55 +05:30
pukkandan 998a3cae0c
[cleanup] Misc fixes 2022-06-22 03:47:41 +05:30
MMM 36195c4461
[dash] Show fragment count with `--live-from-start` (#3493)
Authored by: flashdagger
2022-06-07 05:44:08 -07:00
pukkandan 1890fc6389
[cleanup] Misc fixes
Cherry-picks from: #3498, #3947
Related: #3949, https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1140313836
Authored by: pukkandan, flashdagger, gamer191
2022-06-03 21:45:35 +05:30
pukkandan 666c36d58d
Bugfix for 23326151c4 2022-05-20 21:03:19 +05:30
pukkandan 23326151c4
Add option --retry-sleep (#3059)
Closes #2852
2022-05-19 20:00:31 +05:30
pukkandan 3fe75fdc80
[cleanup] Misc fixes (see desc)
* Do not warn when fixup is skipped for existing file
* [fragment] Fix `--skip-unavailable-fragments` for HTTP Errors
* [utils] write_string: Fix bug in 59f943cd50
* [utils] parse_codecs: Subtitle codec is generally referred to as `scodec`. https://github.com/yt-dlp/yt-dlp/pull/2174#discussion_r790156048
* [docs] Remove note about permissions. Closes #3597
2022-05-01 04:58:38 +05:30
pukkandan 1d485a1a79
[cleanup] Misc fixes
Closes #3565, https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
2022-04-29 07:39:33 +05:30
pukkandan 19a0394044
[cleanup] Misc cleanup and refactor (#2173) 2022-04-18 02:28:28 +05:30
Lesmiscore (Naoya Ozaki) c854208ccf
[downloader/fragment] Make single thread download work for --live-from-start (#3446)
Authored by: Lesmiscore
2022-04-16 21:11:09 +09:00
pukkandan e06bd8800f
Fix `--skip-unavailable-fragments`
Bug in d71fd41249
Closes #3437
2022-04-15 23:47:37 +05:30
pukkandan 3b9d9f4374
Do not change fragment chunk-size when `--test`
Closes #3434
2022-04-15 17:05:56 +05:30
pukkandan f82711587c
[cleanup] Sort imports
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
2022-04-12 05:32:52 +05:30
pukkandan 86e5f3ed2e
[cleanup] Upgrade syntax
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
2022-04-12 05:32:51 +05:30
pukkandan a44ca5a470
[cleanup] Misc fixes
Closes https://github.com/yt-dlp/yt-dlp/pull/3213, Closes https://github.com/yt-dlp/yt-dlp/pull/3117

Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114, https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671, a825ffbffa (commitcomment-68538986), https://github.com/yt-dlp/yt-dlp/issues/2360, 5fa3c9a88f (r70393519), 5fa3c9a88f (r70393254)
2022-04-05 18:12:18 +05:30
Lesmiscore (Naoya Ozaki) d71fd41249
[fragment] Read downloaded fragments only when needed (#3069)
Authored by: Lesmiscore
2022-03-15 12:27:41 +09:00
Emanuel Hoogeveen 45806d44a7
[downloader] Obey `--file-access-retries` when deleting/renaming (#2224)
Authored by: ehoogeveen-medweb
2022-03-03 06:33:32 -08:00
Lesmiscore (Naoya Ozaki) 93c8410d33
[downloader/fragment] Fix bugs around resuming with Range (#2901)
Authored by: Lesmiscore
2022-02-28 13:10:54 +09:00
Lesmiscore 195c22840c
[downloader/fragment] Ignore `FileNotFoundError` when downloading livestreams
when `--live-from-start` is used for YouTube and the live ends, request for the last segment prematurely ends (or 404, 403).
this is causing lack of the file and `FileNotFoundError`
lacking segment doesn't have any data, so it's safe to ignore
2022-02-26 12:34:36 +09:00
Lesmiscore f0734e1190
[downloader/fragment] Fix problem where multiple threads can share one iterator
which causes "ValueError: generator already executing" error

Closes #2881
2022-02-25 13:22:17 +09:00
Lesmiscore (Naoya Ozaki) a539f06570
[downloader/fragment] Improve `--live-from-start` for YouTube livestreams (#2870) 2022-02-25 02:00:46 +09:00
pukkandan 1d3586d0d5
[aes] Add unpad_pkcs7 2022-02-01 00:29:36 +05:30
pukkandan 185bf31070
[youtube] End `live-from-start` properly when stream ends with 403
Closes #2089
2021-12-26 16:14:00 +05:30
Emanuel Hoogeveen 205a0654c0
Add option `--file-access-retries` (#2066)
Closes #517
Authored by: ehoogeveen-medweb
2021-12-23 07:59:03 +05:30
The Hatsune Daishi adbc4ec4bb
[dash,youtube] Download live from start to end (#888)
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms

Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats

Closes #1521
Authored by: nao20010128nao, pukkandan
2021-12-20 11:36:46 +05:30
pukkandan 59a7a13ef9
[docs] Minor documentation improvements
Closes #1583, #1599
2021-11-10 04:19:52 +05:30
pukkandan bd93fd5d45
[fragment] Fix progress display in fragmented downloads
Closes #1517
2021-11-03 16:45:58 +05:30
pukkandan 49a57e70a9
[cleanup] misc 2021-10-23 02:09:10 +05:30
pukkandan b4b855ebc7
[fragment] Print error message when skipping fragment 2021-10-19 22:58:26 +05:30
pukkandan 2cda6b401d
Revert "[fragments] Pad fragments before decrypting (#1298)"
This reverts commit 373475f035.
2021-10-19 22:58:25 +05:30
shirt 373475f035
[fragments] Pad fragments before decrypting (#1298)
Closes #197, #1297, #1007
Authored by: shirt-dev
2021-10-18 08:14:20 +05:30
pukkandan 819e05319b
Improved progress reporting (See desc) (#1125)
* Separate `--console-title` and `--no-progress`
* Add option `--progress` to show progress-bar even in quiet mode
* Fix and refactor `minicurses`
* Use `minicurses` for all progress reporting
* Standardize use of terminal sequences and enable color support for windows 10
* Add option `--progress-template` to customize progress-bar and console-title
* Add postprocessor hooks and progress reporting

Closes: #906, #901, #1085, #1170
2021-10-09 00:41:59 +05:30
shirt 7687c8ac6e
[HLS] Fix decryption issues (#1117)
* Unpad HLS fragments with PKCS#7 according to datatracker.ietf.org/doc/html/rfc8216
* media_sequence should only be incremented in for media fragments
* The native decryption should only be used if ffmpeg is unavailable since it is significantly slower. Closes #1086

Authored by: shirt-dev, pukkandan
2021-09-29 00:23:24 +05:30
The Hatsune Daishi bd50a52b0d
Basic framework for simultaneous download of multiple formats (#1036)
Authored by: nao20010128nao
2021-09-22 19:42:04 +05:30
pukkandan 1009f67c2a
[fragment,aria2c] Generalize and refactor some code 2021-09-22 05:27:07 +05:30
pukkandan d9d8b85747
[fragment] Fix range header when using `-N` and media sequence (#1048)
Authored by: shirt
2021-09-22 04:19:45 +05:30