etc/yt-dlp
0
0
Fork 0
mirror of https://github.com/yt-dlp/yt-dlp.git synced 2024-11-04 23:35:04 +00:00
Code Issues Projects Releases Wiki Activity
15195 commits 3 branches 99 tags 82 MiB
Commit graph

500 commits

Author SHA1 Message Date
Jaime Marquínez Ferrándiz
360e1ca5cc [youtube] Convert to new subtitles system 
The automatic captions are stored in the 'automactic_captions' field, which is used if no normal subtitles are found for an specific language.
 2015-02-16 22:47:39 +01:00
Jaime Marquínez Ferrándiz
c84dd8a90d [YoutubeDL] store the subtitles to download in the 'requested_subtitles' field 
We need to keep the orginal subtitles information, so that the '--load-info' option can be used to list or select the subtitles again.
We'll also be able to have a separate field for storing the automatic captions info.
 2015-02-16 21:51:08 +01:00
Jaime Marquínez Ferrándiz
a504ced097 Improve subtitles support 
For each language the extractor builds a list with the available formats sorted (like for video formats), then YoutubeDL selects one of them using the '--sub-format' option which now allows giving the format preferences (for example 'ass/srt/best').
For each format the 'url' field can be set so that we only download the contents if needed, or if the contents needs to be processed (like in crunchyroll) the 'data' field can be used.

The reasons for this change are:
* We weren't checking that the format given with '--sub-format' was available, checking it in each extractor would be repetitive.
* It allows to easily support giving a format preference.
* The subtitles were automatically downloaded in the extractor, but I think that if you use for example the '--dump-json' option you want to finish as fast as possible.

Currently only the ted extractor has been updated, but the old system still works.
 2015-02-16 21:51:03 +01:00
Philipp Hagemeister
73fac4e911 [ffmpeg] Add --ffmpeg-location 2015-02-16 04:05:53 +01:00
Philipp Hagemeister
442c37b7a9 [YoutubeDL] Do not perform filter matching on partial results (Fixes #4921) 2015-02-10 11:28:28 +01:00
Philipp Hagemeister
7e5db8c930 [options] Add --no-color 2015-02-10 04:22:10 +01:00
Philipp Hagemeister
347de4931c [YoutubeDL] Add generic video filtering (Fixes #4916) 
This functionality is intended to eventually encompass the current format filtering.
 2015-02-10 03:32:24 +01:00
Philipp Hagemeister
43f244b6d5 [YoutubeDL] Do not show worst in --list-formats output 
Nobody wants to know what the worst possible format is. And if they do, they can still provide -f worst.
 2015-02-09 15:57:42 +01:00
Sergey M․
2ec19e9558 [YoutubeDL] Allow filtering by audio sampling rate 2015-02-09 01:09:45 +06:00
Sergey M․
9ddb6925bf [YoutubeDL] Allow filtering by string properties (#4906) 2015-02-09 01:07:43 +06:00
Jaime Marquínez Ferrándiz
b3a286d69d [YoutubeDL] _calc_cookies: add get_header method to _PseudoRequest (#4861) 2015-02-06 22:23:06 +01:00
Sergey M․
300753a069 [YoutubeDL] Fix video+audio format field (Closes #4880) 2015-02-06 00:51:16 +06:00
Philipp Hagemeister
f6c24009be [YoutubeDL] Calculate thumbnail IDs automatically 2015-02-03 10:52:22 +01:00
Philipp Hagemeister
8f9312c387 Appease pyflakes8-3 2015-02-01 11:30:56 +01:00
Sergey M․
6d593c3276 [YoutubeDL] Fix video+audio format_id (Closes #4824) 2015-01-31 03:50:11 +06:00
codesparkle
5468ff4d91 Remove duplicate dictionary keys 2015-01-30 20:11:51 +11:00
Jaime Marquínez Ferrándiz
a7a14d9586 [YoutubeDL] set the 'thumbnails' field if the info_dict has the 'thumbnails' field 
Since the '--write-thumbnail' uses the 'thumbnails' field and we didn't updated the info_dict, it wouldn't detect the thumbnail. (fixes #4812)
 2015-01-29 20:15:38 +01:00
Sergey M․
dcf53d4408 [YoutubeDL] Set format_id for video+audio (Closes #3634) 2015-01-29 00:44:40 +06:00
Sergey M․
3a0d2f520a [YoutubeDL] Temporary fix for subprocess encoding issues on python2 @ Windows (Closes #4787) 
For now filenames will be encoded with preferrefencoding before written to disk
 2015-01-27 22:38:28 +06:00
Philipp Hagemeister
9f0df77ab1 [YoutubeDL] Allow format filtering by fps 2015-01-26 14:36:22 +01:00
Jaime Marquínez Ferrándiz
e72c7e4123 [YoutubeDL] Always set the '_filename' field in the info_dict (reported in #4053) 
It's also useful when you use the '--write-info-json' option.
 2015-01-26 12:01:43 +01:00
Philipp Hagemeister
ee69b99af6 [YoutubeDL] clarify hook documentation 2015-01-25 06:15:54 +01:00
Philipp Hagemeister
881e6a1f5c Add --xattr-set-filesize option (Fixes #1348) 2015-01-25 04:49:44 +01:00
Philipp Hagemeister
c14e88f0f5 [YoutubeDL] Add --playlist-items option (Fixes #2662) 2015-01-25 04:24:55 +01:00
Philipp Hagemeister
ec82d85acd [YoutubeDL] Implement --write-all-thumbnails (Closes #2269) 2015-01-25 03:11:12 +01:00
Philipp Hagemeister
cfb56d1af3 Add --list-thumbnails 2015-01-25 02:43:19 +01:00
Jaime Marquínez Ferrándiz
1070711d60 [YoutubeDL._calc_cookies] Restore the 'is_unverifiable' 
I should have check everything was copied before commiting 4b405cfc6e.
 2015-01-24 20:12:47 +01:00
Jaime Marquínez Ferrándiz
4b405cfc6e [YoutubeDL._calc_cookies] Restore the 'has_header' method 
I didn't copied it from downloader/external
 2015-01-24 20:08:24 +01:00
Jaime Marquínez Ferrándiz
e5660ee6ae [YoutubeDL] Fill the info dict 'http_headers' field with all the headers available 
Useful for external tools using the json output.

The methods '_calc_headers' and '_calc_cookies' have been copied from the downloader/external, now they just use "info_dict['http_headers']".
 2015-01-24 18:56:04 +01:00
Philipp Hagemeister
222516d97d [downloader] Lay groundwork for external downloaders. 
This comes with a very simply implementation for wget; the real work is in setting up the infrastructure.
 2015-01-24 01:38:48 +01:00
Philipp Hagemeister
a055469faf [downloader] Improve downloader selection 2015-01-23 23:50:31 +01:00
Philipp Hagemeister
62cd676c74 [youtube] Fixup DASH m4a headers 
This fixes #2288, #2506, #2607, #3681, #4741, #4767.
 2015-01-23 18:39:12 +01:00
Philipp Hagemeister
5f0d813d93 Merge remote-tracking branch 'rupertbaxter2/master' 
Conflicts:
	youtube_dl/__init__.py
	youtube_dl/downloader/common.py
 2015-01-23 12:05:01 +01:00
Philipp Hagemeister
083c9df93b [YoutubeDL] Allow filtering by properties (Fixes #4584) 2015-01-23 00:04:05 +01:00
Pierre
4340decad2 check for overwriting files in the downloader (fixes #3916, closes #3829) 2015-01-11 12:02:27 +01:00
Jaime Marquínez Ferrándiz
f3ff1a3696 YoutubeDL: Make the decision about removing the original file after each postprocessor is run (fixes #2261) 
If one of the processors said the file should be kept, it wouldn't pay
attention to the response from the following processors. This was wrong if the
'keep_video' option was False, if the first extractor modifies the original file
and then we extract its audio we don't want to keep the original video file.
 2015-01-11 11:35:18 +01:00
Philipp Hagemeister
8bfa75451b [options] Add --no-call-home 
While we're at it, also drop "experimental" moniker for --call-home - should work fine.
 2015-01-10 21:09:18 +01:00
Philipp Hagemeister
58b1f00d19 [YoutubeDL] Add new --call-home option for debugging 2015-01-10 21:02:27 +01:00
Philipp Hagemeister
be4a824d74 Add new option --source-address 
Closes #3618, fixes #721, fixes #2481, fixes #4551, closes #1020.
 2015-01-10 19:56:51 +01:00
Philipp Hagemeister
6271f1cad9 [youtube|ffmpeg] Automatically correct video with non-square pixels (Fixes #4674) 2015-01-10 05:45:51 +01:00
Philipp Hagemeister
45598f1578 Merge remote-tracking branch 'aft90/merge-output-format' 
Conflicts:
	youtube_dl/YoutubeDL.py
 2015-01-10 01:59:14 +01:00
Andrei Troie
d02115f837 Use the option in preparing the merge output filename 2015-01-10 00:29:06 +00:00
Philipp Hagemeister
f90ad27375 [YoutubeDL] Copy over format metadata when merging (Fixes #4671) 2015-01-09 20:50:23 +01:00
Philipp Hagemeister
0217c78377 [YoutubeDL] Allow selection by more extensions 2015-01-08 16:14:50 +01:00
Philipp Hagemeister
0590062925 Respect age_limit when listing extractors (Fixes #4653) 2015-01-07 07:20:20 +01:00
Philipp Hagemeister
e65566a9cc [youtube] Correct handling when DASH manifest is not necessary to find all formats 2015-01-03 18:33:38 +01:00
Jaime Marquínez Ferrándiz
f00fd51dae Don't write the description file if info_dict['description'] is None (#3166) 2014-12-21 20:49:14 +01:00
Jaime Marquínez Ferrándiz
0e4cb4f406 YoutubeDL: style fix 2014-12-17 14:55:27 +01:00
AndroKev
cd58dc3e56 Update YoutubeDL.py 2014-12-17 13:21:22 +01:00
AndroKev
c33bcf2051 only add video-id to archive, when successful 
Example:
no space left--> youtube-dl adds the id to archive, but the video isn't fully downloaded
 2014-12-17 13:05:19 +01:00
Philipp Hagemeister
734ea11e3c Drop hash character in downloader output (#4484) 2014-12-16 00:37:42 +01:00
Philipp Hagemeister
d86007873e [YoutubeDL] Document where details for format can be found 2014-12-16 00:22:23 +01:00
Philipp Hagemeister
71b640cc5b [YoutubeDL] Add declarative version of progress hooks 2014-12-15 01:26:20 +01:00
Philipp Hagemeister
4f026fafbc [YoutubeDL] Make postprocessors declarative 
Instead of having to configure PPs in code, this allows us and embedding programs not to worry about imports or finer details, similarly to how we handle IEs.
 2014-12-15 01:06:25 +01:00
Mark Schreiber
ff815fe65a Download playlist items in reverse order 
Series of videos are typically uploaded to YouTube playlists in
chronological order.  By default, these videos are downloaded
latest-to-earliest; this is great for seeing the latest videos in a
series, but prevents streaming video in the order that the videos were
produced.  Add an option to download videos in reverse order,
earliest-to-latest.

Conflicts:
	youtube_dl/YoutubeDL.py
	youtube_dl/__init__.py
 2014-12-12 16:56:29 +01:00
Philipp Hagemeister
412c617d0f [cnet] Update to new theplatform infrastructure (Fixes #2736) 2014-12-12 15:55:55 +01:00
Philipp Hagemeister
f17e4c9c28 [screenwavemedia] Simplify (#3766) 2014-12-12 02:11:58 +01:00
cryptonaut
16ae61f655 Handle --get-url with merged formats (fixes #2883) 
Outputs one URL per line
 2014-12-06 12:55:07 -08:00
Philipp Hagemeister
b82f815f37 Allow iterators for playlist result entries 2014-12-06 14:02:19 +01:00
Sergey M․
706d7d4ee7 [YoutubeDL] Avoid negative timestamps on Windows 2014-12-02 21:18:07 +06:00
Philipp Hagemeister
4aae94f9d0 [YoutubeDL] Remove incorrect documentation 2014-11-26 11:25:43 +01:00
Philipp Hagemeister
acda92f6bc Clarify --no-playlist documentation (Closes #4309) 2014-11-26 10:51:03 +01:00
Jouke Waleson
9e1a5b8455 PEP8: applied even more rules 2014-11-23 21:39:15 +01:00
Jouke Waleson
8bcc875676 PEP8: more applied 2014-11-23 21:20:46 +01:00
Jouke Waleson
5f6a1245ff PEP8 applied 2014-11-23 20:41:03 +01:00
Philipp Hagemeister
7d4111ed14 Provide guidance when called with a YouTube ID starting with a dash. 
Reported at https://news.ycombinator.com/item?id=8648121
 2014-11-23 10:51:09 +01:00
Philipp Hagemeister
42e12102a9 [YoutubeDL] Fix multi_video check 2014-11-21 22:39:57 +01:00
Philipp Hagemeister
dfd5313afd [YoutubeDL] Support new _type multi_video 2014-11-21 00:25:46 +01:00
Philipp Hagemeister
c9bf41145f [YoutubeDL] Warn if an extractor returns compat_list 2014-11-20 16:29:31 +01:00
Jaime Marquínez Ferrándiz
92120217eb [cache] Fix writing to paths with unicode characters 
* Use "compat_getenv"
* "write_json_file" now expects the filename to be a string
 2014-11-19 00:02:24 +01:00
Philipp Hagemeister
c6afed48ff [YoutubeDL] guard against strange sys.stdouts 2014-11-12 15:30:26 +01:00
Jaime Marquínez Ferrándiz
c295490830 [YoutubeDL] Fix bug in the detection of formats that don't contain video (fixes #4150) 
If the format requested was not available, we called the method '.get' in None.
 2014-11-12 09:42:35 +01:00
Philipp Hagemeister
a1cf99d03a [YoutubeDL] Add playlist_id and playlist_title fields (Fixes #4139) 2014-11-09 22:32:35 +01:00
Jaime Marquínez Ferrándiz
f866e474f3 [YoutubeDL] Don't dowload formats for merging if the first doesn't contain the video (#4132) 2014-11-09 10:59:56 +01:00
Philipp Hagemeister
8c25f81bee [util] Move compatibility functions out of util 
utils is large enough without these compatibility functions.

Everything that is present in newer versions of Python (i.e. with dev Python it's just an import) goes into compat.py .
Everything else (i.e. youtube-dl-specific helpers) goes into utils.py .
 2014-11-02 11:23:42 +01:00
Philipp Hagemeister
4c83c96795 [YoutubeDL] Include rtmpdump in exe versions -v output 2014-11-02 10:55:36 +01:00
Philipp Hagemeister
fbb21cf528 [youtube] Add formats 298, 299 (Fixes #4056) 2014-10-30 09:34:13 +01:00
Philipp Hagemeister
3511266bc3 [YoutubeDL] Simplify API of YoutubeDL 
Calling add_default_extractors twice should be harmless since the first set of extractors will match.
 2014-10-28 12:54:29 +01:00
Philipp Hagemeister
488447455d [ffmpeg] Warn if ffmpeg/avconv version is too old (Fixes #4026) 2014-10-26 16:46:34 +01:00
Philipp Hagemeister
d28b517154 [YoutubeDL] Output avconv/ffmpeg versions if -v is given 2014-10-26 16:31:52 +01:00
Sergey M․
f889cea109 Merge branch 'compat-getenv-and-expanduser' of https://github.com/dstftw/youtube-dl into dstftw-compat-getenv-and-expanduser 
Conflicts:
	test/test_utils.py
	youtube_dl/__init__.py
 2014-10-26 19:56:52 +07:00
Philipp Hagemeister
63e0be3415 New option --dump-single-json (#4003) 2014-10-25 00:30:57 +02:00
Philipp Hagemeister
e82c1e9a6e [YoutubeDL] Do not apply playlist info to videos when extract_flat is set (#4003) 2014-10-24 16:13:45 +02:00
Philipp Hagemeister
057a5206cc Add --flat-playlist option (Closes #4003) 2014-10-24 14:48:12 +02:00
George Boyle
53d9009bdb KeyError on initialising YoutubeDL in python3 #3910 2014-10-10 10:03:24 +01:00
George Boyle
1b725173a5 Fixed typo 2014-10-10 09:35:41 +01:00
Sergey M․
4644ac5527 [core] Decode environment variables with filesystem encoding (Fixes #3854, Fixes #3217, Fixes #2918) 
Introduces compat versions of os.getenv and os.path.expanduser
 2014-09-30 22:27:53 +07:00
Sergey M․
ee0d90707a [YoutubeDL] Fix string check for python3 2014-09-28 02:48:41 +07:00
Sergey M․
68b0973046 [YoutubeDL] Expect all kind of strings in urlopen 
Now it doesn't fail if req is python2's str
 2014-09-28 02:07:42 +07:00
Philipp Hagemeister
1de33fafd9 [YoutubeDL] Allow downloading multiple formats with , 2014-09-18 18:43:49 +02:00
Philipp Hagemeister
e2e5dae64d Add -f m4a 2014-09-18 18:40:19 +02:00
Sergey M․
d05cfe0600 [YoutubeDL/utils] Clarify rationale for URL escaping in comment, move escape routines to utils and add some tests 2014-09-13 20:59:16 +07:00
Sergey M․
37419b4f99 [YoutubeDL] Escape non-ASCII characters in URLs 
urllib chokes on URLs with non-ASCII characters (see http://bugs.python.org/issue3991)
Working around by replacing request's original URL with escaped one
 2014-09-12 23:20:17 +07:00
Philipp Hagemeister
a0e07d3161 [youtube] Move cache into its own module 2014-09-03 17:29:19 +02:00
Philipp Hagemeister
241f7a8ade Merge remote-tracking branch 'JGjorgji/fix-leading-zeroes' 2014-08-25 13:59:19 +02:00
Philipp Hagemeister
8d31fa3cce [execafterdownload] Simplify (#3569) 2014-08-25 10:18:01 +02:00
Gjorgji Jankovski
c6b4132a0a renamed for consistency 2014-08-24 18:49:04 +02:00
Gjorgji Jankovski
ad260c90ab Filenames are padded according to the playlist length 2014-08-24 18:23:32 +02:00
Philipp Hagemeister
be843678b1 [YouTubeDL] Correct handling of age_limit = None in result 2014-08-22 17:46:57 +02:00
Philipp Hagemeister
e8ee972c6e Allow playlist test definitions in test_download. 
This moves playlist tests where they belong, i.e. to the extractors themselves.
Additionally, all our network interaction configuration for tests in test_download now applies to playlist tests as well.
 2014-08-21 11:52:07 +02:00
Philipp Hagemeister
11b85ce62e [YouTubeDL] Best practices (Closes #3370) 2014-07-25 23:37:32 +02:00
Philipp Hagemeister
4192b51c7c Replace failure handling with up-front check. 
The only time that write_string should fail is if the Python is completely braindead.
Check for that condition and output a more accurate warning.

See #3326 for details.
 2014-07-24 13:29:44 +02:00
rupertbaxter2
b7f8116406 Deletes temp files after postprocess merge unless -k option is specified 2014-07-23 02:53:44 +02:00
Philipp Hagemeister
b0472057a3 [YoutubeDL] Make sure we really, really get out the encoding string 
Fixes #3326
Apparently, on some platforms, even outputting this fails already.
 2014-07-23 02:24:52 +02:00
Philipp Hagemeister
9732d77ed2 [snotr] PEP8 and minor fixes (#3296) 2014-07-21 12:02:44 +02:00
Philipp Hagemeister
8d5797b00f [YoutubeDL] Show download URL when -v is set 
This will allow us to debug issues like #3204
 2014-07-06 11:28:51 +02:00
Philipp Hagemeister
be6d722904 [cnn] Improve thumbnail extraction 2014-06-07 15:39:21 +02:00
Philipp Hagemeister
d551980823 [spiegeltv] Simplify and PEP8 2014-06-07 15:35:13 +02:00
Philipp Hagemeister
acd69589a5 [YoutubeDL] Do not require default output template to be set 2014-04-30 10:02:08 +02:00
Philipp Hagemeister
c57f775710 [YoutubeDL] Add simple tests for format_note (Closes #2825) 2014-04-30 02:02:41 +02:00
robbie
e4db19511a Fix subtitle download error reporting (Fixes #2724) 2014-04-08 15:59:27 +01:00
Philipp Hagemeister
b58ddb32ba [utils] Completely rewrite Windows output (Fixes #2672) 2014-04-07 22:48:13 +02:00
Philipp Hagemeister
734f90bb41 Use --encoding when outputting 2014-04-07 19:57:42 +02:00
Philipp Hagemeister
d26e981df4 Correct check for empty dirname (Fixes #2683) 2014-04-03 15:28:41 +02:00
Philipp Hagemeister
bec1fad223 [YouTubeDL] Throw an early error if the info_dict result is invalid 2014-04-03 14:38:16 +02:00
Philipp Hagemeister
cce929eaac [franceculture] Add extractor (Fixes #2669) 2014-04-03 08:55:38 +02:00
Philipp Hagemeister
62fec3b2ff Add new --encoding option (Fixes #2650) 2014-03-30 06:08:22 +02:00
Philipp Hagemeister
ad8915b729 Add --no-warnings option (Fixes #2630) 2014-03-26 00:43:46 +01:00
Philipp Hagemeister
ea38e55fff [instagram] Add support for user profiles (Fixes #2606) 2014-03-23 16:06:07 +01:00
Philipp Hagemeister
7e8c0af004 Add --prefer-insecure option (Fixes #2364) 2014-03-21 00:37:10 +01:00
Philipp Hagemeister
1a4895453a [YoutubeDL] Improve error message 2014-03-20 16:33:46 +01:00
Jaime Marquínez Ferrándiz
bc6d597828 Add bestvideo and worstvideo to special format names (#2163) 2014-03-14 17:01:47 +01:00
Philipp Hagemeister
955c451456 Rename upload_timestamp to timestamp 2014-03-13 18:45:14 +01:00
Philipp Hagemeister
9d2ecdbc71 [vevo] Centralize timestamp handling 2014-03-13 15:30:25 +01:00
Philipp Hagemeister
db95dc13a1 [playvid] Simplify (#2539) 2014-03-10 20:55:47 +01:00
Jaime Marquínez Ferrándiz
a0792b738e Don't install the global url opener 
All the code uses now the urlopen method of YoutubeDL
 2014-03-10 19:04:51 +01:00
Jaime Marquínez Ferrándiz
19a41fc613 Don't set the global socket timeout 
Use the timeout argument of the `OpenerDirector.open` method instead
 2014-03-10 19:03:37 +01:00
Philipp Hagemeister
805ef3c60b Correct automatic resolution determination 2014-03-10 10:29:25 +01:00
Jaime Marquínez Ferrándiz
6d07ce0162 YoutubeDL: If the logger is set call its warning method in report_warning 2014-03-09 15:16:54 +01:00
Jaime Marquínez Ferrándiz
e9c092f125 YoutubeDL: Use its urlopen method for downloading the thumbnail. 2014-03-07 16:43:34 +01:00
Philipp Hagemeister
17b75c0de1 Document width, height, and resolution (#1445) 2014-03-04 03:49:33 +01:00
Philipp Hagemeister
c9ae7b9565 [youtube] Add support for search result URLs (Fixes #2495) 2014-03-04 03:32:28 +01:00
Philipp Hagemeister
0afef30b23 Add display_id field 2014-03-03 12:06:28 +01:00
Jaime Marquínez Ferrándiz
f89197d73e Some pep8 style fixes 2014-01-25 15:33:23 +01:00
Philipp Hagemeister
1394ce65b4 [youtube] Add new formats (Fixes #2221) 2014-01-23 23:54:06 +01:00
Philipp Hagemeister
fd28827864 Do not count unmatched videos for --max-downloads (Fixes #2211) 2014-01-23 19:04:22 +01:00
Philipp Hagemeister
d3e5bbf437 Correct --max-downloads with --ignore-errors 2014-01-23 10:36:47 +01:00
Philipp Hagemeister
65697b3bf3 Merge branch 'paged-lists' 
Conflicts:
	test/test_utils.py
	youtube_dl/extractor/youtube.py
 2014-01-22 20:00:16 +01:00
Philipp Hagemeister
50317b111d Merge branch 'youtube-dash-manifest' 
Conflicts:
	youtube_dl/extractor/youtube.py
 2014-01-22 19:58:31 +01:00
Philipp Hagemeister
de3ef3ed58 Default to -f best-audio when only audio is requested 2014-01-22 14:53:23 +01:00
Philipp Hagemeister
ba7678f9cc Add -f bestaudio (Fixes #2163) 2014-01-22 14:47:29 +01:00
Philipp Hagemeister
04b4d394d9 Add new --default-search option (#2193) 2014-01-22 14:16:43 +01:00
Philipp Hagemeister
7b0817e8e1 [servingsys] Add support 
This also adds support for brightcove advertisements.
Fixes #2181
 2014-01-21 02:09:51 +01:00
Philipp Hagemeister
b7ab059084 Add infrastructure for paged lists 
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
    youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes #2175
 2014-01-20 11:36:47 +01:00
Philipp Hagemeister
dd27fd1739 [youtube] Download DASH manifest 
If given, download and parse the DASH manifest file, in order to get ultra-HQ formats.
Fixes #2166
 2014-01-19 05:47:20 +01:00
Jaime Marquínez Ferrándiz
58c3c7ae38 Don’t try to merge the formats if ffmpeg or avconv are not installed 2014-01-15 12:59:15 +01:00
Jaime Marquínez Ferrándiz
76b1bd672d Add ‘--prefer-avconv’ and ‘--prefer-ffmpeg’ options (#2115) 
Affects the ffmpeg post processors, if ‘--prefer-ffmpeg’ is given and both avconv and ffmpeg are installed, it will use ffmpeg. Otherwise it will follow the old behaviour.
 2014-01-08 17:53:34 +01:00
Philipp Hagemeister
282962bd36 --list-formats: Only add "@" if vbr is given 2014-01-07 08:08:48 +01:00
Philipp Hagemeister
56327689a2 Move postprocessor into its own package 2014-01-07 05:49:17 +01:00
Philipp Hagemeister
a9c5e5ca6e Set required properties for format merging 2014-01-05 03:44:08 +01:00
Philipp Hagemeister
a015dce0e2 Merge remote-tracking branch 'jaimeMF/merge-formats' 2014-01-05 02:06:48 +01:00
Philipp Hagemeister
6febd1c1df Prepare widespread unicode literal use 2014-01-05 01:52:03 +01:00
Jaime Marquínez Ferrándiz
6350728be2 Allow merging formats (closes #1612) 
Multiple formats can be requested using `-f 137+139`, each one is downloaded and then the two are merged with ffmpeg.
 2014-01-04 13:13:51 +01:00
Philipp Hagemeister
eadaf08c16 Merge remote-tracking branch 'origin/master' 2014-01-01 15:30:46 +01:00
Jaime Marquínez Ferrándiz
241bce7aaf Merge pull request #2061 from rzhxeo/var 
Correct variable name in YoutubeDL.list_formats
 2014-01-01 03:33:34 -08:00
Philipp Hagemeister
33ec2ae8d9 Merge remote-tracking branch 'origin/master' 2014-01-01 10:43:58 +01:00
Jaime Marquínez Ferrándiz
e9f9a10fba Fix initialization of YoutubeDL with params set to None 
Set it to an empty dictionary because it’s directly accessed when setting some properties
 2013-12-31 13:34:52 +01:00
rzhxeo
1cdfc31e1f Correct variable name in YoutubeDL 2013-12-30 06:50:12 +01:00
Philipp Hagemeister
a0ddb8a2fa Add new --print-traffic option 2013-12-29 15:28:32 +01:00
Philipp Hagemeister
7217e148fb [yahoo] Use centralized sorting, and add tbr field 2013-12-25 15:18:40 +01:00
Philipp Hagemeister
c7deaa4c74 [zdf] Use centralized sorting 2013-12-24 23:32:04 +01:00
Philipp Hagemeister
4bcc7bd1f2 Add temporary _sort_formats helper function 2013-12-24 12:31:42 +01:00
Philipp Hagemeister
f49d89ee04 Add a resolution field and improve general --list-formats output 2013-12-24 11:56:02 +01:00
Philipp Hagemeister
1538eff6d8 [bliptv] Remove support for direct downloads 
This is now handled by the generic IE
 2013-12-23 15:49:21 +01:00
Jaime Marquínez Ferrándiz
933605d7e8 YoutubeDL: rename _fd_progress_hooks back to _progress_hooks 
In the future it may report more things.
 2013-12-23 10:37:27 +01:00
Jaime Marquínez Ferrándiz
b3d9ef88ec YoutubeDL: only set the ‘formats’ field of the info_dict if it was already set before 
It caused a circular reference error, when trying to dump it to json (for example with the test video for myvideo.de or any other video without formats)
 2013-12-23 10:23:13 +01:00
Philipp Hagemeister
9fc3bef87a Merge remote-tracking branch 'jaimeMF/split-downloaders' 2013-12-23 05:03:32 +01:00
Philipp Hagemeister
f82b18efc1 Merge remote-tracking branch 'rzhxeo/youtube' 2013-12-23 04:37:40 +01:00
Philipp Hagemeister
196938835a Remove debugging code 
Introduced by accident in 5d681e960d
 2013-12-23 04:30:57 +01:00
Philipp Hagemeister
5d681e960d Use bidiv instead of fribidi if available (Fixes #1912) 2013-12-23 04:19:50 +01:00
rzhxeo
62d68c43ed Make prefer_free_formats sorting more robust 2013-12-18 21:25:13 +01:00
rzhxeo
bfaae0a768 Filter and sort videos before calling list_formats 2013-12-18 21:24:39 +01:00
rzhxeo
4ea3be0a5c [YoutubeIE] Externalize format selection 2013-12-18 03:30:55 +01:00
Philipp Hagemeister
29eb517403 Add webpage_url_basename info_dict field (Fixes #1938) 2013-12-17 04:13:36 +01:00
alimirjamali
0a9ce268ba Incorrect variable is used to check whether thumbnail exists 
Dear @phihag

I believe in line 848, the correct variable to check is 'thumb_filename' rather than 'infofn'

Kindly advise

Mit freundlichen Gruessen
Ali
 2013-12-16 20:14:28 +03:30
Philipp Hagemeister
a19fd00cc4 Simplify --playlist-start / --playlist-end interface 2013-12-16 13:16:20 +01:00
Philipp Hagemeister
7b6fefc9d4 Apply --no-overwrites for --write-* files as well (Fixes #1980) 2013-12-16 04:39:13 +01:00
Philipp Hagemeister
525ef9227f Add --get-duration (Fixes #859) 2013-12-16 04:15:10 +01:00
Philipp Hagemeister
5fe18bdbde Add --min-views / --max-views (Fixes #1979) 2013-12-16 03:09:49 +01:00
Jaime Marquínez Ferrándiz
3bc2ddccc8 Move FileDownloader to its own module and create a new class for each download process 
A suitable downloader can be found using the 'get_suitable_downloader' function.

Each subclass implements 'real_download', for downloading an info dict you call the 'download' method, which first checks if the video has already been downloaded
 2013-12-11 16:18:48 +01:00
Jaime Marquínez Ferrándiz
8ab470f1b2 Now a new FileDownloader is created when downloading a video 
The progress hooks can be added using the method "add_downloader_progress_hook"
 2013-12-11 16:04:42 +01:00
Philipp Hagemeister
357ddadbf5 Fix thumbnail filename determination (Fixes #1945) 2013-12-11 08:54:48 +01:00
Philipp Hagemeister
45598aab08 [YoutubeDL] Simplify filename preparation 2013-12-10 11:23:35 +01:00
Jaime Marquínez Ferrándiz
26e6393134 Set 'NA' as the default value for missing fields in the output template (fixes #1931) 
Remove the `except KeyError` clause, it won't get raised anymore
 2013-12-09 22:00:42 +01:00
Philipp Hagemeister
1c088fa89d Improve --bidi-workaround support 2013-12-09 18:29:07 +01:00
Philipp Hagemeister
395293a889 [--load-info] Always read file as UTF-8 
This allows editing the file (and not escaping non-ASCII characters) and reloading it in.
 2013-12-09 04:59:51 +01:00
Philipp Hagemeister
db4da14027 Merge remote-tracking branch 'jaimeMF/load-info' 2013-12-09 04:55:02 +01:00
Philipp Hagemeister
2101830c0d Remove unused imports 2013-12-09 04:53:23 +01:00
Philipp Hagemeister
a0d96c9843 Add filename to --dump-json output (Fixes #1908) 2013-12-09 04:31:18 +01:00
Philipp Hagemeister
0783b09b92 Add a workaround for terminals without bidi support (Fixes #1912) 2013-12-09 04:08:51 +01:00
Philipp Hagemeister
ef4fd84857 [wistia] Add extractor 2013-12-06 09:15:04 +01:00
Philipp Hagemeister
7fc3fa0545 [9gag] Add extractor 2013-12-05 14:29:08 +01:00
Jaime Marquínez Ferrándiz
d494389821 Option '--load-info': if the download fails, try extracting the info with the 'webpage_url' field of the info dict 
The video url may have expired.
 2013-12-03 20:16:52 +01:00
Jaime Marquínez Ferrándiz
1dcc4c0cad Add --load-info option (#972) 
It just calls the 'YoutubeDL.process_ie_result' with the dictionary from the json file
 2013-12-03 20:15:20 +01:00
Philipp Hagemeister
6ad14cab59 Add --socket-timeout option 2013-12-02 13:37:05 +01:00
Philipp Hagemeister
e344693b65 Make socket timeout configurable, and bump default to 10 minutes (#1862) 2013-12-01 11:42:02 +01:00
Philipp Hagemeister
a3fb4675fb Do not mutate default arguments 
In this case, it looks rather harmless (since the conditions for --restrict-filenames should not change while a process is running), but just to be sure.
This also simplifies the interface for callers, who can just pass in the idiomatic None for "I don't care, whatever is the default".
 2013-11-29 15:25:11 +01:00
Jaime Marquínez Ferrándiz
4a98cdbf3b YoutubeDL: set the 'params' property before any message/warning/error is sent (fixes #1840) 
If it sets the 'restrictfilenames' param, it will first report a warning. It will try to get the logger from the 'params' property, which would be set at that moment to None, raising the error 'AttributeError: 'NoneType' object has no attribute 'get''
 2013-11-26 18:54:14 +01:00
Jaime Marquínez Ferrándiz
d31209a144 Use the 'extractor_key' field for the download archive file 
It has the same value as the ie_key.
 2013-11-25 22:57:15 +01:00
Jaime Marquínez Ferrándiz
529a2e2cc3 Fix typo in the documentation of the 'download_archive' param 2013-11-25 22:52:09 +01:00
Philipp Hagemeister
fb04e40396 [soundcloud] Support for listing of audio-only files 2013-11-25 22:34:56 +01:00
Philipp Hagemeister
b0b9eaa196 Merge pull request #1829 from jaimeMF/ydl-empty-params 
Allow to initialize a YoutubeDL object without parameters
 2013-11-25 13:19:59 -08:00
Philipp Hagemeister
0c75c3fa7a Do not warn about fixed output template if --max-downloads is 1 
Fixes #1828
 2013-11-25 22:15:33 +01:00
Jaime Marquínez Ferrándiz
a3927cf7ee Allow to initialize a YoutubeDL object without parameters 
Having to pass the 'outtmpl' parameter feels really strange when you just want to extract the info of a video.
 2013-11-25 22:03:39 +01:00
Philipp Hagemeister
d46cc192d7 Reduce socket timeout 2013-11-25 19:11:01 +01:00
Philipp Hagemeister
5db07df634 Fix --download-archive (Fixes #1826) 2013-11-25 15:46:54 +01:00
Philipp Hagemeister
79d09f47c2 Merge branch 'opener-to-ydl' 2013-11-25 03:30:37 +01:00
Philipp Hagemeister
c059bdd432 Remove quality_name field and improve zdf extractor 2013-11-25 03:28:55 +01:00
Philipp Hagemeister
02dbf93f0e [zdf/common] Use API in ZDF extractor. 
This also comes with a lot of extra format fields
Fixes #1518
 2013-11-25 03:13:22 +01:00
Philipp Hagemeister
e03db0a077 Merge branch 'master' into opener-to-ydl 2013-11-24 15:18:44 +01:00
Philipp Hagemeister
a1ee09e815 Document proxy 2013-11-24 15:03:25 +01:00
Philipp Hagemeister
8bf9319e9c Simplify logger code(#1811) 2013-11-24 06:08:11 +01:00
Itay Brandes
43afe28588 Log to an external logger (fixes #1810) 
Sadly applications using youtube-dl's python sources can't directly
access it's log stream. It's pretty much limited to stdout and stderr
only.

It should log to logging.Logger instance passed to YoutubeDL's params
dictionary.
 2013-11-23 10:22:18 +02:00
Philipp Hagemeister
7012b23c94 Match --download-archive during playlist processing (Fixes #1745) 2013-11-22 22:46:46 +01:00
Philipp Hagemeister
00ea0f11eb Print full title in --get-title output (#1806) 2013-11-22 20:00:35 +01:00
Philipp Hagemeister
dca0872056 Move the opener to the YoutubeDL object. 
This is the first step towards being able to just import youtube_dl and start using it.
Apart from removing global state, this would fix problems like #1805.
 2013-11-22 19:57:52 +01:00
Philipp Hagemeister
86bd5f2ca9 Merge remote-tracking branch 'dz0ny/patch-1' 2013-11-20 06:21:05 +01:00
Philipp Hagemeister
8694c60000 import json for --dump-json 2013-11-20 06:18:24 +01:00
Mohamedh Fazal
9d1538182f Add an option to dump json information 2013-11-20 06:14:57 +01:00
Jaime Marquínez Ferrándiz
efd6c574a2 Correctly write and restore the console title on the stack (fixes #1782) 2013-11-18 16:35:41 +01:00
Philipp Hagemeister
749febf4d1 Allow --console-title when --quiet is given (Fixes #1783) 2013-11-17 21:12:50 +01:00
Philipp Hagemeister
bdde425cbe Save and restore console title (Fixes #1782) 2013-11-17 21:10:11 +01:00
Philipp Hagemeister
ce02ed60f2 Remove * imports 2013-11-17 16:47:52 +01:00
Philipp Hagemeister
1e5b9a95fd Move console_title to YoutubeDL 2013-11-17 11:39:52 +01:00
Philipp Hagemeister
b5349e8721 Fix indentation of (best) and (worst) in --list-formats 2013-11-16 01:39:45 +01:00
Philipp Hagemeister
7150858d49 [spiegel] Implement format selection 2013-11-16 01:33:12 +01:00
Philipp Hagemeister
91c7271aab Add automatic generation of format note based on bitrate and codecs 2013-11-16 01:08:43 +01:00
Jaime Marquínez Ferrándiz
feee2ecfa9 Pass the 'download' argument to 'process_video_result' (fixes #1769) 2013-11-15 11:04:26 +01:00
Janez Troha
9771cceb2c Fix filename extension leaking to json filename 
Makes writeinfojson behaving exactly as writethumbnail in case where filename contains mediafile extension.

Case:

video.mp4 converted to music.mp3 would yield music.mp4.info.json instead music.mp3.info.json or music.info.json
 2013-11-13 18:34:03 +01:00
Jaime Marquínez Ferrándiz
ca715127a2 Don't assume the 'subtitlesformat' is set in the params dict (fixes #1750) 2013-11-13 17:14:10 +01:00
Jaime Marquínez Ferrándiz
897d6cc43a Improve format listing for long format ids 
Now arte.tv videos have quite long ids.
 2013-11-09 19:07:34 +01:00
Jaime Marquínez Ferrándiz
be97abc247 Set the 'extractor_key' field in the info_dict 
It's the string returned by the class method 'ie_key', which allows to retrieve the extractor with 'get_info_extractor'
 2013-11-03 12:14:44 +01:00
Jaime Marquínez Ferrándiz
9103bbc5cd Add the 'webpage_url' field to info_dict 
The url for the video page, it must allow to reproduce the result.
It's automatically set by YoutubeDL if it's missing.
 2013-11-03 12:11:13 +01:00
Jaime Marquínez Ferrándiz
b6c45014ae Set the extra_info inside YoutubeDL.process_ie_result and set only if the keys are missing 2013-11-03 11:57:04 +01:00
Philipp Hagemeister
7193498811 Use index in formt string (Fixes vevo test on Python 2.6) 2013-10-30 01:17:00 +01:00
Philipp Hagemeister
94badb2599 Fix output indenting for --list-formats 2013-10-30 01:09:26 +01:00
Philipp Hagemeister
57dd9a8f2f Nicer --list-formats output 2013-10-29 15:09:45 +01:00
Jaime Marquínez Ferrándiz
702665c085 tests: build the filename from the info_dict if the 'file' key is missing 
It will need to have the 'id' and 'ext' keys to work.
 2013-10-28 22:01:37 +01:00
rzhxeo
216d71d001 Check if description and thumbnail are None to prevent crash 2013-10-28 16:28:35 +01:00
Philipp Hagemeister
78a3a9f89e Make "requested format not available" expected (#1655) 2013-10-28 11:41:59 +01:00
Philipp Hagemeister
a7685f3bf4 mixcloud does not do any format selection 2013-10-28 11:41:32 +01:00
Philipp Hagemeister
8abeeb9449 Nicer --list-formats output 2013-10-28 11:31:12 +01:00
Philipp Hagemeister
c1002e96e9 Let extractors omit ext in formats 2013-10-28 11:28:02 +01:00
rzhxeo
7df286540f [YouPornIE] Extract all encrypted links and remove doubles at the end 2013-10-26 21:57:10 +02:00
Philipp Hagemeister
7853cc5ae1 Merge remote-tracking branch 'origin/master' 
Conflicts:
	youtube_dl/YoutubeDL.py
 2013-10-22 22:30:06 +02:00
Philipp Hagemeister
586a91b67f Expand tilde in template (Fixes #1639) 2013-10-22 22:28:26 +02:00
Jaime Marquínez Ferrándiz
fe7e0c9825 Style fixes in YoutubeDL.py 
Fixed some of the problems reported by pep8
 2013-10-22 14:49:34 +02:00
Jaime Marquínez Ferrándiz
12893efe01 Respect the download parameter in YoutubeDL.process_video_result if the extractor handle the format selection 2013-10-22 00:01:59 +02:00
Jaime Marquínez Ferrándiz
8c51aa6506 The 'format' field now defaults to '{format_id} - {width}x{height}{format_note}' 
Following the YoutubeIE format. The 'format_note' gives additional info about the format, for example '3D' or 'DASH video'.
 2013-10-21 14:42:06 +02:00
Jaime Marquínez Ferrándiz
3fd39e37f2 YoutubeDL: remove method that came from FileDownloader 2013-10-21 13:52:24 +02:00