gitlab-com-dstftw/youtube-dl
1
0
Fork 0
mirror of https://gitlab.com/dstftw/youtube-dl.git synced 2020-11-16 09:42:26 +00:00
Code Issues Releases Wiki Activity
16052 commits 3 branches 1078 tags 43 MiB
Commit graph

513 commits

Author SHA1 Message Date
Sergey M․ c89b49f743
[extractor/common] Add manifest_url for explicit group rendition formats 2017-04-28 03:00:14 +07:00
Sergey M․ ff99fe529e
Don't list master m3u8 playlists in format list (closes #12832) 2017-04-27 21:53:17 +07:00
Sergey M․ ac9c69ace7
[extractor/common] Improve jwplayer regex 2017-04-25 23:46:05 +07:00
Tithen-Firion 40e41780f1 [phantomjs] add cookie support 2017-04-25 15:12:54 +02:00
Sergey M․ 3019cb0c99
[extractor/common] Rephrase comment 2017-04-23 11:52:07 +07:00
Sergey M․ ddd258f922
[test_InfoExtractor] Add m3u8 parsing test for NAME attribute in EXT-X-STREAM-INF tag 2017-04-23 11:49:57 +07:00
Sergey M․ 9c99bef704
[extractor/common] Use float for scaled tbr 2017-04-23 11:33:49 +07:00
Sergey M․ cb2520802d
[extractor/common] Improve m3u8 extraction (closes #12211) 
* Extract m3u8 parsing to separate method
* Improve rendition groups extraction
* Build stream name according stream GROUP-ID
* Ignore reference to AUDIO group without URI when stream has no CODECS
+ Add test coverage for parsing m3u8 from #11507, #11995, #12211 and twitch vod
 2017-04-22 07:01:00 +07:00
Sergey M․ bae1404893
[extractor/common] Add support for video of WebPage context in _json_ld (closes #12778) 2017-04-18 22:21:38 +07:00
Remita Amine bf1b87cd91 [common] Relax JWPlayer regex and remove duplicate urls(#12768) 2017-04-17 08:48:24 +01:00
Remita Amine 40fcba5edb improve coding style 2017-04-12 20:38:43 +01:00
Sergey M․ fd47550885
[extractor/common] Add coding cookie 2017-04-02 04:42:10 +07:00
Sergey M․ 4457823dda
[extractor/common] Move censorship checks to a separate method and add check for just another ISP 2017-04-02 03:57:44 +07:00
Random User 4f06c1c9fc Merge branch 'master' of github.com-rndusr:rg3/youtube-dl into fix/str-item-assignment 2017-03-25 21:36:59 +01:00
Random User c73e330e7a _find_jwplayer_data() returns dict or None 
This simplifies code for callers of `_find_jwplayer_data()` which no longer have
to run `_parse_json()` on the return value.

It also makes sure that `_find_jwplayer_data()` returns either a `dict` or
`None` and nothing else.
 2017-03-25 19:38:30 +01:00
John Hawkinson 46b18f2349 [BostonGlobe] New. Nonstandard version of Brightcove. 
Has a "data-brightcove-video-id" instead of a "data-video-id," otherwise
pretty much just Brightcove. Except the Globe isn't all Brightcove
videos, so fallback to Generic, too.

Also, abstract playlist_from_matches() from generic.py to common.py, and use
it here.

History of these changes can be found in
51170427d4b1143572a498dedaee61863a5b2c5b.
 2017-03-19 20:40:31 +08:00
Sergey M․ b51dc9db0e
[extractor/common] Extract SMIL formats from jwplayer 2017-03-16 03:30:53 +07:00
Sergey M․ 1a2192cb90
[extractor/common] Pass arguments to _parse_jwplayer_formats and PEP8 2017-03-05 23:29:17 +07:00
Sergey M․ 0236cd0dfd
[extractor/common] Improve height extraction and extract bitrate 2017-03-05 23:25:03 +07:00
Sergey M․ ed0cf9b383
[extractor/common] Move jwplayer formats extraction in separate method 2017-03-05 23:22:27 +07:00
Yen Chi Hsuan eeb0a95684
[extractor/common] Add 'preference' to _parse_html5_media_entries 
Some websites, like NJPWorld, put different qualities on different
player pages.
 2017-02-25 18:40:05 +08:00
Sergey M․ eea0716cae
[extractor/common] Print origin country for fake IP 2017-02-21 23:14:33 +07:00
Sergey M․ 336a76551b
[extractor/common] Do not quit _initialize_geo_bypass on empty countries 2017-02-21 23:09:41 +07:00
Sergey M․ dc0a869e5e
[extractor/common] Fix typo 2017-02-21 23:05:31 +07:00
Sergey M․ e39b5d4ab8
[extractor/common] Allow calling _initialize_geo_bypass from extractors (#11970) 2017-02-21 23:00:43 +07:00
Sergey M․ 3ccdde8cb7
[extractor/common] Emphasize geo bypass APIs are experimental 2017-02-20 23:21:15 +07:00
Sergey M․ 4248dad92b Improve geo bypass mechanism 
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
 2017-02-19 05:10:08 +08:00
Sergey M․ 0a840f584c Rename bypass geo restriction options 2017-02-19 05:10:08 +08:00
Sergey M․ 0016b84e16 Add faked X-Forwarded-For to formats' HTTP headers 2017-02-19 05:10:08 +08:00
Sergey M․ 773f291dcb Add experimental geo restriction bypass mechanism 
Based on faking X-Forwarded-For HTTP header
 2017-02-19 05:10:08 +08:00
Yen Chi Hsuan a4a554a793
[generic] Try parsing JWPlayer embedded videos (closes #12030) 2017-02-16 23:44:03 +08:00
Remita Amine 08a00eef79 [extractor/common] skip m3u8 manifests protected with Adobe Flash Access 2017-02-10 17:00:09 +01:00
Sergey M․ 242a14a1f6
[extractor/common] Fix audio only with audio group in m3u8 (closes #11995) 2017-02-07 00:22:16 +07:00
Sergey M․ 2aec7256ae
[extractor/common] Speed-up media tags regex (closes #11979) 2017-02-06 00:20:30 +07:00
Mattias Wadman 45024183ae [infoq] Add audio only format if available (#11565) 
* [infoq] Add audio only format if available

Refactor cookie code into a function.
Renamed formats to http_video, http_audio, rtmp_video
Renamed extract functions to video instead of videos as they return
one or no video.

* [infoq] Rename to _extract_cookies as it more than one

* [infoq] Remove redundant determine_ext

* [infoq] Add comment about hardcoded URL

* [infoq] Use _hidden_inputs instead of messy regex

* [infoq] Probe if audio URL is valid

Make it possible to pass headers to _is_valid_url

* [infoq] Add audio only test
 2017-02-03 12:10:13 +08:00
Sergey M․ e228616c6e
[extractor/common] Fix initialization template (closes #11605, closes #11825) 2017-01-29 06:57:39 +07:00
Sergey M․ c58c2d63cb
[extractor/common] Document forgotten fragment base and path interfaces 2017-01-29 05:57:53 +07:00
Sergey M․ d04621daf4
[extractor/common] Fix duration per dash segment (closes #11868) 2017-01-29 05:36:53 +07:00
Remita Amine c4251b9aaa [common] add possibility to customize akamai manifest host 2017-01-13 10:21:36 +01:00
Yen Chi Hsuan 4606c34e19 [extractor/common] Allow non-lang in subtitles' keys 
See 264e77c406
 2016-12-25 01:50:50 +08:00
Remita Amine a07588369f [common] improve detection for video only formats and m3u8 manifest(fixes #11507) 2016-12-22 10:02:56 +01:00
Remita Amine 7fe1592073 [common] fix dash codec information for mixed videos and fragment url construction(#11490) 2016-12-20 12:35:03 +01:00
Sergey M․ 87a449c1ed
[extractor/common] Recognize DASH formats in html5 media entries 2016-12-17 23:03:13 +07:00
Remita Amine 8821a718cf [common] recognize hls manifests that contain video only formats(#11394) 2016-12-10 17:22:15 +01:00
Sergey M․ 4ea4c0bb22
[extractor/common] Fix Bandwidth substitution in media template (closes #11175) 2016-11-13 05:43:34 +07:00
Sergey M․ f076d7972c
[extractor/common] Improve thumbnail extraction from JSON-LD 2016-11-12 23:01:05 +07:00
Sergey M․ c4c9b8440c
[extractor/common] Tolerate malformed RESOLUTION attribute in m3u8 manifests (closes #11113) 2016-11-04 05:02:31 +07:00
Sergey M․ 3365ea8929
[extractor/common] Remove unused code 2016-11-02 02:34:23 +07:00
Sergey M․ 1616f9b452
[extractor/common] Fix typo 2016-11-02 02:30:25 +07:00
Sergey M․ 02dc0a36b7
[utils] Introduce base_url 2016-11-02 02:30:18 +07:00
Remita Amine b2758123c5 add Basic support for Smooth Streaming protocol(#8118) 2016-11-02 01:54:45 +07:00
Sergey M․ b82c33dd67
[extractor/common] Improve mpd base URL extraction (closes #10909, closes #11079) 2016-11-01 01:15:46 +07:00
Remita Amine 0384932e3d [extractor/common] try to extract non smil wowza mpd manifests 2016-10-19 14:57:12 +01:00
Remita Amine edd6074cea [extractor/common] detect f4m audio only formats 2016-10-19 14:42:48 +01:00
Sergey M․ 8d76bdf12b
[extractor/common] Mention podcast in series fields section 2016-10-16 18:37:17 +07:00
Yen Chi Hsuan cea364f70c
[extractor/common] Support HTML media elements without child nodes 2016-10-12 01:40:28 +08:00
Yen Chi Hsuan 98763ee354
[extractor/common] Add id and title helpers for generic IEs 2016-10-07 19:20:53 +08:00
Remita Amine e71a450956 [common] add hdcore sign to akamai f4m formats 2016-09-24 21:55:53 +01:00
Yen Chi Hsuan 5968d7d2fe
[extractor/common] Improved support for HTML5 subtitles 
Ref: #10625

In a strict sense, <track>s with kind=captions are not subtitles. [1]
openload misuses this attribute, and I guess there will be more
examples, so I add it to common.py.

Also allow extracting information for subtitles-only <video> or <audio>
tags, which is the case of openload.

[1] https://www.w3.org/TR/html5/embedded-content-0.html#attr-track-kind
 2016-09-24 14:20:42 +08:00
Sergey M․ 30d0b549be
[extractor/common] Add manifest_url for hls and hds formats 2016-09-17 21:33:38 +07:00
Sergey M․ 86f4d14f81
Refactor fragments interface and dash segments downloader 
- Eliminate segment_urls and initialization_url
+ Introduce manifest_url (manifest may contain unfragmented data in this case url will be used for direct media URL and manifest_url for manifest itself correspondingly)
* Rewrite dashsegments downloader to use fragments data
* Improve generic mpd extraction
 2016-09-17 20:35:22 +07:00
Sergey M․ b4c1d6e800
[extractor/common] Expose fragments interface for dashsegments formats 2016-09-17 18:31:18 +07:00
Sergey M․ a0d5077c8d
[extractor/common] Introduce fragments interface 2016-09-17 18:31:09 +07:00
Remita Amine 6ad0219556 [common] add helper method for Wowza Streaming Engine format extraction 2016-09-16 19:30:38 +01:00
Sergey M․ dcce092e0a
[extractor/common] Simplify _get_netrc_login_info and carry long lines 2016-09-15 22:35:12 +07:00
Sergey M․ 32443dd346
[extractor/common] Update _get_login_info's comment 2016-09-15 22:34:29 +07:00
Sergey M․ 2133565cec
[extractor/common] Simplify _get_login_info 2016-09-15 22:26:37 +07:00
Sergey M․ c849836854
[utils] Improve _hidden_inputs 2016-09-15 21:54:48 +07:00
Remita Amine 1b6712ab23 [adobepass] add specific options for adobe pass authentication 
- add --ap-username and --ap-password option to specify
TV provider username and password in the cmd line
- add --ap-retries option to limit the number of retries
- add --list-ap-msi-ids to list the supported TV Providers
 2016-09-13 22:16:01 +01:00
Remita Amine 0def758782 [internetvideoarchive] extract all formats 2016-09-04 11:45:29 +01:00
Christian Pointner dedb177029
Fix parsing of HTML5 media elements 
This fixes an error in _parse_html5_media_entries in case
an audio or video tag directly uses a src attribute insted
of <source> elements in it's body.
 2016-09-03 16:09:35 +07:00
Sergey M․ e816c9d158
[extractor/common] Simplify _extract_m3u8_formats 2016-09-01 22:18:16 +07:00
Sergey M․ 9250181f37
[extractor/common] Restore NAME usage from EXT-X-MEDIA tag for formats codes in _extract_m3u8_formats (Closes #10522) 2016-09-01 21:37:25 +07:00
Remita Amine f39ffc5877 [common] extract formats from #EXT-X-MEDIA tags 2016-08-25 08:49:44 +01:00
Remita Amine c7c43a93ba [common] add helper method to extract akamai m3u8 and f4m formats 2016-08-22 07:49:34 +01:00
Yen Chi Hsuan ad120ae1c5
[extractor/common] Change the default m3u8 protocol in HTML5 
Helper functions should have consistent default values
 2016-08-22 02:26:07 +08:00
Yen Chi Hsuan 520251c093
[extractor/common] Recognize m3u8 manifests in HTML5 multimedia tags 2016-08-19 23:53:47 +08:00
Remita Amine 2118fdd1a9 [common] add separate method for getting netrc ligin info 2016-08-14 11:55:28 +01:00
Sergey M․ 321b5e082a
[extractor/common] Respect default in _search_json_ld 2016-08-08 22:36:18 +07:00
Remita Amine 37768f9242 [common] correctly lower the preference of m3u8 master manifest format 2016-08-07 10:59:09 +01:00
Remita Amine d16b3c6677 [common] extract partOfTVSeries info in json-ld 2016-08-06 18:58:38 +01:00
Remita Amine c57244cdb1 [common] lower the preference of m3u8 master manifest format 2016-08-06 18:55:05 +01:00
Sergey M․ 46933a15d6
[extractor/common] Support root JSON-LD lists (Closes #10203) 2016-08-05 23:14:32 +07:00
Remita Amine 8b40854529 [common] lower proto_preference of rtsp formats 
Most of the time the RtspFD fail to download videos but it report
success of the download with this output:
[mpv] 0 bytes
[download] 100% of 0.00B
 2016-08-04 09:39:06 +01:00
Sergey M․ b070564efb
[extractor/common] Support multiple properties in _og_search_property 2016-08-02 22:55:14 +07:00
Sergey M․ 40090e8d51
[extractor/common] Improve is_suitable 
In order to fix breakage introduced by a3aa814b77
 2016-07-26 23:54:06 +07:00
Sergey M․ f094834857
[extractor/common] Add support for $ in SegmentTemplate in MPD manifests 2016-07-24 10:27:16 +07:00
Remita Amine 00f4764cb7 [common] extract vbr, abr and fps for Unified Streaming Platform m3u8 manifests 2016-07-13 15:58:24 +01:00
Yen Chi Hsuan 0b68de3cc1 Merge pull request #8876 from remitamine/html5_media 
[extractor/common] add helper method to extract html5 media entries
 2016-07-10 23:40:45 +08:00
Sergey M․ 95b31e266b
[extractor/common] Add expected_type in json ld routines 2016-07-09 03:28:04 +07:00
Sergey M․ 6b3a3098b5
[extractor/common] Extract more metadata for VideoObject in _json_ld 2016-07-09 03:27:11 +07:00
Sergey M․ 2de624fdd5
[extractor/common] Introduce filesize metafield for thumbnails 2016-07-09 03:24:36 +07:00
Déstin Reed 3fee7f636c
[flipagram] Add extractor 2016-07-09 03:23:32 +07:00
Yen Chi Hsuan 38cce791c7
Rename --cn-verfication-proxy to --geo-verification-proxy 
And deprecate the former one

Since commit f138873900, this option is
not limited to China websites, so rename it.
 2016-07-03 23:29:56 +08:00
Sergey M․ bf8dd79045
[extractor/common] Fix sorting with custom field preference 2016-06-26 21:09:07 +07:00
remitamine 59bbe4911a [extractor/common] add helper method to extract html5 media entries 2016-06-26 14:04:08 +01:00
Sergey M․ 88d9f6c0c4
[utils] Add support for name list in _html_search_meta 2016-06-26 16:57:14 +07:00
remitamine e154c65128 [downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader 2016-06-19 01:01:40 +01:00
Yen Chi Hsuan a6571f1073
[common] Fix <bootstrapInfo> detection in F4M manifests 
Regression since 0a5685b26f
 2016-06-08 00:19:33 +08:00
Sergey M․ 77b8b4e696
[extractor/common] Borrow quality metadata from parent set-level manifest for f4m 2016-05-27 01:47:44 +06:00