Commit graph

15470 commits

Author SHA1 Message Date
Sergey M․ c7178f0f7a
[extractor/common] Output error for invalid URLs in _is_valid_url (refs #21400, refs #24151, refs #25617, refs #25618, refs #25586, refs #26068, refs #27072) 2020-11-18 23:31:35 +07:00
Remita Amine 5c3f7014ef [lbry] add support for odysee.com domain(closes #26806) 2020-11-18 00:41:07 +01:00
Remita Amine d65628ef03 [francetv] improve info extraction 2020-11-17 23:16:04 +01:00
gdzx 8a6c5b0806
[francetv] Add fallback video url extraction (#27047)
Fallback on another API endpoint when no video formats are found.

Closes ytdl-org#22561
2020-11-17 23:06:19 +01:00
Sergey M․ 9360936f26
release 2020.11.18 2020-11-18 04:15:30 +07:00
Sergey M․ 2d7a29081c
[spiegel] Fix extraction (closes #24206, closes #24767)
Code picked from PR #24767 since original repo is not available due to takedown.
2020-11-18 04:09:02 +07:00
Sergey M․ 2864179293
[youtube] Improve extraction
+ Add support for --no-playlist (closes #27009)
* Improve playlist and mix extraction (closes #26390, closes #26509, closes #26534, closes #27011)
+ Extract playlist uploader data
* Update tests
2020-11-18 03:34:08 +07:00
Remita Amine 284f8306df [youtube:tab] fix view_count extraction(closes #27051) 2020-11-17 20:32:50 +01:00
Remita Amine aa613ef7e1 [malltv] fix extraction(closes #27035) 2020-11-17 19:13:38 +01:00
Remita Amine cb2b9a22a5 [bandcamp] extract playlist_description(closes #22684) 2020-11-17 14:46:02 +01:00
Remita Amine 5b867c15a8 [urplay] fix extraction(closes #26828) 2020-11-17 13:12:20 +01:00
Sergey M․ f8c749f12c
[youtube:tab] Fix playlist title extraction (closes #27015) 2020-11-17 07:01:41 +07:00
Sergey M․ 7d509c613b
[youtube] Fix chapters extraction (closes #26005) 2020-11-17 04:28:35 +07:00
Sergey M․ b92e95aa01
release 2020.11.17 2020-11-17 03:59:54 +07:00
Sergey M․ 6699b6ce41
[youtube:tab] Fix extraction with cookies provided (closes #27005) 2020-11-17 03:40:57 +07:00
Remita Amine 91dcde8a38 [lrt] fix extraction with empty tags(closes #20264) 2020-11-16 21:28:12 +01:00
Sergey M․ 11f3471c4b
[ndr:embed:base] Extract subtitles (closes #25447, closes #26106) 2020-11-17 03:24:23 +07:00
Sergey M․ f4093b34f6
[servus] Add support for pm-wissen.com (closes #25869) 2020-11-17 03:02:07 +07:00
Sergey M․ a80b23c373
[servus] Fix extraction (closes #26872, closes #26967, closes #26983, closes #27000) 2020-11-17 02:57:26 +07:00
Sergey M․ 2e7fa18bb9
[xtube] Fix extraction (closes #26996) 2020-11-17 01:32:48 +07:00
Sergey M․ fe07e788bf
[utils] Skip ! prefixed code in js_to_json 2020-11-17 01:30:43 +07:00
Remita Amine 6d3bdcf217 [lrt] fix extraction 2020-11-16 19:17:10 +01:00
Remita Amine ec2a2ab441 [lbry] Add new extractor 2020-11-16 19:04:55 +01:00
Remita Amine 9448a20312 [condenast] fix extraction and extract subtitles 2020-11-16 18:57:56 +01:00
Remita Amine 3f1748b944 [bandcamp] fix extraction 2020-11-16 17:26:09 +01:00
Remita Amine fe13087cd1 [rai] fix RaiPlay extraction 2020-11-16 17:17:52 +01:00
Remita Amine f22fa82d7f [extractors] Remove VLivePlaylistIE import 2020-11-16 16:36:56 +01:00
Remita Amine 059fa9aa81 [vlive] fix extraction 2020-11-16 16:35:08 +01:00
Remita Amine 650aec4a98 [usanetwork] fix extraction 2020-11-16 16:21:48 +01:00
Remita Amine 2ea9c97432 [nbc] fix NBCNews/Today/MSNBC extraction 2020-11-16 16:18:37 +01:00
Remita Amine 1737ea69b9 [cnbc] fix extraction 2020-11-16 15:55:07 +01:00
Sergey M․ 1fb034d029
[youtube] Remove RIAA copyrighted media from tests as per [1]
1. Github dmca and 1201 notice re youtube-dl 9-21-20
2020-11-16 21:08:16 +07:00
Sergey M․ 28f9568a84
release 2020.11.12 2020-11-12 06:23:46 +07:00
Sergey M․ 2de2ca6659
[youtube] Rework extractors
WIP
2020-11-12 06:16:37 +07:00
Sergey M․ 051071203c
release 2020.11.01.1 2020-11-01 08:58:40 +07:00
Sergey M․ 34299510bb
release 2020.11.01 2020-11-01 08:52:27 +07:00
Sergey M․ 6d4733ce7b
[youtube] Fix JS player URL extraction 2020-11-01 06:52:00 +07:00
Sergey M․ 416da574ec
[ytsearch] Fix extraction (closes #26920) 2020-10-23 21:31:37 +07:00
Toan Nguyen 48c5663c5f
[afreecatv] Fix typo (#26970) 2020-10-22 19:15:05 +07:00
Hannu Hartikainen 7d740e7dc7
[23video] Relax _VALID_URL (#26870) 2020-10-20 00:56:23 +07:00
Kevin O'Connor 4eda10499e
[utils] Don't attempt to coerce JS strings to numbers in js_to_json (#26851)
The current logic in `js_to_json` tries to rewrite octal/hex numbers to
decimal. However, when the logic actually happens the `"` or `'` have
already been trimmed off. This causes what were originally strings, that
happen to look like octal/hex numbers, to get rewritten to decimal and
returned as a number rather than a string.

In practive something like:

```js
{
  "0x40": "foo",
  "040": "bar",
}
```

would get rewritten as:

```json
{
  64: "foo",
  32: "bar
}
```

This is problematic since this isn't valid JSON as you cannot have
non-string keys.
2020-10-18 00:10:41 +07:00
Sergio Livi 605535776a
[ustream] Add support for video.ibm.com (#26894) 2020-10-17 23:14:46 +07:00
Felix Yan 1050e0d09f
[iqiyi] Fix typo (#26884) 2020-10-17 23:02:17 +07:00
Sergey M․ d65d89183f
[expressen] Add support for di.se (closes #26670) 2020-09-24 07:37:10 +07:00
Surkal 0c92f1e96b
[iprima] Improve video id extraction (#26507) (closes #26494) 2020-09-24 06:46:58 +07:00
Sergey M․ c5764b3f89
[downloader/http] Properly handle missing message in SSLError (closes #26646) 2020-09-22 07:01:59 +07:00
Sergey M․ 0837992a22
[downloader/http] Fix access to not yet opened stream in retry 2020-09-22 06:44:14 +07:00
Sergey M․ b55715934b
release 2020.09.20 2020-09-20 12:30:45 +07:00
nixxo 1ca5f821c8
[redtube] Extend _VALID_URL (#26506) 2020-09-20 11:39:42 +07:00
Sergey M․ defc820b70
[twitch] Switch streams to GraphQL and refactor (closes #26535) 2020-09-20 10:05:00 +07:00
Sergey M․ 82ef02e936
[telequebec] Fix issues (closes #26368) 2020-09-19 07:56:00 +07:00
Patrick Dessalle b856b3997c
[telequebec] Add support for brightcove videos (closes #25833) 2020-09-19 07:52:57 +07:00
Sergey M․ cd85a1bb8b
[pornhub] Extract metadata from JSON-LD (closes #26614) 2020-09-19 06:34:34 +07:00
Sergey M․ ce5b904050
[extractor/common] Relax interaction count extraction in _json_ld 2020-09-19 06:33:17 +07:00
Sergey M․ ad06b99dd4
[extractor/common] Extract author as uploader for VideoObject in _json_ld 2020-09-19 06:13:42 +07:00
JChris246 540b9f5164
[pornhub] Fix view count extraction (#26621) (refs #26614) 2020-09-19 05:59:19 +07:00
Stefan Pöschel 6e65a2a67e
[downloader/hls] Fix incorrect end byte in Range HTTP header for media segments with EXT-X-BYTERANGE (#24512) (closes #14748)
The end of the byte range is the first byte that is NOT part of the to
be downloaded range. So don't include it into the requested HTTP
download range, as this additional byte leads to a broken TS packet and
subsequently to e.g. visible video corruption.

Fixes #14748.
2020-09-18 05:26:56 +07:00
Sergey M․ f8c7bed133
[extractor/common] Handle ssl.CertificateError in _request_webpage (closes #26601)
ssl.CertificateError is raised on some python versions <= 3.7.x
2020-09-18 03:41:16 +07:00
Sergey M․ cdc55e666f
[downloader/http] Improve timeout detection when reading block of data (refs #10935) 2020-09-18 03:32:54 +07:00
Ori Avtalion 86b7c00adc
[downloader/http] Retry download when urlopen times out (#26603) (refs #10935) 2020-09-18 03:15:44 +07:00
Sergey M․ e8c5d40bc8
release 2020.09.14 2020-09-14 03:37:36 +07:00
Sergey M․ bff857a8af
[postprocessor/embedthumbnail] Fix issues (closes #25717)
* Fix WebP with wrong extension processing
* Fix embedding of thumbnails with % character in path
2020-09-14 03:28:31 +07:00
Alex Merkel a31a022efd
[postprocessor/embedthumbnail] Add support for non jpeg/png thumbnails (closes #25687) 2020-09-14 03:10:01 +07:00
Sergey M․ 45f6362464
[rtlnl] Extend _VALID_URL for new embed URL schema 2020-09-13 21:42:06 +07:00
Derek Land 97f34a48d7
[rtlnl] Extend _VALID_URL (#26549) (closes #25821) 2020-09-13 21:38:16 +07:00
Daniel Peukert ea74e00b3a
[youtube] Fix empty description extraction (#26575) (closes #26006) 2020-09-13 21:23:21 +07:00
Sergey M․ 06cd4cdb25
[srgssr] Extend _VALID_URL (closes #26555, closes #26556, closes #26578) 2020-09-13 21:07:25 +07:00
Sergey M․ da2069fb22
[googledrive] Use redirect URLs for source format (closes #18877, closes #23919, closes #24689, closes #26565) 2020-09-13 20:49:32 +07:00
Sergey M․ 95c9810015
[svtplay] Fix id extraction (closes #26576) 2020-09-13 18:59:37 +07:00
Remita Amine b03eebdb6a [redbulltv] improve support for rebull.com TV localized URLS(#22063) 2020-09-13 11:26:11 +01:00
Remita Amine 1f7675451c [redbulltv] Add support for new redbull.com TV URLs(closes #22037)(closes #22063) 2020-09-12 19:27:58 +01:00
tfvlrue aa27253556
[soundcloud] Reduce pagination limit to fix 502 Bad Gateway errors when listing a user's tracks. (#26557)
Per the documentation here https://developers.soundcloud.com/blog/offset-pagination-deprecated the maximum limit is 200, so let's respect that (even if a higher value sometimes works).

Co-authored-by: tfvlrue <tfvlrue>
2020-09-12 09:35:11 +00:00
Sergey M․ d51e23d9fc
release 2020.09.06 2020-09-06 13:00:41 +07:00
Sergey M․ 50e9fcc1fd
[nrktv:episode] Improve video id extraction (closes #25594, closes #26369, closes #26409) 2020-09-06 12:43:50 +07:00
random-nick 16ee69c1b7
[youtube] Fix age gate content detection (#26100) (closes #26152, closes #26311, closes #26384) 2020-09-06 11:44:53 +07:00
Sergey M․ 67171ed7e9
[youtube:user] Extend _VALID_URL (closes #26443) 2020-09-06 11:31:28 +07:00
Sergey M․ 1d9bf655e6
[utils] Recognize wav mimetype (closes #26463) 2020-09-06 11:19:53 +07:00
TheRealDude2 62ae19ff76
[xhamster] Improve initials regex (#26526) (closes #26353) 2020-09-06 11:10:27 +07:00
Sergey M․ 5ed05f26ad
[svtplay] Fix svt id extraction (closes #26425, closes #26428, closes #26438) 2020-09-06 10:45:57 +07:00
Sergey M․ 841b683804
[twitch] Rework extractors (closes #12297, closes #20414, closes #20604, closes #21811, closes #21812, closes #22979, closes #24263, closes #25010, closes #25553, closes #25606)
* Switch to GraphQL.
+ Add support for collections.
+ Add support for clips and collections playlists.
2020-09-06 10:45:34 +07:00
Remita Amine f5863a3ea0 [biqle] improve video_ext extraction 2020-08-27 19:20:41 +01:00
Sergey M․ 10709fc7c6
[xhamster] Extend _VALID_URL (closes #25927) 2020-08-12 21:51:50 +07:00
TheRealDude2 a7e348556a
[xhamster] Fix extraction (closes #26157) (#26254) 2020-08-12 21:42:17 +07:00
JChris246 6cb30ea5ed
[xhamster] Extend _VALID_URL (closes #25789) (#25804) 2020-08-12 21:37:22 +07:00
Sergey M․ a4ed50bb84
release 2020.07.28 2020-07-28 05:13:03 +07:00
Sergey M․ e450f6cb63
[youtube] Fix sigfunc name extraction (closes #26134, closes #26135, closes #26136, closes #26137) 2020-07-28 05:05:38 +07:00
MRWITEK a115e07594
[youtube] Improve description extraction (closes #25937) (#25980) 2020-07-14 12:01:15 +01:00
Sergey M․ 718393c632
[wistia] Restrict embed regex (closes #25969) 2020-07-11 18:27:19 +07:00
Glenn Slayden 07af16b92e
[youtube] Prevent excess HTTP 301 (#25786) 2020-07-01 02:56:16 +07:00
Sergey M․ e942cfd1a7
[youtube:playlists] Extend _VALID_URL (closes #25810) 2020-06-28 10:30:03 +07:00
Remita Amine 9a7e5cb88a [bellmedia] add support for cp24.com clip URLs(closes #25764) 2020-06-23 15:09:13 +01:00
Sergey M․ 2391941f28
[brightcove] Improve embed detection (closes #25674) 2020-06-16 17:38:25 +07:00
Sergey M․ 9ff6165a81
release 2020.06.16.1 2020-06-16 06:22:01 +07:00
Sergey M․ ee0b726cd7
[youtube] Force old layout (closes #25682, closes #25683, closes #25680, closes #25686) 2020-06-16 06:17:53 +07:00
Sergey M․ dbeafce5d5
[youtube] Fix categories and improve tags extraction 2020-06-16 03:13:39 +07:00
Sergey M․ ed604ce7bc
release 2020.06.16 2020-06-16 02:53:33 +07:00
Sergey M․ a6211d237b
[youtube] Fix uploader id and uploader URL extraction 2020-06-16 02:43:09 +07:00
Sergey M․ 7b16239a49
[youtube] Improve view count extraction 2020-06-16 02:38:45 +07:00
Sergey M․ 37357d21a9
[youtube] Fix upload date extraction 2020-06-16 02:37:19 +07:00
Sergey M․ b477fc1314
[youtube] Fix thumbnails extraction and remove uploader id extraction warning (closes #25676) 2020-06-16 02:29:04 +07:00