You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1992 lines
80 KiB

10 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
11 years ago
11 years ago
11 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
10 years ago
10 years ago
  1. # encoding: utf-8
  2. from __future__ import unicode_literals
  3. import os
  4. import re
  5. import sys
  6. from .common import InfoExtractor
  7. from .youtube import YoutubeIE
  8. from ..compat import (
  9. compat_etree_fromstring,
  10. compat_urllib_parse_unquote,
  11. compat_urlparse,
  12. compat_xml_parse_error,
  13. )
  14. from ..utils import (
  15. determine_ext,
  16. ExtractorError,
  17. float_or_none,
  18. HEADRequest,
  19. is_html,
  20. orderedSet,
  21. sanitized_Request,
  22. smuggle_url,
  23. unescapeHTML,
  24. unified_strdate,
  25. unsmuggle_url,
  26. UnsupportedError,
  27. url_basename,
  28. xpath_text,
  29. )
  30. from .brightcove import (
  31. BrightcoveLegacyIE,
  32. BrightcoveNewIE,
  33. )
  34. from .nbc import NBCSportsVPlayerIE
  35. from .ooyala import OoyalaIE
  36. from .rutv import RUTVIE
  37. from .tvc import TVCIE
  38. from .sportbox import SportBoxEmbedIE
  39. from .smotri import SmotriIE
  40. from .myvi import MyviIE
  41. from .condenast import CondeNastIE
  42. from .udn import UDNEmbedIE
  43. from .senateisvp import SenateISVPIE
  44. from .svt import SVTIE
  45. from .pornhub import PornHubIE
  46. from .xhamster import XHamsterEmbedIE
  47. from .vimeo import VimeoIE
  48. from .dailymotion import DailymotionCloudIE
  49. from .onionstudios import OnionStudiosIE
  50. from .snagfilms import SnagFilmsEmbedIE
  51. from .screenwavemedia import ScreenwaveMediaIE
  52. from .mtv import MTVServicesEmbeddedIE
  53. from .pladform import PladformIE
  54. from .videomore import VideomoreIE
  55. from .googledrive import GoogleDriveIE
  56. from .jwplatform import JWPlatformIE
  57. from .digiteka import DigitekaIE
  58. class GenericIE(InfoExtractor):
  59. IE_DESC = 'Generic downloader that works on some sites'
  60. _VALID_URL = r'.*'
  61. IE_NAME = 'generic'
  62. _TESTS = [
  63. # Direct link to a video
  64. {
  65. 'url': 'http://media.w3.org/2010/05/sintel/trailer.mp4',
  66. 'md5': '67d406c2bcb6af27fa886f31aa934bbe',
  67. 'info_dict': {
  68. 'id': 'trailer',
  69. 'ext': 'mp4',
  70. 'title': 'trailer',
  71. 'upload_date': '20100513',
  72. }
  73. },
  74. # Direct link to media delivered compressed (until Accept-Encoding is *)
  75. {
  76. 'url': 'http://calimero.tk/muzik/FictionJunction-Parallel_Hearts.flac',
  77. 'md5': '128c42e68b13950268b648275386fc74',
  78. 'info_dict': {
  79. 'id': 'FictionJunction-Parallel_Hearts',
  80. 'ext': 'flac',
  81. 'title': 'FictionJunction-Parallel_Hearts',
  82. 'upload_date': '20140522',
  83. },
  84. 'expected_warnings': [
  85. 'URL could be a direct video link, returning it as such.'
  86. ]
  87. },
  88. # Direct download with broken HEAD
  89. {
  90. 'url': 'http://ai-radio.org:8000/radio.opus',
  91. 'info_dict': {
  92. 'id': 'radio',
  93. 'ext': 'opus',
  94. 'title': 'radio',
  95. },
  96. 'params': {
  97. 'skip_download': True, # infinite live stream
  98. },
  99. 'expected_warnings': [
  100. r'501.*Not Implemented'
  101. ],
  102. },
  103. # Direct link with incorrect MIME type
  104. {
  105. 'url': 'http://ftp.nluug.nl/video/nluug/2014-11-20_nj14/zaal-2/5_Lennart_Poettering_-_Systemd.webm',
  106. 'md5': '4ccbebe5f36706d85221f204d7eb5913',
  107. 'info_dict': {
  108. 'url': 'http://ftp.nluug.nl/video/nluug/2014-11-20_nj14/zaal-2/5_Lennart_Poettering_-_Systemd.webm',
  109. 'id': '5_Lennart_Poettering_-_Systemd',
  110. 'ext': 'webm',
  111. 'title': '5_Lennart_Poettering_-_Systemd',
  112. 'upload_date': '20141120',
  113. },
  114. 'expected_warnings': [
  115. 'URL could be a direct video link, returning it as such.'
  116. ]
  117. },
  118. # RSS feed
  119. {
  120. 'url': 'http://phihag.de/2014/youtube-dl/rss2.xml',
  121. 'info_dict': {
  122. 'id': 'http://phihag.de/2014/youtube-dl/rss2.xml',
  123. 'title': 'Zero Punctuation',
  124. 'description': 're:.*groundbreaking video review series.*'
  125. },
  126. 'playlist_mincount': 11,
  127. },
  128. # RSS feed with enclosure
  129. {
  130. 'url': 'http://podcastfeeds.nbcnews.com/audio/podcast/MSNBC-MADDOW-NETCAST-M4V.xml',
  131. 'info_dict': {
  132. 'id': 'pdv_maddow_netcast_m4v-02-27-2015-201624',
  133. 'ext': 'm4v',
  134. 'upload_date': '20150228',
  135. 'title': 'pdv_maddow_netcast_m4v-02-27-2015-201624',
  136. }
  137. },
  138. # SMIL from http://videolectures.net/promogram_igor_mekjavic_eng
  139. {
  140. 'url': 'http://videolectures.net/promogram_igor_mekjavic_eng/video/1/smil.xml',
  141. 'info_dict': {
  142. 'id': 'smil',
  143. 'ext': 'mp4',
  144. 'title': 'Automatics, robotics and biocybernetics',
  145. 'description': 'md5:815fc1deb6b3a2bff99de2d5325be482',
  146. 'upload_date': '20130627',
  147. 'formats': 'mincount:16',
  148. 'subtitles': 'mincount:1',
  149. },
  150. 'params': {
  151. 'force_generic_extractor': True,
  152. 'skip_download': True,
  153. },
  154. },
  155. # SMIL from http://www1.wdr.de/mediathek/video/livestream/index.html
  156. {
  157. 'url': 'http://metafilegenerator.de/WDR/WDR_FS/hds/hds.smil',
  158. 'info_dict': {
  159. 'id': 'hds',
  160. 'ext': 'flv',
  161. 'title': 'hds',
  162. 'formats': 'mincount:1',
  163. },
  164. 'params': {
  165. 'skip_download': True,
  166. },
  167. },
  168. # SMIL from https://www.restudy.dk/video/play/id/1637
  169. {
  170. 'url': 'https://www.restudy.dk/awsmedia/SmilDirectory/video_1637.xml',
  171. 'info_dict': {
  172. 'id': 'video_1637',
  173. 'ext': 'flv',
  174. 'title': 'video_1637',
  175. 'formats': 'mincount:3',
  176. },
  177. 'params': {
  178. 'skip_download': True,
  179. },
  180. },
  181. # SMIL from http://adventure.howstuffworks.com/5266-cool-jobs-iditarod-musher-video.htm
  182. {
  183. 'url': 'http://services.media.howstuffworks.com/videos/450221/smil-service.smil',
  184. 'info_dict': {
  185. 'id': 'smil-service',
  186. 'ext': 'flv',
  187. 'title': 'smil-service',
  188. 'formats': 'mincount:1',
  189. },
  190. 'params': {
  191. 'skip_download': True,
  192. },
  193. },
  194. # SMIL from http://new.livestream.com/CoheedandCambria/WebsterHall/videos/4719370
  195. {
  196. 'url': 'http://api.new.livestream.com/accounts/1570303/events/1585861/videos/4719370.smil',
  197. 'info_dict': {
  198. 'id': '4719370',
  199. 'ext': 'mp4',
  200. 'title': '571de1fd-47bc-48db-abf9-238872a58d1f',
  201. 'formats': 'mincount:3',
  202. },
  203. 'params': {
  204. 'skip_download': True,
  205. },
  206. },
  207. # XSPF playlist from http://www.telegraaf.nl/tv/nieuws/binnenland/24353229/__Tikibad_ontruimd_wegens_brand__.html
  208. {
  209. 'url': 'http://www.telegraaf.nl/xml/playlist/2015/8/7/mZlp2ctYIUEB.xspf',
  210. 'info_dict': {
  211. 'id': 'mZlp2ctYIUEB',
  212. 'ext': 'mp4',
  213. 'title': 'Tikibad ontruimd wegens brand',
  214. 'description': 'md5:05ca046ff47b931f9b04855015e163a4',
  215. 'thumbnail': 're:^https?://.*\.jpg$',
  216. 'duration': 33,
  217. },
  218. 'params': {
  219. 'skip_download': True,
  220. },
  221. },
  222. # MPD from http://dash-mse-test.appspot.com/media.html
  223. {
  224. 'url': 'http://yt-dash-mse-test.commondatastorage.googleapis.com/media/car-20120827-manifest.mpd',
  225. 'md5': '4b57baab2e30d6eb3a6a09f0ba57ef53',
  226. 'info_dict': {
  227. 'id': 'car-20120827-manifest',
  228. 'ext': 'mp4',
  229. 'title': 'car-20120827-manifest',
  230. 'formats': 'mincount:9',
  231. },
  232. 'params': {
  233. 'format': 'bestvideo',
  234. },
  235. },
  236. # google redirect
  237. {
  238. 'url': 'http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CCUQtwIwAA&url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DcmQHVoWB5FY&ei=F-sNU-LLCaXk4QT52ICQBQ&usg=AFQjCNEw4hL29zgOohLXvpJ-Bdh2bils1Q&bvm=bv.61965928,d.bGE',
  239. 'info_dict': {
  240. 'id': 'cmQHVoWB5FY',
  241. 'ext': 'mp4',
  242. 'upload_date': '20130224',
  243. 'uploader_id': 'TheVerge',
  244. 'description': 're:^Chris Ziegler takes a look at the\.*',
  245. 'uploader': 'The Verge',
  246. 'title': 'First Firefox OS phones side-by-side',
  247. },
  248. 'params': {
  249. 'skip_download': False,
  250. }
  251. },
  252. {
  253. # redirect in Refresh HTTP header
  254. 'url': 'https://www.facebook.com/l.php?u=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DpO8h3EaFRdo&h=TAQHsoToz&enc=AZN16h-b6o4Zq9pZkCCdOLNKMN96BbGMNtcFwHSaazus4JHT_MFYkAA-WARTX2kvsCIdlAIyHZjl6d33ILIJU7Jzwk_K3mcenAXoAzBNoZDI_Q7EXGDJnIhrGkLXo_LJ_pAa2Jzbx17UHMd3jAs--6j2zaeto5w9RTn8T_1kKg3fdC5WPX9Dbb18vzH7YFX0eSJmoa6SP114rvlkw6pkS1-T&s=1',
  255. 'info_dict': {
  256. 'id': 'pO8h3EaFRdo',
  257. 'ext': 'mp4',
  258. 'title': 'Tripeo Boiler Room x Dekmantel Festival DJ Set',
  259. 'description': 'md5:6294cc1af09c4049e0652b51a2df10d5',
  260. 'upload_date': '20150917',
  261. 'uploader_id': 'brtvofficial',
  262. 'uploader': 'Boiler Room',
  263. },
  264. 'params': {
  265. 'skip_download': False,
  266. },
  267. },
  268. {
  269. 'url': 'http://www.hodiho.fr/2013/02/regis-plante-sa-jeep.html',
  270. 'md5': '85b90ccc9d73b4acd9138d3af4c27f89',
  271. 'info_dict': {
  272. 'id': '13601338388002',
  273. 'ext': 'mp4',
  274. 'uploader': 'www.hodiho.fr',
  275. 'title': 'R\u00e9gis plante sa Jeep',
  276. }
  277. },
  278. # bandcamp page with custom domain
  279. {
  280. 'add_ie': ['Bandcamp'],
  281. 'url': 'http://bronyrock.com/track/the-pony-mash',
  282. 'info_dict': {
  283. 'id': '3235767654',
  284. 'ext': 'mp3',
  285. 'title': 'The Pony Mash',
  286. 'uploader': 'M_Pallante',
  287. },
  288. 'skip': 'There is a limit of 200 free downloads / month for the test song',
  289. },
  290. # embedded brightcove video
  291. # it also tests brightcove videos that need to set the 'Referer' in the
  292. # http requests
  293. {
  294. 'add_ie': ['BrightcoveLegacy'],
  295. 'url': 'http://www.bfmtv.com/video/bfmbusiness/cours-bourse/cours-bourse-l-analyse-technique-154522/',
  296. 'info_dict': {
  297. 'id': '2765128793001',
  298. 'ext': 'mp4',
  299. 'title': 'Le cours de bourse : l’analyse technique',
  300. 'description': 'md5:7e9ad046e968cb2d1114004aba466fd9',
  301. 'uploader': 'BFM BUSINESS',
  302. },
  303. 'params': {
  304. 'skip_download': True,
  305. },
  306. },
  307. {
  308. # https://github.com/rg3/youtube-dl/issues/2253
  309. 'url': 'http://bcove.me/i6nfkrc3',
  310. 'md5': '0ba9446db037002366bab3b3eb30c88c',
  311. 'info_dict': {
  312. 'id': '3101154703001',
  313. 'ext': 'mp4',
  314. 'title': 'Still no power',
  315. 'uploader': 'thestar.com',
  316. 'description': 'Mississauga resident David Farmer is still out of power as a result of the ice storm a month ago. To keep the house warm, Farmer cuts wood from his property for a wood burning stove downstairs.',
  317. },
  318. 'add_ie': ['BrightcoveLegacy'],
  319. },
  320. {
  321. 'url': 'http://www.championat.com/video/football/v/87/87499.html',
  322. 'md5': 'fb973ecf6e4a78a67453647444222983',
  323. 'info_dict': {
  324. 'id': '3414141473001',
  325. 'ext': 'mp4',
  326. 'title': 'Видео. Удаление Дзагоева (ЦСКА)',
  327. 'description': 'Онлайн-трансляция матча ЦСКА - "Волга"',
  328. 'uploader': 'Championat',
  329. },
  330. },
  331. {
  332. # https://github.com/rg3/youtube-dl/issues/3541
  333. 'add_ie': ['BrightcoveLegacy'],
  334. 'url': 'http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1',
  335. 'info_dict': {
  336. 'id': '3866516442001',
  337. 'ext': 'mp4',
  338. 'title': 'Leer mij vrouwen kennen: Aflevering 1',
  339. 'description': 'Leer mij vrouwen kennen: Aflevering 1',
  340. 'uploader': 'SBS Broadcasting',
  341. },
  342. 'skip': 'Restricted to Netherlands',
  343. 'params': {
  344. 'skip_download': True, # m3u8 download
  345. },
  346. },
  347. # ooyala video
  348. {
  349. 'url': 'http://www.rollingstone.com/music/videos/norwegian-dj-cashmere-cat-goes-spartan-on-with-me-premiere-20131219',
  350. 'md5': '166dd577b433b4d4ebfee10b0824d8ff',
  351. 'info_dict': {
  352. 'id': 'BwY2RxaTrTkslxOfcan0UCf0YqyvWysJ',
  353. 'ext': 'mp4',
  354. 'title': '2cc213299525360.mov', # that's what we get
  355. 'duration': 238.231,
  356. },
  357. 'add_ie': ['Ooyala'],
  358. },
  359. {
  360. # ooyala video embedded with http://player.ooyala.com/iframe.js
  361. 'url': 'http://www.macrumors.com/2015/07/24/steve-jobs-the-man-in-the-machine-first-trailer/',
  362. 'info_dict': {
  363. 'id': 'p0MGJndjoG5SOKqO_hZJuZFPB-Tr5VgB',
  364. 'ext': 'mp4',
  365. 'title': '"Steve Jobs: Man in the Machine" trailer',
  366. 'description': 'The first trailer for the Alex Gibney documentary "Steve Jobs: Man in the Machine."',
  367. 'duration': 135.427,
  368. },
  369. 'params': {
  370. 'skip_download': True,
  371. },
  372. },
  373. # multiple ooyala embeds on SBN network websites
  374. {
  375. 'url': 'http://www.sbnation.com/college-football-recruiting/2015/2/3/7970291/national-signing-day-rationalizations-itll-be-ok-itll-be-ok',
  376. 'info_dict': {
  377. 'id': 'national-signing-day-rationalizations-itll-be-ok-itll-be-ok',
  378. 'title': '25 lies you will tell yourself on National Signing Day - SBNation.com',
  379. },
  380. 'playlist_mincount': 3,
  381. 'params': {
  382. 'skip_download': True,
  383. },
  384. 'add_ie': ['Ooyala'],
  385. },
  386. # embed.ly video
  387. {
  388. 'url': 'http://www.tested.com/science/weird/460206-tested-grinding-coffee-2000-frames-second/',
  389. 'info_dict': {
  390. 'id': '9ODmcdjQcHQ',
  391. 'ext': 'mp4',
  392. 'title': 'Tested: Grinding Coffee at 2000 Frames Per Second',
  393. 'upload_date': '20140225',
  394. 'description': 'md5:06a40fbf30b220468f1e0957c0f558ff',
  395. 'uploader': 'Tested',
  396. 'uploader_id': 'testedcom',
  397. },
  398. # No need to test YoutubeIE here
  399. 'params': {
  400. 'skip_download': True,
  401. },
  402. },
  403. # funnyordie embed
  404. {
  405. 'url': 'http://www.theguardian.com/world/2014/mar/11/obama-zach-galifianakis-between-two-ferns',
  406. 'info_dict': {
  407. 'id': '18e820ec3f',
  408. 'ext': 'mp4',
  409. 'title': 'Between Two Ferns with Zach Galifianakis: President Barack Obama',
  410. 'description': 'Episode 18: President Barack Obama sits down with Zach Galifianakis for his most memorable interview yet.',
  411. },
  412. },
  413. # RUTV embed
  414. {
  415. 'url': 'http://www.rg.ru/2014/03/15/reg-dfo/anklav-anons.html',
  416. 'info_dict': {
  417. 'id': '776940',
  418. 'ext': 'mp4',
  419. 'title': 'Охотское море стало целиком российским',
  420. 'description': 'md5:5ed62483b14663e2a95ebbe115eb8f43',
  421. },
  422. 'params': {
  423. # m3u8 download
  424. 'skip_download': True,
  425. },
  426. },
  427. # TVC embed
  428. {
  429. 'url': 'http://sch1298sz.mskobr.ru/dou_edu/karamel_ki/filial_galleries/video/iframe_src_http_tvc_ru_video_iframe_id_55304_isplay_false_acc_video_id_channel_brand_id_11_show_episodes_episode_id_32307_frameb/',
  430. 'info_dict': {
  431. 'id': '55304',
  432. 'ext': 'mp4',
  433. 'title': 'Дошкольное воспитание',
  434. },
  435. },
  436. # SportBox embed
  437. {
  438. 'url': 'http://www.vestifinance.ru/articles/25753',
  439. 'info_dict': {
  440. 'id': '25753',
  441. 'title': 'Вести Экономика ― Прямые трансляции с Форума-выставки "Госзаказ-2013"',
  442. },
  443. 'playlist': [{
  444. 'info_dict': {
  445. 'id': '370908',
  446. 'title': 'Госзаказ. День 3',
  447. 'ext': 'mp4',
  448. }
  449. }, {
  450. 'info_dict': {
  451. 'id': '370905',
  452. 'title': 'Госзаказ. День 2',
  453. 'ext': 'mp4',
  454. }
  455. }, {
  456. 'info_dict': {
  457. 'id': '370902',
  458. 'title': 'Госзаказ. День 1',
  459. 'ext': 'mp4',
  460. }
  461. }],
  462. 'params': {
  463. # m3u8 download
  464. 'skip_download': True,
  465. },
  466. },
  467. # Myvi.ru embed
  468. {
  469. 'url': 'http://www.kinomyvi.tv/news/detail/Pervij-dublirovannij-trejler--Uzhastikov-_nOw1',
  470. 'info_dict': {
  471. 'id': 'f4dafcad-ff21-423d-89b5-146cfd89fa1e',
  472. 'ext': 'mp4',
  473. 'title': 'Ужастики, русский трейлер (2015)',
  474. 'thumbnail': 're:^https?://.*\.jpg$',
  475. 'duration': 153,
  476. }
  477. },
  478. # XHamster embed
  479. {
  480. 'url': 'http://www.numisc.com/forum/showthread.php?11696-FM15-which-pumiscer-was-this-%28-vid-%29-%28-alfa-as-fuck-srx-%29&s=711f5db534502e22260dec8c5e2d66d8',
  481. 'info_dict': {
  482. 'id': 'showthread',
  483. 'title': '[NSFL] [FM15] which pumiscer was this ( vid ) ( alfa as fuck srx )',
  484. },
  485. 'playlist_mincount': 7,
  486. },
  487. # Embedded TED video
  488. {
  489. 'url': 'http://en.support.wordpress.com/videos/ted-talks/',
  490. 'md5': '65fdff94098e4a607385a60c5177c638',
  491. 'info_dict': {
  492. 'id': '1969',
  493. 'ext': 'mp4',
  494. 'title': 'Hidden miracles of the natural world',
  495. 'uploader': 'Louie Schwartzberg',
  496. 'description': 'md5:8145d19d320ff3e52f28401f4c4283b9',
  497. }
  498. },
  499. # Embedded Ustream video
  500. {
  501. 'url': 'http://www.american.edu/spa/pti/nsa-privacy-janus-2014.cfm',
  502. 'md5': '27b99cdb639c9b12a79bca876a073417',
  503. 'info_dict': {
  504. 'id': '45734260',
  505. 'ext': 'flv',
  506. 'uploader': 'AU SPA: The NSA and Privacy',
  507. 'title': 'NSA and Privacy Forum Debate featuring General Hayden and Barton Gellman'
  508. }
  509. },
  510. # nowvideo embed hidden behind percent encoding
  511. {
  512. 'url': 'http://www.waoanime.tv/the-super-dimension-fortress-macross-episode-1/',
  513. 'md5': '2baf4ddd70f697d94b1c18cf796d5107',
  514. 'info_dict': {
  515. 'id': '06e53103ca9aa',
  516. 'ext': 'flv',
  517. 'title': 'Macross Episode 001 Watch Macross Episode 001 onl',
  518. 'description': 'No description',
  519. },
  520. },
  521. # arte embed
  522. {
  523. 'url': 'http://www.tv-replay.fr/redirection/20-03-14/x-enius-arte-10753389.html',
  524. 'md5': '7653032cbb25bf6c80d80f217055fa43',
  525. 'info_dict': {
  526. 'id': '048195-004_PLUS7-F',
  527. 'ext': 'flv',
  528. 'title': 'X:enius',
  529. 'description': 'md5:d5fdf32ef6613cdbfd516ae658abf168',
  530. 'upload_date': '20140320',
  531. },
  532. 'params': {
  533. 'skip_download': 'Requires rtmpdump'
  534. }
  535. },
  536. # francetv embed
  537. {
  538. 'url': 'http://www.tsprod.com/replay-du-concert-alcaline-de-calogero',
  539. 'info_dict': {
  540. 'id': 'EV_30231',
  541. 'ext': 'mp4',
  542. 'title': 'Alcaline, le concert avec Calogero',
  543. 'description': 'md5:61f08036dcc8f47e9cfc33aed08ffaff',
  544. 'upload_date': '20150226',
  545. 'timestamp': 1424989860,
  546. 'duration': 5400,
  547. },
  548. 'params': {
  549. # m3u8 downloads
  550. 'skip_download': True,
  551. },
  552. 'expected_warnings': [
  553. 'Forbidden'
  554. ]
  555. },
  556. # Condé Nast embed
  557. {
  558. 'url': 'http://www.wired.com/2014/04/honda-asimo/',
  559. 'md5': 'ba0dfe966fa007657bd1443ee672db0f',
  560. 'info_dict': {
  561. 'id': '53501be369702d3275860000',
  562. 'ext': 'mp4',
  563. 'title': 'Honda’s New Asimo Robot Is More Human Than Ever',
  564. }
  565. },
  566. # Dailymotion embed
  567. {
  568. 'url': 'http://www.spi0n.com/zap-spi0n-com-n216/',
  569. 'md5': '441aeeb82eb72c422c7f14ec533999cd',
  570. 'info_dict': {
  571. 'id': 'k2mm4bCdJ6CQ2i7c8o2',
  572. 'ext': 'mp4',
  573. 'title': 'Le Zap de Spi0n n°216 - Zapping du Web',
  574. 'uploader': 'Spi0n',
  575. },
  576. 'add_ie': ['Dailymotion'],
  577. },
  578. # YouTube embed
  579. {
  580. 'url': 'http://www.badzine.de/ansicht/datum/2014/06/09/so-funktioniert-die-neue-englische-badminton-liga.html',
  581. 'info_dict': {
  582. 'id': 'FXRb4ykk4S0',
  583. 'ext': 'mp4',
  584. 'title': 'The NBL Auction 2014',
  585. 'uploader': 'BADMINTON England',
  586. 'uploader_id': 'BADMINTONEvents',
  587. 'upload_date': '20140603',
  588. 'description': 'md5:9ef128a69f1e262a700ed83edb163a73',
  589. },
  590. 'add_ie': ['Youtube'],
  591. 'params': {
  592. 'skip_download': True,
  593. }
  594. },
  595. # MTVSercices embed
  596. {
  597. 'url': 'http://www.gametrailers.com/news-post/76093/north-america-europe-is-getting-that-mario-kart-8-mercedes-dlc-too',
  598. 'md5': '35727f82f58c76d996fc188f9755b0d5',
  599. 'info_dict': {
  600. 'id': '0306a69b-8adf-4fb5-aace-75f8e8cbfca9',
  601. 'ext': 'mp4',
  602. 'title': 'Review',
  603. 'description': 'Mario\'s life in the fast lane has never looked so good.',
  604. },
  605. },
  606. # YouTube embed via <data-embed-url="">
  607. {
  608. 'url': 'https://play.google.com/store/apps/details?id=com.gameloft.android.ANMP.GloftA8HM',
  609. 'info_dict': {
  610. 'id': '4vAffPZIT44',
  611. 'ext': 'mp4',
  612. 'title': 'Asphalt 8: Airborne - Update - Welcome to Dubai!',
  613. 'uploader': 'Gameloft',
  614. 'uploader_id': 'gameloft',
  615. 'upload_date': '20140828',
  616. 'description': 'md5:c80da9ed3d83ae6d1876c834de03e1c4',
  617. },
  618. 'params': {
  619. 'skip_download': True,
  620. }
  621. },
  622. # Camtasia studio
  623. {
  624. 'url': 'http://www.ll.mit.edu/workshops/education/videocourses/antennas/lecture1/video/',
  625. 'playlist': [{
  626. 'md5': '0c5e352edabf715d762b0ad4e6d9ee67',
  627. 'info_dict': {
  628. 'id': 'Fenn-AA_PA_Radar_Course_Lecture_1c_Final',
  629. 'title': 'Fenn-AA_PA_Radar_Course_Lecture_1c_Final - video1',
  630. 'ext': 'flv',
  631. 'duration': 2235.90,
  632. }
  633. }, {
  634. 'md5': '10e4bb3aaca9fd630e273ff92d9f3c63',
  635. 'info_dict': {
  636. 'id': 'Fenn-AA_PA_Radar_Course_Lecture_1c_Final_PIP',
  637. 'title': 'Fenn-AA_PA_Radar_Course_Lecture_1c_Final - pip',
  638. 'ext': 'flv',
  639. 'duration': 2235.93,
  640. }
  641. }],
  642. 'info_dict': {
  643. 'title': 'Fenn-AA_PA_Radar_Course_Lecture_1c_Final',
  644. }
  645. },
  646. # Flowplayer
  647. {
  648. 'url': 'http://www.handjobhub.com/video/busty-blonde-siri-tit-fuck-while-wank-6313.html',
  649. 'md5': '9d65602bf31c6e20014319c7d07fba27',
  650. 'info_dict': {
  651. 'id': '5123ea6d5e5a7',
  652. 'ext': 'mp4',
  653. 'age_limit': 18,
  654. 'uploader': 'www.handjobhub.com',
  655. 'title': 'Busty Blonde Siri Tit Fuck While Wank at HandjobHub.com',
  656. }
  657. },
  658. # Multiple brightcove videos
  659. # https://github.com/rg3/youtube-dl/issues/2283
  660. {
  661. 'url': 'http://www.newyorker.com/online/blogs/newsdesk/2014/01/always-never-nuclear-command-and-control.html',
  662. 'info_dict': {
  663. 'id': 'always-never',
  664. 'title': 'Always / Never - The New Yorker',
  665. },
  666. 'playlist_count': 3,
  667. 'params': {
  668. 'extract_flat': False,
  669. 'skip_download': True,
  670. }
  671. },
  672. # MLB embed
  673. {
  674. 'url': 'http://umpire-empire.com/index.php/topic/58125-laz-decides-no-thats-low/',
  675. 'md5': '96f09a37e44da40dd083e12d9a683327',
  676. 'info_dict': {
  677. 'id': '33322633',
  678. 'ext': 'mp4',
  679. 'title': 'Ump changes call to ball',
  680. 'description': 'md5:71c11215384298a172a6dcb4c2e20685',
  681. 'duration': 48,
  682. 'timestamp': 1401537900,
  683. 'upload_date': '20140531',
  684. 'thumbnail': 're:^https?://.*\.jpg$',
  685. },
  686. },
  687. # Wistia embed
  688. {
  689. 'url': 'http://education-portal.com/academy/lesson/north-american-exploration-failed-colonies-of-spain-france-england.html#lesson',
  690. 'md5': '8788b683c777a5cf25621eaf286d0c23',
  691. 'info_dict': {
  692. 'id': '1cfaf6b7ea',
  693. 'ext': 'mov',
  694. 'title': 'md5:51364a8d3d009997ba99656004b5e20d',
  695. 'duration': 643.0,
  696. 'filesize': 182808282,
  697. 'uploader': 'education-portal.com',
  698. },
  699. },
  700. {
  701. 'url': 'http://thoughtworks.wistia.com/medias/uxjb0lwrcz',
  702. 'md5': 'baf49c2baa8a7de5f3fc145a8506dcd4',
  703. 'info_dict': {
  704. 'id': 'uxjb0lwrcz',
  705. 'ext': 'mp4',
  706. 'title': 'Conversation about Hexagonal Rails Part 1 - ThoughtWorks',
  707. 'duration': 1715.0,
  708. 'uploader': 'thoughtworks.wistia.com',
  709. },
  710. },
  711. # Soundcloud embed
  712. {
  713. 'url': 'http://nakedsecurity.sophos.com/2014/10/29/sscc-171-are-you-sure-that-1234-is-a-bad-password-podcast/',
  714. 'info_dict': {
  715. 'id': '174391317',
  716. 'ext': 'mp3',
  717. 'description': 'md5:ff867d6b555488ad3c52572bb33d432c',
  718. 'uploader': 'Sophos Security',
  719. 'title': 'Chet Chat 171 - Oct 29, 2014',
  720. 'upload_date': '20141029',
  721. }
  722. },
  723. # Livestream embed
  724. {
  725. 'url': 'http://www.esa.int/Our_Activities/Space_Science/Rosetta/Philae_comet_touch-down_webcast',
  726. 'info_dict': {
  727. 'id': '67864563',
  728. 'ext': 'flv',
  729. 'upload_date': '20141112',
  730. 'title': 'Rosetta #CometLanding webcast HL 10',
  731. }
  732. },
  733. # LazyYT
  734. {
  735. 'url': 'http://discourse.ubuntu.com/t/unity-8-desktop-mode-windows-on-mir/1986',
  736. 'info_dict': {
  737. 'id': '1986',
  738. 'title': 'Unity 8 desktop-mode windows on Mir! - Ubuntu Discourse',
  739. },
  740. 'playlist_mincount': 2,
  741. },
  742. # Cinchcast embed
  743. {
  744. 'url': 'http://undergroundwellness.com/podcasts/306-5-steps-to-permanent-gut-healing/',
  745. 'info_dict': {
  746. 'id': '7141703',
  747. 'ext': 'mp3',
  748. 'upload_date': '20141126',
  749. 'title': 'Jack Tips: 5 Steps to Permanent Gut Healing',
  750. }
  751. },
  752. # Cinerama player
  753. {
  754. 'url': 'http://www.abc.net.au/7.30/content/2015/s4164797.htm',
  755. 'info_dict': {
  756. 'id': '730m_DandD_1901_512k',
  757. 'ext': 'mp4',
  758. 'uploader': 'www.abc.net.au',
  759. 'title': 'Game of Thrones with dice - Dungeons and Dragons fantasy role-playing game gets new life - 19/01/2015',
  760. }
  761. },
  762. # embedded viddler video
  763. {
  764. 'url': 'http://deadspin.com/i-cant-stop-watching-john-wall-chop-the-nuggets-with-th-1681801597',
  765. 'info_dict': {
  766. 'id': '4d03aad9',
  767. 'ext': 'mp4',
  768. 'uploader': 'deadspin',
  769. 'title': 'WALL-TO-GORTAT',
  770. 'timestamp': 1422285291,
  771. 'upload_date': '20150126',
  772. },
  773. 'add_ie': ['Viddler'],
  774. },
  775. # Libsyn embed
  776. {
  777. 'url': 'http://thedailyshow.cc.com/podcast/episodetwelve',
  778. 'info_dict': {
  779. 'id': '3377616',
  780. 'ext': 'mp3',
  781. 'title': "The Daily Show Podcast without Jon Stewart - Episode 12: Bassem Youssef: Egypt's Jon Stewart",
  782. 'description': 'md5:601cb790edd05908957dae8aaa866465',
  783. 'upload_date': '20150220',
  784. },
  785. },
  786. # jwplayer YouTube
  787. {
  788. 'url': 'http://media.nationalarchives.gov.uk/index.php/webinar-using-discovery-national-archives-online-catalogue/',
  789. 'info_dict': {
  790. 'id': 'Mrj4DVp2zeA',
  791. 'ext': 'mp4',
  792. 'upload_date': '20150212',
  793. 'uploader': 'The National Archives UK',
  794. 'description': 'md5:a236581cd2449dd2df4f93412f3f01c6',
  795. 'uploader_id': 'NationalArchives08',
  796. 'title': 'Webinar: Using Discovery, The National Archives’ online catalogue',
  797. },
  798. },
  799. # rtl.nl embed
  800. {
  801. 'url': 'http://www.rtlnieuws.nl/nieuws/buitenland/aanslagen-kopenhagen',
  802. 'playlist_mincount': 5,
  803. 'info_dict': {
  804. 'id': 'aanslagen-kopenhagen',
  805. 'title': 'Aanslagen Kopenhagen | RTL Nieuws',
  806. }
  807. },
  808. # Zapiks embed
  809. {
  810. 'url': 'http://www.skipass.com/news/116090-bon-appetit-s5ep3-baqueira-mi-cor.html',
  811. 'info_dict': {
  812. 'id': '118046',
  813. 'ext': 'mp4',
  814. 'title': 'EP3S5 - Bon Appétit - Baqueira Mi Corazon !',
  815. }
  816. },
  817. # Kaltura embed
  818. {
  819. 'url': 'http://www.monumentalnetwork.com/videos/john-carlson-postgame-2-25-15',
  820. 'info_dict': {
  821. 'id': '1_eergr3h1',
  822. 'ext': 'mp4',
  823. 'upload_date': '20150226',
  824. 'uploader_id': 'MonumentalSports-Kaltura@perfectsensedigital.com',
  825. 'timestamp': int,
  826. 'title': 'John Carlson Postgame 2/25/15',
  827. },
  828. },
  829. # Kaltura embed (different embed code)
  830. {
  831. 'url': 'http://www.premierchristianradio.com/Shows/Saturday/Unbelievable/Conference-Videos/Os-Guinness-Is-It-Fools-Talk-Unbelievable-Conference-2014',
  832. 'info_dict': {
  833. 'id': '1_a52wc67y',
  834. 'ext': 'flv',
  835. 'upload_date': '20150127',
  836. 'uploader_id': 'PremierMedia',
  837. 'timestamp': int,
  838. 'title': 'Os Guinness // Is It Fools Talk? // Unbelievable? Conference 2014',
  839. },
  840. },
  841. # Kaltura embed protected with referrer
  842. {
  843. 'url': 'http://www.disney.nl/disney-channel/filmpjes/achter-de-schermen#/videoId/violetta-achter-de-schermen-ruggero',
  844. 'info_dict': {
  845. 'id': '1_g4fbemnq',
  846. 'ext': 'mp4',
  847. 'title': 'Violetta - Achter De Schermen - Ruggero',
  848. 'description': 'Achter de schermen met Ruggero',
  849. 'timestamp': 1435133761,
  850. 'upload_date': '20150624',
  851. 'uploader_id': 'echojecka',
  852. },
  853. },
  854. # Eagle.Platform embed (generic URL)
  855. {
  856. 'url': 'http://lenta.ru/news/2015/03/06/navalny/',
  857. 'info_dict': {
  858. 'id': '227304',
  859. 'ext': 'mp4',
  860. 'title': 'Навальный вышел на свободу',
  861. 'description': 'md5:d97861ac9ae77377f3f20eaf9d04b4f5',
  862. 'thumbnail': 're:^https?://.*\.jpg$',
  863. 'duration': 87,
  864. 'view_count': int,
  865. 'age_limit': 0,
  866. },
  867. },
  868. # ClipYou (Eagle.Platform) embed (custom URL)
  869. {
  870. 'url': 'http://muz-tv.ru/play/7129/',
  871. 'info_dict': {
  872. 'id': '12820',
  873. 'ext': 'mp4',
  874. 'title': "'O Sole Mio",
  875. 'thumbnail': 're:^https?://.*\.jpg$',
  876. 'duration': 216,
  877. 'view_count': int,
  878. },
  879. },
  880. # Pladform embed
  881. {
  882. 'url': 'http://muz-tv.ru/kinozal/view/7400/',
  883. 'info_dict': {
  884. 'id': '100183293',
  885. 'ext': 'mp4',
  886. 'title': 'Тайны перевала Дятлова • 1 серия 2 часть',
  887. 'description': 'Документальный сериал-расследование одной из самых жутких тайн ХХ века',
  888. 'thumbnail': 're:^https?://.*\.jpg$',
  889. 'duration': 694,
  890. 'age_limit': 0,
  891. },
  892. },
  893. # Playwire embed
  894. {
  895. 'url': 'http://www.cinemablend.com/new/First-Joe-Dirt-2-Trailer-Teaser-Stupid-Greatness-70874.html',
  896. 'info_dict': {
  897. 'id': '3519514',
  898. 'ext': 'mp4',
  899. 'title': 'Joe Dirt 2 Beautiful Loser Teaser Trailer',
  900. 'thumbnail': 're:^https?://.*\.png$',
  901. 'duration': 45.115,
  902. },
  903. },
  904. # 5min embed
  905. {
  906. 'url': 'http://techcrunch.com/video/facebook-creates-on-this-day-crunch-report/518726732/',
  907. 'md5': '4c6f127a30736b59b3e2c19234ee2bf7',
  908. 'info_dict': {
  909. 'id': '518726732',
  910. 'ext': 'mp4',
  911. 'title': 'Facebook Creates "On This Day" | Crunch Report',
  912. },
  913. },
  914. # SVT embed
  915. {
  916. 'url': 'http://www.svt.se/sport/ishockey/jagr-tacklar-giroux-under-intervjun',
  917. 'info_dict': {
  918. 'id': '2900353',
  919. 'ext': 'flv',
  920. 'title': 'Här trycker Jagr till Giroux (under SVT-intervjun)',
  921. 'duration': 27,
  922. 'age_limit': 0,
  923. },
  924. },
  925. # Crooks and Liars embed
  926. {
  927. 'url': 'http://crooksandliars.com/2015/04/fox-friends-says-protecting-atheists',
  928. 'info_dict': {
  929. 'id': '8RUoRhRi',
  930. 'ext': 'mp4',
  931. 'title': "Fox & Friends Says Protecting Atheists From Discrimination Is Anti-Christian!",
  932. 'description': 'md5:e1a46ad1650e3a5ec7196d432799127f',
  933. 'timestamp': 1428207000,
  934. 'upload_date': '20150405',
  935. 'uploader': 'Heather',
  936. },
  937. },
  938. # Crooks and Liars external embed
  939. {
  940. 'url': 'http://theothermccain.com/2010/02/02/video-proves-that-bill-kristol-has-been-watching-glenn-beck/comment-page-1/',
  941. 'info_dict': {
  942. 'id': 'MTE3MjUtMzQ2MzA',
  943. 'ext': 'mp4',
  944. 'title': 'md5:5e3662a81a4014d24c250d76d41a08d5',
  945. 'description': 'md5:9b8e9542d6c3c5de42d6451b7d780cec',
  946. 'timestamp': 1265032391,
  947. 'upload_date': '20100201',
  948. 'uploader': 'Heather',
  949. },
  950. },
  951. # NBC Sports vplayer embed
  952. {
  953. 'url': 'http://www.riderfans.com/forum/showthread.php?121827-Freeman&s=e98fa1ea6dc08e886b1678d35212494a',
  954. 'info_dict': {
  955. 'id': 'ln7x1qSThw4k',
  956. 'ext': 'flv',
  957. 'title': "PFT Live: New leader in the 'new-look' defense",
  958. 'description': 'md5:65a19b4bbfb3b0c0c5768bed1dfad74e',
  959. },
  960. },
  961. # UDN embed
  962. {
  963. 'url': 'http://www.udn.com/news/story/7314/822787',
  964. 'md5': 'fd2060e988c326991037b9aff9df21a6',
  965. 'info_dict': {
  966. 'id': '300346',
  967. 'ext': 'mp4',
  968. 'title': '中一中男師變性 全校師生力挺',
  969. 'thumbnail': 're:^https?://.*\.jpg$',
  970. }
  971. },
  972. # Ooyala embed
  973. {
  974. 'url': 'http://www.businessinsider.com/excel-index-match-vlookup-video-how-to-2015-2?IR=T',
  975. 'info_dict': {
  976. 'id': '50YnY4czr4ms1vJ7yz3xzq0excz_pUMs',
  977. 'ext': 'mp4',
  978. 'description': 'VIDEO: INDEX/MATCH versus VLOOKUP.',
  979. 'title': 'This is what separates the Excel masters from the wannabes',
  980. 'duration': 191.933,
  981. },
  982. 'params': {
  983. # m3u8 downloads
  984. 'skip_download': True,
  985. }
  986. },
  987. # Contains a SMIL manifest
  988. {
  989. 'url': 'http://www.telewebion.com/fa/1263668/%D9%82%D8%B1%D8%B9%D9%87%E2%80%8C%DA%A9%D8%B4%DB%8C-%D9%84%DB%8C%DA%AF-%D9%82%D9%87%D8%B1%D9%85%D8%A7%D9%86%D8%A7%D9%86-%D8%A7%D8%B1%D9%88%D9%BE%D8%A7/%2B-%D9%81%D9%88%D8%AA%D8%A8%D8%A7%D9%84.html',
  990. 'info_dict': {
  991. 'id': 'file',
  992. 'ext': 'flv',
  993. 'title': '+ Football: Lottery Champions League Europe',
  994. 'uploader': 'www.telewebion.com',
  995. },
  996. 'params': {
  997. # rtmpe downloads
  998. 'skip_download': True,
  999. }
  1000. },
  1001. # Brightcove URL in single quotes
  1002. {
  1003. 'url': 'http://www.sportsnet.ca/baseball/mlb/sn-presents-russell-martin-world-citizen/',
  1004. 'md5': '4ae374f1f8b91c889c4b9203c8c752af',
  1005. 'info_dict': {
  1006. 'id': '4255764656001',
  1007. 'ext': 'mp4',
  1008. 'title': 'SN Presents: Russell Martin, World Citizen',
  1009. 'description': 'To understand why he was the Toronto Blue Jays’ top off-season priority is to appreciate his background and upbringing in Montreal, where he first developed his baseball skills. Written and narrated by Stephen Brunt.',
  1010. 'uploader': 'Rogers Sportsnet',
  1011. },
  1012. },
  1013. # Dailymotion Cloud video
  1014. {
  1015. 'url': 'http://replay.publicsenat.fr/vod/le-debat/florent-kolandjian,dominique-cena,axel-decourtye,laurence-abeille,bruno-parmentier/175910',
  1016. 'md5': '49444254273501a64675a7e68c502681',
  1017. 'info_dict': {
  1018. 'id': '5585de919473990de4bee11b',
  1019. 'ext': 'mp4',
  1020. 'title': 'Le débat',
  1021. 'thumbnail': 're:^https?://.*\.jpe?g$',
  1022. }
  1023. },
  1024. # OnionStudios embed
  1025. {
  1026. 'url': 'http://www.clickhole.com/video/dont-understand-bitcoin-man-will-mumble-explanatio-2537',
  1027. 'info_dict': {
  1028. 'id': '2855',
  1029. 'ext': 'mp4',
  1030. 'title': 'Don’t Understand Bitcoin? This Man Will Mumble An Explanation At You',
  1031. 'thumbnail': 're:^https?://.*\.jpe?g$',
  1032. 'uploader': 'ClickHole',
  1033. 'uploader_id': 'clickhole',
  1034. }
  1035. },
  1036. # SnagFilms embed
  1037. {
  1038. 'url': 'http://whilewewatch.blogspot.ru/2012/06/whilewewatch-whilewewatch-gripping.html',
  1039. 'info_dict': {
  1040. 'id': '74849a00-85a9-11e1-9660-123139220831',
  1041. 'ext': 'mp4',
  1042. 'title': '#whilewewatch',
  1043. }
  1044. },
  1045. # AdobeTVVideo embed
  1046. {
  1047. 'url': 'https://helpx.adobe.com/acrobat/how-to/new-experience-acrobat-dc.html?set=acrobat--get-started--essential-beginners',
  1048. 'md5': '43662b577c018ad707a63766462b1e87',
  1049. 'info_dict': {
  1050. 'id': '2456',
  1051. 'ext': 'mp4',
  1052. 'title': 'New experience with Acrobat DC',
  1053. 'description': 'New experience with Acrobat DC',
  1054. 'duration': 248.667,
  1055. },
  1056. },
  1057. # ScreenwaveMedia embed
  1058. {
  1059. 'url': 'http://www.thecinemasnob.com/the-cinema-snob/a-nightmare-on-elm-street-2-freddys-revenge1',
  1060. 'md5': '24ace5baba0d35d55c6810b51f34e9e0',
  1061. 'info_dict': {
  1062. 'id': 'cinemasnob-55d26273809dd',
  1063. 'ext': 'mp4',
  1064. 'title': 'cinemasnob',
  1065. },
  1066. },
  1067. # BrightcoveInPageEmbed embed
  1068. {
  1069. 'url': 'http://www.geekandsundry.com/tabletop-bonus-wils-final-thoughts-on-dread/',
  1070. 'info_dict': {
  1071. 'id': '4238694884001',
  1072. 'ext': 'flv',
  1073. 'title': 'Tabletop: Dread, Last Thoughts',
  1074. 'description': 'Tabletop: Dread, Last Thoughts',
  1075. 'duration': 51690,
  1076. },
  1077. },
  1078. # JWPlayer with M3U8
  1079. {
  1080. 'url': 'http://ren.tv/novosti/2015-09-25/sluchaynyy-prohozhiy-poymal-avtougonshchika-v-murmanske-video',
  1081. 'info_dict': {
  1082. 'id': 'playlist',
  1083. 'ext': 'mp4',
  1084. 'title': 'Случайный прохожий поймал автоугонщика в Мурманске. ВИДЕО | РЕН ТВ',
  1085. 'uploader': 'ren.tv',
  1086. },
  1087. 'params': {
  1088. # m3u8 downloads
  1089. 'skip_download': True,
  1090. }
  1091. }
  1092. ]
  1093. def report_following_redirect(self, new_url):
  1094. """Report information extraction."""
  1095. self._downloader.to_screen('[redirect] Following redirect to %s' % new_url)
  1096. def _extract_rss(self, url, video_id, doc):
  1097. playlist_title = doc.find('./channel/title').text
  1098. playlist_desc_el = doc.find('./channel/description')
  1099. playlist_desc = None if playlist_desc_el is None else playlist_desc_el.text
  1100. entries = []
  1101. for it in doc.findall('./channel/item'):
  1102. next_url = xpath_text(it, 'link', fatal=False)
  1103. if not next_url:
  1104. enclosure_nodes = it.findall('./enclosure')
  1105. for e in enclosure_nodes:
  1106. next_url = e.attrib.get('url')
  1107. if next_url:
  1108. break
  1109. if not next_url:
  1110. continue
  1111. entries.append({
  1112. '_type': 'url',
  1113. 'url': next_url,
  1114. 'title': it.find('title').text,
  1115. })
  1116. return {
  1117. '_type': 'playlist',
  1118. 'id': url,
  1119. 'title': playlist_title,
  1120. 'description': playlist_desc,
  1121. 'entries': entries,
  1122. }
  1123. def _extract_camtasia(self, url, video_id, webpage):
  1124. """ Returns None if no camtasia video can be found. """
  1125. camtasia_cfg = self._search_regex(
  1126. r'fo\.addVariable\(\s*"csConfigFile",\s*"([^"]+)"\s*\);',
  1127. webpage, 'camtasia configuration file', default=None)
  1128. if camtasia_cfg is None:
  1129. return None
  1130. title = self._html_search_meta('DC.title', webpage, fatal=True)
  1131. camtasia_url = compat_urlparse.urljoin(url, camtasia_cfg)
  1132. camtasia_cfg = self._download_xml(
  1133. camtasia_url, video_id,
  1134. note='Downloading camtasia configuration',
  1135. errnote='Failed to download camtasia configuration')
  1136. fileset_node = camtasia_cfg.find('./playlist/array/fileset')
  1137. entries = []
  1138. for n in fileset_node.getchildren():
  1139. url_n = n.find('./uri')
  1140. if url_n is None:
  1141. continue
  1142. entries.append({
  1143. 'id': os.path.splitext(url_n.text.rpartition('/')[2])[0],
  1144. 'title': '%s - %s' % (title, n.tag),
  1145. 'url': compat_urlparse.urljoin(url, url_n.text),
  1146. 'duration': float_or_none(n.find('./duration').text),
  1147. })
  1148. return {
  1149. '_type': 'playlist',
  1150. 'entries': entries,
  1151. 'title': title,
  1152. }
  1153. def _real_extract(self, url):
  1154. if url.startswith('//'):
  1155. return {
  1156. '_type': 'url',
  1157. 'url': self.http_scheme() + url,
  1158. }
  1159. parsed_url = compat_urlparse.urlparse(url)
  1160. if not parsed_url.scheme:
  1161. default_search = self._downloader.params.get('default_search')
  1162. if default_search is None:
  1163. default_search = 'fixup_error'
  1164. if default_search in ('auto', 'auto_warning', 'fixup_error'):
  1165. if '/' in url:
  1166. self._downloader.report_warning('The url doesn\'t specify the protocol, trying with http')
  1167. return self.url_result('http://' + url)
  1168. elif default_search != 'fixup_error':
  1169. if default_search == 'auto_warning':
  1170. if re.match(r'^(?:url|URL)$', url):
  1171. raise ExtractorError(
  1172. 'Invalid URL: %r . Call youtube-dl like this: youtube-dl -v "https://www.youtube.com/watch?v=BaW_jenozKc" ' % url,
  1173. expected=True)
  1174. else:
  1175. self._downloader.report_warning(
  1176. 'Falling back to youtube search for %s . Set --default-search "auto" to suppress this warning.' % url)
  1177. return self.url_result('ytsearch:' + url)
  1178. if default_search in ('error', 'fixup_error'):
  1179. raise ExtractorError(
  1180. '%r is not a valid URL. '
  1181. 'Set --default-search "ytsearch" (or run youtube-dl "ytsearch:%s" ) to search YouTube'
  1182. % (url, url), expected=True)
  1183. else:
  1184. if ':' not in default_search:
  1185. default_search += ':'
  1186. return self.url_result(default_search + url)
  1187. url, smuggled_data = unsmuggle_url(url)
  1188. force_videoid = None
  1189. is_intentional = smuggled_data and smuggled_data.get('to_generic')
  1190. if smuggled_data and 'force_videoid' in smuggled_data:
  1191. force_videoid = smuggled_data['force_videoid']
  1192. video_id = force_videoid
  1193. else:
  1194. video_id = compat_urllib_parse_unquote(os.path.splitext(url.rstrip('/').split('/')[-1])[0])
  1195. self.to_screen('%s: Requesting header' % video_id)
  1196. head_req = HEADRequest(url)
  1197. head_response = self._request_webpage(
  1198. head_req, video_id,
  1199. note=False, errnote='Could not send HEAD request to %s' % url,
  1200. fatal=False)
  1201. if head_response is not False:
  1202. # Check for redirect
  1203. new_url = head_response.geturl()
  1204. if url != new_url:
  1205. self.report_following_redirect(new_url)
  1206. if force_videoid:
  1207. new_url = smuggle_url(
  1208. new_url, {'force_videoid': force_videoid})
  1209. return self.url_result(new_url)
  1210. full_response = None
  1211. if head_response is False:
  1212. request = sanitized_Request(url)
  1213. request.add_header('Accept-Encoding', '*')
  1214. full_response = self._request_webpage(request, video_id)
  1215. head_response = full_response
  1216. # Check for direct link to a video
  1217. content_type = head_response.headers.get('Content-Type', '')
  1218. m = re.match(r'^(?P<type>audio|video|application(?=/(?:ogg$|(?:vnd\.apple\.|x-)?mpegurl)))/(?P<format_id>.+)$', content_type)
  1219. if m:
  1220. upload_date = unified_strdate(
  1221. head_response.headers.get('Last-Modified'))
  1222. formats = []
  1223. if m.group('format_id').endswith('mpegurl'):
  1224. formats = self._extract_m3u8_formats(url, video_id, 'mp4')
  1225. else:
  1226. formats = [{
  1227. 'format_id': m.group('format_id'),
  1228. 'url': url,
  1229. 'vcodec': 'none' if m.group('type') == 'audio' else None
  1230. }]
  1231. return {
  1232. 'id': video_id,
  1233. 'title': compat_urllib_parse_unquote(os.path.splitext(url_basename(url))[0]),
  1234. 'direct': True,
  1235. 'formats': formats,
  1236. 'upload_date': upload_date,
  1237. }
  1238. if not self._downloader.params.get('test', False) and not is_intentional:
  1239. force = self._downloader.params.get('force_generic_extractor', False)
  1240. self._downloader.report_warning(
  1241. '%s on generic information extractor.' % ('Forcing' if force else 'Falling back'))
  1242. if not full_response:
  1243. request = sanitized_Request(url)
  1244. # Some webservers may serve compressed content of rather big size (e.g. gzipped flac)
  1245. # making it impossible to download only chunk of the file (yet we need only 512kB to
  1246. # test whether it's HTML or not). According to youtube-dl default Accept-Encoding
  1247. # that will always result in downloading the whole file that is not desirable.
  1248. # Therefore for extraction pass we have to override Accept-Encoding to any in order
  1249. # to accept raw bytes and being able to download only a chunk.
  1250. # It may probably better to solve this by checking Content-Type for application/octet-stream
  1251. # after HEAD request finishes, but not sure if we can rely on this.
  1252. request.add_header('Accept-Encoding', '*')
  1253. full_response = self._request_webpage(request, video_id)
  1254. # Maybe it's a direct link to a video?
  1255. # Be careful not to download the whole thing!
  1256. first_bytes = full_response.read(512)
  1257. if not is_html(first_bytes):
  1258. self._downloader.report_warning(
  1259. 'URL could be a direct video link, returning it as such.')
  1260. upload_date = unified_strdate(
  1261. head_response.headers.get('Last-Modified'))
  1262. return {
  1263. 'id': video_id,
  1264. 'title': compat_urllib_parse_unquote(os.path.splitext(url_basename(url))[0]),
  1265. 'direct': True,
  1266. 'url': url,
  1267. 'upload_date': upload_date,
  1268. }
  1269. webpage = self._webpage_read_content(
  1270. full_response, url, video_id, prefix=first_bytes)
  1271. self.report_extraction(video_id)
  1272. # Is it an RSS feed, a SMIL file, an XSPF playlist or a MPD manifest?
  1273. try:
  1274. doc = compat_etree_fromstring(webpage.encode('utf-8'))
  1275. if doc.tag == 'rss':
  1276. return self._extract_rss(url, video_id, doc)
  1277. elif re.match(r'^(?:{[^}]+})?smil$', doc.tag):
  1278. return self._parse_smil(doc, url, video_id)
  1279. elif doc.tag == '{http://xspf.org/ns/0/}playlist':
  1280. return self.playlist_result(self._parse_xspf(doc, video_id), video_id)
  1281. elif re.match(r'(?i)^(?:{[^}]+})?MPD$', doc.tag):
  1282. return {
  1283. 'id': video_id,
  1284. 'title': compat_urllib_parse_unquote(os.path.splitext(url_basename(url))[0]),
  1285. 'formats': self._parse_mpd_formats(
  1286. doc, video_id, mpd_base_url=url.rpartition('/')[0]),
  1287. }
  1288. except compat_xml_parse_error:
  1289. pass
  1290. # Is it a Camtasia project?
  1291. camtasia_res = self._extract_camtasia(url, video_id, webpage)
  1292. if camtasia_res is not None:
  1293. return camtasia_res
  1294. # Sometimes embedded video player is hidden behind percent encoding
  1295. # (e.g. https://github.com/rg3/youtube-dl/issues/2448)
  1296. # Unescaping the whole page allows to handle those cases in a generic way
  1297. webpage = compat_urllib_parse_unquote(webpage)
  1298. # it's tempting to parse this further, but you would
  1299. # have to take into account all the variations like
  1300. # Video Title - Site Name
  1301. # Site Name | Video Title
  1302. # Video Title - Tagline | Site Name
  1303. # and so on and so forth; it's just not practical
  1304. video_title = self._html_search_regex(
  1305. r'(?s)<title>(.*?)</title>', webpage, 'video title',
  1306. default='video')
  1307. # Try to detect age limit automatically
  1308. age_limit = self._rta_search(webpage)
  1309. # And then there are the jokers who advertise that they use RTA,
  1310. # but actually don't.
  1311. AGE_LIMIT_MARKERS = [
  1312. r'Proudly Labeled <a href="http://www.rtalabel.org/" title="Restricted to Adults">RTA</a>',
  1313. ]
  1314. if any(re.search(marker, webpage) for marker in AGE_LIMIT_MARKERS):
  1315. age_limit = 18
  1316. # video uploader is domain name
  1317. video_uploader = self._search_regex(
  1318. r'^(?:https?://)?([^/]*)/.*', url, 'video uploader')
  1319. # Helper method
  1320. def _playlist_from_matches(matches, getter=None, ie=None):
  1321. urlrs = orderedSet(
  1322. self.url_result(self._proto_relative_url(getter(m) if getter else m), ie)
  1323. for m in matches)
  1324. return self.playlist_result(
  1325. urlrs, playlist_id=video_id, playlist_title=video_title)
  1326. # Look for Brightcove Legacy Studio embeds
  1327. bc_urls = BrightcoveLegacyIE._extract_brightcove_urls(webpage)
  1328. if bc_urls:
  1329. self.to_screen('Brightcove video detected.')
  1330. entries = [{
  1331. '_type': 'url',
  1332. 'url': smuggle_url(bc_url, {'Referer': url}),
  1333. 'ie_key': 'BrightcoveLegacy'
  1334. } for bc_url in bc_urls]
  1335. return {
  1336. '_type': 'playlist',
  1337. 'title': video_title,
  1338. 'id': video_id,
  1339. 'entries': entries,
  1340. }
  1341. # Look for Brightcove New Studio embeds
  1342. bc_urls = BrightcoveNewIE._extract_urls(webpage)
  1343. if bc_urls:
  1344. return _playlist_from_matches(bc_urls, ie='BrightcoveNew')
  1345. # Look for embedded rtl.nl player
  1346. matches = re.findall(
  1347. r'<iframe[^>]+?src="((?:https?:)?//(?:www\.)?rtl\.nl/system/videoplayer/[^"]+(?:video_)?embed[^"]+)"',
  1348. webpage)
  1349. if matches:
  1350. return _playlist_from_matches(matches, ie='RtlNl')
  1351. vimeo_url = VimeoIE._extract_vimeo_url(url, webpage)
  1352. if vimeo_url is not None:
  1353. return self.url_result(vimeo_url)
  1354. vid_me_embed_url = self._search_regex(
  1355. r'src=[\'"](https?://vid\.me/[^\'"]+)[\'"]',
  1356. webpage, 'vid.me embed', default=None)
  1357. if vid_me_embed_url is not None:
  1358. return self.url_result(vid_me_embed_url, 'Vidme')
  1359. # Look for embedded YouTube player
  1360. matches = re.findall(r'''(?x)
  1361. (?:
  1362. <iframe[^>]+?src=|
  1363. data-video-url=|
  1364. <embed[^>]+?src=|
  1365. embedSWF\(?:\s*|
  1366. new\s+SWFObject\(
  1367. )
  1368. (["\'])
  1369. (?P<url>(?:https?:)?//(?:www\.)?youtube(?:-nocookie)?\.com/
  1370. (?:embed|v|p)/.+?)
  1371. \1''', webpage)
  1372. if matches:
  1373. return _playlist_from_matches(
  1374. matches, lambda m: unescapeHTML(m[1]))
  1375. # Look for lazyYT YouTube embed
  1376. matches = re.findall(
  1377. r'class="lazyYT" data-youtube-id="([^"]+)"', webpage)
  1378. if matches:
  1379. return _playlist_from_matches(matches, lambda m: unescapeHTML(m))
  1380. # Look for embedded Dailymotion player
  1381. matches = re.findall(
  1382. r'<(?:(?:embed|iframe)[^>]+?src=|input[^>]+id=[\'"]dmcloudUrlEmissionSelect[\'"][^>]+value=)(["\'])(?P<url>(?:https?:)?//(?:www\.)?dailymotion\.com/(?:embed|swf)/video/.+?)\1', webpage)
  1383. if matches:
  1384. return _playlist_from_matches(
  1385. matches, lambda m: unescapeHTML(m[1]))
  1386. # Look for embedded Dailymotion playlist player (#3822)
  1387. m = re.search(
  1388. r'<iframe[^>]+?src=(["\'])(?P<url>(?:https?:)?//(?:www\.)?dailymotion\.[a-z]{2,3}/widget/jukebox\?.+?)\1', webpage)
  1389. if m:
  1390. playlists = re.findall(
  1391. r'list\[\]=/playlist/([^/]+)/', unescapeHTML(m.group('url')))
  1392. if playlists:
  1393. return _playlist_from_matches(
  1394. playlists, lambda p: '//dailymotion.com/playlist/%s' % p)
  1395. # Look for embedded Wistia player
  1396. match = re.search(
  1397. r'<(?:meta[^>]+?content|iframe[^>]+?src)=(["\'])(?P<url>(?:https?:)?//(?:fast\.)?wistia\.net/embed/iframe/.+?)\1', webpage)
  1398. if match:
  1399. embed_url = self._proto_relative_url(
  1400. unescapeHTML(match.group('url')))
  1401. return {
  1402. '_type': 'url_transparent',
  1403. 'url': embed_url,
  1404. 'ie_key': 'Wistia',
  1405. 'uploader': video_uploader,
  1406. 'title': video_title,
  1407. 'id': video_id,
  1408. }
  1409. match = re.search(r'(?:id=["\']wistia_|data-wistia-?id=["\']|Wistia\.embed\(["\'])(?P<id>[^"\']+)', webpage)
  1410. if match:
  1411. return {
  1412. '_type': 'url_transparent',
  1413. 'url': 'http://fast.wistia.net/embed/iframe/{0:}'.format(match.group('id')),
  1414. 'ie_key': 'Wistia',
  1415. 'uploader': video_uploader,
  1416. 'title': video_title,
  1417. 'id': match.group('id')
  1418. }
  1419. # Look for SVT player
  1420. svt_url = SVTIE._extract_url(webpage)
  1421. if svt_url:
  1422. return self.url_result(svt_url, 'SVT')
  1423. # Look for embedded condenast player
  1424. matches = re.findall(
  1425. r'<iframe\s+(?:[a-zA-Z-]+="[^"]+"\s+)*?src="(https?://player\.cnevids\.com/embed/[^"]+")',
  1426. webpage)
  1427. if matches:
  1428. return {
  1429. '_type': 'playlist',
  1430. 'entries': [{
  1431. '_type': 'url',
  1432. 'ie_key': 'CondeNast',
  1433. 'url': ma,
  1434. } for ma in matches],
  1435. 'title': video_title,
  1436. 'id': video_id,
  1437. }
  1438. # Look for Bandcamp pages with custom domain
  1439. mobj = re.search(r'<meta property="og:url"[^>]*?content="(.*?bandcamp\.com.*?)"', webpage)
  1440. if mobj is not None:
  1441. burl = unescapeHTML(mobj.group(1))
  1442. # Don't set the extractor because it can be a track url or an album
  1443. return self.url_result(burl)
  1444. # Look for embedded Vevo player
  1445. mobj = re.search(
  1446. r'<iframe[^>]+?src=(["\'])(?P<url>(?:https?:)?//(?:cache\.)?vevo\.com/.+?)\1', webpage)
  1447. if mobj is not None:
  1448. return self.url_result(mobj.group('url'))
  1449. # Look for embedded Viddler player
  1450. mobj = re.search(
  1451. r'<(?:iframe[^>]+?src|param[^>]+?value)=(["\'])(?P<url>(?:https?:)?//(?:www\.)?viddler\.com/(?:embed|player)/.+?)\1',
  1452. webpage)
  1453. if mobj is not None:
  1454. return self.url_result(mobj.group('url'))
  1455. # Look for NYTimes player
  1456. mobj = re.search(
  1457. r'<iframe[^>]+src=(["\'])(?P<url>(?:https?:)?//graphics8\.nytimes\.com/bcvideo/[^/]+/iframe/embed\.html.+?)\1>',
  1458. webpage)
  1459. if mobj is not None:
  1460. return self.url_result(mobj.group('url'))
  1461. # Look for Libsyn player
  1462. mobj = re.search(
  1463. r'<iframe[^>]+src=(["\'])(?P<url>(?:https?:)?//html5-player\.libsyn\.com/embed/.+?)\1', webpage)
  1464. if mobj is not None:
  1465. return self.url_result(mobj.group('url'))
  1466. # Look for Ooyala videos
  1467. mobj = (re.search(r'player\.ooyala\.com/[^"?]+[?#][^"]*?(?:embedCode|ec)=(?P<ec>[^"&]+)', webpage) or
  1468. re.search(r'OO\.Player\.create\([\'"].*?[\'"],\s*[\'"](?P<ec>.{32})[\'"]', webpage) or
  1469. re.search(r'SBN\.VideoLinkset\.ooyala\([\'"](?P<ec>.{32})[\'"]\)', webpage) or
  1470. re.search(r'data-ooyala-video-id\s*=\s*[\'"](?P<ec>.{32})[\'"]', webpage))
  1471. if mobj is not None:
  1472. return OoyalaIE._build_url_result(smuggle_url(mobj.group('ec'), {'domain': url}))
  1473. # Look for multiple Ooyala embeds on SBN network websites
  1474. mobj = re.search(r'SBN\.VideoLinkset\.entryGroup\((\[.*?\])', webpage)
  1475. if mobj is not None:
  1476. embeds = self._parse_json(mobj.group(1), video_id, fatal=False)
  1477. if embeds:
  1478. return _playlist_from_matches(
  1479. embeds, getter=lambda v: OoyalaIE._url_for_embed_code(smuggle_url(v['provider_video_id'], {'domain': url})), ie='Ooyala')
  1480. # Look for Aparat videos
  1481. mobj = re.search(r'<iframe .*?src="(http://www\.aparat\.com/video/[^"]+)"', webpage)
  1482. if mobj is not None:
  1483. return self.url_result(mobj.group(1), 'Aparat')
  1484. # Look for MPORA videos
  1485. mobj = re.search(r'<iframe .*?src="(http://mpora\.(?:com|de)/videos/[^"]+)"', webpage)
  1486. if mobj is not None:
  1487. return self.url_result(mobj.group(1), 'Mpora')
  1488. # Look for embedded NovaMov-based player
  1489. mobj = re.search(
  1490. r'''(?x)<(?:pagespeed_)?iframe[^>]+?src=(["\'])
  1491. (?P<url>http://(?:(?:embed|www)\.)?
  1492. (?:novamov\.com|
  1493. nowvideo\.(?:ch|sx|eu|at|ag|co)|
  1494. videoweed\.(?:es|com)|
  1495. movshare\.(?:net|sx|ag)|
  1496. divxstage\.(?:eu|net|ch|co|at|ag))
  1497. /embed\.php.+?)\1''', webpage)
  1498. if mobj is not None:
  1499. return self.url_result(mobj.group('url'))
  1500. # Look for embedded Facebook player
  1501. mobj = re.search(
  1502. r'<iframe[^>]+?src=(["\'])(?P<url>https://www\.facebook\.com/video/embed.+?)\1', webpage)
  1503. if mobj is not None:
  1504. return self.url_result(mobj.group('url'), 'Facebook')
  1505. # Look for embedded VK player
  1506. mobj = re.search(r'<iframe[^>]+?src=(["\'])(?P<url>https?://vk\.com/video_ext\.php.+?)\1', webpage)
  1507. if mobj is not None:
  1508. return self.url_result(mobj.group('url'), 'VK')
  1509. # Look for embedded ivi player
  1510. mobj = re.search(r'<embed[^>]+?src=(["\'])(?P<url>https?://(?:www\.)?ivi\.ru/video/player.+?)\1', webpage)
  1511. if mobj is not None:
  1512. return self.url_result(mobj.group('url'), 'Ivi')
  1513. # Look for embedded Huffington Post player
  1514. mobj = re.search(
  1515. r'<iframe[^>]+?src=(["\'])(?P<url>https?://embed\.live\.huffingtonpost\.com/.+?)\1', webpage)
  1516. if mobj is not None:
  1517. return self.url_result(mobj.group('url'), 'HuffPost')
  1518. # Look for embed.ly
  1519. mobj = re.search(r'class=["\']embedly-card["\'][^>]href=["\'](?P<url>[^"\']+)', webpage)
  1520. if mobj is not None:
  1521. return self.url_result(mobj.group('url'))
  1522. mobj = re.search(r'class=["\']embedly-embed["\'][^>]src=["\'][^"\']*url=(?P<url>[^&]+)', webpage)
  1523. if mobj is not None:
  1524. return self.url_result(compat_urllib_parse_unquote(mobj.group('url')))
  1525. # Look for funnyordie embed
  1526. matches = re.findall(r'<iframe[^>]+?src="(https?://(?:www\.)?funnyordie\.com/embed/[^"]+)"', webpage)
  1527. if matches:
  1528. return _playlist_from_matches(
  1529. matches, getter=unescapeHTML, ie='FunnyOrDie')
  1530. # Look for BBC iPlayer embed
  1531. matches = re.findall(r'setPlaylist\("(https?://www\.bbc\.co\.uk/iplayer/[^/]+/[\da-z]{8})"\)', webpage)
  1532. if matches:
  1533. return _playlist_from_matches(matches, ie='BBCCoUk')
  1534. # Look for embedded RUTV player
  1535. rutv_url = RUTVIE._extract_url(webpage)
  1536. if rutv_url:
  1537. return self.url_result(rutv_url, 'RUTV')
  1538. # Look for embedded TVC player
  1539. tvc_url = TVCIE._extract_url(webpage)
  1540. if tvc_url:
  1541. return self.url_result(tvc_url, 'TVC')
  1542. # Look for embedded SportBox player
  1543. sportbox_urls = SportBoxEmbedIE._extract_urls(webpage)
  1544. if sportbox_urls:
  1545. return _playlist_from_matches(sportbox_urls, ie='SportBoxEmbed')
  1546. # Look for embedded PornHub player
  1547. pornhub_url = PornHubIE._extract_url(webpage)
  1548. if pornhub_url:
  1549. return self.url_result(pornhub_url, 'PornHub')
  1550. # Look for embedded XHamster player
  1551. xhamster_urls = XHamsterEmbedIE._extract_urls(webpage)
  1552. if xhamster_urls:
  1553. return _playlist_from_matches(xhamster_urls, ie='XHamsterEmbed')
  1554. # Look for embedded Tvigle player
  1555. mobj = re.search(
  1556. r'<iframe[^>]+?src=(["\'])(?P<url>(?:https?:)?//cloud\.tvigle\.ru/video/.+?)\1', webpage)
  1557. if mobj is not None:
  1558. return self.url_result(mobj.group('url'), 'Tvigle')
  1559. # Look for embedded TED player
  1560. mobj = re.search(
  1561. r'<iframe[^>]+?src=(["\'])(?P<url>https?://embed(?:-ssl)?\.ted\.com/.+?)\1', webpage)
  1562. if mobj is not None:
  1563. return self.url_result(mobj.group('url'), 'TED')
  1564. # Look for embedded Ustream videos
  1565. mobj = re.search(
  1566. r'<iframe[^>]+?src=(["\'])(?P<url>http://www\.ustream\.tv/embed/.+?)\1', webpage)
  1567. if mobj is not None:
  1568. return self.url_result(mobj.group('url'), 'Ustream')
  1569. # Look for embedded arte.tv player
  1570. mobj = re.search(
  1571. r'<script [^>]*?src="(?P<url>http://www\.arte\.tv/playerv2/embed[^"]+)"',
  1572. webpage)
  1573. if mobj is not None:
  1574. return self.url_result(mobj.group('url'), 'ArteTVEmbed')
  1575. # Look for embedded francetv player
  1576. mobj = re.search(
  1577. r'<iframe[^>]+?src=(["\'])(?P<url>(?:https?://)?embed\.francetv\.fr/\?ue=.+?)\1',
  1578. webpage)
  1579. if mobj is not None:
  1580. return self.url_result(mobj.group('url'))
  1581. # Look for embedded smotri.com player
  1582. smotri_url = SmotriIE._extract_url(webpage)
  1583. if smotri_url:
  1584. return self.url_result(smotri_url, 'Smotri')
  1585. # Look for embedded Myvi.ru player
  1586. myvi_url = MyviIE._extract_url(webpage)
  1587. if myvi_url:
  1588. return self.url_result(myvi_url)
  1589. # Look for embedded soundcloud player
  1590. mobj = re.search(
  1591. r'<iframe\s+(?:[a-zA-Z0-9_-]+="[^"]+"\s+)*src="(?P<url>https?://(?:w\.)?soundcloud\.com/player[^"]+)"',
  1592. webpage)
  1593. if mobj is not None:
  1594. url = unescapeHTML(mobj.group('url'))
  1595. return self.url_result(url)
  1596. # Look for embedded vulture.com player
  1597. mobj = re.search(
  1598. r'<iframe src="(?P<url>https?://video\.vulture\.com/[^"]+)"',
  1599. webpage)
  1600. if mobj is not None:
  1601. url = unescapeHTML(mobj.group('url'))
  1602. return self.url_result(url, ie='Vulture')
  1603. # Look for embedded mtvservices player
  1604. mtvservices_url = MTVServicesEmbeddedIE._extract_url(webpage)
  1605. if mtvservices_url:
  1606. return self.url_result(mtvservices_url, ie='MTVServicesEmbedded')
  1607. # Look for embedded yahoo player
  1608. mobj = re.search(
  1609. r'<iframe[^>]+?src=(["\'])(?P<url>https?://(?:screen|movies)\.yahoo\.com/.+?\.html\?format=embed)\1',
  1610. webpage)
  1611. if mobj is not None:
  1612. return self.url_result(mobj.group('url'), 'Yahoo')
  1613. # Look for embedded sbs.com.au player
  1614. mobj = re.search(
  1615. r'''(?x)
  1616. (?:
  1617. <meta\s+property="og:video"\s+content=|
  1618. <iframe[^>]+?src=
  1619. )
  1620. (["\'])(?P<url>https?://(?:www\.)?sbs\.com\.au/ondemand/video/.+?)\1''',
  1621. webpage)
  1622. if mobj is not None:
  1623. return self.url_result(mobj.group('url'), 'SBS')
  1624. # Look for embedded Cinchcast player
  1625. mobj = re.search(
  1626. r'<iframe[^>]+?src=(["\'])(?P<url>https?://player\.cinchcast\.com/.+?)\1',
  1627. webpage)
  1628. if mobj is not None:
  1629. return self.url_result(mobj.group('url'), 'Cinchcast')
  1630. mobj = re.search(
  1631. r'<iframe[^>]+?src=(["\'])(?P<url>https?://m(?:lb)?\.mlb\.com/shared/video/embed/embed\.html\?.+?)\1',
  1632. webpage)
  1633. if not mobj:
  1634. mobj = re.search(
  1635. r'data-video-link=["\'](?P<url>http://m.mlb.com/video/[^"\']+)',
  1636. webpage)
  1637. if mobj is not None:
  1638. return self.url_result(mobj.group('url'), 'MLB')
  1639. mobj = re.search(
  1640. r'<(?:iframe|script)[^>]+?src=(["\'])(?P<url>%s)\1' % CondeNastIE.EMBED_URL,
  1641. webpage)
  1642. if mobj is not None:
  1643. return self.url_result(self._proto_relative_url(mobj.group('url'), scheme='http:'), 'CondeNast')
  1644. mobj = re.search(
  1645. r'<iframe[^>]+src="(?P<url>https?://new\.livestream\.com/[^"]+/player[^"]+)"',
  1646. webpage)
  1647. if mobj is not None:
  1648. return self.url_result(mobj.group('url'), 'Livestream')
  1649. # Look for Zapiks embed
  1650. mobj = re.search(
  1651. r'<iframe[^>]+src="(?P<url>https?://(?:www\.)?zapiks\.fr/index\.php\?.+?)"', webpage)
  1652. if mobj is not None:
  1653. return self.url_result(mobj.group('url'), 'Zapiks')
  1654. # Look for Kaltura embeds
  1655. mobj = (re.search(r"(?s)kWidget\.(?:thumb)?[Ee]mbed\(\{.*?'wid'\s*:\s*'_?(?P<partner_id>[^']+)',.*?'entry_?[Ii]d'\s*:\s*'(?P<id>[^']+)',", webpage) or
  1656. re.search(r'(?s)(?P<q1>["\'])(?:https?:)?//cdnapi(?:sec)?\.kaltura\.com/.*?(?:p|partner_id)/(?P<partner_id>\d+).*?(?P=q1).*?entry_?[Ii]d\s*:\s*(?P<q2>["\'])(?P<id>.+?)(?P=q2)', webpage))
  1657. if mobj is not None:
  1658. return self.url_result(smuggle_url(
  1659. 'kaltura:%(partner_id)s:%(id)s' % mobj.groupdict(),
  1660. {'source_url': url}), 'Kaltura')
  1661. # Look for Eagle.Platform embeds
  1662. mobj = re.search(
  1663. r'<iframe[^>]+src="(?P<url>https?://.+?\.media\.eagleplatform\.com/index/player\?.+?)"', webpage)
  1664. if mobj is not None:
  1665. return self.url_result(mobj.group('url'), 'EaglePlatform')
  1666. # Look for ClipYou (uses Eagle.Platform) embeds
  1667. mobj = re.search(
  1668. r'<iframe[^>]+src="https?://(?P<host>media\.clipyou\.ru)/index/player\?.*\brecord_id=(?P<id>\d+).*"', webpage)
  1669. if mobj is not None:
  1670. return self.url_result('eagleplatform:%(host)s:%(id)s' % mobj.groupdict(), 'EaglePlatform')
  1671. # Look for Pladform embeds
  1672. pladform_url = PladformIE._extract_url(webpage)
  1673. if pladform_url:
  1674. return self.url_result(pladform_url)
  1675. # Look for Videomore embeds
  1676. videomore_url = VideomoreIE._extract_url(webpage)
  1677. if videomore_url:
  1678. return self.url_result(videomore_url)
  1679. # Look for Playwire embeds
  1680. mobj = re.search(
  1681. r'<script[^>]+data-config=(["\'])(?P<url>(?:https?:)?//config\.playwire\.com/.+?)\1', webpage)
  1682. if mobj is not None:
  1683. return self.url_result(mobj.group('url'))
  1684. # Look for 5min embeds
  1685. mobj = re.search(
  1686. r'<meta[^>]+property="og:video"[^>]+content="https?://embed\.5min\.com/(?P<id>[0-9]+)/?', webpage)
  1687. if mobj is not None:
  1688. return self.url_result('5min:%s' % mobj.group('id'), 'FiveMin')
  1689. # Look for Crooks and Liars embeds
  1690. mobj = re.search(
  1691. r'<(?:iframe[^>]+src|param[^>]+value)=(["\'])(?P<url>(?:https?:)?//embed\.crooksandliars\.com/(?:embed|v)/.+?)\1', webpage)
  1692. if mobj is not None:
  1693. return self.url_result(mobj.group('url'))
  1694. # Look for NBC Sports VPlayer embeds
  1695. nbc_sports_url = NBCSportsVPlayerIE._extract_url(webpage)
  1696. if nbc_sports_url:
  1697. return self.url_result(nbc_sports_url, 'NBCSportsVPlayer')
  1698. # Look for Google Drive embeds
  1699. google_drive_url = GoogleDriveIE._extract_url(webpage)
  1700. if google_drive_url:
  1701. return self.url_result(google_drive_url, 'GoogleDrive')
  1702. # Look for UDN embeds
  1703. mobj = re.search(
  1704. r'<iframe[^>]+src="(?P<url>%s)"' % UDNEmbedIE._PROTOCOL_RELATIVE_VALID_URL, webpage)
  1705. if mobj is not None:
  1706. return self.url_result(
  1707. compat_urlparse.urljoin(url, mobj.group('url')), 'UDNEmbed')
  1708. # Look for Senate ISVP iframe
  1709. senate_isvp_url = SenateISVPIE._search_iframe_url(webpage)
  1710. if senate_isvp_url:
  1711. return self.url_result(senate_isvp_url, 'SenateISVP')
  1712. # Look for Dailymotion Cloud videos
  1713. dmcloud_url = DailymotionCloudIE._extract_dmcloud_url(webpage)
  1714. if dmcloud_url:
  1715. return self.url_result(dmcloud_url, 'DailymotionCloud')
  1716. # Look for OnionStudios embeds
  1717. onionstudios_url = OnionStudiosIE._extract_url(webpage)
  1718. if onionstudios_url:
  1719. return self.url_result(onionstudios_url)
  1720. # Look for SnagFilms embeds
  1721. snagfilms_url = SnagFilmsEmbedIE._extract_url(webpage)
  1722. if snagfilms_url:
  1723. return self.url_result(snagfilms_url)
  1724. # Look for JWPlatform embeds
  1725. jwplatform_url = JWPlatformIE._extract_url(webpage)
  1726. if jwplatform_url:
  1727. return self.url_result(jwplatform_url, 'JWPlatform')
  1728. # Look for ScreenwaveMedia embeds
  1729. mobj = re.search(ScreenwaveMediaIE.EMBED_PATTERN, webpage)
  1730. if mobj is not None:
  1731. return self.url_result(unescapeHTML(mobj.group('url')), 'ScreenwaveMedia')
  1732. # Look for Digiteka embeds
  1733. digiteka_url = DigitekaIE._extract_url(webpage)
  1734. if digiteka_url:
  1735. return self.url_result(self._proto_relative_url(digiteka_url), DigitekaIE.ie_key())
  1736. # Look for Limelight embeds
  1737. mobj = re.search(r'LimelightPlayer\.doLoad(Media|Channel|ChannelList)\(["\'](?P<id>[a-z0-9]{32})', webpage)
  1738. if mobj:
  1739. lm = {
  1740. 'Media': 'media',
  1741. 'Channel': 'channel',
  1742. 'ChannelList': 'channel_list',
  1743. }
  1744. return self.url_result('limelight:%s:%s' % (
  1745. lm[mobj.group(1)], mobj.group(2)), 'Limelight%s' % mobj.group(1), mobj.group(2))
  1746. # Look for AdobeTVVideo embeds
  1747. mobj = re.search(
  1748. r'<iframe[^>]+src=[\'"]((?:https?:)?//video\.tv\.adobe\.com/v/\d+[^"]+)[\'"]',
  1749. webpage)
  1750. if mobj is not None:
  1751. return self.url_result(
  1752. self._proto_relative_url(unescapeHTML(mobj.group(1))),
  1753. 'AdobeTVVideo')
  1754. def check_video(vurl):
  1755. if YoutubeIE.suitable(vurl):
  1756. return True
  1757. vpath = compat_urlparse.urlparse(vurl).path
  1758. vext = determine_ext(vpath)
  1759. return '.' in vpath and vext not in ('swf', 'png', 'jpg', 'srt', 'sbv', 'sub', 'vtt', 'ttml')
  1760. def filter_video(urls):
  1761. return list(filter(check_video, urls))
  1762. # Start with something easy: JW Player in SWFObject
  1763. found = filter_video(re.findall(r'flashvars: [\'"](?:.*&)?file=(http[^\'"&]*)', webpage))
  1764. if not found:
  1765. # Look for gorilla-vid style embedding
  1766. found = filter_video(re.findall(r'''(?sx)
  1767. (?:
  1768. jw_plugins|
  1769. JWPlayerOptions|
  1770. jwplayer\s*\(\s*["'][^'"]+["']\s*\)\s*\.setup
  1771. )
  1772. .*?
  1773. ['"]?file['"]?\s*:\s*["\'](.*?)["\']''', webpage))
  1774. if not found:
  1775. # Broaden the search a little bit
  1776. found = filter_video(re.findall(r'[^A-Za-z0-9]?(?:file|source)=(http[^\'"&]*)', webpage))
  1777. if not found:
  1778. # Broaden the findall a little bit: JWPlayer JS loader
  1779. found = filter_video(re.findall(
  1780. r'[^A-Za-z0-9]?(?:file|video_url)["\']?:\s*["\'](http(?![^\'"]+\.[0-9]+[\'"])[^\'"]+)["\']', webpage))
  1781. if not found:
  1782. # Flow player
  1783. found = filter_video(re.findall(r'''(?xs)
  1784. flowplayer\("[^"]+",\s*
  1785. \{[^}]+?\}\s*,
  1786. \s*\{[^}]+? ["']?clip["']?\s*:\s*\{\s*
  1787. ["']?url["']?\s*:\s*["']([^"']+)["']
  1788. ''', webpage))
  1789. if not found:
  1790. # Cinerama player
  1791. found = re.findall(
  1792. r"cinerama\.embedPlayer\(\s*\'[^']+\',\s*'([^']+)'", webpage)
  1793. if not found:
  1794. # Try to find twitter cards info
  1795. found = filter_video(re.findall(
  1796. r'<meta (?:property|name)="twitter:player:stream" (?:content|value)="(.+?)"', webpage))
  1797. if not found:
  1798. # We look for Open Graph info:
  1799. # We have to match any number spaces between elements, some sites try to align them (eg.: statigr.am)
  1800. m_video_type = re.findall(r'<meta.*?property="og:video:type".*?content="video/(.*?)"', webpage)
  1801. # We only look in og:video if the MIME type is a video, don't try if it's a Flash player:
  1802. if m_video_type is not None:
  1803. found = filter_video(re.findall(r'<meta.*?property="og:video".*?content="(.*?)"', webpage))
  1804. if not found:
  1805. # HTML5 video
  1806. found = re.findall(r'(?s)<(?:video|audio)[^<]*(?:>.*?<source[^>]*)?\s+src=["\'](.*?)["\']', webpage)
  1807. if not found:
  1808. REDIRECT_REGEX = r'[0-9]{,2};\s*(?:URL|url)=\'?([^\'"]+)'
  1809. found = re.search(
  1810. r'(?i)<meta\s+(?=(?:[a-z-]+="[^"]+"\s+)*http-equiv="refresh")'
  1811. r'(?:[a-z-]+="[^"]+"\s+)*?content="%s' % REDIRECT_REGEX,
  1812. webpage)
  1813. if not found:
  1814. # Look also in Refresh HTTP header
  1815. refresh_header = head_response.headers.get('Refresh')
  1816. if refresh_header:
  1817. # In python 2 response HTTP headers are bytestrings
  1818. if sys.version_info < (3, 0) and isinstance(refresh_header, str):
  1819. refresh_header = refresh_header.decode('iso-8859-1')
  1820. found = re.search(REDIRECT_REGEX, refresh_header)
  1821. if found:
  1822. new_url = compat_urlparse.urljoin(url, unescapeHTML(found.group(1)))
  1823. self.report_following_redirect(new_url)
  1824. return {
  1825. '_type': 'url',
  1826. 'url': new_url,
  1827. }
  1828. if not found:
  1829. raise UnsupportedError(url)
  1830. entries = []
  1831. for video_url in found:
  1832. video_url = video_url.replace('\\/', '/')
  1833. video_url = compat_urlparse.urljoin(url, video_url)
  1834. video_id = compat_urllib_parse_unquote(os.path.basename(video_url))
  1835. # Sometimes, jwplayer extraction will result in a YouTube URL
  1836. if YoutubeIE.suitable(video_url):
  1837. entries.append(self.url_result(video_url, 'Youtube'))
  1838. continue
  1839. # here's a fun little line of code for you:
  1840. video_id = os.path.splitext(video_id)[0]
  1841. entry_info_dict = {
  1842. 'id': video_id,
  1843. 'uploader': video_uploader,
  1844. 'title': video_title,
  1845. 'age_limit': age_limit,
  1846. }
  1847. ext = determine_ext(video_url)
  1848. if ext == 'smil':
  1849. entry_info_dict['formats'] = self._extract_smil_formats(video_url, video_id)
  1850. elif ext == 'xspf':
  1851. return self.playlist_result(self._extract_xspf_playlist(video_url, video_id), video_id)
  1852. elif ext == 'm3u8':
  1853. entry_info_dict['formats'] = self._extract_m3u8_formats(video_url, video_id, ext='mp4')
  1854. elif ext == 'mpd':
  1855. entry_info_dict['formats'] = self._extract_mpd_formats(video_url, video_id)
  1856. else:
  1857. entry_info_dict['url'] = video_url
  1858. entries.append(entry_info_dict)
  1859. if len(entries) == 1:
  1860. return entries[0]
  1861. else:
  1862. for num, e in enumerate(entries, start=1):
  1863. # 'url' results don't have a title
  1864. if e.get('title') is not None:
  1865. e['title'] = '%s (%d)' % (e['title'], num)
  1866. return {
  1867. '_type': 'playlist',
  1868. 'entries': entries,
  1869. }