You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

427 lines
16 KiB

10 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
10 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
10 years ago
  1. # coding: utf-8
  2. from __future__ import unicode_literals
  3. import re
  4. import json
  5. import itertools
  6. from .common import InfoExtractor
  7. from ..compat import compat_str
  8. from ..utils import (
  9. ExtractorError,
  10. determine_ext,
  11. int_or_none,
  12. parse_iso8601,
  13. sanitized_Request,
  14. str_to_int,
  15. unescapeHTML,
  16. )
  17. class DailymotionBaseInfoExtractor(InfoExtractor):
  18. @staticmethod
  19. def _build_request(url):
  20. """Build a request with the family filter disabled"""
  21. request = sanitized_Request(url)
  22. request.add_header('Cookie', 'family_filter=off; ff=off')
  23. return request
  24. def _download_webpage_handle_no_ff(self, url, *args, **kwargs):
  25. request = self._build_request(url)
  26. return self._download_webpage_handle(request, *args, **kwargs)
  27. def _download_webpage_no_ff(self, url, *args, **kwargs):
  28. request = self._build_request(url)
  29. return self._download_webpage(request, *args, **kwargs)
  30. class DailymotionIE(DailymotionBaseInfoExtractor):
  31. _VALID_URL = r'(?i)(?:https?://)?(?:(www|touch)\.)?dailymotion\.[a-z]{2,3}/(?:(embed|#)/)?video/(?P<id>[^/?_]+)'
  32. IE_NAME = 'dailymotion'
  33. _FORMATS = [
  34. ('stream_h264_ld_url', 'ld'),
  35. ('stream_h264_url', 'standard'),
  36. ('stream_h264_hq_url', 'hq'),
  37. ('stream_h264_hd_url', 'hd'),
  38. ('stream_h264_hd1080_url', 'hd180'),
  39. ]
  40. _TESTS = [
  41. {
  42. 'url': 'https://www.dailymotion.com/video/x2iuewm_steam-machine-models-pricing-listed-on-steam-store-ign-news_videogames',
  43. 'md5': '2137c41a8e78554bb09225b8eb322406',
  44. 'info_dict': {
  45. 'id': 'x2iuewm',
  46. 'ext': 'mp4',
  47. 'title': 'Steam Machine Models, Pricing Listed on Steam Store - IGN News',
  48. 'description': 'Several come bundled with the Steam Controller.',
  49. 'thumbnail': 're:^https?:.*\.(?:jpg|png)$',
  50. 'duration': 74,
  51. 'timestamp': 1425657362,
  52. 'upload_date': '20150306',
  53. 'uploader': 'IGN',
  54. 'uploader_id': 'xijv66',
  55. 'age_limit': 0,
  56. 'view_count': int,
  57. 'comment_count': int,
  58. }
  59. },
  60. # Vevo video
  61. {
  62. 'url': 'http://www.dailymotion.com/video/x149uew_katy-perry-roar-official_musi',
  63. 'info_dict': {
  64. 'title': 'Roar (Official)',
  65. 'id': 'USUV71301934',
  66. 'ext': 'mp4',
  67. 'uploader': 'Katy Perry',
  68. 'upload_date': '20130905',
  69. },
  70. 'params': {
  71. 'skip_download': True,
  72. },
  73. 'skip': 'VEVO is only available in some countries',
  74. },
  75. # age-restricted video
  76. {
  77. 'url': 'http://www.dailymotion.com/video/xyh2zz_leanna-decker-cyber-girl-of-the-year-desires-nude-playboy-plus_redband',
  78. 'md5': '0d667a7b9cebecc3c89ee93099c4159d',
  79. 'info_dict': {
  80. 'id': 'xyh2zz',
  81. 'ext': 'mp4',
  82. 'title': 'Leanna Decker - Cyber Girl Of The Year Desires Nude [Playboy Plus]',
  83. 'uploader': 'HotWaves1012',
  84. 'age_limit': 18,
  85. }
  86. },
  87. # geo-restricted, player v5
  88. {
  89. 'url': 'http://www.dailymotion.com/video/xhza0o',
  90. 'only_matching': True,
  91. },
  92. # with subtitles
  93. {
  94. 'url': 'http://www.dailymotion.com/video/x20su5f_the-power-of-nightmares-1-the-rise-of-the-politics-of-fear-bbc-2004_news',
  95. 'only_matching': True,
  96. }
  97. ]
  98. def _real_extract(self, url):
  99. video_id = self._match_id(url)
  100. webpage = self._download_webpage_no_ff(
  101. 'https://www.dailymotion.com/video/%s' % video_id, video_id)
  102. age_limit = self._rta_search(webpage)
  103. description = self._og_search_description(webpage) or self._html_search_meta(
  104. 'description', webpage, 'description')
  105. view_count = str_to_int(self._search_regex(
  106. [r'<meta[^>]+itemprop="interactionCount"[^>]+content="UserPlays:(\d+)"',
  107. r'video_views_count[^>]+>\s+([\d\.,]+)'],
  108. webpage, 'view count', fatal=False))
  109. comment_count = int_or_none(self._search_regex(
  110. r'<meta[^>]+itemprop="interactionCount"[^>]+content="UserComments:(\d+)"',
  111. webpage, 'comment count', fatal=False))
  112. player_v5 = self._search_regex(
  113. [r'buildPlayer\(({.+?})\);\n', # See https://github.com/rg3/youtube-dl/issues/7826
  114. r'playerV5\s*=\s*dmp\.create\([^,]+?,\s*({.+?})\);',
  115. r'buildPlayer\(({.+?})\);'],
  116. webpage, 'player v5', default=None)
  117. if player_v5:
  118. player = self._parse_json(player_v5, video_id)
  119. metadata = player['metadata']
  120. self._check_error(metadata)
  121. formats = []
  122. for quality, media_list in metadata['qualities'].items():
  123. for media in media_list:
  124. media_url = media.get('url')
  125. if not media_url:
  126. continue
  127. type_ = media.get('type')
  128. if type_ == 'application/vnd.lumberjack.manifest':
  129. continue
  130. ext = determine_ext(media_url)
  131. if type_ == 'application/x-mpegURL' or ext == 'm3u8':
  132. m3u8_formats = self._extract_m3u8_formats(
  133. media_url, video_id, 'mp4', m3u8_id='hls', fatal=False)
  134. if m3u8_formats:
  135. formats.extend(m3u8_formats)
  136. elif type_ == 'application/f4m' or ext == 'f4m':
  137. f4m_formats = self._extract_f4m_formats(
  138. media_url, video_id, preference=-1, f4m_id='hds', fatal=False)
  139. if f4m_formats:
  140. formats.extend(f4m_formats)
  141. else:
  142. f = {
  143. 'url': media_url,
  144. 'format_id': quality,
  145. }
  146. m = re.search(r'H264-(?P<width>\d+)x(?P<height>\d+)', media_url)
  147. if m:
  148. f.update({
  149. 'width': int(m.group('width')),
  150. 'height': int(m.group('height')),
  151. })
  152. formats.append(f)
  153. self._sort_formats(formats)
  154. title = metadata['title']
  155. duration = int_or_none(metadata.get('duration'))
  156. timestamp = int_or_none(metadata.get('created_time'))
  157. thumbnail = metadata.get('poster_url')
  158. uploader = metadata.get('owner', {}).get('screenname')
  159. uploader_id = metadata.get('owner', {}).get('id')
  160. subtitles = {}
  161. subtitles_data = metadata.get('subtitles', {}).get('data', {})
  162. if subtitles_data and isinstance(subtitles_data, dict):
  163. for subtitle_lang, subtitle in subtitles_data.items():
  164. subtitles[subtitle_lang] = [{
  165. 'ext': determine_ext(subtitle_url),
  166. 'url': subtitle_url,
  167. } for subtitle_url in subtitle.get('urls', [])]
  168. return {
  169. 'id': video_id,
  170. 'title': title,
  171. 'description': description,
  172. 'thumbnail': thumbnail,
  173. 'duration': duration,
  174. 'timestamp': timestamp,
  175. 'uploader': uploader,
  176. 'uploader_id': uploader_id,
  177. 'age_limit': age_limit,
  178. 'view_count': view_count,
  179. 'comment_count': comment_count,
  180. 'formats': formats,
  181. 'subtitles': subtitles,
  182. }
  183. # vevo embed
  184. vevo_id = self._search_regex(
  185. r'<link rel="video_src" href="[^"]*?vevo.com[^"]*?video=(?P<id>[\w]*)',
  186. webpage, 'vevo embed', default=None)
  187. if vevo_id:
  188. return self.url_result('vevo:%s' % vevo_id, 'Vevo')
  189. # fallback old player
  190. embed_page = self._download_webpage_no_ff(
  191. 'https://www.dailymotion.com/embed/video/%s' % video_id,
  192. video_id, 'Downloading embed page')
  193. timestamp = parse_iso8601(self._html_search_meta(
  194. 'video:release_date', webpage, 'upload date'))
  195. info = self._parse_json(
  196. self._search_regex(
  197. r'var info = ({.*?}),$', embed_page,
  198. 'video info', flags=re.MULTILINE),
  199. video_id)
  200. self._check_error(info)
  201. formats = []
  202. for (key, format_id) in self._FORMATS:
  203. video_url = info.get(key)
  204. if video_url is not None:
  205. m_size = re.search(r'H264-(\d+)x(\d+)', video_url)
  206. if m_size is not None:
  207. width, height = map(int_or_none, (m_size.group(1), m_size.group(2)))
  208. else:
  209. width, height = None, None
  210. formats.append({
  211. 'url': video_url,
  212. 'ext': 'mp4',
  213. 'format_id': format_id,
  214. 'width': width,
  215. 'height': height,
  216. })
  217. self._sort_formats(formats)
  218. # subtitles
  219. video_subtitles = self.extract_subtitles(video_id, webpage)
  220. title = self._og_search_title(webpage, default=None)
  221. if title is None:
  222. title = self._html_search_regex(
  223. r'(?s)<span\s+id="video_title"[^>]*>(.*?)</span>', webpage,
  224. 'title')
  225. return {
  226. 'id': video_id,
  227. 'formats': formats,
  228. 'uploader': info['owner.screenname'],
  229. 'timestamp': timestamp,
  230. 'title': title,
  231. 'description': description,
  232. 'subtitles': video_subtitles,
  233. 'thumbnail': info['thumbnail_url'],
  234. 'age_limit': age_limit,
  235. 'view_count': view_count,
  236. 'duration': info['duration']
  237. }
  238. def _check_error(self, info):
  239. if info.get('error') is not None:
  240. raise ExtractorError(
  241. '%s said: %s' % (self.IE_NAME, info['error']['title']), expected=True)
  242. def _get_subtitles(self, video_id, webpage):
  243. try:
  244. sub_list = self._download_webpage(
  245. 'https://api.dailymotion.com/video/%s/subtitles?fields=id,language,url' % video_id,
  246. video_id, note=False)
  247. except ExtractorError as err:
  248. self._downloader.report_warning('unable to download video subtitles: %s' % compat_str(err))
  249. return {}
  250. info = json.loads(sub_list)
  251. if (info['total'] > 0):
  252. sub_lang_list = dict((l['language'], [{'url': l['url'], 'ext': 'srt'}]) for l in info['list'])
  253. return sub_lang_list
  254. self._downloader.report_warning('video doesn\'t have subtitles')
  255. return {}
  256. class DailymotionPlaylistIE(DailymotionBaseInfoExtractor):
  257. IE_NAME = 'dailymotion:playlist'
  258. _VALID_URL = r'(?:https?://)?(?:www\.)?dailymotion\.[a-z]{2,3}/playlist/(?P<id>.+?)/'
  259. _MORE_PAGES_INDICATOR = r'(?s)<div class="pages[^"]*">.*?<a\s+class="[^"]*?icon-arrow_right[^"]*?"'
  260. _PAGE_TEMPLATE = 'https://www.dailymotion.com/playlist/%s/%s'
  261. _TESTS = [{
  262. 'url': 'http://www.dailymotion.com/playlist/xv4bw_nqtv_sport/1#video=xl8v3q',
  263. 'info_dict': {
  264. 'title': 'SPORT',
  265. 'id': 'xv4bw_nqtv_sport',
  266. },
  267. 'playlist_mincount': 20,
  268. }]
  269. def _extract_entries(self, id):
  270. video_ids = set()
  271. processed_urls = set()
  272. for pagenum in itertools.count(1):
  273. page_url = self._PAGE_TEMPLATE % (id, pagenum)
  274. webpage, urlh = self._download_webpage_handle_no_ff(
  275. page_url, id, 'Downloading page %s' % pagenum)
  276. if urlh.geturl() in processed_urls:
  277. self.report_warning('Stopped at duplicated page %s, which is the same as %s' % (
  278. page_url, urlh.geturl()), id)
  279. break
  280. processed_urls.add(urlh.geturl())
  281. for video_id in re.findall(r'data-xid="(.+?)"', webpage):
  282. if video_id not in video_ids:
  283. yield self.url_result('http://www.dailymotion.com/video/%s' % video_id, 'Dailymotion')
  284. video_ids.add(video_id)
  285. if re.search(self._MORE_PAGES_INDICATOR, webpage) is None:
  286. break
  287. def _real_extract(self, url):
  288. mobj = re.match(self._VALID_URL, url)
  289. playlist_id = mobj.group('id')
  290. webpage = self._download_webpage(url, playlist_id)
  291. return {
  292. '_type': 'playlist',
  293. 'id': playlist_id,
  294. 'title': self._og_search_title(webpage),
  295. 'entries': self._extract_entries(playlist_id),
  296. }
  297. class DailymotionUserIE(DailymotionPlaylistIE):
  298. IE_NAME = 'dailymotion:user'
  299. _VALID_URL = r'https?://(?:www\.)?dailymotion\.[a-z]{2,3}/(?!(?:embed|#|video|playlist)/)(?:(?:old/)?user/)?(?P<user>[^/]+)'
  300. _PAGE_TEMPLATE = 'http://www.dailymotion.com/user/%s/%s'
  301. _TESTS = [{
  302. 'url': 'https://www.dailymotion.com/user/nqtv',
  303. 'info_dict': {
  304. 'id': 'nqtv',
  305. 'title': 'Rémi Gaillard',
  306. },
  307. 'playlist_mincount': 100,
  308. }, {
  309. 'url': 'http://www.dailymotion.com/user/UnderProject',
  310. 'info_dict': {
  311. 'id': 'UnderProject',
  312. 'title': 'UnderProject',
  313. },
  314. 'playlist_mincount': 1800,
  315. 'expected_warnings': [
  316. 'Stopped at duplicated page',
  317. ],
  318. 'skip': 'Takes too long time',
  319. }]
  320. def _real_extract(self, url):
  321. mobj = re.match(self._VALID_URL, url)
  322. user = mobj.group('user')
  323. webpage = self._download_webpage(
  324. 'https://www.dailymotion.com/user/%s' % user, user)
  325. full_user = unescapeHTML(self._html_search_regex(
  326. r'<a class="nav-image" title="([^"]+)" href="/%s">' % re.escape(user),
  327. webpage, 'user'))
  328. return {
  329. '_type': 'playlist',
  330. 'id': user,
  331. 'title': full_user,
  332. 'entries': self._extract_entries(user),
  333. }
  334. class DailymotionCloudIE(DailymotionBaseInfoExtractor):
  335. _VALID_URL_PREFIX = r'http://api\.dmcloud\.net/(?:player/)?embed/'
  336. _VALID_URL = r'%s[^/]+/(?P<id>[^/?]+)' % _VALID_URL_PREFIX
  337. _VALID_EMBED_URL = r'%s[^/]+/[^\'"]+' % _VALID_URL_PREFIX
  338. _TESTS = [{
  339. # From http://www.francetvinfo.fr/economie/entreprises/les-entreprises-familiales-le-secret-de-la-reussite_933271.html
  340. # Tested at FranceTvInfo_2
  341. 'url': 'http://api.dmcloud.net/embed/4e7343f894a6f677b10006b4/556e03339473995ee145930c?auth=1464865870-0-jyhsm84b-ead4c701fb750cf9367bf4447167a3db&autoplay=1',
  342. 'only_matching': True,
  343. }, {
  344. # http://www.francetvinfo.fr/societe/larguez-les-amarres-le-cobaturage-se-developpe_980101.html
  345. 'url': 'http://api.dmcloud.net/player/embed/4e7343f894a6f677b10006b4/559545469473996d31429f06?auth=1467430263-0-90tglw2l-a3a4b64ed41efe48d7fccad85b8b8fda&autoplay=1',
  346. 'only_matching': True,
  347. }]
  348. @classmethod
  349. def _extract_dmcloud_url(self, webpage):
  350. mobj = re.search(r'<iframe[^>]+src=[\'"](%s)[\'"]' % self._VALID_EMBED_URL, webpage)
  351. if mobj:
  352. return mobj.group(1)
  353. mobj = re.search(
  354. r'<input[^>]+id=[\'"]dmcloudUrlEmissionSelect[\'"][^>]+value=[\'"](%s)[\'"]' % self._VALID_EMBED_URL,
  355. webpage)
  356. if mobj:
  357. return mobj.group(1)
  358. def _real_extract(self, url):
  359. video_id = self._match_id(url)
  360. webpage = self._download_webpage_no_ff(url, video_id)
  361. title = self._html_search_regex(r'<title>([^>]+)</title>', webpage, 'title')
  362. video_info = self._parse_json(self._search_regex(
  363. r'var\s+info\s*=\s*([^;]+);', webpage, 'video info'), video_id)
  364. # TODO: parse ios_url, which is in fact a manifest
  365. video_url = video_info['mp4_url']
  366. return {
  367. 'id': video_id,
  368. 'url': video_url,
  369. 'title': title,
  370. 'thumbnail': video_info.get('thumbnail_url'),
  371. }