You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

220 lines
7.4 KiB

Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
10 years ago
  1. # encoding: utf-8
  2. from __future__ import unicode_literals
  3. import base64
  4. import re
  5. import time
  6. from .common import InfoExtractor
  7. from ..utils import (
  8. ExtractorError,
  9. float_or_none,
  10. remove_end,
  11. sanitized_Request,
  12. std_headers,
  13. struct_unpack,
  14. )
  15. def _decrypt_url(png):
  16. encrypted_data = base64.b64decode(png.encode('utf-8'))
  17. text_index = encrypted_data.find(b'tEXt')
  18. text_chunk = encrypted_data[text_index - 4:]
  19. length = struct_unpack('!I', text_chunk[:4])[0]
  20. # Use bytearray to get integers when iterating in both python 2.x and 3.x
  21. data = bytearray(text_chunk[8:8 + length])
  22. data = [chr(b) for b in data if b != 0]
  23. hash_index = data.index('#')
  24. alphabet_data = data[:hash_index]
  25. url_data = data[hash_index + 1:]
  26. alphabet = []
  27. e = 0
  28. d = 0
  29. for l in alphabet_data:
  30. if d == 0:
  31. alphabet.append(l)
  32. d = e = (e + 1) % 4
  33. else:
  34. d -= 1
  35. url = ''
  36. f = 0
  37. e = 3
  38. b = 1
  39. for letter in url_data:
  40. if f == 0:
  41. l = int(letter) * 10
  42. f = 1
  43. else:
  44. if e == 0:
  45. l += int(letter)
  46. url += alphabet[l]
  47. e = (b + 3) % 4
  48. f = 0
  49. b += 1
  50. else:
  51. e -= 1
  52. return url
  53. class RTVEALaCartaIE(InfoExtractor):
  54. IE_NAME = 'rtve.es:alacarta'
  55. IE_DESC = 'RTVE a la carta'
  56. _VALID_URL = r'http://www\.rtve\.es/(m/)?alacarta/videos/[^/]+/[^/]+/(?P<id>\d+)'
  57. _TESTS = [{
  58. 'url': 'http://www.rtve.es/alacarta/videos/balonmano/o-swiss-cup-masculina-final-espana-suecia/2491869/',
  59. 'md5': '1d49b7e1ca7a7502c56a4bf1b60f1b43',
  60. 'info_dict': {
  61. 'id': '2491869',
  62. 'ext': 'mp4',
  63. 'title': 'Balonmano - Swiss Cup masculina. Final: España-Suecia',
  64. 'duration': 5024.566,
  65. },
  66. }, {
  67. 'note': 'Live stream',
  68. 'url': 'http://www.rtve.es/alacarta/videos/television/24h-live/1694255/',
  69. 'info_dict': {
  70. 'id': '1694255',
  71. 'ext': 'flv',
  72. 'title': 'TODO',
  73. },
  74. 'skip': 'The f4m manifest can\'t be used yet',
  75. }, {
  76. 'url': 'http://www.rtve.es/m/alacarta/videos/cuentame-como-paso/cuentame-como-paso-t16-ultimo-minuto-nuestra-vida-capitulo-276/2969138/?media=tve',
  77. 'only_matching': True,
  78. }]
  79. def _real_initialize(self):
  80. user_agent_b64 = base64.b64encode(std_headers['User-Agent'].encode('utf-8')).decode('utf-8')
  81. manager_info = self._download_json(
  82. 'http://www.rtve.es/odin/loki/' + user_agent_b64,
  83. None, 'Fetching manager info')
  84. self._manager = manager_info['manager']
  85. def _real_extract(self, url):
  86. mobj = re.match(self._VALID_URL, url)
  87. video_id = mobj.group('id')
  88. info = self._download_json(
  89. 'http://www.rtve.es/api/videos/%s/config/alacarta_videos.json' % video_id,
  90. video_id)['page']['items'][0]
  91. if info['state'] == 'DESPU':
  92. raise ExtractorError('The video is no longer available', expected=True)
  93. png_url = 'http://www.rtve.es/ztnr/movil/thumbnail/%s/videos/%s.png' % (self._manager, video_id)
  94. png_request = sanitized_Request(png_url)
  95. png_request.add_header('Referer', url)
  96. png = self._download_webpage(png_request, video_id, 'Downloading url information')
  97. video_url = _decrypt_url(png)
  98. if not video_url.endswith('.f4m'):
  99. video_url = video_url.replace(
  100. 'resources/', 'auth/resources/'
  101. ).replace('.net.rtve', '.multimedia.cdn.rtve')
  102. subtitles = None
  103. if info.get('sbtFile') is not None:
  104. subtitles = self.extract_subtitles(video_id, info['sbtFile'])
  105. return {
  106. 'id': video_id,
  107. 'title': info['title'],
  108. 'url': video_url,
  109. 'thumbnail': info.get('image'),
  110. 'page_url': url,
  111. 'subtitles': subtitles,
  112. 'duration': float_or_none(info.get('duration'), scale=1000),
  113. }
  114. def _get_subtitles(self, video_id, sub_file):
  115. subs = self._download_json(
  116. sub_file + '.json', video_id,
  117. 'Downloading subtitles info')['page']['items']
  118. return dict(
  119. (s['lang'], [{'ext': 'vtt', 'url': s['src']}])
  120. for s in subs)
  121. class RTVEInfantilIE(InfoExtractor):
  122. IE_NAME = 'rtve.es:infantil'
  123. IE_DESC = 'RTVE infantil'
  124. _VALID_URL = r'https?://(?:www\.)?rtve\.es/infantil/serie/(?P<show>[^/]*)/video/(?P<short_title>[^/]*)/(?P<id>[0-9]+)/'
  125. _TESTS = [{
  126. 'url': 'http://www.rtve.es/infantil/serie/cleo/video/maneras-vivir/3040283/',
  127. 'md5': '915319587b33720b8e0357caaa6617e6',
  128. 'info_dict': {
  129. 'id': '3040283',
  130. 'ext': 'mp4',
  131. 'title': 'Maneras de vivir',
  132. 'thumbnail': 'http://www.rtve.es/resources/jpg/6/5/1426182947956.JPG',
  133. 'duration': 357.958,
  134. },
  135. }]
  136. def _real_extract(self, url):
  137. video_id = self._match_id(url)
  138. info = self._download_json(
  139. 'http://www.rtve.es/api/videos/%s/config/alacarta_videos.json' % video_id,
  140. video_id)['page']['items'][0]
  141. webpage = self._download_webpage(url, video_id)
  142. vidplayer_id = self._search_regex(
  143. r' id="vidplayer([0-9]+)"', webpage, 'internal video ID')
  144. png_url = 'http://www.rtve.es/ztnr/movil/thumbnail/default/videos/%s.png' % vidplayer_id
  145. png = self._download_webpage(png_url, video_id, 'Downloading url information')
  146. video_url = _decrypt_url(png)
  147. return {
  148. 'id': video_id,
  149. 'ext': 'mp4',
  150. 'title': info['title'],
  151. 'url': video_url,
  152. 'thumbnail': info.get('image'),
  153. 'duration': float_or_none(info.get('duration'), scale=1000),
  154. }
  155. class RTVELiveIE(InfoExtractor):
  156. IE_NAME = 'rtve.es:live'
  157. IE_DESC = 'RTVE.es live streams'
  158. _VALID_URL = r'http://www\.rtve\.es/(?:deportes/directo|noticias|television)/(?P<id>[a-zA-Z0-9-]+)'
  159. _TESTS = [{
  160. 'url': 'http://www.rtve.es/noticias/directo-la-1/',
  161. 'info_dict': {
  162. 'id': 'directo-la-1',
  163. 'ext': 'flv',
  164. 'title': 're:^La 1 de TVE [0-9]{4}-[0-9]{2}-[0-9]{2}Z[0-9]{6}$',
  165. },
  166. 'params': {
  167. 'skip_download': 'live stream',
  168. }
  169. }]
  170. def _real_extract(self, url):
  171. mobj = re.match(self._VALID_URL, url)
  172. start_time = time.gmtime()
  173. video_id = mobj.group('id')
  174. webpage = self._download_webpage(url, video_id)
  175. player_url = self._search_regex(
  176. r'<param name="movie" value="([^"]+)"/>', webpage, 'player URL')
  177. title = remove_end(self._og_search_title(webpage), ' en directo')
  178. title += ' ' + time.strftime('%Y-%m-%dZ%H%M%S', start_time)
  179. vidplayer_id = self._search_regex(
  180. r' id="vidplayer([0-9]+)"', webpage, 'internal video ID')
  181. png_url = 'http://www.rtve.es/ztnr/movil/thumbnail/default/videos/%s.png' % vidplayer_id
  182. png = self._download_webpage(png_url, video_id, 'Downloading url information')
  183. video_url = _decrypt_url(png)
  184. return {
  185. 'id': video_id,
  186. 'ext': 'flv',
  187. 'title': title,
  188. 'url': video_url,
  189. 'app': 'rtve-live-live?ovpfv=2.1.2',
  190. 'player_url': player_url,
  191. 'rtmp_live': True,
  192. }