You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

199 lines
7.3 KiB

Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
Switch codebase to use sanitized_Request instead of compat_urllib_request.Request [downloader/dash] Use sanitized_Request [downloader/http] Use sanitized_Request [atresplayer] Use sanitized_Request [bambuser] Use sanitized_Request [bliptv] Use sanitized_Request [brightcove] Use sanitized_Request [cbs] Use sanitized_Request [ceskatelevize] Use sanitized_Request [collegerama] Use sanitized_Request [extractor/common] Use sanitized_Request [crunchyroll] Use sanitized_Request [dailymotion] Use sanitized_Request [dcn] Use sanitized_Request [dramafever] Use sanitized_Request [dumpert] Use sanitized_Request [eitb] Use sanitized_Request [escapist] Use sanitized_Request [everyonesmixtape] Use sanitized_Request [extremetube] Use sanitized_Request [facebook] Use sanitized_Request [fc2] Use sanitized_Request [flickr] Use sanitized_Request [4tube] Use sanitized_Request [gdcvault] Use sanitized_Request [extractor/generic] Use sanitized_Request [hearthisat] Use sanitized_Request [hotnewhiphop] Use sanitized_Request [hypem] Use sanitized_Request [iprima] Use sanitized_Request [ivi] Use sanitized_Request [keezmovies] Use sanitized_Request [letv] Use sanitized_Request [lynda] Use sanitized_Request [metacafe] Use sanitized_Request [minhateca] Use sanitized_Request [miomio] Use sanitized_Request [meovideo] Use sanitized_Request [mofosex] Use sanitized_Request [moniker] Use sanitized_Request [mooshare] Use sanitized_Request [movieclips] Use sanitized_Request [mtv] Use sanitized_Request [myvideo] Use sanitized_Request [neteasemusic] Use sanitized_Request [nfb] Use sanitized_Request [niconico] Use sanitized_Request [noco] Use sanitized_Request [nosvideo] Use sanitized_Request [novamov] Use sanitized_Request [nowness] Use sanitized_Request [nuvid] Use sanitized_Request [played] Use sanitized_Request [pluralsight] Use sanitized_Request [pornhub] Use sanitized_Request [pornotube] Use sanitized_Request [primesharetv] Use sanitized_Request [promptfile] Use sanitized_Request [qqmusic] Use sanitized_Request [rtve] Use sanitized_Request [safari] Use sanitized_Request [sandia] Use sanitized_Request [shared] Use sanitized_Request [sharesix] Use sanitized_Request [sina] Use sanitized_Request [smotri] Use sanitized_Request [sohu] Use sanitized_Request [spankwire] Use sanitized_Request [sportdeutschland] Use sanitized_Request [streamcloud] Use sanitized_Request [streamcz] Use sanitized_Request [tapely] Use sanitized_Request [tube8] Use sanitized_Request [tubitv] Use sanitized_Request [twitch] Use sanitized_Request [twitter] Use sanitized_Request [udemy] Use sanitized_Request [vbox7] Use sanitized_Request [veoh] Use sanitized_Request [vessel] Use sanitized_Request [vevo] Use sanitized_Request [viddler] Use sanitized_Request [videomega] Use sanitized_Request [viewvster] Use sanitized_Request [viki] Use sanitized_Request [vk] Use sanitized_Request [vodlocker] Use sanitized_Request [voicerepublic] Use sanitized_Request [wistia] Use sanitized_Request [xfileshare] Use sanitized_Request [xtube] Use sanitized_Request [xvideos] Use sanitized_Request [yandexmusic] Use sanitized_Request [youku] Use sanitized_Request [youporn] Use sanitized_Request [youtube] Use sanitized_Request [patreon] Use sanitized_Request [extractor/common] Remove unused import [nfb] PEP 8
9 years ago
  1. # encoding: utf-8
  2. from __future__ import unicode_literals
  3. import itertools
  4. from .amp import AMPIE
  5. from ..compat import (
  6. compat_HTTPError,
  7. compat_urlparse,
  8. )
  9. from ..utils import (
  10. ExtractorError,
  11. clean_html,
  12. int_or_none,
  13. sanitized_Request,
  14. urlencode_postdata
  15. )
  16. class DramaFeverBaseIE(AMPIE):
  17. _LOGIN_URL = 'https://www.dramafever.com/accounts/login/'
  18. _NETRC_MACHINE = 'dramafever'
  19. _CONSUMER_SECRET = 'DA59dtVXYLxajktV'
  20. _consumer_secret = None
  21. def _get_consumer_secret(self):
  22. mainjs = self._download_webpage(
  23. 'http://www.dramafever.com/static/51afe95/df2014/scripts/main.js',
  24. None, 'Downloading main.js', fatal=False)
  25. if not mainjs:
  26. return self._CONSUMER_SECRET
  27. return self._search_regex(
  28. r"var\s+cs\s*=\s*'([^']+)'", mainjs,
  29. 'consumer secret', default=self._CONSUMER_SECRET)
  30. def _real_initialize(self):
  31. self._login()
  32. self._consumer_secret = self._get_consumer_secret()
  33. def _login(self):
  34. (username, password) = self._get_login_info()
  35. if username is None:
  36. return
  37. login_form = {
  38. 'username': username,
  39. 'password': password,
  40. }
  41. request = sanitized_Request(
  42. self._LOGIN_URL, urlencode_postdata(login_form))
  43. response = self._download_webpage(
  44. request, None, 'Logging in as %s' % username)
  45. if all(logout_pattern not in response
  46. for logout_pattern in ['href="/accounts/logout/"', '>Log out<']):
  47. error = self._html_search_regex(
  48. r'(?s)class="hidden-xs prompt"[^>]*>(.+?)<',
  49. response, 'error message', default=None)
  50. if error:
  51. raise ExtractorError('Unable to login: %s' % error, expected=True)
  52. raise ExtractorError('Unable to log in')
  53. class DramaFeverIE(DramaFeverBaseIE):
  54. IE_NAME = 'dramafever'
  55. _VALID_URL = r'https?://(?:www\.)?dramafever\.com/drama/(?P<id>[0-9]+/[0-9]+)(?:/|$)'
  56. _TESTS = [{
  57. 'url': 'http://www.dramafever.com/drama/4512/1/Cooking_with_Shin/',
  58. 'info_dict': {
  59. 'id': '4512.1',
  60. 'ext': 'mp4',
  61. 'title': 'Cooking with Shin 4512.1',
  62. 'description': 'md5:a8eec7942e1664a6896fcd5e1287bfd0',
  63. 'episode': 'Episode 1',
  64. 'episode_number': 1,
  65. 'thumbnail': 're:^https?://.*\.jpg',
  66. 'timestamp': 1404336058,
  67. 'upload_date': '20140702',
  68. 'duration': 343,
  69. },
  70. 'params': {
  71. # m3u8 download
  72. 'skip_download': True,
  73. },
  74. }, {
  75. 'url': 'http://www.dramafever.com/drama/4826/4/Mnet_Asian_Music_Awards_2015/?ap=1',
  76. 'info_dict': {
  77. 'id': '4826.4',
  78. 'ext': 'mp4',
  79. 'title': 'Mnet Asian Music Awards 2015 4826.4',
  80. 'description': 'md5:3ff2ee8fedaef86e076791c909cf2e91',
  81. 'episode': 'Mnet Asian Music Awards 2015 - Part 3',
  82. 'episode_number': 4,
  83. 'thumbnail': 're:^https?://.*\.jpg',
  84. 'timestamp': 1450213200,
  85. 'upload_date': '20151215',
  86. 'duration': 5602,
  87. },
  88. 'params': {
  89. # m3u8 download
  90. 'skip_download': True,
  91. },
  92. }]
  93. def _real_extract(self, url):
  94. video_id = self._match_id(url).replace('/', '.')
  95. try:
  96. info = self._extract_feed_info(
  97. 'http://www.dramafever.com/amp/episode/feed.json?guid=%s' % video_id)
  98. except ExtractorError as e:
  99. if isinstance(e.cause, compat_HTTPError):
  100. raise ExtractorError(
  101. 'Currently unavailable in your country.', expected=True)
  102. raise
  103. series_id, episode_number = video_id.split('.')
  104. episode_info = self._download_json(
  105. # We only need a single episode info, so restricting page size to one episode
  106. # and dealing with page number as with episode number
  107. r'http://www.dramafever.com/api/4/episode/series/?cs=%s&series_id=%s&page_number=%s&page_size=1'
  108. % (self._consumer_secret, series_id, episode_number),
  109. video_id, 'Downloading episode info JSON', fatal=False)
  110. if episode_info:
  111. value = episode_info.get('value')
  112. if isinstance(value, list):
  113. for v in value:
  114. if v.get('type') == 'Episode':
  115. subfile = v.get('subfile') or v.get('new_subfile')
  116. if subfile and subfile != 'http://www.dramafever.com/st/':
  117. info.setdefault('subtitles', {}).setdefault('English', []).append({
  118. 'ext': 'srt',
  119. 'url': subfile,
  120. })
  121. episode_number = int_or_none(v.get('number'))
  122. episode_fallback = 'Episode'
  123. if episode_number:
  124. episode_fallback += ' %d' % episode_number
  125. info['episode'] = v.get('title') or episode_fallback
  126. info['episode_number'] = episode_number
  127. break
  128. return info
  129. class DramaFeverSeriesIE(DramaFeverBaseIE):
  130. IE_NAME = 'dramafever:series'
  131. _VALID_URL = r'https?://(?:www\.)?dramafever\.com/drama/(?P<id>[0-9]+)(?:/(?:(?!\d+(?:/|$)).+)?)?$'
  132. _TESTS = [{
  133. 'url': 'http://www.dramafever.com/drama/4512/Cooking_with_Shin/',
  134. 'info_dict': {
  135. 'id': '4512',
  136. 'title': 'Cooking with Shin',
  137. 'description': 'md5:84a3f26e3cdc3fb7f500211b3593b5c1',
  138. },
  139. 'playlist_count': 4,
  140. }, {
  141. 'url': 'http://www.dramafever.com/drama/124/IRIS/',
  142. 'info_dict': {
  143. 'id': '124',
  144. 'title': 'IRIS',
  145. 'description': 'md5:b3a30e587cf20c59bd1c01ec0ee1b862',
  146. },
  147. 'playlist_count': 20,
  148. }]
  149. _PAGE_SIZE = 60 # max is 60 (see http://api.drama9.com/#get--api-4-episode-series-)
  150. def _real_extract(self, url):
  151. series_id = self._match_id(url)
  152. series = self._download_json(
  153. 'http://www.dramafever.com/api/4/series/query/?cs=%s&series_id=%s'
  154. % (self._consumer_secret, series_id),
  155. series_id, 'Downloading series JSON')['series'][series_id]
  156. title = clean_html(series['name'])
  157. description = clean_html(series.get('description') or series.get('description_short'))
  158. entries = []
  159. for page_num in itertools.count(1):
  160. episodes = self._download_json(
  161. 'http://www.dramafever.com/api/4/episode/series/?cs=%s&series_id=%s&page_size=%d&page_number=%d'
  162. % (self._consumer_secret, series_id, self._PAGE_SIZE, page_num),
  163. series_id, 'Downloading episodes JSON page #%d' % page_num)
  164. for episode in episodes.get('value', []):
  165. episode_url = episode.get('episode_url')
  166. if not episode_url:
  167. continue
  168. entries.append(self.url_result(
  169. compat_urlparse.urljoin(url, episode_url),
  170. 'DramaFever', episode.get('guid')))
  171. if page_num == episodes['num_pages']:
  172. break
  173. return self.playlist_result(entries, series_id, title, description)