Class VideoScraperCoreAbstract

The VideoScraperCore class, that can be extended to scrape a video from a website and saves it to a file.

Hierarchy

  • VideoScraperCore

Constructors

constructor

Properties

browser fullScreenSelector logger options playButtonSelector videoDurationSelector

Methods

afterPageLoaded close getFullScreenSelector getPlayButtonSelector getVideoDuration getVideoDurationSelector handleDurationText launch playVideo scrape setBrowserOptions setVideoToFullScreen

Constructors

constructor

  • The constructor of the VideoScraperCore class.

    Parameters

    • options: { browserExecutablePath?: string; debug?: boolean; debugScope?: null | string; windowSize?: { width?: number | undefined; height?: number | undefined; } } = {}

      The BrowserOptions to pass to the instance.

      • Optional browserExecutablePath?: string

        The path to the browser executable. (Default value: '/usr/bin/google-chrome')

      • Optional debug?: boolean

        If true, it will show debug log. (Default value: false)

      • Optional debugScope?: null | string

        The scope given to the euberlog debug logger. (Default value: 'VideoScraperCore')

      • Optional windowSize?: { width?: number | undefined; height?: number | undefined; }

        The object that says how big the window size will be. (Default value: { width: 1920, height: 1080 })

    Returns VideoScraperCore

Properties

Private browser

browser: any = null

Private Readonly fullScreenSelector

fullScreenSelector: string

Private logger

logger: Logger

Private options

options: InternalBrowserOptions

Private Readonly playButtonSelector

playButtonSelector: string

Private Readonly videoDurationSelector

videoDurationSelector: string

Methods

Protected Abstract afterPageLoaded

  • This method is called after the page, with the specified url, is loaded. It can be used for things such as logging in if it is requested before reaching the video page.

    Parameters

    • options: { audio?: boolean; audioBitsPerSecond?: number; debug?: null | boolean; debugScope?: null | string; delayAfterVideoFinished?: number; delayAfterVideoStarted?: number; duration?: null | number; frameSize?: number; fullScreen?: boolean; mimeType?: BrowserMimeType; useGlobalDebug?: boolean; video?: boolean; videoBitsPerSecond?: number }

      The ScrapingOptions that can be used or changed in this method

      • Optional audio?: boolean

        If true, the audio will be recorded. (Default value: true)

      • Optional audioBitsPerSecond?: number

        The chosen bitrate for the audio component of the media. If not specified, it will be adaptive, depending upon the sample rate and the number of channels. (Default value: undefined)

      • Optional debug?: null | boolean

        If null, the debug will be shown by looking at the passed BrowserOptions. Otherwise, if useGlobalDebug is false, this specifies if the debug will be shown.

      • Optional debugScope?: null | string

        If useGlobalDebug is true, this will be ignore. Otherwise, this specifies if the euberlog logger scope for the debug of this scrape.

      • Optional delayAfterVideoFinished?: number

        The delay in milliseconds after that the duration milliseconds are past and before that the recording is stopped. (Default value: 15_000)

      • Optional delayAfterVideoStarted?: number

        The delay in milliseconds after that the play button has been clicked. (Default value: 0)

      • Optional duration?: null | number

        The duration in milliseconds of the recorded video. The recording starts delayAfterVideoStarted milliseconds after that the play button has been clicked and finishes delayAfterVideoFinished milliseconds after that this duration has passed. If the value is null, the duration will be automatically gotten by looking at the page's video duration. (Default value: null)

      • Optional frameSize?: number

        The number of milliseconds to record into each packet. (Default value: 20)

      • Optional fullScreen?: boolean

        If true, the video will be recorded after having put it on fullscreen. (Default value: false)

      • Optional mimeType?: BrowserMimeType

        The mimetype of the recorded video or audio. (Default value: 'video/webm')

      • Optional useGlobalDebug?: boolean

        If true, the global logger will be used, ignoring other debug options in this object. (Default value: true)

      • Optional video?: boolean

        If true, the video will be recorded. (Default value: true)

      • Optional videoBitsPerSecond?: number

        The chosen bitrate for the video component of the media. If not specified, the rate will be 2.5Mbps. (Default value: undefined)

    • page: Page

      The page (puppeteer) that is loaded

    • logger: Logger

      A logger instance to write log

    Returns Promise<void>

close

  • Closes the browser window.

    Returns Promise<void>

Protected Abstract getFullScreenSelector

  • Returns the video full screen selector, which is used by the method setVideoToFullScreen to put the video in full screen.

    Returns string

Protected Abstract getPlayButtonSelector

  • Returns the video play button selector, which is used by the method playVideo to play the video.

    Returns string

Protected getVideoDuration

  • Gets the video duration by parsing the given page. This method can be overridden in case the page's way to display time is not compatible.

    Returns

    The extracted video duration

    Parameters

    • page: Page

      A browser page (of puppeteer)

    • logger: Logger

      A logger instance to write log

    Returns Promise<number>

Protected Abstract getVideoDurationSelector

  • Returns the video duration selector, which is used by the method getVideoDuration to extract the video duration text from the page.

    Returns string

Protected handleDurationText

  • Given the duration text gotten from the page's HTML (e.g. 1:30:23), it returns the duration in milliseconds. This method can be overridden in case the page's time format is not compatible.

    Returns

    The duration in milliseconds.

    Parameters

    • durationText: string

      The duration text gotten from the page's HTML.

    Returns number

launch

  • Launches the browser window.

    Returns Promise<void>

Protected playVideo

  • Plays the video by clicking the play button. This method can be overridden in case the page's way to start the video is not compatible.

    Parameters

    • page: Page

      A browser page (of puppeteer)

    • logger: Logger

      A logger instance to write log

    Returns Promise<void>

scrape

  • Scrapes a video.

    Parameters

    • url: string

      The url of the video to save

    • destPath: string

      The path where the video will be saved. Note that the extension should be webm.

    • options: { audio?: boolean; audioBitsPerSecond?: number; debug?: null | boolean; debugScope?: null | string; delayAfterVideoFinished?: number; delayAfterVideoStarted?: number; duration?: null | number; frameSize?: number; fullScreen?: boolean; mimeType?: BrowserMimeType; useGlobalDebug?: boolean; video?: boolean; videoBitsPerSecond?: number } = {}

      The ScrapingOptions to pass to this method.

      • Optional audio?: boolean

        If true, the audio will be recorded. (Default value: true)

      • Optional audioBitsPerSecond?: number

        The chosen bitrate for the audio component of the media. If not specified, it will be adaptive, depending upon the sample rate and the number of channels. (Default value: undefined)

      • Optional debug?: null | boolean

        If null, the debug will be shown by looking at the passed BrowserOptions. Otherwise, if useGlobalDebug is false, this specifies if the debug will be shown.

      • Optional debugScope?: null | string

        If useGlobalDebug is true, this will be ignore. Otherwise, this specifies if the euberlog logger scope for the debug of this scrape.

      • Optional delayAfterVideoFinished?: number

        The delay in milliseconds after that the duration milliseconds are past and before that the recording is stopped. (Default value: 15_000)

      • Optional delayAfterVideoStarted?: number

        The delay in milliseconds after that the play button has been clicked. (Default value: 0)

      • Optional duration?: null | number

        The duration in milliseconds of the recorded video. The recording starts delayAfterVideoStarted milliseconds after that the play button has been clicked and finishes delayAfterVideoFinished milliseconds after that this duration has passed. If the value is null, the duration will be automatically gotten by looking at the page's video duration. (Default value: null)

      • Optional frameSize?: number

        The number of milliseconds to record into each packet. (Default value: 20)

      • Optional fullScreen?: boolean

        If true, the video will be recorded after having put it on fullscreen. (Default value: false)

      • Optional mimeType?: BrowserMimeType

        The mimetype of the recorded video or audio. (Default value: 'video/webm')

      • Optional useGlobalDebug?: boolean

        If true, the global logger will be used, ignoring other debug options in this object. (Default value: true)

      • Optional video?: boolean

        If true, the video will be recorded. (Default value: true)

      • Optional videoBitsPerSecond?: number

        The chosen bitrate for the video component of the media. If not specified, the rate will be 2.5Mbps. (Default value: undefined)

    Returns Promise<void>

setBrowserOptions

  • Changes the BrowserOptions options.

    Parameters

    • options: { browserExecutablePath?: string; debug?: boolean; debugScope?: null | string; windowSize?: { width?: number | undefined; height?: number | undefined; } }

      The new options.

      • Optional browserExecutablePath?: string

        The path to the browser executable. (Default value: '/usr/bin/google-chrome')

      • Optional debug?: boolean

        If true, it will show debug log. (Default value: false)

      • Optional debugScope?: null | string

        The scope given to the euberlog debug logger. (Default value: 'VideoScraperCore')

      • Optional windowSize?: { width?: number | undefined; height?: number | undefined; }

        The object that says how big the window size will be. (Default value: { width: 1920, height: 1080 })

    Returns void

Protected setVideoToFullScreen

  • Sets the video put the video in fullscreen. This method can be overridden in case the page's way to put the video in fullscreen is not compatible.

    Parameters

    • page: Page

      A browser page (of puppeteer)

    • logger: Logger

      A logger instance to write log

    Returns Promise<void>

Settings

Member Visibility

Theme

Generated using TypeDoc