patreon-dl
A Patreon downloader written in Node.js.
Features
- Access to patron-only content through cookie
- Download posts by user, in a collection or single post
- Download products (aka shop purchases)
- Items included in downloads:
- videos
- images
- audio
- attachments
- Save campaign and content info
- Extensively configurable
You can run patreon-dl
from the command-line or use it as a library for your project. Node.js v16.16.0 or higher required.
Limitations
- Embedded videos, other than those from YouTube, are not supported (e.g. Vimeo). Only info about these embeds is saved.
- Likewise, embedded links are not followed; only info about the embed is saved.
FFmpeg dependency
FFmpeg is required when downloading:
- videos that are provided only in streaming format; and
- embedded YouTube videos.
Not all video downloads require FFmpeg, but you should have it installed on your system anyway.
Embedded YouTube videos - Premium access
patreon-dl
supports downloading embedded YouTube videos. In addition, if you have a YouTube Premium subscription, you can connect patreon-dl
to your account and download videos at qualities available only to Premium accounts (e.g. '1080p Premium'). For CLI users, you would configure patreon-dl
as follows:
$ patreon-dl --configure-youtube
For library usage, see Configuring YouTube connection.
Installation
npm i -g patreon-dl
The -g
option is for installing patreon-dl
globally and have the CLI executable added to the PATH. Depending on your usage, you might not need this.
CLI usage
$ patreon-dl [OPTION]... URL
OPTION
Option | Alias | Description |
---|
--help | -h | Display usage guide |
--config-file <path> | -C | Load configuration file at <path> for setting full options |
--cookie <string> | -c | Cookie for accessing patron-only content; how to obtain cookie. |
--ffmpeg <path> | -f | Path to FFmpeg executable |
--out-dir <path> | -o | Directory to save content |
--log-level <level> | -l | Log level of the console logger: info , debug , warn or error ; set to none to disable the logger. |
--no-prompt | -y | Do not prompt for confirmation to proceed |
--configure-youtube | | Configure YouTube connection. patreon-dl supports downloading embedded YouTube videos. If you have a YouTube Premium account, you can connect patreon-dl to it for downloading Premium-quality streams. |
URL
Supported URL formats
// Download a product
https://www.patreon.com/<creator>/shop/<slug>-<product_id>
// Download posts by creator
https://www.patreon.com/<creator>/posts
// Dowload a single post
https://www.patreon.com/posts/<slug>-<post_id>
// Download posts in a collection
https://www.patreon.com/collection/<collection_id>
Multiple URLs
You may specify multiple URLs by separating them with a comma. E.g.:
// First download posts by johndoe, followed by posts by janedoe.
$ patreon-dl "https://www.patreon.com/johndoe/posts,https://www.patreon.com/janedoe/posts"
Supplying URLs through file
You can also use a file to supply URLs to patreon-dl
. For example, you can have a urls.txt
that has the following content:
# Each URL is placed in its own line
# Comments (lines starting with '#') will be ignored
https://www.patreon.com/johndoe/posts
https://www.patreon.com/janedoe/posts
You can then pass urls.txt
to patreon-dl
:
$ patreon-dl urls.txt
Directory structure
Content is saved with the following directory structure:
out-dir
├── campaign
├── campaign_info
├── posts
│ ├── post 1
│ │ ├── post_info
│ │ ├── images
│ │ ├── ...
│ ├── post 2
│ ├── post_info
│ ├── images
│ ├── ...
├──shop
├── product 1
├── product_info
├── content_media
├── ...
Configuration file
Command-line options are limited. To access the full range of options, create a configuration file and pass it to patreon-dl
with the (capital) -C
option.
Refer to the example config to see what options are offered. Also see How to obtain Cookie.
Note that you can override an option from a configuration file with one provided at the command-line, provided of course that a command-line equivalent is available.
Library usage
To use patreon-dl
in your own project:
import PatreonDownloader from 'patreon-dl';
const url = '....';
const downloader = await PatreonDownloader.getInstance(url, [options]);
await downloader.start();
Here, we first obtain a downloader instance by calling PatreonDownloader.getInstance()
, passing to it the URL we want to download from (one of the supported URL formats) and downloader options, if any.
Then, we call start()
on the downloader instance to begin the download process. The start()
method returns a Promise that resolves when the download process has ended.
Downloader options
An object with the following properties (all optional):
Option | Description |
---|
cookie | Cookie to include in requests; required for accessing patron-only content. See How to obtain Cookie. |
useStatusCache | Whether to use status cache to quickly determine whether a target that had been downloaded before has changed since the last download. Default: true |
pathToFFmpeg | Path to ffmpeg executable. If not specified, ffmpeg will be called directly when needed, so make sure it is in the PATH. |
pathToYouTubeCredentials | Path to file storing YouTube credentials for connecting to a YouTube account when downloading embedded YouTube videos. Its purpose is to allow YouTube Premium accounts to download videos at higher than normal qualities. For more information, see Configuring YouTube connection. |
outDir | Path to directory where content is saved. Default: current working directory |
dirNameFormat | How to name directories: (object) |
filenameFormat | Naming of files: (object) |
include | What to include in the download: (object) lockedContent : whether to process locked content. Default: true postsWithMediaType : sets the media type criteria for downloading posts. Values can be:any : download posts regardless of the type of media they contain. Also applies to posts that do not contain any media.none : only download posts that do not contain media.- Array<
image | video | audio | attachment >: only download posts that contain the specified media type(s). Default: any campaignInfo : whether to save campaign info. Default: true contentInfo : whether to save content info. Default: true contentMedia : the type of content media to download (images, videos, audio, attachments, excluding previews). Values can be:true : download all content media.false : do not download content media.- Array<
image | video | audio | attachment | file >: only download the specified media type(s). Default: true previewMedia : the type of preview media to download, if available. Values can be:true : download all preview media.false : do not download preview media.- Array<
image | video | audio >: only download the specified media type(s). Default: true allMediaVariants : whether to download all media variants, if available. If false , only the best quality variant will be downloaded. Default: false
|
request | Rate limiting and retry on error: (object)maxRetries : maximum number of retries if a request or download fails. Default: 3maxConcurrent : maximum number of concurrent downloads. Default: 10minTime : minimum time to wait between starting requests or downloads (milliseconds). Default: 333
|
fileExistsAction | What to do when a target file already exists: (object)info : in the context of saving info (such as campaign or post info), the action to take when a file belonging to the info already exists. Default: saveAsCopyIfNewer infoAPI : API data is saved as part of info. Because it changes frequently, and usually used for debugging purpose only, you can set a different action when saving an API data file that already exists. Default: overwrite content : in the context of downloading content, the action to take when a file belonging to the content already exists. Default: skip
Supported actions: overwrite : overwrite existing file.skip : skip saving the file.saveAsCopy : save the file under incremented filename (e.g. "abc.jpg" becomes "abc (1).jpg").saveAsCopyIfNewer : like saveAsCopy , but only do so if the contents have actually changed.
|
logger | See Logger |
Campaign directory name format
Format to apply when naming campaign directories. A format is a string pattern consisting of fields enclosed in curly braces.
What is a campaign directory?
When you download content, a directory is created for the campaign that hosts the content. Content directories, which stores the downloaded content, are then placed under the campaign directory.
If campaign info could not be obtained from content, then content directory
will be created directly under outDir
.
A format must contain at least one of the following fields:
creator.vanity
creator.name
creator.id
campaign.name
campaign.id
Characters enclosed in square brackets followed by a question mark denote
conditional separators. If the value of a field could not be obtained or
is empty, the conditional separator immediately adjacent to it will be
omitted from the name.
Default: '{creator.vanity}[ - ]?{campaign.name}'
Fallback: 'campaign-{campaign.id}'
Content directory name format
Format to apply when naming content directories. A format is a string pattern consisting of fields enclosed in curly braces.
What is a content directory?
Content can be a post or product. A directory is created for each piece of content. Downloaded items for the content are placed under this directory.
A format must contain at least one of the following unique identifier fields:
content.id
: ID of contentcontent.slug
: last segment of the content URL
In addition, a format can contain the following fields:
content.name
: post title or product namecontent.type
: type of content ('product' or 'post')content.publishDate
: publish date (ISO UTC format)
Characters enclosed in square brackets followed by a question mark denote conditional separators. If the value of a field could not be obtained or is empty, the conditional separator immediately adjacent to it will be omitted from the name.
Default: '{content.id}[ - ]?{content.name}'
Fallback: '{content.type}-{content.id}'
Media filename format
Filename format of a downloaded item. A format is a string pattern consisting of fields enclosed in curly braces.
A format must contain at least one of the following fields:
media.id
: ID of the item downloaded (assigned by Patreon)media.filename
: can be one of the following, in order of availability:
- original filename included in the item's API data; or
- filename derived from the header of the response to the HTTP download request.
In addition, a format can contain the following fields:
media.type
: type of item (e.g. 'image' or 'video')media.variant
: where applicable, the variant of the item (e.g. 'original', 'thumbnailSmall'...for images)
If media.variant
is not included in the format, it will be appended to it if allMediaVariants
is true
.
Sometimes media.filename
could not be obtained, in which case it will be replaced with media.id
, unless it is already present in the format.
Characters enclosed in square brackets followed by a question mark denote conditional separators. If the value of a field could not be obtained or is empty, the conditional separator immediately adjacent to it will be omitted from the name.
Default: '{media.filename}'
Fallback: '{media.type}-{media.id}'
Configuring YouTube connection
In its simplest form, the process of connecting patreon-dl
to a YouTube account is as follows:
- Obtain credentials by having the user visit a Google page that links his or her account to a 'device' (which in this case is actually
patreon-dl
). - Save the credentials, as a JSON string, to a file.
- Pass the path of the file to
PatreonDownloader.getInstance()
To obtain credentials, you can use the YouTubeCredentialsCapturer
class:
import { YouTubeCredentialsCapturer } from 'patreon-dl';
// Note: you should wrap the following logic inside an async
// process, and resolve when the credentials have been saved.
const capturer = new YouTubeCredentialsCapturer();
/**
* 'pending' event emitted when verification data is ready and waiting
* for user to carry out the verification process.
*/
capturer.on('pending', (data) => {
// `data` is an object: { verificationURL: <string>, code: <string> }
// Use `data` to provide instructions to the user:
console.log(
`In a browser, go to the following Verification URL and enter Code:
- Verification URL: ${data.verificationURL}
- Code: ${data.code}
Then wait for this script to complete.`);
});
/**
* 'capture' event emitted when the user has completed verification and the
* credentials have been relayed back to the capturer.
*/
capturer.on('capture', (credentials) => {
// `credentials` is an object which you need to save to file as JSON string.
fs.writeFileSync('/path/to/yt-credentials.json', JSON.stringify(credentials));
console.log('Credentials saved!');
});
// When you have added the listeners, start the capture process.
capturer.begin();
Then, pass the path of the file to PatreonDownloader.getInstance()
:
const downloader = await PatreonDownloader.getInstance(url, {
...
pathToYouTubeCredentials: '/path/to/yt-credentials.json'
});
You should ensure the credentials file is writable, as it needs to be updated with new credentials when the current ones expire. The process of renewing credentials is done automatically by the downloader.
Logger
Logging is optional, but provides useful information about the download process. You can implement your own logger by extending the Logger
abstract class:
import { Logger } from 'patreon-dl';
class MyLogger extends Logger {
log(entry) {
// Do something with log entry
}
// Called when downloader ends, so you can
// clean up the logger process if necessary.
end() {
// This is not an abstract function, so you don't have to
// implement it if there is no action to be taken here. Default is
// to resolve right away.
return Promise.resolve();
}
}
Each entry passed to log()
is an object with the following properties:
level
: info
, debug
, warn
or error
, indicating the severity of the log message.originator
: (string or undefined
) where the message is coming from.message
: array of elements comprising the message. An element can be anything such as a string, Error or object.
Built-in loggers
The patreon-dl
library comes with the following Logger
implementations that you may utilize:
-
ConsoleLogger
Outputs messages to the console:
import { ConsoleLogger } from 'patreon-dl';
const myLogger = new ConsoleLogger([options]);
const downloader = await PatreonDownloader.getInstance(url, {
...
logger: myLogger
});
options
: (object)
Option | Description |
---|
enabled | Whether to enable this logger. Default: true |
logLevel | info , debug , warn or error . Default: info
Output messages up to the specified severity level. |
include | What to include in log messages: (object)dateTime : show date / time of log messages. Default: true level : show the severity level. Default: true originator : show where the messsage came from. Default: true errorStack : for Errors, whether to show the full error stack. Default: false
|
dateTimeFormat | The pattern to format data-time strings, when include.dateTime is true . Date-time formatting is provided by dateformat library. Refer to the README of that project for pattern rules. Default: 'mmm dd HH:MM:ss' |
-
FileLogger
Like ConsoleLogger
, but writes messages to file.
import { FileLogger } from 'patreon-dl';
const myLogger = new FileLogger(init, [options]);
const downloader = await PatreonDownloader.getInstance(url, {
...
logger: myLogger
});
init
: values that determine the name of the log file (object)
Property | Description |
---|
targetURL | The url passed to PatreonDownloader.getInstance() |
outDir | Value of outDir specified in PatreonDownloader.getInstance() options, or undefined if none specified (in which case defaults to current working directory). |
date | (optional) Date instance representing the creation date / time of the logger. Default: current date-time You might want to provide this if you are creating multiple FileLogger instances and filenames are to be formatted with the date, otherwise the date-time part of the filenames might have different values. |
options
: all ConsoleLogger
options plus the following:
Option | Description |
---|
logDir | Path to directory of the log file. The path can be a string pattern consisting of the following fields enclosed in curly braces: out.dir : value of outDir provided in init (or the default current working directory if none provided).target.url.path : the pathname of targetURL provided in init , sanitized as necessary.datetime.<date-time format> : the date-time of logger creation, as represented by date in init and formatted according to <date-time format> (using pattern rules defined by the dateformat library).
|
logFilename | Name of the log file. The path can be a string pattern consisting of the following fields enclosed in curly braces: target.url.path : the pathname of targetURL provided in init , sanitized as necessary.datetime.<date-time format> : the date-time of logger creation, as represented by date in init and formatted according to <date-time format> (using pattern rules defined by the dateformat library).
Default: '{datetime.yyyymmdd}-{log.level}.log' |
fileExistsAction | What to do if log file already exists? One of the following values: append : append logs to existing fileoverwrite : overwrite the existing file
Default: append |
-
ChainLogger
Combines multiple loggers into one single logger.
import { ConsoleLogger, FileLogger, ChainLogger } from 'patreon-dl';
const consoleLogger = new ConsoleLogger(...);
const fileLogger = new FileLogger(...);
const chainLogger = new ChainLogger([ consoleLogger, fileLogger ]);
const downloader = await PatreonDownloader.getInstance(url, {
...
logger: chainLogger
});
Aborting
To prematurely end a download process, use AbortController
to send an abort signal to the downloader instance.
const downloader = await PatreonDownloader.getInstance(...);
const abortController = new AbortController();
downloader.start({
signal: abortController.signal
});
...
abortController.abort();
// Downloader aborts current and pending tasks, then ends.
Workflow and Events
Workflow
- Downloader analyzes given URL and determines what targets to fetch.
- Downloader begins fetching data from Patreon servers. Emits
fetchBegin
event. - Downloader obtains the target(s) from the fetched data for downloading.
- For each target (which can be a campaign, product or post):
- Downloader emits
targetBegin
event. - Downloader determines whether the target needs to be downloaded, based on downloader configuration and target info such as accessibility.
- If target is to be skipped, downloader emits
targetEnd
event with isSkipped: true
. It then proceeds to the next target, if any.
- If target is to be downloaded, downloader saves target info (subject to downloader configuration), and emits
phaseBegin
event with phase: saveInfo
. When done, downloader emits phaseEnd
event. - Downloader begins saving media belonging to target (again, subject to downloader configuration). Emits
phaseBegin
event with phase: saveMedia
.
- Downloader saves files that do not need to be downloaded, e.g. embedded video / link info.
- Downloader proceeds to download files (images, videos, audio, attachments, etc.) belonging to the target in batches. For each batch, downloader emits
phaseBegin
event with phase: batchDownload
. When done, downloader emits phaseEnd
event with phase: batchDownload
.
- In this
phaseBegin
event, you can attach listeners to the download batch to monitor events for each download. See Download Task Batch.
- Downloader emits
phaseEnd
event with phase: saveMedia
. - Downloader emits
targetEnd
event with isSkipped: false
, and proceeds to the next target.
- When there are no more targets to be processed, or a fatal error occurred, downloader ends with
end
event.
Events
const downloader = await PatreonDownloader.getInstance(...);
downloader.on('fetchBegin', (payload) => {
...
});
downloader.start();
Each event emitted by a PatreonDownloader
instance has a payload, which is an object with properties containing information about the event.
Event | Description |
---|
fetchBegin | Emitted when downloader begins fetching data about target(s). Payload properties: targetType : the type of target being fetched; one of product , post or post .
|
targetBegin | Emitted when downloader begins processing a target. Payload properties: |
targetEnd | Emitted when downloader is done processing a target. Payload properties: target : the target processed; one of Campaign, Product or Post.isSkipped : whether target was skipped.
If isSkipped is true , the following additional properties are available: skipReason : the reason for skipping the target; one of the following enums:TargetSkipReason.Inaccessible TargetSkipReason.AlreadyDownloaded TargetSkipReason.UnmetMediaTypeCriteria
skipMessage : description of the skip reason.
|
phaseBegin | Emitted when downloader begins a phase in the processing of a target. Payload properties: target : the subject target of the phase; one of Campaign, Product or Post.phase : the phase that is about to begin; one of saveInfo or batchDownload .
If phase is batchDownload , the following additional property is available: batch : an object representing the batch of downloads to be executed by the downloader. For monitoring downloads in the batch, see Download Task Batch.
|
phaseEnd | Emitted when a phase ends for a target. Payload properties: target : the subject target of the phase; one of Campaign, Product or Post.phase : the phase that has ended; one of saveInfo or batchDownload .
|
end | Emitted when downloader ends. Payload properties: aborted : boolean indicating whether the downloader is ending because of an abort requesterror : if downloader ends because of an error, then error will be the captured error. Note that error is not necessarily an Error object; it can be anything other than undefined .
|
Download Task Batch
Files are downloaded in batches. Each batch is provided in the payload of phaseBegin
event with phase: batchDownload
. You can monitor events of individual downloads in the batch as follows:
downloader.on('phaseBegin', (payload) => {
if (payload.phase === 'batchDownload') {
const batch = payload.batch;
batch.on(event, listener);
}
})
Note that you don't have to remove listeners yourself. They will be removed once the batch ends and is destroyed by the downloader.
Download Task
Each download task in a batch is represented by an object with the following properties:
Property | Description |
---|
id | ID assigned to the task. |
src | The source of the download; URL or otherwise file path if downloading video from a previously-downloaded m3u8 playlist. |
srcEntity | The Downloadable item from which the download task was created. |
retryCount | The current retry count if download failed previously. |
resolvedDestFilename | The resolved destination filename of the download, or null if it has not yet been resolved. |
resolvedDestFilename | The resolved destination file path of the download, or null if it has not yet been reoslved. |
getProgress() | Function that returns the download progress. |
Events
Each event emitted by a download task batch has a payload, which is an object with properties containing information about the event.
Event | Description |
---|
taskStart | Emitted when a download starts. Payload properties: |
taskProgress | Emitted when a download progress is updated. Payload properties: task : the download taskprogress : (object)destFilename : the destination filename of the downloaddestFilePath : the destination file path of the downloadlengthUnit : the unit measuring the progress. Generally, it would be 'byte', but for videos the unit would be 'second'.length : length downloaded, measured in lengthUnit .percent : percent downloadedsizeDownloaded : size of file downloaded (kb)speed : download speed (kb/s)
|
taskComplete | Emitted when a download is complete. Payload properties: |
taskError | Emitted when a download error occurs. Payload properties: error : (object)task : the download taskcause : Error object or undefined
willRetry : whether the download will be reattempted
|
taskAbort | Emitted when a download is aborted. Payload properties: |
taskSkip | Emitted when a download is skipped. Payload properties: task : the download taskreason : (object)name : destFileExists or other message : string indicating the skip reason
If reason.name is destFileExists , reason will also contain the following property: existingDestFilePath : the existing file path that is causing the download to skip
|
taskSpawn | Emitted when a download task is spawned from another task. Payload properties: origin : the original download taskspawn : the spawned download task
|
complete | Emitted when the batch is complete and there are no more downloads pending. Payload properties: none |
Changelog
v1.3.0
- Add support for multiple target URLs
- Add
content.publishDate
field to the content dir name format (PR #12 by kazuoteramoto) - Bug fixes
v1.2.2
- Fix wrong file extension for some content types
- Fix YouTube API requests throwing errors due to YT changes
v1.2.1
v1.2.0
- Add support for granular control over:
- posts to include in download based on type of media contained
- the type of media to download
- Bug fixes
v1.1.1
- Fix initial data parsing following Patreon changes
v1.1.0
- Add support for downloading embedded YouTube videos
v1.0.1
- Fix missing types when importing as library
- Fix link in this README
v1.0.0