Security News
Cloudflare Adds Security.txt Setup Wizard
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
mwn is a modern MediaWiki bot framework in NodeJS, orginally adapted from mwbot.
Development status: Unstable. Versioning: while mwn is in version 0, changes may be made to the public interface with a change in the minor version number.
Documentation given below is incomplete. There are a number of additional classes such as bot.title
, bot.wikitext
, bot.page
, etc that provide useful functionality but aren't documented. You can learn about these by looking through the source code.
Amongst the major highlights are batchOperation
and seriesBatchOperation
which allow you run a large number of tasks with control over concurrency and sleep time between tasks. Failing actions can be automatically retried.
To install, run npm install mwn
.
Or obtain the latest development copy:
git clone https://github.com/siddharthvp/mwn.git
cd mwn
npm install # install dependencies
mwn is written with Node.js 13 in hand. While everything may still work in older versions of Node, you can consider upgrading to Node.js 13. If your bot is hosted on Toolforge, you can install the latest node.js in your home directory, using:
npm install npm@latest # update npm first to the latest version
npm install n
export N_PREFIX=~
./node_modules/n/bin/n latest
export PATH=~/bin:$PATH
Check that your .profile
or .bashrc
file includes the line PATH="$HOME/bin:$PATH"
, so that the path includes your home directory every time you open the shell.
mwn is written for and tested on the latest version of MediaWiki used on WMF wikis.
mwn supports authentication via both BotPasswords and via OAuth. Use of OAuth is recommended as it does away the need for separate API requests for logging in, and is also a bit more secure.
Bot passwords may be a bit easier to set up. To generate one, go to the wiki's Special:BotPasswords page.
If you're migrating from mwbot, note that:
edit
in mwbot is different from edit
in mwn. You want to use save
instead.Importing mwn:
In JavaScript:
const {mwn} = require('mwn');
Note: Prior to mwn v0.8.0, import was via const mwn = require('mwn');
In TypeScript:
import {mwn} from 'mwn';
Create a new bot instance:
const bot = new mwn({
apiUrl: 'https://en.wikipedia.org/w/api.php',
username: 'YourBotUsername',
password: 'YourBotPassword'
});
await bot.login();
Or to use OAuth:
const bot = new mwn({
apiUrl: 'https://en.wikipedia.org/w/api.php',
oauth_consumer_token: "16_DIGIT_ALPHANUMERIC_KEY",
oauth_consumer_secret: "20_DIGIT_ALPHANUMERIC_KEY",
oauth_access_token: "16_DIGIT_ALPHANUMERIC_KEY",
oauth_access_secret: "20_DIGIT_ALPHANUMERIC_KEY"
});
bot.initOAuth(); // does not involve an API call
// Any errors in authentication will surface when the first actual API call is made
A more complete constructor syntax:
const bot = new mwn({
apiUrl: 'https://en.wikipedia.org/w/api.php',
username: 'YourBotUsername',
password: 'YourBotPassword',
userAgent: 'myCoolToolName 1.0 ([[link to bot user page or tool documentation]])',
defaultParams: {
assert: 'user' // API parameter to ensure we're logged in
}
});
Set default parameters to be sent to be included in every API request:
bot.setDefaultParams({
assert: 'bot',
maxlag: 4 // mwn default is 5
});
Set bot options. The default values for each is specified below:
bot.setOptions({
silent: false, // suppress messages (except error messages)
maxlagPause: 5000, // pause for 5000 milliseconds (5 seconds) on maxlag error.
maxlagMaxRetries: 3, // attempt to retry a request failing due to maxlag upto 3 times
apiUrl: null // set the API URL, can also be set by a bot.setApiUrl
});
Maxlag: The default maxlag parameter used by mwn is 5 seconds. Requests failing due to maxlag will be automatically retried after pausing for a duration specified by maxlagPause
(default 5 seconds). A maximum of maxlagMaxRetries
will take place (default 3).
Fetch an CSRF token required for most write operations.
bot.getCsrfToken();
The token, once obtained is stored in the bot state so that it can be reused any number of times.
If an action fails due to an expired or missing token, the action will be automatically retried after fetching a new token.
For convenience, you can log in and get the edit token together as:
bot.loginGetToken();
If your bot doesn't need to log in, you can simply set the API url using:
bot.setApiUrl('https://en.wikipedia.org/w/api.php');
Set your user agent (required for WMF wikis):
bot.setUserAgent('myCoolToolName v1.0 ([[w:en:User:Example]])/mwn');
Edit a page. Edit conflicts are raised as errors.
bot.edit('Page title', rev => {
// rev.content gives the revision text
// rev.timestamp gives the revision timestamp
var text = rev.content.replace(/foo/g, 'bar');
return { // return parameters needed for [[mw:API:Edit]]
text: text,
summary: 'replacing foo with bar',
minor: true
};
});
Save a page with the given content without loading it first. Simpler verion of edit
. Does not offer any edit conflict detection.
bot.save('Page title', 'Page content', 'Edit summary');
Create a new page.
bot.create('Page title', 'Page content', 'Edit summary');
Post a new section to a talk page:
bot.newSection('Page title', 'New section header', 'Section content', additionalOptions);
Read the contents of a page:
bot.read('Page title');
Read a page along with metadata:
bot.read('Page title', {
rvprop: ['content', 'timestamp', 'user', 'comment']
});
Read multiple pages using a single API call:
bot.read(['Page 1', 'Page 2', 'Page 3']).then(pages => {
// pages[0], pages[1], pages[2]
});
Delete a page:
bot.delete('Page title', 'deletion log summary', additionalOptions);
Restore all deleted versions:
bot.undelete('Page title', 'log summary', additionalOptions);
Move a page along with its subpages:
bot.move('Old page title', 'New page title', 'move summary', {
movesubpages: true,
movetalk: true
});
Parse wikitext (see API:Parse for additionalOptions)
bot.parseWikitext('Input wikitext', additonalOptions);
Parse the contents of a given page
bot.parseTitle('Page name', additionalOptions);
Upload a file from your system to the wiki:
bot.upload('File title', '/path/to/file', 'comment', customParams);
Directly query the API. See mw:API for options. You can create and test your queries in the API sandbox
. Be sure to set formatversion: 2 in the options for format=json!
Example: get all images used on the article Foo
bot.request({
"action": "query",
"prop": "images",
"titles": "Foo"
}).then(data => {
return data.query.pages[0].images.map(im => im.title);
});
Send an API query, and continue re-sending it with the continue parameters received in the response, until there are no more results (or till maxCalls
limit is reached). The return value is a promise resolved with the array of responses to individual API calls.
bot.continousQuery(apiQueryObject, maxCalls=10)
Example: get a list of all active users on the wiki using continuedQuery
(using API:Allusers):
bot.continuedQuery({
"action": "query",
"list": "allusers",
"auactiveusers": 1,
"aulimit": "max"
}, 40).then(jsons => {
return jsons.reduce((activeusers, json) => {
return activeusers.concat(json.query.allusers.map(user => user.name));
}, []);
});
A simpler way is to use bot.continuedQueryGen
which is an asynchronous generator.
var activeusers = [];
for await (let json of bot.continuedQueryGen({
"action": "query",
"list": "allusers",
"auactiveusers": 1,
"aulimit": "max"
})) {
let users = json.query.allusers.map(user => user.name);
activeusers = activeusers.concat(users);
}
Use of continuedQueryGen
is recommended for several reasons:
continuedQueryGen
gets the result of each API call and processes them one at a time.MediaWiki sets a limit of 500 (50 for non-bots) on the number of pages that can be queried in a single API call. To query more than that, the massQuery
function can be used, which splits the page list into batches of 500 and sends individual queries and returns a promise resolved with the array of all individual API call responses.
Example: get the protection status of a large number of pages:
bot.massQuery({
"action": "query",
"format": "json",
"prop": "info",
"titles": ['Page1', 'Page2', ... , 'Page1300'], // array of page names
"inprop": "protection"
}) // 2nd parameter is taken as 'titles' by default
.then(jsons => {
// jsons is the array of individual JSON responses.
});
The 3rd parameter hasApiHighLimit
is set true
by default. If you get the API error 'toomanyvalues' (or similar), your account doesn't have the required user right, so set the parameter as false
.
Any errors in the individual API calls will not cause the entire massQuery to fail, but the data at the array index corresponding to that API call will be error object.
Perform asynchronous tasks (involving API usage) over a number of pages (or other arbitrary items). batchOperation
uses a default concurrency of 5. Customise this according to how expensive the API operation is. Higher concurrency limits could lead to more frequent API errors.
The workerFunction
must return a promise.
bot.batchOperation(pageList, (page, idx) => {
// do something with each page
// the index of the page in pageList is available as the 2nd argument
// return a promise in the end
}, 5, 2); // set the concurrency as the third parameter, number of retries as 4th parameter
Perform asynchronous tasks (involving API usage) over a number of pages one after the other, with a sleep duration between each task (default 5 seconds)
The workerFunction
must return a promise.
bot.seriesBatchOperation(pageList, (page, idx) => {
// do something with each page
// the index of the page in pageList is available as the 2nd argument
// return a promise in the end
}, 5000, 2); // set the sleep duration in milliseconds as the third parameter, max number of retries for each action is set as the 4th parameter
Note that seriesBatchOperation
with delay=0 is same as batchOperation
with concurrency=1.
FAQs
JavaScript & TypeScript MediaWiki bot framework for Node.js
The npm package mwn receives a total of 267 weekly downloads. As such, mwn popularity was classified as not popular.
We found that mwn demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Cloudflare has launched a setup wizard allowing users to easily create and manage a security.txt file for vulnerability disclosure on their websites.
Security News
The Socket Research team breaks down a malicious npm package targeting the legitimate DOMPurify library. It uses obfuscated code to hide that it is exfiltrating browser and crypto wallet data.
Security News
ENISA’s 2024 report highlights the EU’s top cybersecurity threats, including rising DDoS attacks, ransomware, supply chain vulnerabilities, and weaponized AI.