MWoffliner

MWoffliner is a tool for creating a local offline HTML snapshot of any online MediaWiki instance. It scrapes all pages (or a selection if specified) and creates the corresponding ZIM file. While primarily targeted for Wikimedia projects like Wikipedia and Wiktionary, MWoffliner also supports any recent MediaWiki instance (version 1.27+), though instances with custom skins or highly unusual configurations may have limitations.

Read CONTRIBUTING.md to learn more about MWoffliner development.

User help is available in the FAQ.

Features

Scrape with or without image thumbnails
Scrape with or without audio/video multimedia content
S3 cache (optional)
Image size optimization and WebP conversion
Scrape all pages in namespaces or title list based
Specify additional/non-main namespaces to scrape

Run mwoffliner --help to see all available options.

Quick Start

Prerequisites

Docker (or Docker-based engine)
amd64 or arm64 architecture

To turn the Bambara Wikipedia into an offline ZIM:

mkdir -p output
docker run -v $(pwd)/output:/output ghcr.io/openzim/mwoffliner \
  mwoffliner --mwUrl=https://bm.wikipedia.org --adminEmail=you@example.com \
  --outputDirectory=/output

NOTE: In order to avoid 429 responses, be sure to use an actual email address as dummy addresses like you@example.com will be flagged by the Wiki servers.

That's it. When it finishes you'll find a .zim file in the output/ directory. Open it with Kiwix Reader.

The Four Decisions

Every scrape comes down to four choices:

1. Which wiki?

--mwUrl=https://en.wikipedia.org        # English Wikipedia
--mwUrl=https://fr.wiktionary.org       # French Wiktionary
--mwUrl=https://terraria.wiki.gg        # Terraria Wiki
--mwUrl=https://proofwiki.org           # ProofWiki
--mwUrl=https://wiki.my-org.com         # Any MediaWiki site

2. Which API path?

Most MediaWiki sites use the default API path /w/api.php, but many don't. Check by visiting <wiki-url>/w/api.php in your browser. If that returns a 404, you need to set --mwActionApiPath to the correct path.

Tip

Try https://terraria.wiki.gg/api.php to see a wiki where --mwActionApiPath=/api.php is required.

3. What content?

If you want…	Add this
Everything (text, images, audio, video)	(nothing — this is the default)
Everything except video and audio	`--format=novid:maxi`
No pictures, no video, no audio	`--format=nopic:nopic`
Head paragraphs only, no media at all	`--format=nodet,nopic:mini`

4. Where does it go?

The ZIM is written to /output inside the container. Map that to a folder on your machine:

docker run -v /path/on/my/machine:/output ghcr.io/openzim/mwoffliner ...

NOTE: --adminEmail=you@example.com is also required. It is included in the HTTP User-Agent so wiki operators know who is scraping.

A Few More Things You Might Want

Scrape only specific pages

Pass a comma-separated list of page titles directly:

--pageList="Main Page,Earth,Albert Einstein"

Or put one page title per line in a text file and point to it:

--pageList=./my-pages.txt

Scrape a private wiki

Provide credentials with --mwUsername and --mwPassword:

--mwUsername=jdoe --mwPassword=s3cret

Preferably use a bot password rather than a regular user account.

If authentication requires a separate domain, also pass --mwDomain:

--mwDomain=corp --mwUsername=jdoe --mwPassword=s3cret

Customising the Result

Want your ZIM to have a specific title, description, or icon?

--customZimTitle="My Offline Wiki"
--customZimDescription="A hand-picked selection of articles"
--customZimFavicon=https://example.com/icon.png

Adjusting the scrape speed

Since version 2.0.0, the default request rate (speed 1) is fine for most wikis. The --speed option controls the climb rate — how aggressively mwoffliner ramps up its request concurrency:

If you see lots of HTTP errors (e.g. 429 Too Many Requests) in the logs, try lowering the speed e.g --speed=0.5 can help prevent the wiki from rate-limiting you.

Going Further

These and all other options are listed in mwoffliner --help.

Also, see the FAQ for detailed explanations of the command line options and common issues.

Need help?

Installation

The recommended way to install and run mwoffliner is using the pre-built Docker container:

docker pull ghcr.io/openzim/mwoffliner

Run software locally / Build from source

Prerequisites for local execution

*NIX Operating System (GNU/Linux, macOS, etc.)
Redis — in-memory data store
Node.js version 24 (we support only one single Node.js version; other versions might work or might not)
Libzim — C++ library for creating ZIM files (automatically downloaded on GNU/Linux & macOS)
Various build tools which are probably already installed on your machine:
- libjpeg-dev — JPEG image processing
- libglu1 — OpenGL utility library
- autoconf — automatic configuration system
- automake — Makefile generator
- gcc — C compiler
(These packages are for Debian/Ubuntu systems)

An online MediaWiki instance with its API available.

Installation methods

Build your own container

Clone the repository locally:

git clone https://github.com/openzim/mwoffliner.git && cd mwoffliner

Build the image:

docker build . -f docker/Dockerfile -t ghcr.io/openzim/mwoffliner

Run the software locally using NPM

[!WARNING] Local installation requires several system dependencies (see above). Using the Docker image is strongly recommended to avoid setup issues.

Setting up MWoffliner locally for development can be tricky due to several dependencies and version requirements. Follow these steps carefully to avoid common errors.

1. Node.js Version

MWoffliner requires Node.js 24 (other versions may fail).

Compatible Node 24 ranges: >=24 <24.6 or >=24.7 <25.

Check your version:

node -v

If your version does not match, use nvm to install the correct Node.js version.

2. libzim Dependency

MWoffliner depends on @openzim/libzim, which requires the C++ libzim library.

On Linux/macOS, MWoffliner can download libzim automatically.
On Windows, you must install libzim manually because there are no prebuilt binaries. See the libzim installation guide for details.

3. Compiler Requirements (Windows)

Node 24 on Windows officially supports Visual Studio 2019 (v16) or Visual Studio 2022 (v17).

Ensure C++ build tools are installed and environment variables are set correctly. See Windows Setup for node-gyp for detailed instructions.

4. Node-gyp

MWoffliner uses node-gyp, which enforces strict checks for Node and compiler versions. Make sure you have:

Proper Visual Studio version (Windows) — see Visual Studio versions
Required C++ headers, e.g., zim/archive.h — see libzim documentation
Python 3.10+ (required by node-gyp; a recent version is preferred for compatibility)

Additional troubleshooting steps if errors persist:

Clear npm cache — a corrupted cache can cause cryptic install failures:
```
npm cache clean --force
```
Delete node_modules and reinstall — stale or partially installed dependencies are a common source of errors:
```
rm -rf node_modules package-lock.json
npm install
```
Check that all environment variables are set — especially on Windows, PATH, INCLUDE, and LIB must point to the correct Visual Studio and libzim directories. Reopen your terminal after installing new tools.
Verify Redis is running before starting MWoffliner — MWoffliner will fail immediately if it cannot connect to Redis:
```
redis-cli ping   # expected output: PONG
```
Run npm install with verbose logging to see exactly where it fails:
```
npm install --verbose
```

5. Common Errors & Troubleshooting

Error	Cause	Solution
Node.js version error	Node.js version incompatible	Install Node 24 with nvm
Cannot find module @openzim/libzim	libzim not installed	Follow libzim installation guide; Windows users must install manually
node-gyp rebuild failed	Wrong Node or compiler version	Check Node.js version, Visual Studio version, Python 3.x
zim/archive.h not found	C++ headers missing	Install libzim system-wide, verify include paths

[!NOTE] Even with these steps, other setup errors may occur. Using Docker is strongly recommended for a smoother experience.

Installation via NPM

npm i -g mwoffliner

[!WARNING] You might need to run this command with the sudo command, depending on how your npm / OS is configured. npm permission checking can be a bit annoying for newcomers. Please read the npm script documentation if you encounter issues.

Contribute

If you've retrieved the MWoffliner source code (e.g., via a git clone), you can install and run it locally with your modifications:

npm i
npm run mwoffliner -- --help

Detailed contribution documentation and guidelines are available.

API

MWoffliner provides an API and can be used as a Node.js library. Here's a stub example for your index.mjs file:

import * as mwoffliner from 'mwoffliner'

const parameters = {
  mwUrl: 'https://es.wikipedia.org',
  adminEmail: 'foo@bar.net',
  verbose: true,
  format: 'nopic',
  pageList: './pageList',
}

mwoffliner.execute(parameters) // returns a Promise

MathJax support

Warning

MathJax support is UNSTABLE: the --mathJax* CLI parameters described below may change, even in a minor release. Setting them up also requires wiki-specific technical preparation, so this is aimed at developers rather than end users.

MWoffliner can bundle MathJax into the ZIM so that math formulas keep rendering offline, on wikis that rely on it (e.g. via the SimpleMathJax extension).

MathJax 2, 3 and 4 are very different beasts: each ships its own set of files and, more importantly, each requires its own incompatible configuration format (which defines things like math delimiters and custom macros). Because of this, MWoffliner cannot auto-detect and configure MathJax for you — you have to supply a matching MathJax build and configuration yourself, via three CLI parameters:

--mathJaxSource: local path or HTTP(S) URL to a ZIP archive of a compiled MathJax distribution. Its content is extracted and pushed to the ZIM (under an internal _mathjax_/ namespace).
--mathJaxConfig: local path or HTTP(S) URL to an HTML file containing a single <script> tag with the MathJax configuration (this must be copied from the wiki, see below). Its content is injected inline, before the MathJax library, on every page that needs it. If the configuration needs to reference a path inside the MathJax archive (e.g. MathJax.Ajax.config.path[...]), it cannot use an absolute path since ZIMs have no fixed root URL. Write __MATHJAX_ROOT__ instead of the leading slash and mwoffliner will replace it, on every page, with the correct relative path to the root of the MathJax archive you provided. For instance use MathJax.Ajax.config.path["Contrib"] = "__MATHJAX_ROOT__/MathJaxExtensions/legacy"; if the resources are located in a MathJaxExtensions/legacy folder in the archive (inside the root folder of the archive if there is a single folder as usual).
--mathJaxEntryPoint: path, relative to the root of the extracted archive, to the MathJax script to load (e.g. es5/tex-chtml.js for MathJax 3). Defaults to MathJax.js (MathJax 2 entry point).
--mathJaxAllPages: inject the config/entry-point <script> tags on every page instead of only on pages detected to need MathJax (see below). Some wikis don't list MathJax in their page JS modules, which defeats the automatic detection; this flag works around that at the cost of adding the scripts to every page.

--mathJaxConfig, --mathJaxEntryPoint and --mathJaxAllPages all require --mathJaxSource to also be set. A page is considered to "need" MathJax, and only then gets the config/entry-point <script> tags injected, when one of the JS modules MediaWiki reports for that page matches mathjax (case-insensitive), unless --mathJaxAllPages is set, in which case every page gets them. The extracted library files themselves are always pushed to the ZIM as soon as --mathJaxSource is set, regardless of which pages use them.

Preparing the parameters for a given wiki

Find the live MathJax version and configuration. Open a wiki page that renders math formulas, open your browser's developer console and type MathJax.version to get the exact version. Then find the configuration, typically a <script> block setting window.MathJax = {...} (MathJax 3/4) or MathJax.Hub.Config({...}) (MathJax 2) in the page source — copy it as-is into a local file, e.g. mathjax-config.html.
Find the exact entry point. In your browser's network tab, find the request loading the MathJax library itself (typically named MathJax.js for MathJax 2, or a tex-chtml.js/tex-svg.js/... for MathJax 3/4) and note its full path, including any query string.
Build a matching MathJax ZIP, the exact steps depend on the major version in use (see below).

Run the scraper with the three parameters, e.g.:

mwoffliner --mwUrl=https://your.wiki --adminEmail=foo@bar.net \
  --mathJaxSource=./mathjax-source.zip \
  --mathJaxConfig=./mathjax-config.html \
  --mathJaxEntryPoint=es5/tex-chtml.js

Building the ZIP — MathJax 2

MathJax 2 is not published to npm as an installable package; it is only distributed as source on GitHub. Download a release archive directly from the MathJax releases page (e.g. 2.7.9) and use it as-is as --mathJaxSource, no re-zipping needed:

curl -Lo mathjax-source.zip https://github.com/mathjax/MathJax/archive/refs/tags/2.7.9.zip

The archive has a single top-level folder (e.g. MathJax-2.7.9/); mwoffliner strips it automatically when extracting.

Unlike MathJax 3/4, MathJax 2 loads its extensions/output-processor via a config= query parameter on the MathJax.js request itself (e.g. MathJax.js?config=TeX-MML-AM_CHTML) rather than solely through the injected configuration script — this is the URL you captured in step 2 above. Set --mathJaxEntryPoint to that same path and query string, e.g.:

--mathJaxEntryPoint="MathJax.js?config=TeX-MML-AM_CHTML"

The referenced combined-configuration file (here config/TeX-MML-AM_CHTML.js) is part of the standard MathJax 2 distribution, so it is already included in the ZIP from the release archive.

Building the ZIP — MathJax 3

MathJax 3 is published to npm as mathjax-full, which bundles both the compiled runtime (under es5/) and the TypeScript sources (which mwoffliner automatically ignores when an es5/ folder is present):

npm install mathjax-full@3.2.2
cd node_modules/mathjax-full && zip -r ../../mathjax-source.zip . && cd ../..

Entry point example: --mathJaxEntryPoint=es5/tex-chtml.js.

Building the ZIP — MathJax 4

MathJax 4 moved to scoped npm packages. Use the plain mathjax package (deployment-ready bundle), not @mathjax/src (the TypeScript source package meant for building MathJax itself, which requires separately installing font packages):

npm install mathjax@4
cd node_modules/mathjax && zip -r ../../mathjax-source.zip . && cd ../..

Unlike MathJax 3, entry-point files live at the root of the package (no es5/ folder), e.g. --mathJaxEntryPoint=tex-chtml.js.

Background

Complementary information about MWoffliner:

MediaWiki software is used by thousands of wikis, the most famous ones being the Wikimedia ones, including Wikipedia.
MediaWiki is a PHP wiki runtime engine.
Wikitext is the markup language that MediaWiki uses.
MediaWiki parser converts Wikitext to HTML, which displays in your browser.
Read the scraper functional architecture for more details.

License

GPLv3 or later, see LICENSE for more details.

Acknowledgements

This project received funding through NGI Zero Core, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. Learn more at the NLnet project page.

Navigation Menu

Uh oh!

Uh oh!

MWoffliner

Features

Quick Start

Prerequisites

The Four Decisions

1. Which wiki?

2. Which API path?

3. What content?

4. Where does it go?

A Few More Things You Might Want

Scrape only specific pages

Scrape a private wiki

Customising the Result

Adjusting the scrape speed

Going Further

Installation

Prerequisites for local execution

Installation methods

Build your own container

Run the software locally using NPM

1. Node.js Version

2. libzim Dependency

3. Compiler Requirements (Windows)

4. Node-gyp

Additional troubleshooting steps if errors persist:

5. Common Errors & Troubleshooting

Installation via NPM

Contribute

API

MathJax support

Preparing the parameters for a given wiki

Building the ZIP — MathJax 2

Building the ZIP — MathJax 3

Building the ZIP — MathJax 4

Background

License

Acknowledgements

Uh oh!

Uh oh!

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

MWoffliner

Features

Quick Start

Prerequisites

The Four Decisions

1. Which wiki?

2. Which API path?

3. What content?

4. Where does it go?

A Few More Things You Might Want

Scrape only specific pages

Scrape a private wiki

Customising the Result

Adjusting the scrape speed

Going Further

Installation

Prerequisites for local execution

Installation methods

Build your own container

Run the software locally using NPM

1. Node.js Version

2. libzim Dependency

3. Compiler Requirements (Windows)

4. Node-gyp

Additional troubleshooting steps if errors persist:

5. Common Errors & Troubleshooting

Installation via NPM

Contribute

API

MathJax support

Preparing the parameters for a given wiki

Building the ZIP — MathJax 2

Building the ZIP — MathJax 3

Building the ZIP — MathJax 4

Background

License

Acknowledgements