Automating Complex Export Tasks in MediaWiki Using Lua

MediaWiki's Special:Export gives you raw XML; working with that data often means leaving the platform and parsing it elsewhere. For structured, repeatable exports — CSV rows from a cargo table, JSON from infobox fields, generated documentation files — Scribunto's Lua modules let you do the job inside the wiki, where the data lives. This article covers the pattern, the limits, and where the sandbox stops being the right tool.

Getting Scribunto in place

Scribunto is bundled with MediaWiki (it powers Wikipedia's complex templates). Enable it in LocalSettings.php:

wfLoadExtension( 'Scribunto' );

No extra server-side interpreter is needed — MediaWiki ships a sandboxed Lua environment. Modules live in the Module: namespace, and templates invoke them with {{#invoke:MyModule|function|args}}.

The frame object and its toolkit

Every module function receives the frame — the access point to everything the module may touch:

• frame:preprocess( 'text' ) — expands templates and parser functions inside a string, returning rendered wikitext; the key to reading template-generated content
• frame:getParent() — the caller frame; useful for reading the calling template's arguments
• mw.title.new( 'Page' ) — title objects with exists, categories, subpageTitle etc.
• mw.text / mw.ustring — string handling that works with Unicode

A CSV export module

The classic pattern: iterate title objects, pull structured fields, build rows:

local export = {}

-- Iterate a category and emit CSV of a template field
function export.csv( frame )
    local rows = { 'Title,Author,Year' }
    for _, title in ipairs( mw.site... ) do
        if title.exists then
            local page = mw.title.new( title )
            local text = page:getContent() or ''
            -- extract fields from the template call with Lua patterns
            local author = text:match( '|author = ([^|]+)' ) or ''
            rows[#rows + 1] = table.concat( { page.fullText, author }, ',' )
        end
    end
    return table.concat( rows, '\n' )
end

return export

Real-world modules read Cargo tables with mw.ext.cargo.query() or SMW values with mw.smw.getProperty() (both provided by the respective extensions), which is far more robust than text patterns. The output is returned as wikitext: wrap it in <syntaxhighlight lang="csv"> or a plain text page to make it copyable.

What the sandbox cannot do

Lua in MediaWiki is deliberately restricted — a key part of why Scribunto is safe on shared wikis:

• No file or network I/O — modules cannot read the filesystem, fetch URLs or send HTTP requests; exports are generated on page render
• Strict limits — per-module CPU time, memory and expansion depth are capped (configurable via $wgScribuntoEngineConf); loop-heavy exports on big categories hit them
• No persistent state — each render starts from scratch; caches (template transclusion caching) are the only 'memory'

When to use maintenance scripts instead

Large or scheduled exports (nightly backups of structured data, full-wiki dumps, API syncs) do not belong in page-render Lua. The right tool is a PHP maintenance script or an external script calling the Action API: arbitrary data access, no page-render penalties, and it can write files. The split is simple:

• Lua for small, request-driven, wiki-side exports that templates and users consume interactively
• Scripts/API for bulk, scheduled, machine-oriented extraction

The Scribunto reference manual documents every library function, and Cargo's format=template/json output already covers many 'export' use cases before you write a line of Lua.