User:Inductiveload/Script development

Developing scripts live on-wiki can be frustrating due to the need for saving a page for each change.

Instead, I usually serve scripts directly from my machine, which is faster to load and also doesn't require changes to be saved on-wiki.

Serving

HTTPS

Because Wikisource is served over HTTPS, your scripts should be do. This can be quite tricky to set up.

I have the following script to set up the certificates.

#! /bin/sh
set -e

echo "Generating key/cert"
openssl req -x509 -nodes -new -sha256 -days 1024 -newkey rsa:2048 -keyout RootCA.key -out RootCA.pem -subj "/C=US/CN=Example-Root-CA"
openssl x509 -outform pem -in RootCA.pem -out RootCA.crt

openssl req -new -nodes -newkey rsa:2048 -keyout localhost.key -out localhost.csr -subj "/C=US/ST=YourState/L=YourCity/O=Example-Certificates/CN=localhost.local"

echo "Generate localhost.crt"
openssl x509 -req -sha256 -days 1024 -in localhost.csr -CA RootCA.pem -CAkey RootCA.key -CAcreateserial -extfile domains.ext -out localhost.crt

openssl pkcs12 -export -in RootCA.crt -inkey RootCA.key -out server.p12

echo "Now import to Firefox with Settings->Privacy & Security->Certificates->Servers->Add Exception"

Serving

It is easy to server the contents of your script development directory, along with the above HTTPS certs:

#! /usr/bin/env python
#
import http.server
import ssl

import logging
import argparse

import os
import re


class NoCacheHTTPRequestHandler(http.server.SimpleHTTPRequestHandler):
    def end_headers(self):
        self.send_my_headers()
        http.server.SimpleHTTPRequestHandler.end_headers(self)

    def send_my_headers(self):
        self.send_header("Cache-Control", "no-cache, no-store, must-revalidate")
        self.send_header("Pragma", "no-cache")
        self.send_header("Expires", "0")

    def do_GET(self):
        logging.info("Sending headers")
        # Sending an '200 OK' response
        self.send_response(200)

        if self.path.endswith(".js"):
            ct = "text/javascript"
        elif self.path.endswith(".css"):
            ct = "text/css"
        else:
            ct = "text/html"

        # Setting the header
        self.send_header("Content-type", ct)

        # Whenever using 'send_header', you also have to call 'end_headers'
        self.end_headers()

        path = re.sub(r'^/', '', self.path)

        with open(os.path.join(".", path), 'r') as content_file:

            content = content_file.read()

            content = re.sub(r'/\* *debug-replace: *(.*?)\*/', r'\1', content)

            # Writing the HTML contents with UTF-8
            self.wfile.write(bytes(content, 'utf-8'))

        return


if __name__ == "__main__":

    parser = argparse.ArgumentParser(description='')
    parser.add_argument('-v', '--verbose', action='store_true',
                        help='show debugging information')
    parser.add_argument('-p', '--port', type=int, default=5555,
                        help='The port to serve on')
    parser.add_argument('-k', '--key-file', default="localhost.key",
                        help='The SSL keyfile')
    parser.add_argument('-c', '--cert-file', default="localhost.crt",
                        help='The SSL certificate file')

    args = parser.parse_args()

    logging.basicConfig(
        encoding='utf-8',
        level=logging.VERBOSE if args.verbose else logging.INFO)

    logging.info("Serving over HTTPS on port %d" % args.port)
    logging.info("Using cert: %s" % args.cert_file)

    server_address = ("0.0.0.0", args.port)
    httpd = http.server.HTTPServer(
        server_address, NoCacheHTTPRequestHandler)

    context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    context.load_cert_chain(args.cert_file, args.key_file)
    httpd.socket = context.wrap_socket(
        httpd.socket,
        server_side=True
    )
    httpd.serve_forever()

NoCacheHTTPRequestHandler prevents the browser caching the JS, so it updates immediately after change without having to flush browser caches.

Caching

The above method of serving files indicates to browsers to never cache the files. This is usually what you want for development purposes.

If you use other methods of serving the code, it's very common for your browser to cache script files even when they are served locally. This can prevent you seeing the latest version of a script. Disabling this caching depends on the browser.

In Firefox, open the Network debugging pane (Ctrl+Shift+E) and select Disable cache. This may make pages slower to load, but it will discard all cached script versions and load from fresh from your development server.

When loading scripts from the wiki, it can take a minute or two for the server-side caches to update after changing the file.

Scripts

Selecting local or on-wiki using common.js

Your common.js can be set up to load from this local development server if possible, or fall back to on-wiki defaults.

The following code attempts to load a file called ws_common.js from your development server, and falls back if that fails (e.g. the server is offline):

// Load local dev script if available, or fall back to an on-wiki version
mw.loader.getScript( 'https://localhost:5555/ws_common.js' )
    .then( function () {
    	console.log( 'Local ws_common.js script loaded' );
    }, function ( e ) {
        console.log( 'On-wiki common.js script loaded' );
        // ....load your on-wiki scripts here
	    mw.loader.load( '//en.wikisource.org/w/index.php?title=User:You/script.js&action=raw&ctype=text/javascript' );

        // and you can load gadgets too
        mw.loader.load( 'ext.gadget.some_gadget' );
    } );

You can load gadgets from your fallback code, or based on other logic, with mw.loader.load(). This can be useful when you want to load the "normal" gadgets when not developing

mw.loader.load( [ 'ext.gadget.PageNumbers' ] );

Switching between local code and gadgets

When developing gadgets, it can be helpful to turn them on and off. The following code can use a local script if a gadget is turned off. Then all you have to do to switch over is to enable or disable the gasget in your preferences:

// See if a gadget is configured on or off
if ( !mw.user.options.get( 'gadget-PageNumbers' ) ) {
   mw.loader.load( 'https://localhost:5555/local_script_pagenumbers.js' );
} else {
  console.log( 'Using on-wiki gadget: pagenumbers' );
}

Simulating ResourceLoader dependencies

Many gadgets have dependencies that they assume are loaded before the gadget runs. When developing, you should do this too or you find it works differently locally to when it is a gadget. To do this, wrap the mw.loader.load() call with mw.loader.using().

// Equivalent to:
// sandbox[ResourceLoader|dependencies=mediawiki.util,mediawiki.cookie]|sandbox.js

mw.loader.using( [ 'mediawiki.util', 'mediawiki.cookie' ] ), function () {
  mw.loader.load( 'https://localhost:5555/local_script_sandbox.js' );
} );

Loading CSS components

When your gadget contains CSS files, you can use mw.loader.load() to load them like this:

// Equivalent to:
// sandbox[ResourceLoader]|sandbox.js|sandbox.css

mw.loader.load('https://localhost:5555/local_script_sandbox.js');
mw.loader.load('https://localhost:5555/local_script_sandbox.css', 'text/css');

When these are a proper gadget, the gadget's ResourceLoader dependencies in MediaWiki:Gadgets-definition will include this and you won't need to load two files - just use mw.loader.load( 'ext.gadget.the_gadget' ) or enable in preferences and the CSS will be included.

IIFEs

Gadgets are automatically wrapped in a Immediately invoked function expression (IIFE) by ResourceLoader. Locally served scripts do not have this. If you want to isolate your local scripts in the same way, you can add your own IIFE. If used as-is in a gadget, there will be two levels of IIFE, but this is not an issue in practice.

// this variable is in the global scope when served locally
// but inside an IIFE when served by ResourceLoader as a gadget
var global;
( function ( $, mw ) {
   var i; // this variable is NOT in the global scope locally or in a gadget
} ( mediaWiki, jQuery ) );

Worry box

• CSP: is this going to destroy this method? phab:T28508.