Building a portfolio with Gulp and Metalsmith

Written on 27th August 2017, with 1605 words (approx reading time of 6 min) by David Wood.

Welcome to my new website! It long overdue, but I've finally gotten around to rebuilding my portfolio and personal website. This writing should be an introduction to how I've approached building my site and the technologies I've used - it won't be an exhaustive guide by any means, but if you've been considering using Metalsmith and stumbled upon this, it might clear up some things.

Why a static site with Gulp and Metalsmith?

My main considerations when building this site was to keep it simple, I wanted to avoid having to have a server running somewhere that would need patched or could contain any security vulnerabilities. Static sites are ideal for that - using a tool like Jekyll, Hugo or Metalsmith allow me to maintain a single Git repo, containing my templating, styles and content that could be built using a CI pipeline (or a service such as Netlify) to produce the finished product.

I was already familiar with using Gulp for automating workflow tasks and would almost certainly be using it for CSS and Javascript minification and processing - so choosing a Javascript-based static site generator that would work with that pipeline was a consideration.

Further, I wanted something that wasn't prescriptive as to the contents of the site - as much as I have blog posts, I don't necessarily consider this website a blog. After some research, Metalsmith seemed like a good fit. I've got experience with Jekyll and I always found it was too geared towards blogs.

How does it work?

After some research, I decided to use Metalsmith in the context of a Gulp pipeline. For this project, I also chose to use the upcoming Gulp 4 with ES2015, you can see an excerpt of that below. If you're interested in seeing the whole source, here's a link to the repository.

Configuration

I tend to specify any settings any Node.js projects in the package.json file, importing them as below.

import fs from 'fs';
const pkg = JSON.parse(fs.readFileSync('./package.json'));
"settings": {
  "meta": {
    "site": {
      "title": "David Wood",
      "url": "https://davidtw.co",
      "description": "Personal website of David Wood"
    },
    "links": [
      {
        "name": "GitLab",
        "icon": "gitlab",
        "link": "https://gitlab.com/davidtwco"
      }
    ]
}

I use this configuration to specify any options that isn't config - including output directories, meta information, favicon generation information, etc.

Tasks

Gulp organizes pipelines from tasks, each task processes a set of files and produces a result. In Gulp 4, tasks are specified using the CommonJS export notation and since we're using the ES2015 syntax, we can do that by adding the export keyword before the function definition.

export function scripts() {
    const outputPath = path.join(__dirname, pkg.settings.assets, 'scripts');
    return gulp.src([
        pkg.settings.src.scripts + '/**/*.js'
    ], {since: gulp.lastRun(scripts)})
        .pipe(sourcemaps.init())
        .pipe(babel())
        .pipe(concat('app.min.js'))
        .pipe(uglify())
        .pipe(sourcemaps.write())
        .pipe(gulp.dest(outputPath));
}

In the above snippet, I define the scripts task that processes any Javascript code. In this case, we create sourcemaps, convert our ES2015 Javascript into Javascript that is supported by current browsers, concatenate into a single file, minify and write to our output directory. Gulp code typically relies on a lot of third party plugins to provide functionality, these are specified in the package.json, installed with npm, imported and used as normal.

BrowserSync

In building the site, I make heavy use of BrowserSync - BrowserSync allows me to have the page refreshed automatically when changes are made, in combination with Gulp, I can have all of the affected assets from my change rebuilt before this reload occurs.

In development, I run the gulp serve command, which is defined below. This command triggers the serve task: this triggers the build task, so that I have everything built from the current state; then the watch task that ensures that the correct tasks are re-run when any files are changed and finally the serve task, that starts BrowserSync to serve the distribution folder.

// Fixes issue with browsersync in Gulp 4 only reloading once.
const reload = (callback) => { browserSync.reload(); callback(); }
export function watch(callback) {
    // Watch for file changes.
    gulp.watch(['gulpfile.babel.js', 'package.json'],
        gulp.series(gulp.parallel(styles, scripts, favicon, fonts), metalsmith, reload));

    gulp.watch([pkg.settings.src.fonts + '/**/*'],
        gulp.series(fonts, metalsmith, reload));

    gulp.watch([pkg.settings.src.images + '**/*'],
        gulp.series(images, metalsmith, reload));

    gulp.watch([pkg.settings.src.favicon],
        gulp.series(favicon, metalsmith, reload));

    gulp.watch([pkg.settings.src.styles + '/**/*.scss'],
        gulp.series(styles, metalsmith, reload));

    gulp.watch([pkg.settings.src.scripts + '/**/*.js'],
        gulp.series(scripts, metalsmith, reload));

    gulp.watch([
        pkg.settings.src.content + '/**/*.md',
        pkg.settings.src.layouts + '/**/*.njk'
    ], gulp.series(metalsmith, reload));

    callback();
}

// Serve the built files and ensure that the watch
// task is running so that they are always up-to-date.
function server() {
    browserSync.init({
        server: {
            baseDir: pkg.settings.dist
        },
        ui: {
            port: 8080
        }
    });
}
export const serve = gulp.series(build, watch, server);

Metalsmith

Metalsmith works similarly to Gulp, it takes a folder full of files, runs a series of functions on them, transforming the contents and them outputs the result in a directory.

export function metalsmith(callback) {
  let layoutDir = pkg.settings.src.layouts;
  let loader = new nunjucks.FileSystemLoader(layoutDir);
  let environment = new nunjucks.Environment(loader);
  // We can add our own globals and filters to Nunjucks here.

  const m = Metalsmith(__dirname)
    .metadata(pkg.settings.meta)
    .source(pkg.settings.src.content)
    .destination(pkg.settings.dist)
    .clean(true)
    // ...a bunch more plugins...
    .use(markdown({
      gfm: true,
      tables: true
    }))
    // ...a bunch more plugins...
    .use(layouts({
      engine: 'nunjucks',
      directory: pkg.settings.src.layouts,
      nunjucksEnv: environment
    }))
    // ...maybe one more plugin...
    .use(assets({
      source: pkg.settings.assets,
      dest: '.'
    }))
    .build((err) => {
      if (err) throw err;
      callback();
    });
}

Our metalsmith task is defined similarly to other Gulp tasks, inside we call Metalsmith, chaining use calls and passing in a function each time. Metalsmith's use function takes a function as an argument, that function takes three arguments - files, an array of file objects containing metadata and the contents, this is where the majority of the work is performed; metalsmith, access to the metalsmith state and variables; and done, a callback used to indicate completion. Each metalsmith plugin typically takes a map with settings and returns this function configured as per our specification.

Tips, Tricks and Gotchas

Metalsmith hasn't got a lot of documentation and I had to rely on a handful of blog posts (like this one) that details how another developer used it. Here are a handful of little things that tripped me up..

  1. Order is important: The order of your plugins are important, plugins that operate on HTML files will need to come after the metalsmith-markdown plugin.

  2. Watch your frontmatter: I found that the metalsmith-markdown plugin likes to fail silently, if you find that you're missing the metadata attributes from your files (perhaps you accidentally indented using tabs in the YAML rather than spaces) then you'll find some missing attributes and no message explaining why.

  3. Add a short debug snippet: Metalsmith's plugins are incredibly simple, they are just a function! Remember the following snippet and insert it to help track down where unexpected behaviour occurs.

  // ...previous plugins...
  .use((files, metalsmith, done) => {
    console.log(files);
  })
  // ...upcoming plugins...

Templating

Metalsmith has two plugins that you can use for templating - metalsmith-layouts and metalsmith-in-place.

metalsmith-layouts works by checking the layout attribute in your Markdown files, and then rendering with that template. This works similarly to Django and Flask's templating if you are familiar with that - but instead you specify the context and layout in the frontmatter (the body is added as a contents property).

metalsmith-in-place allows for in-place templating. I opted to use the metalsmith-layouts plugin (though I believe you can use them both together if required) and so I'm not going to go into how to use metalsmith-in-place.

Both these plugins are very flexible in that they let you use any templating language that works with consolidate.js - and that's quite a few.

I went with Nunjucks, by Mozilla - it's built to look like Jinja2, which might sound familiar if you've used Flask before. Nunjucks supports template inheritance and is very easy to extend with custom filters and globals.

export function metalsmith(callback) {
  let layoutDir = pkg.settings.src.layouts;
  let loader = new nunjucks.FileSystemLoader(layoutDir);
  let environment = new nunjucks.Environment(loader);
  // nunjucksDate is a handy npm package that lets allows
  // formatting of dates in the templates using
  // moment.js
  environment.addFilter('date', nunjucksDate);
  // evaluates a global function on an object.
  // {{ item.ancestry.self | evaluate(isBaseLabCollection) }}
  environment.addFilter('evaluate', function(obj, fn) {
    return fn(obj);
  });
  // filters a list based on a fn that returns true/false.
  environment.addFilter('filter', function(objs, fn, negate = false) {
    if (objs.constructor !== Array) return objs;

    if (negate)
      return objs.filter((obj) => !fn(obj));
    return objs.filter(fn);
  });
  // Checks if the path is at the root of a directory.
  environment.addGlobal('isBaseLabCollection', function(obj) {
    return obj.paths.dir.indexOf('/') === -1;
  });
  // Returns the path after the first directory.
  environment.addGlobal('getLabCollection', function (obj) {
    let index = obj.paths.dir.indexOf('/');
    if (index === -1) return obj.paths.dir;

    return obj.paths.dir.substring(index + 1);
  });

  const m = Metalsmith(__dirname)
  // ...the first few Metalsmith plugins...
    .use(layouts({
      engine: 'nunjucks',
      directory: pkg.settings.src.layouts,
      nunjucksEnv: environment
    }))
  // ..the final few Metalsmith plugins...
}

If Nunjucks works out of the box for you then you won't even need the environment setup! In that case you can just leave the nunjucksEnv property out of the layouts() call.

Misc

There are various other smaller tasks used in building this website, such as Favicon and manifest.json generation, style compilation, font processing and more. You can check out the full source for the website to see how these are defined.

Deployment

I chose to deploy the website using Netlify's Pro tier. Luckily Netlify offer that tier for free to open source/non-commercial projects. If you'd prefer to keep your source closed, Netlify's regular free tier covers custom domains and HTTPS - all you need to do is link it to your GitHub, GitLab or Bitbucket and specify the build command.

You can also use GitHub Pages or GitLab Pages - both of which are excellent.


Thanks for reading and checking out my new site!