Generating Pugpig XML feeds with Grunt

We're currently nearing completion of a webapp build that is being integrated into Pugpig to produce a downloadable digital publication.

As part of the Pugpig build, the framework requires three XML feeds - contents, issues and editions.

During this project, I wrote a few Grunt plugins to help with automation when building the XML feeds required by Pugpig for publishing the digital publication.

 

The need for automation

Quick deployment and testing

We needed to rapidly update and deploy our webapp to an Amazon S3 endpoint, then download it via the Pugpig app to use it within the framework and on our test devices.

As we first did a proof of concept both of the webapp and the deployment process, we created mock XML feeds for the Pugpig app and pushed those up manually to the endpoint.

We kept tweaking and testing the contents, editions and issues XML files until we got them right - providing everything that Pugpig needed in order for the app to download and unpack our content.

Avoiding repetition

It was a little tedious doing this at first; it served its purpose for the proof of concept, but as the webapp grew, content and pages were added and we needed an automated solution to generate the required files, preferably as part of the front end build process.

As our requirements were pretty specific to a Pugpig project, there weren't any existing solutions, so I spent some time writing Grunt plugins for the XML feeds Pugpig requires.

Avoiding repetition

Just kidding.

The Feeds

Contents

The contents feed dictates the pages that appear in the Pugpig app - the pages accessible from the native page navigation.

As well as this, it includes meta for each page:

  • id
  • the alternate link
  • title
  • summary
  • updated timestamp
  • published timestamp

This feed lives in with the zipped-up files that are pushed to the endpoint, and all pages in the feed are relative to its location.

[gist]https://gist.github.com/furzeface/e611e0fff84158ae0469[/gist]

Editions

The editions file lives at the root of the directory, and holds meta about the publication in its entirety and each individual edition, for example, the 2014 edition, 2015 edition etc.

This requires dynamically produced, updated and issued dates, and needs to reference a cover image for Pugpig, as well as the aforementioned issue.xml file.

[gist]https://gist.github.com/furzeface/0629109f91efe3c85e43[/gist]

Issues

The issue.xml file sits in the same directory as the zipped up files, but outside the package. This is used to point to the source for unpacking - the archive.

It also needs the size of the zip and modified date - things that we needed to produce dynamically on the Grunt build.

[gist]https://gist.github.com/furzeface/fb7cf256889af5f3a453[/gist]

 

The Tasks

Contents

What it does

All three Pugpig feed plugins require Node XMLBuilder which generates the final XML feed and its contents. They also require the standard fs Node module for traversing the file system, chalk for some pretty colours and superb for some positive reinforcement and kind words when you've done a task right.

The grunt_pugpig_contents_xml task starts by reading the package.json file from the root of your project, combining some of the aspects of it with options passed in from the Grunt config to kick off the building of the XML feed.

The task then loops the files in the path passed in the options and creates an <entry> for each file - in our case the pages that need to be within the final digital publication.

Finally the contents task formats the XML string and writes to a file - defaulted to content.xml - gives you a superb phrase and writes a success message to the console along with creating the file in your destination directory.

Usage example

A basic example using some default and some custom options:

[gist]https://gist.github.com/furzeface/d2ff02a2ef9526d4d09c[/gist]


Known Issues / contributing

You can check out the GitHub issues or the Waffle.io board for the issues and contribution needs. The main issue is the fact that there's a tiny teeny hack to get around the nesting of generated XML feeds within Node XMLBuilder - have a look at the source if you like and have a go at fixing it!

Editions

What it does

The grunt_pugpig_editions_xml task again merges your options defined in the Grunt config with sensible defaults and starts to build an XML string to make up the editions.xml file.

The majority of the data in this file is from the user input in the Grunt config, other than the dynamic updated and issued dates.

Usage example

[gist]https://gist.github.com/furzeface/d5d3b183d7008f946e7d[/gist]

Known Issues / contributing

The same mini hack as before is in place in this task, to get around nesting <entry>s. You can check out the GitHub issues or the Waffle.io board for the issues and contribution needs.

Issues

What it does

The grunt_pugpig_issues_xml task is the simplest of the three. This again merges the Grunt config options with defaults and proceeds to build the XML string.

The nice bit of this task however when it looks for the source of the zip file defined in the config and writes the file size to the XML feed - an attribute required by Pugpig for publishing our app.

It finally writes out a nice success message for you using the stupendous Superb, and writes the file to your destination ready for deployment.

Usage example

[gist]https://gist.github.com/furzeface/212ff7ba5199cbbfa73b[/gist]


Known Issues / contributing

As with the other plugins, more test are always good, and there's a little @todo in the code and on the board also. You can check out the GitHub issues or the Waffle.io board for the issues and contribution needs.

 

Our overall process

Our Grunt build does a fair bit but the part of the process that creates the Pugpig feeds and builds our webapp for production goes a little something like this:

  • Pages are Assembled
  • Pugpig contents.xml is built
  • Meta files are stripped out of the /deploy directory that are no longer needed.
  • Files are packaged up into a zip file ready for deploy
  • The Pugpig issue.xml is built
  • The Pugpig editions.xml is built
  • Entire directory deployed to an endpoint ready for downloading and unpackaging via the Pugpig app

Next time we do a Pugpig publication it will be extremely easy to reuse these Grunt plugins to automatically generate the required feeds for the publication.

As issues come up we'll be adding to the functionality of the tasks and building on them.

If you find these useful please let us know and again, feel free to fork and contribute on GitHub!

Any questions?

If you need more information or have any questions just get in touch and we'd be happy to answer them for you.