Node Express builds using Gulp

The 'Express application generator' at https://expressjs.com/en/starter/generator.html is a great starting point, providing a basic Express web application server framework for the node.js platform. But as the Express document states, it's just one of many ways to structure Express apps.

The present project is concerned with developing the default application created using the 'Express application generator'. This will provide a full SASS implementation, Pug templating and a comprehensive Gulp build process of front-end resources under Express 'development' or 'production' environments. The project can be cloned from: https://github.com/nickbergquist/node-express-sass-workflow

Using Express middleware

One way to build resources is, for example, to use node-sass and postCSS, etc. - middleware within an Express application which is itself a series of middleware function calls. The default application built with the "Express application generator" using the --css=sass switch currently (as of 2017) does just this, using node-sass-middleware to simply compile a *.sass file to CSS in the same directory with a source-map. Note that by default the generator sets indentedSyntax: true in the root app.js for the older syntax of SASS:

var sassMiddleware = require('node-sass-middleware');

app.use(sassMiddleware({
  src: path.join(__dirname, 'public'),
  dest: path.join(__dirname, 'public'),
  indentedSyntax: true, // true = .sass and false = .scss
  sourceMap: true
}));

Furthermore, the build processing request can be passed from one middleware function in the stack to the next. So for example, in the node-sass-middleware documentation there is an example of a way in which SASS stylesheets may be compiled in one middleware function and then CSS vendor-prefixing is accomplished in the subsequent middleware function:

var connect = require('connect');
var sassMiddleware = require('node-sass-middleware');
var postcssMiddleware = require('postcss-middleware');
var autoprefixer = require('autoprefixer');
var path = require('path');
var http = require('http');
var app = connect();
var destPath = __dirname + '/public';
app.use(sassMiddleware({
    src: __dirname
  , response: false
  , dest: destPath
  , outputStyle: 'extended'
}));
app.use(postcssMiddleware({
  plugins: [autoprefixer()],
  src: function(req) {
    return path.join(destPath, req.url);
  }
}));

http.createServer(app).listen(3000);

This is all useful to know but in practical terms there is usually extensive build processing to be done on front-end resources. And ultimately Gulp is about creating files whereas Express is responsible for routing and serving them. So rather than have this done in app.js which should focus on application logic it seems sensible to engage a dedicated task-runner such as Gulp which can also be used to start both the Express server and the application itself. In Express 4.0 this process is begun with a movement of the server object creation, server listening and server error handling away from app.js into bin/www.

Using Gulp

Using the task-runner Gulp to build the front-end resources provides a straightforward way to:

• differentiate on the build files required for 'development' or 'production' environments
• apply linting or uniform coding standards to particular filetypes
• compile certain files and perform post-compilation tasks
• set up Gulp 'watch' tasks to automatically recompile and/or build specific files when they are edited
• restart the application when server-side Javascript files change using node-mon
• concatenate client-side Javascript files for fewer HTTP requests
• minify files removing all unneccessary code from build files without affecting functionality
• write source-maps so that debugging tools can map the built code back to the original source code

In order to do this the application structure has been changed to facilitate building resources into two root directories, build or dist rather than the default public one that served as both the source and destination in the default application provided by the 'Express application generator'. A new source directory assets forms the working directory.

Creating the builds

Once the project has been cloned and dependencies installed via npm open up a CLI in the root directory and either enter gulp for a 'development' build or npm start for a 'production' build. In either case the build directories and files will be cleared out prior to the build taking place so that in effect the application is running in either environment with only the build files required for that environment. In a real instance you may wish to have both builds coexisting on disk and only the application environment switched.

If a 'development' build is made then entering gulp will run the Gulp 'default' task. If a 'production' build is required then running npm start node will search for a scripts object and then run the command in the associated value of a key named start in package.json:

"scripts": {
    "start": "Set NODE_ENV=production&& gulp"
}

Note: As can be seen this sets Express to run in a 'production' environment by setting NODE_ENV=production but note that if running under Linux or Mac OSX then the set command should be substituted with export. Also, resist the temptation to insert a space prior to the Javascript logical && operator as this will cause the Express environment to be set to 'production ' (note space). Subsequent expectation that the string will be 'production' will be foiled without trimming.

The command associated with the start key also runs a Gulp 'default' task. The difference being that there are two Gulp 'default' tasks in the root gulpfile.js conditionally switched dependent on environment:

// gulp default tasks
if(process.env.NODE_ENV === 'production'){
    gulp.task('default', ['clean', 'publish']);
} else {
    gulp.task('default', ['clean', 'build', 'watch']);
}

gulp.task('build', ['build-css', 'build-pug', 'build-fonts', 'build-js'], () => {
    gulpUtil.log(gulpUtil.colors.green('Application built'));
    nodemonInit();
});

gulp.task('publish', ['pub-css', 'pub-pug', 'pub-fonts', 'pub-js'], () => {
    gulpUtil.log(gulpUtil.colors.green('Application published. No watch on assets'));
    nodeInit();
});

Development build details

In a 'development' build the following takes place:

• linting (SASS) takes place
• compilation takes place
• post-compilation process of writing browser vendor-prefixes to CSS
• source-maps are written
• gulp-plumber is used to prevent Gulp 'watch' tasks from ending if there is a syntax error, i.e., in SASS compilation, etc.
• files and directories in assets are processed or copied to the build directory
• files are not concatenated nor minified
• a Gulp 'watch' is established so that any changes to files in assets results in an immediate build process of those files following the steps above.
• node-mon restarts the application when changes to server-side Javascript files occur

A notable absence from this build is use of gulp-livereload in the pipe. This automatically reloads the browser when there are changes to files. However, it requires browser-specific extensions, doesn't always work without saving a file twice and personally I find the delay in waiting for the reload greater than simply switching to the browser and hitting the F5 key... but you may wish to use this!

Production build details

In a 'production' build the following takes place:

• linting is absent - you may wish to change this as there's a presumption that any warnings or errors have been picked up in a previous 'development' build
• compilation takes place
• files and directories in assets are processed or copied to the dist directory
• files are minified or concatenated and minified
• uses Node's child_process module to execute the command, node ./bin/www to start the application

Note: Although the application can be started in the course of a 'production' build as outlined above, a real instance would make use of a process manager such as StrongLoop which will not only restart the application if and when it crashes but also manage other deployment issues and metrics.

Pug templating

Having chosen Pug as the templating system there is a need to differentiate which built front-end resources should be loaded dependent on which Express environment the application is running under. For example, in a 'development' build more files result as there is no concatenation; in a 'production' build there may be a requirement for certain common libraries to be resourced from a Content Delivery Network plus also filenames of built files may differ with *.min suffixes.

The project uses two approaches for comparison: firstly a 'HTML build block processor', in this case gulp-processjade (Jade being the old name for Pug). In assets/views/layout.pug:

doctype html
html(lang='en')
  head
    meta(charset='utf-8')
    meta(name='viewport' content='width=device-width, initial-scale=1.0')
    title #{title} :: My website
    // build:css stylesheets/main.min.css
    link(rel='stylesheet', href='stylesheets/main.css')
    // /build

And secondly, the path to the front-end Javascript files also in assets/views/layout.pug is managed by use of a pug conditional block.

block scripts
    if settings.env === 'production'
        script(src='https://code.jquery.com/jquery-3.2.1.min.js')
        script(src='javascripts/site.min.js')
    else
        script(src='javascripts/static/jquery-3.2.1.min.js')
        //- required by IE9 to process javascript-based media queries - applies 
        //- patch if window.matchMedia() is unsupported
        script(src='javascripts/static/matchMedia.js')
        //- 'enquire.js' for javascript-based media queries.. requires 
        //- window.matchMedia() support.
        script(src='javascripts/static/enquire.min.js')
        script(src='javascripts/site.js')

And a third, even better option to differentially load front-end resources would be to use a dedicated Express asset manager tool such as express-asset-manager via npm.

Have a go

This is just a working example with plenty of scope for further development but it will hopefully get you set up with a much further advanced front-end for your Node Express application dependent on what is to be achieved. There are interesting alternative approaches such as using npm as a build tool and using Webpack to bundle and process front-end resources. If you wish to examine the repo further then please clone from the GitHub repository at: https://github.com/nickbergquist/node-express-sass-workflow