gulp-smake

gulp-smake

A node.js tool for building solidity projects.

Installing

npm install gulp-smake

Usage

var gulp = require('gulp);
var gsm = require('gulp-smake');

gulp.task('compile-solidity', function(){
    gulp.src(myfileglobs).pipe(gsm.build(options, exports));
}

The gsm.build(options, exports) function is what you use to build. It only allows virtual files (no strings or streams). It will read the relative path of all incoming files and put them in an array. When all the files has arrived, that array is formatted into a string and appended to the solc command string, the solc command is run (with the given options), and then the files are moved into the proper build folder (as defined in the options).

The options object:

{
  "paths": [
    "./contracts/src/**/*.sol",
  ],
  "root": "./contracts",
  "sourceRoot": "src",
  "buildDir": "build",
  "docsDir": "docs",
  "compilerFlags": "--optimize 1 --binary file --json-abi file --ast-json file --natspec-dev file"
}

paths: This is passed to gulp.src and should be formatted for that. In this case I want it to get every .sol file it can find in the source folder and all of its sub-folders.

root: Root of the Solidity project.

sourceDir: Root dir of the source files. It must be a sub-folder of root, and would in this case be: ./contracts/src

buildDir: Output directory. This is where the built code-files will end up. It must be a sub-folder of root, and would in this case be: ./contracts/build

docsDir: Output directory. This is where the built doc-files will end up. It must be a sub-folder of root, and would in this case be: ./contracts/docs

compilerFlags: This is a string of compiler flags. It will be included when running solc. In this case, i'd like to build the .binary, .abi, .ast, and .devdoc (developer documentation) files for each source file, and to enable the optimizer.

The exports object:

{
  "base": "/tmp/project_sources",
}

base: If an exports object is passed into build, and it has a base field, it will override the source root in options. More info below.

The solidity compiler

The gulp-smake method takes an options and an exports object to find out where the root of the contracts are, then runs solc in that folder with the given commands, with the output ending up in the same folder. It then manually moves the compiled files out into the build folder.

The reason for this is that the Solidity compiler tool (solc) is not very advanced yet, so we must do some things manually. The first step is sometimes to set up a temporary folder and copy the sources into that (along with any external dependencies). This can be done in a different task on which the main build task depends. Since it's likely that the user would want to use the OS' temp folder to store the temporary source folder, it makes sense to generate the folder in code, make sure it's empty and then put the path into an exports object so as to leave the options object un-touched.

Here's an example of a build script that moves the sources into a temp folder and builds.

var gulp = require('gulp);
var gsm = require('gulp-smake');

var os = require('os);
var fs = require('fs-extra');
var path = require('path');

// We keep the options in a separate file.
var options = require('./options.json');

gulp.task('pre-build', function(){
    var temp = path.join(os.tmpdir(), "my_sources");
    fs.emptyDirSync(temp);
    // Create the path to the root source folder.
    var base = path.join(options.root, options.sourceDir);
    return gulp.src(options.paths, {base: base}).pipe(gulp.dest(temp));
});

gulp.task('build', ['pre-build'], function(){
    gulp.src(myTempFileGlobs).pipe(gsm.build(options, exports));
}

Setting an alternative solidity compile function

To compile the solidity code by default you will need solc on your path, or you will get an error. It is possible to overwrite the default compile function. The default function looks like this:

var solc = function (cmd, cwd, callback) {
    var exec = cp.exec;
    exec(cmd, {cwd: cwd}, function (error, stdout) {
        callback(error, stdout);
    });
};

Using gsm.setSolcFn you may set the function that is called when compiling. An example of when this is useful is when running a remote compiler, so the compile function would perhaps send an http request instead of starting a child process.

Solidity project structure

There is no standard way of setting up a solidity project. Personally I stick to a few simple rules.

Folder structure

Use a source folder named src, and a docs folder named docs, and a build folder named build. Keep all the internal sources in the src folder. Use sub-directories to organize. I normally use a "post build" task that runs after the build to move docs into the docs folder, and to perhaps filter out files that is generated as part of the build but I don't need.

External imports / builds

This is handled differently depending on if I want to use the projects compiled contracts in my DApp, or if I want to import the sources and use them when I compile my own contracts.

If I want to use the compiled files from another project I will run the gulp build task from that project, then manually copy them into a subfolder in my own build folder that is named after the project. If the project name is stack, and the file I want is Stack.binary, then it would be in my build folder like this: build/stack/Stack.binary

If I want to import the sources of another project to my own, I will manually move them into my source folder as a subfolder named after that project. I then include them in the 'paths' and just build as normal. An example would be when I use the assertions library from my Solidity unit testing library. It has one single file (Asserter.sol) in its root source folder. When I put it in my temp source folder for building it would be: temp_src_folder/assertions/Asserter.sol, and the way I would access it from my solidity files would be through: import "assertions/Asserter.sol.

That is generally how I handle external sources: Their source root becomes a sub-folder in my root, and its name is the same as the name (or id) of the project. It's the same both for compiled files or if i pull in the sources for importing.

Dependency trees

The manual importing works fine for now, but it should be automated so that every dependency is pulled in automatically as part of the build (by copying in the files, or pulling from git using gulp-git, or fetch a zip file and decompressing, or whatever). The problem with this is that the external dependencies may have dependencies of their own, so I can't get them without being able to run their build scripts from within mine.

This is certainly possible, of course, but needs some work and standardization of build scripts before it's possible to automate.

Different configurations

This is possible already. I use a different script when building tests. This is why it's good to make the basic build scripts modular, so they can be re-used between different types of builds (pre-build, post-build, all of that).

Licence

LGPL 3.0, included in the LICENCE file.

Made this plugin with 'gulp-concat' as a starting point, so probably still some code/comments that's the same. Thanks gulp-concat.