My JavaScript book is out! Don't miss the opportunity to upgrade your beginner or average dev skills.

Wednesday, May 25, 2011

JavaScript Builder from Falsy Values

Update: uglify-js as third option


Thanks to @kitgoncharov JSBuilder now includes the option to run uglify-js through Rhino ... still cross platform, a builder for any taste.




as promised during my workshop, I have created a Google Code Project with the builder used to demonstrate most advanced and common techniques to make a library, application, snippet, small and fast.

What Is JSBuilder

It's a truly simple Python module based over a certain hierarchy able to combine multiple files and produce a minified version of the application.
A similar approach has been used since ages with jQuery, and I am pretty sure there are tons of builders out there able to do similar task.
I personally created ages ago a similar project compatible with all windows platform.
However, that project was missing the possibility to put everything together, preserve some piece of code if necessary, replace some string runtime, and produce if necessary different outputs through the simplified module approach.
Here a build.py file example, the same you can find in the repository.

# JSBuilder example

# project name (from the root folder)
copyright = '(C) WebReflection JSBuilder from Falsy Values'
max_js = 'build/max.js'
min_js = 'build/min.js'

# file list (from the root/src folder)
files = [
"intro.js", #beginning of the closure
"variables.js", #local scope variables
"main.js", #some application logic
"functions.js", #functions declaration
"debug.js", #something useful or evil to debug
"outro.js" #end of the closure
]

# execute the task
import JSBuilder
JSBuilder.compile(
copyright,
max_js,
min_js,
files,
# optional list of serches
[ #mustaches in comments, hell yeah!
'/*{namespace}*/'
],
# with relative optional list of replaces
[
'var ' #produce var MyLib = ...

# it could be 'this.'
# this.MyLib = ...

# it could be 'my.personal.stuff.'
# my.personal.stuff.MyLib = ...
]
)

# let me read the result ...
import time
time.sleep(2)


Why Python

First of all the version is the 2.6 or higher but not compatible (yet) with version 3+.
Python to me is a sort of server side lingua-franca and it is missing by default only on Windows platform.
I would say the same about Java, with the slightly difference that in Python we don't need to create a class to perform a simple task as the one performed by JSBuilder.
If Python is not present in our system, we can easily find the installer in the python.org website while the build.py example code should be easy to understand even for non programmers.

Java As Well

Both YUICompressor and Google Closure Compiler come with a jar package which is a classic build out of one or more Java files. JSBuilder uses Java as well in order to call one compiler or another one with proper arguments and in a cross platform way so ... just in case you do not have Java installed, here the link to download what we need.

The Current Folder Structure

projectFolder
build
builder
src
Above structure is exactly the same you can find in the repository but don't worry, as you can spot in the build.py file we are free to change destination folders or paths without problems.
JSBuilder assumes in any case relative paths from the root of the application but if you really need a different structure feel free to replace the string '../' with the one you prefer ... it is an easily replaceable part of the script, just find and replace and that's it.

Multiple Projects

The build.py file is just an example but obviously we can create as many examples as we want, even inside the build.py file itself, changing just the part to replace, destination files, and/or copyright if necessary.
JSBuilder should be suitable for all sort of cases ... at least for all cases I have been facing since I have started with JavaScript (10 years of cases, I hope it's enough)

The Concept Behind

As you can see there are different sources behind the scene and the application is distributed across multiple files.
While this concept may be not that familiar or weird, this has been used by jQuery library since ever and even if teams may think it's not easy to maintain such distributed code, a proper structure following some standard could be more than we need to make everything simple.
As example, the file called variables.js contains all local scope variables reused inside other files as well.
Of course if a variable has a meaning for more than a file, it makes perfect sense to put it in the precedent one, even as undefined variables, to eventually assign it later.

Closure Compiler VS YUICompressor

If you checkout the whole project and you try to build once with YUI and once with Closure Compiler, simply swapping comment on line 63 of JSBuilder.py file, you will realize these minifiers are able to produce a completely different output.
I leave you read all comments inside JSBuilder.py in order to understand what happens there exactly, and I hope you will realize there is even more than zou have imagined.

Debugging Code

Sometimes you may need a portion of code for debugging purpose, and the not minified version of the file is the only place where it should be.
When it comes to debug, we would like to be able to export outside private variables or functions, still understanding their original names rather than the minified one.
The debug.js files as example is able to modify internal variables thanks to evil eval function.
This piece of code is surrounded by this comment:

//^
... the debug/evil code ...
//$

If you are familiar with regular expressions you can spot that ^ as "start evil code" and $ as "end evil code" have semantic meaning ... and this is what will never be included in the final minified version of our app.

Give It A Try

I hope those present in my Falsy Values workshops got the advantages a build system could bring on our daily basis work but I have written as many comments as possible to let you understand what is going on, how does it work, and why this simple script is freaking cool and it could potentially become our favorite build system.

What's Next

In my personal projects I have included an extra line at the end of the build.py able to create, per each build, documentation.
This kinda helps me to do not forget I have to document new or modified stuff, plus it gives me extra hints if some documentation has been lazily copied and pasted rather than being rewritten ... well, I honestly don't really like jsDocToolkit since JavaScript syntax has nothing to do with Java one but I agree is, at least, a tool flexible enough and still widely adopted and/or extended so ... if you all agree, I will put the magic documentation part there inside a build-with-doc.py file example.

7 comments:

Christopher said...

why closure compiler and YUIMin, but not the better/easier/& faster Uglify? Plus, it's written in JavaScript?

Andrea Giammarchi said...

Christopher I don't get the "easier" part since the usage of the minifier is totally transparent.

Moreover, I doubt Uglify works on WHS ( Windows Host Scripting ) so in any case I should use Rhino to create runtime a cross-platform JS environment and run Uglify through it.

I may update the file and put this 3rd option as well in any case but I hope you can appreciate the simplicity and flexibility of the current JSBuilder solution offers.

Anonymous said...

Hey - I am really happy to discover this. great job!

Christopher said...

Hi Andrea —

I don't know about running node on Windows... I guess you'd need to run uglify in Rhino, then. By easier, I guess it is subjective. For me, it is easier, for you, sounds like it is more difficult if you develop primarily on Windows.

Personally, while I see the great value of a solution like this, and while, as usual, I think you have done a great job, and as you mentioned, elegant, I would not be inclined to use it. There are many for this, but the main one being that most of what I do is Node JS development, and I use a Mac. So I build my code with JavaScript itself and then uglify the client-side code.

Thanks, I've learned a lot from your blog over the years, esp. about IE.

Andrea Giammarchi said...

I would still pack files for server side modules as well since HD cache is not that massive.

I am sure with node.js ready environment it's easier to minify via uglify.js but at the same time I am not sure uglify is better than closure or yui.

In any case you need a similar system offered by JSBuilder to be able to combine multiple fragmented files so why bother? If node.js is the environment, python and java are there by default ;)

As summary, JSbuilder.py would be ideal to build your own jQuery pieces of library or any other library with stand alone or fragmented JS files and cross platform, not everybody has a mac.

Of course, feel free to use node only, at least remember this page the day you'll need more portability

Andrea Giammarchi said...

btw Christopher, I have just added support for uglify-js ;-)

Christopher said...

nice1 Andrea!