The DataHero Blog

Using Grasp.js

April 1st, 2014

Using_GraspJS

One of the key parts of our technology at DataHero is that we integrate with partners to let users seamlessly and easily analyze data from a variety of sources. Recently, we ran into a technical problem with one of our integrations, this post is about how a new tool called Grasp.js helped us propagate the fix.

First I’ll show you the problem and its solution, then I’ll show you how we used Grasp.js to spread the solution fast.

The Problem

When a user tried to import a dataset, they saw that it started to load and then after a certain point it would simply stop. Internally, we also saw that the dataset would stop importing midway the import. I’ll explain why this happened below, but first we need some background on our infrastructure.

At DataHero we use Node.js — Node.js uses modules that are loaded at runtime. One of the peculiarities of javascript is that it is prototypical. This means when you invoke an object’s method, the javascript engine looks up the object’s prototype chain for the method.

For more information on our stack visit our introductory post

Our integration stopped working because a module (the HTML5 module) was overriding the last function on the array prototype. This made a separate module, that was also overriding last on an array instance call the wrong function. Because in Node.js you pass callbacks to functions when they are doing IO, the callback was never being called, so the dataset import never finished.

The Solution

In order to solve this we needed to stop the HTML5 module from overriding the array prototype’s last function. Here are a couple examples of the required transformation:

tree.open_elements.last() → last(tree.open_elements)
current_token.data.last().nodeValue → last(current_token.data).nodeValue

However, there were several files where the change would have to be made and there were several variants on how the array was accessed than just the two examples above. Because there were a lot of places the transformation would have to occur, doing it manually would be tedious and error prone.   There were also several variants, meaning regex would not be the best solution either (since javascript isn’t a regular language).

Enter Grasp.js

I had recently heard about a tool called Grasp.js from a javascript newsletter. Grasp.js is a search/replace tool specifically for javascript. It works by parsing the code into an abstract syntax tree (AST) and then lets you define transformations on the tree. Thus, it doesn’t have the same limitations as regex. The benefits of using a tool like Grasp.js is that you don’t need to worry about typos, it’s automated, and the replacements you can make are much more expressive.

In short, with Grasp.js I could automate the transformations above and fix all bad calls to last at once.

Using Grasp.js (The Dirty)

To use Grasp.js you specify the transformation you want in terms of the abstract syntax tree. Specifically I used the feature where you provide example javascript to match against:

grasp --e 'some.javascript.toMatch()'

The other important feature to note is that it allows setting wildcards to match anything:
grasp --e '__.open_elements'
grasp --e '$captureMe.open_elements'

To do the refactoring I would need to match any call to the last function, like these two examples:
tree.open_elements.last()
current_token.data.last().nodeValue

My initial attempt to match the previous code used a wildcard and looked like this:

grasp --e '__last()'

This was supposed to match anything that ended in last() but no matches were printed to the console. It certainly looked right. I reached out to the Grasp.js creator George Zahariev for help. He explained that the problem was I was thinking of the match in regex terms instead of AST terms, I needed to modify it to:

grasp --e '__.last()'

The “.” made a big difference. To see the difference under the hood we can use a javascript parser and see the code as Grasp.js sees it. To do that I installed Acorn (a javascript parser) and ran both examples through it:

acorn = require('acorn');
 
a = acorn.parse('last()');
JSON.stringify(a, null, 2);
 
b = acorn.parse('__.last()');
JSON.stringify(b, null, 2);

The result is the structure printed as JSON, but I’ve reproduced it in a visual format here:

AST for last():

AST for __.last():

As you can see, the two examples are very different. For last() there is no MemberExpression, and no notion of the left Identifier node. Since we used a wildcard (“__“) the left node can match anything — it can match tree.open_elements, attributes, or other code. That means that our full __.last() can match any of these:

tree.open_elements.last()
attributes.last()
current_token.data.last()

Now that I was able to match any variation to a call to the last() function, the command had to be completed with the replacement that should be applied. In Grasp.js it is:

grasp -r -i -e '$leftNode.last()' -R 'last({{leftNode}})' ./lib

The above code instead of just using a wildcard to match, captures the left node into the leftNode variable. It then replaces it (the “-R”) with last(leftNode) and does this in the lib folder.

In the end, Grasp.js replaced almost 90 occurrences of the problem. Also, because it was automated it was guaranteed that the replacements were correct! Grasp.js is awesome because it lets you express refactorings at a higher level without much work. It’s fast and now we use it all the time.

The result was a totally clean and quick commit that was done instantly.

Do you have any questions about Grasp.js or want to learn more? Leave a comment and we’ll continue the discussion.

Want to join the team? We’re Hiring!

By Islam Sharabash

Create my Free DataHero Account

Get the fastest, easiest way to understand your data today.