When building an application with Node.JS you usually reach a point where you want to extract logic from the application and put it into an isolated module. This is typically done when the code turns out to be usable, but not strictly coupled to the application. So instead of keeping it in the app, it gets moved to its own node module, with its own version control repository, where it can be treated separately from the rest of the application. By doing this you gain all kinds of benefits: the code can be shared across multiple consumers, the test runs are scoped to the module by default, the changes of the module are versioned independently from its consumer, open-sourcing becomes an option, which allows collaboration with external people — and so on.
A bit too many cross-functional modules
Module management problems
So, while moving isolated logic into its own module generates a lot of advantages, it also makes it more complex to change them in the scope of the entire application and to ensure that such changes are not breaking the application. Typically this is the time when you start cloning the module repository to some place and start doing the npm link dance, which allows you to inject npm modules — that are located somewhere else on your hard drive — into an application. This is a good solution for smaller applications which only have small amount of modules, but it’s getting very frustrating when you have a lot of them, or if the module’s sub-modules are subject to change as well. It is getting even worse if your code contains instanceof checks which makes it crucial to only ever have a single installation of a module.
A possible solution
The following code snippet is about reading an entry via a node module and about handling its success and error case. In the error case it is furthermore checking if the error is of a specific error instance (which live in our errors module) and running different code depending on the result.
A typical problem with this approach can be seen when an application that uses this code and the module that allows reading content entries are running a different installation of the errors module – even though they might use the same version. Suddenly the errors are no longer inheriting from the same module but are two entirely independent objects in memory and therefore the instanceof check is about to fail. This happened to me (and my colleagues) so often when using
npm link and running
npm install inside the module’s code directory that we decided to search for a way that allows modification of the modules while staying inside the application.
Since the overall idea was about being able to modify the node modules used in an application while having full version control over the changes, we were thinking about replacing a node module with its respective repository. Essentially this meant:
Go to the module directory that needs to be changed
Move the dependencies of that module to somewhere else
Remove the module’s directory
Create a clone of the repository
Checkout the right version of the repository
Move the dependencies of that module back
Since doing this manually for a longer time was a no-go, we looked into automating this process. The result was gitify-dependencies, which is not only replacing the module directories, but also utilizes git-new-workdir to allow collaborative editing on the same repository across multiple working directories.
Let’s imagine the following application and its dependencies:
npm dedupe on this will remove all the duplications and reduces the dependencies to the following:
gitify-deps vs npm link
So let’s imagine a situation where a change needs to be introduced to the errors and the validations module. With
npm link one would probably do this:
An interesting side effect of this approach can be seen when
npm ls is called inside the application folder.
It states that suddenly gitify-dependencies-test-errors is installed as a child of the validations module again, although it was gone previously. This happens because of the
npm install step. So while this can be fixed manually, it’s already obvious that this approach is generating all kinds of unwanted side effects.
With gitify-dependencies it looks like this:
Running this will generate the following output:
After converting every matching module into its respective git repository, it is now possible to cd into them and make changes which are tracked by git.
By doing modifications directly inside the node_modules directory, it is now possible to change the application’s dependencies while keeping the possibility to run the (integration) tests of the app.
Options of gitify-deps
As seen in the example above
gitify-deps can be called with some additional option flags. The only required and most important one is the
--gitify-url-pattern flag. While iterating through every of the installed packages gitify-deps is comparing the package’s URL property with the flag value and converts the module to its respective git repository if the comparison succeeds. In addition to this, you can use the following ones:
Speed up your workflow now
Installing and getting started with gitify-dependencies is simple and generates a huge value when working in complex applications with lots of dependencies. It allows you to manipulate dependencies as part of your application while keeping track of the changes via git. Just run
npm install -g gitify-dependencies; gitify-deps -p <needle> and — profit!
Thanks for reading, and let us know what you think.