Javascript all the things! Updated SDKs and import-export tools
A long time ago, Contentful released a tool called Space-sync. It allowed users to sync contents across spaces and make mirror copies which helped when dealing with development environments.
This tool, however, had some inherent architectural flaws. It was born from the community, and only later adopted by Contentful: this meant that maintaining it in its form was less than ideal. Eventually, we figured that the best solution was just to deprecate its use and come up with new tools, which we could design from the ground-up to perfectly match our customers’ needs.
To make the tools more flexible, we decided to split the functionality into smaller, more manageable parts — we’re trained to think in a modular way, after all! That’s why we’ve come up with contentful-import and contentful-export, two npm packages that can be used as standalone CLI tools or as modules in your application. This was a few months ago.
Although we had officially deprecated the use of Space-sync, our alternatives weren’t as robust as we wanted them to be. So more recently, we worked work on making these tools more reliable, performant, and flexible. Today I’m extremely happy to announce that the day has come, when you can set your mind at ease and just execute contentful-import, because now your import will not stop halfway through because of rate limits or an unexpected error.
In the process of improving these tools, we also took some time to work on our Javascript SDKs, which the import and export modules uses behind the scenes. So let’s take a look at the improvements!
What we updated
Our Javascript SDKs are universal: this means that they’re designed to work in a Node environment and in browsers. Thankfully, both these worlds move and evolve faster than ever, so what seemed cutting edge one year ago can be a stable feature today. This is why we periodically look at these landscapes and see what we can do to take advantage of recent improvements while still retaining the biggest possible support.
NodeJS and browser support
In a Node environment, the minimum required version was bumped to 4.7. This underlying change touched both SDKs and import and export tools. Because of this, we were able to remove a lot of boilerplate and polyfill dependencies: we now rely on built-in Node features, which means less code in the final bundles that you ship, and less code to maintain and bugfix. After all, less is more!
In an ideal world, we could remove all the polyfills, but we must also deal with browser support. As you can see in the .babelrc configuration file, we currently commit to supporting the latest 3 versions of evergreen browsers, and Internet Explorer 11. Please be aware of this when using our Javascript libraries in a browser environment.
Better error management
Another significant improvement is a better error management. This applies to SDK libraries, and import and export tools: rate limits and other unexpected errors are now handled by default:
As Contentful implements rate limits as safeguards, it may be possible that a script exceeds its allowance in terms of requests per second or per hour. However, as the API response includes useful headers that say when it will be possible to make the next request, the SDK uses the value in those headers and it will pause the execution until it receives a green light.
In the event of an unexpected server error, the libraries will automatically wait before retrying. As these errors are unpredictable by the library, it will retry using increasing time-outs before actually giving up. This means that, should the API fail for a few seconds, the library would use a “wait-and-see” approach, which is perfect for example during a mass import.
A little but nice-to-have feature we’ve added to the import tool is the ability to manage existing entries in the destination space. By using a PUT request rather than a default POST, we’re able to specify the ID of an entry in the target space, which will be the same as in the source space. But what happens when the ID is already taken? A PUT request actually serves us well even in this situation, because it will try to update the entry using the data from the source space. Of course, if the content type has changed this would probably fail, for instance if we’re trying to set a value to a field that was removed. But besides that situation, the ability of overriding entries is useful in situations where two spaces needs to be synced continuously, and it also helps when managing links between entries.
Great! What's next?
This was the round-up of the changes we recently published. With more lightweight modules, better error management, and a slew of minor improvements, we’re confident that we can finally meet the needs of our users in a truly satisfying way. And don't forget, all these modules are open sourced on our Github page, which means that you can open issues, create pull requests, and follow the overall development. Don't shy away from letting us know what you think!
Here's where you can get all these goodies:
