Content model changes at scale – TELUS and CMS as Code

While catering to a variety of channels is challenging from a content creation perspective, the shipping and delivery of content rich applications at scale is a very demanding task for every engineering department too. Serving every distribution channel often requires custom technical solutions to deal with daily content change requirements and development of features in parallel.

The only solution to overcome these challenges is to reduce the role that humans play in the delivery process. A continuous delivery pipeline that includes automated tests for several stages of the development process is key to successful product delivery.

In this article, we’ll share how TELUS (a Canadian telecommunications company) handled the described challenges—running around 500 migrations while constantly shipping new features.

CMS as Code – content at scale

In a fast-moving development process, you need to be able to quickly modify your content in response to product needs. “The problem starts when you run your content in production” is a sentence we often hear when meeting customers because challenges tend to occur in moments when the content model changes.

• Should you change the content model manually and deploy your code hoping for the best?

• What is the best approach to avoid stressful deployments?

• How do I know who changed something and why?

Until recently, it was very hard to perform content model changes programmatically. This changed with the release of sandbox environments and support from migration CLI tooling. These two essential parts make "CMS as Code", which prescribes development pipelines to make shipping your products faster and more confidently a reality. They enable you to thoroughly test changes in an isolated environment and then apply them to your production environments.

Continuous content evolution and code delivery

Over the last quarter, the Contentful Developer Relations team travelled the globe to give talks about one possible implementation of CMS as code. We presented a delivery pipeline that automatically migrated, tested, and deployed code and content changes.

Martin Fowler describes one of the core characteristics of continuous delivery as follows:

Anybody can get fast, automated feedback on the production readiness of their systems any time somebody makes a change to them.

Changing content structures without affecting production readiness is hard to realize especially when your systems are fed by an API product like Contentful. Sandbox environments are the tool to move forward without affecting your consumer-facing product. This works by making a copy of the content from your master space in a new environment, which you can alter, run tests against, and evaluate to see if your changes lead to the expected outcome.

To realize a delivery pipeline, your Contentful space has to hold versioning information which can be included in a content type that is created only for this purpose. You then can tie your code base to a specific content version by including migration scripts in the code base itself. A CI service like TravisCI will check out a certain branch or version of your code, create a new sandbox environment in Contentful and perform new content migrations in case they have not yet been applied to the master space. This process can be used in testing and staging environments as well as local ones.

We in the Contentful Developer Relations team had planned to release the logic of applying coded migrations automatically as a Node.js package, to make the setup process easier for everybody, when we found out that someone else had run “CMS as Code” successfully in production and already published their tooling. The npm package contentful-migrate helps you to apply automated content model changes to a content delivery pipeline in the same way that we describe in our tutorials

We reached out to Deluan Quintao and Jerry Ma, senior development consultants from ThoughtWorks (a long-time TELUS Partner) who maintain the tooling, to discuss the project and learn how they apply “CMS as Code”.

Digital product evolution – TELUS as an example

At TELUS, they follow “CMS as code” with great success. Deluan and Jerry told us that the secret of the TELUS development teams boils down to characteristics we call CAT.

Your development process including changes of your content structures has to be consistent, automated and transparent at any time.

Consistent content model changes

A lot of our customers go through an intense period of experimenting before they settle on certain content structures that satisfy their requirements. Content modelling is hard, because quite like software development, content structures tend to evolve and change over time.

Code base features are implemented in branches via a version control system like git. And since it’s often that several teams work on development branches in parallel, you have to manage content model changes in several environments, too.

Consistency is indispensable in this scenario to guarantee that every feature branch is based on the same content structure. contentful-migrate uses Contentful’s migration tooling under the hood, which lets you apply scripted changes and opens the door for automation.

An automated process

Making content model changes can range from adding only one new field to a content type, to changing and transforming thousands of entries. Either way—what you want to avoid at all costs is human error.

Content model changes should always be applied via migration scripts. Performing manual changes one after another is neither a scalable nor repeatable solution.

We ran 500 migrations this year. Our content structure is tightly coupled to the code changes we make.

Deluan and Jerry told us that their process includes the creation of migration scripts by a developer, followed by the automated creation of sandbox environments in Contentful to then altering content structures and the connected code bases in the same step. This process leads to software environments that can be tested automatically with minimal human intervention. The production environment stays deployable at all times until everybody is sure that the additions are working as expected.

Transparent content progression

Working in a project at a speedy pace doesn’t just mean moving forwards quickly—a review process to guarantee high quality and documentation both play large roles, too. Knowing when certain code sections or content structures were introduced, but more importantly why, is essential to allow a good collaboration across teams.

Think of a scenario in which different product code bases are connected to a space in Contentful, and one developer comes across an unused field in the product he/she is working on. Finding out why this field is present can take a lot of time because there might be many people and products relying on the space and the question of whether the field is still in use arises (and if so, whether its removal will break a product in production).

A content delivery pipeline is the solution to such uncertainty. Structural content changes are only introduced in parity with code changes as part of the development process. Migration scripts and, more importantly, their history are accessible and analyzable by other developers at any point in time.

The accessible history is not the only advantage though. Code reviews are a common practice in software development to guarantee high quality software. At least four eyes have to confirm that code changes should be applied and by implementing “CMS as Code”, you can apply the same principles to your content structures—ensuring that everything is high quality!

Following these principles content model changes should be automated without involvement of humans and the reasons for changes should be understandable, even months down the road so procedures to realize further requirements can be evaluated. Auditing records of the content progression are the result of treating content as code.

Content and code evolution leading to success

The number of ran migrations at TELUS speaks for itself. Relying on their delivery pipeline lets them ship code and content changes with confidence and rapid pace. Their tool contentful-migrate is perfect to get you started with "CMS as Code" and it even comes with init and bootstrap commands that will set up your Contentful space for automated content changes at scale today. Give it a try and let us know how you set up your delivery pipeline (and perhaps even write about it?)