Versioning applications using migrations

Developing applications involves several phases:

  • Developing from scratch
  • Making a minor change, such as a patch
  • Making a major change, which is a breaking change

For details, see Semantic Versioning 2.0.0.

While developing from scratch is rather straightforward, making minor or major changes requires a structured process.

Continue reading to see a complete development cycle of our Blog in 5 minutes example application, and to use the Contentful migration CLI to script changes to a content model and entries in a structured and reproducible way.

You can also watch this video a hands-on example of content migrations.

Preparation step

Installing the blog example application using the Contentful CLI

Install or update the Contentful CLI to its latest version:

npm install -g contentful-cli

Run the guide:

contentful guide

Follow the steps until you have the app up and running on your workstation.

Navigate to the folder where you stored the blog application code, called contentful-custom-app and do the following:

  • Initialize a git repository, which will be used to reflect changes to the application:
    git init
    git add .
    git commit -m 'Initial commit'
    
  • Export the space ID that was created by the contentful guide command, so you can reuse it:
    export SPACE_ID=my-space-id
    

The initial content model looks as follows:

Migration CLI

To install the migration CLI, which is currently in beta, use the following command:

npm install -g contentful-migration-cli

From v1.0 to v1.1 for a minor change

This change adds a blog-post category that displays on the main page of the blog:

Migrating the content model

Put all of your migrations in one location, so it’s easier for others to find them:

mkdir migrations

Use the following script to add the category field as a Symbol:

module.exports = function (migration) {
  // Create a new category field in the blog post content type.
  const blogPost = migration.editContentType('blogPost');
  blogPost.createField('category')
    .name('Category')
    .type('Symbol');
}

Name the script 01-add-category-field.js, save it in migrations, and run it on your space:

contentful-migration --space-id $SPACE_ID migrations/01-add-category-field.js

You can see the migration plan, and agree or disagree to its execution.

Intializing the blog-post categories

Thus far, existing blog post entries will not have any content in the category field.

The category field of each blog post entry can be manually updated in the web app. If there are many existing blog post entries, this task becomes unmanageable without content transformations.

This is why the migration object comes with functions that transform content entries.

Use transformEntries because it takes each entry of a content type, extracting selected content-type fields, and applies a transformation function on them. The result can replace the content of another set of fields of the same entry.

For our blog post use case, we'll , so that we can have a valid initial repartition of blog posts across a set of initial categories.

Use the following script to initialize the new category field using the existing blog-post tags:

module.exports = function (migration) {
  // Simplistic function deducing a category from a tag name.
  const categoryFromTags = (tagList) => {
    if (tagList.includes('javascript')) {
      return 'Development'
    }
    return 'General'
  }

  // Derives categories based on tags and links these back to blog post entries.
  migration.transformEntries({
    // Start from blog post's tags field
    contentType: 'blogPost',
    from: ['tags'],
    // We'll only create a category using a name for now.
    to: ['category'],
    transformEntryForLocale: async (from, locale) => {
      return {
        category: categoryFromTags(from.tags[locale])
      }
    }
  })
}

Name the script 02-transform-content.js, save it in migrations and run it on your space:

contentful-migration --space-id $SPACE_ID migrations/02-transform-content.js --yes

After the script executes, see the results in the Contentful web app:

open https://app.contentful.com/spaces/$SPACE_ID/entries/2PtC9h1YqIA6kaUaIsWEQ0

The example app changes look as follows:

The blogPost entries have been updated with the category information computed using the tags of the post.

Next, version the changes in Git:

git checkout -b blog-v1.1
git add .
git commit -m 'Add category field to blog posts based on tags.'

Displaying the category in the example app

We can now make some changes to the code of the application to display the category alongside blog posts.

Download this patch

Use the patch to make all required changes on your code at once:

git apply migrations-1.0-1.1.patch

To see the changes, run the example app again:

npm run dev

The example app changes look as follows:

Next, commit the changes in Git:

git add .
git commit -m 'Display blog-post category in the article-preview component.'

At this point, the branch is ready for a pull request for other colleagues to review. This allows any developer to run a migration on their own environment and see how the change looks.

From v1.1 to v1.2 for transforming the category to a reference field

This section will explain how to update the example application to display the list of existing categories on the home page, and add a dedicated page which lists all blog posts that are part of a given category.

The update requires a new category content type, with a URL slug. It is done by creating a migration script to transform our content model as follows:

It also requires to create categories using the existing blog-post information, so that the updated home page changes look as follows:

This approach implements the forward-only migration principle explained in our "Infrastructure as code" article.

Initializing a new branch

Create a separate branch to make these changes:

git checkout -b blog-v1.2

Migrating the content

The following migration script uses a function called deriveLinkedEntries to generate new categories from existing blog posts, by using the original blog-post category field:

module.exports = function (migration) {
  // New category content type.
  const category = migration.createContentType('category')
    .name('Category')
    .displayField('name');
  category.createField('name').type('Symbol').required(true).name('Name');
  category.createField('slug').type('Symbol').required(true).name('URL Slug').validations([{ "unique": true }]);
  category.createField('image').type('Link').linkType('Asset').name('Image');

  // Create a new category field in the blog post content type.
  const blogPost = migration.editContentType('blogPost')
  blogPost.createField('category_ref')  // Using a temporary id to be able to transform entries.
    .name('Category')
    .type('Link')
    .linkType('Entry')
    .validations([
      {
        "linkContentType": ['category']
      }
    ])

  // Derives categories based on the existing category Symbol, and links these back to blog post entries.
  migration.deriveLinkedEntries({
    // Start from blog post's category field
    contentType: 'blogPost',
    from: ['category'],
    // This is the field we created above, which will hold the link to the derived category entries.
    toReferenceField: 'category_ref',
    // The new entries to create are of type 'category'.
    derivedContentType: 'category',
    // We'll only create a category using a name and a slug for now.
    derivedFields: ['name', 'slug'],
    identityKey: async (from) => {
      // The category name will be used as an identity key.
      return from.category['en-US'].toLowerCase()
    },
    deriveEntryForLocale: async (from, locale) => {
      // The structure represents the resulting category entry with the 2 fields mentioned in the `derivedFields` property.
      return {
        name: from.category[locale],
        slug: from.category[locale].toLowerCase()
      }
    }
  })

  // Disable the old field for now so editors will not see it.
  blogPost.editField('category').disabled(true)
}

Name the script 03-category-link.js, save it in migrations and run it on your space:

contentful-migration --space-id $SPACE_ID migrations/03-category-link.js --yes

Updating the code

Download this patch, and apply it to make all required changes on your code at once:

git apply migrations-1.1-1.2.patch

To see the changes, run the example app again:

npm run dev

The new version changes look as follows:

Commit the changes to Git:

git add .
git commit -m 'v2: Category to reference with dedicated page'

The change is now ready to be reviewed and deployed.

Conclusion

This guide explained a process for managing application changes through code, using Contentful tooling. It covered the Migration CLI, as well as the Contentful CLI.

Possible next steps: