Combine external data with Experiences

Overview

Contentful is typically used as one among multiple data sources in a user's systems. In a given experience (usually a full page), you might find youself pulling in and mixing together information from both Contentful and external data sources such as Product Information Management (PIM) systems, Digital Asset Management (DAM) systems, or similar systems for enhancing the user experience while preventing maintenance of the same data (e.g., pricing data) in multiple places.

This comprehensive guide demonstrates how to integrate external data sources with Contentful Experiences, showing you how to create custom components that can fetch and display data from third-party APIs while maintaining optimal performance through server-side rendering.

Table of contents

• Architecture overview
• Prerequisites
• Setting up the project structure
• Creating the product extraction utility
• Building the API route for client-side fetching
• Creating the custom product component
• Setting up the React context
• Implementing the page component
• Registering the custom component
• Testing the implementation
• Caching considerations
• Security considerations

Architecture overview

The solution follows a hybrid approach that combines server-side and client-side data fetching to provide optimal performance for end users while maintaining a smooth editing experience in Contentful Studio.

Key components

• Custom component in Studio with a variable for external system ID (e.g., PIM ID or SKU)
• Server-side data extraction that analyzes the experience structure and fetches external data
• Client-side fallback API for editor preview mode
• React context for passing prefetched data to components
• Caching layer to optimize performance and reduce API calls

Data flow

1. End-user experience:
   1. Experience is fetched on the server side
   2. Product IDs are extracted
   3. External data is fetched
   4. Data is passed to components via context
2. Editor experience:
   1. Experience is passed from Studio
   2. Components fetch data on the client side via API route
   3. Data is displayed with editor context

Prerequisites

Before starting this implementation, make sure you have:

• A Next.js project with App Router
• Contentful Experiences SDK installed (@contentful/experiences-sdk-react)
• Access to an external API (we'll use Shopify's Mock Shop as an example)
• Basic understanding of React Server Components and client components
• React Query by TanStack (@tanstack/react-query) for client-side data fetching

Understanding the Mock Shop integration

The example uses Shopify's Mock Shop as a demonstration PIM system. This service provides:

• GraphQL API: A complete GraphQL endpoint at https://mock.shop/api.
• Sample product data: Pre-populated with realistic product information.
• Product IDs: Simple numeric IDs that are converted to Shopify's Global ID format (e.g., gid://shopify/Product/7982905098262).
• Rich product data: Including titles, descriptions, price ranges, and featured images.

Available product IDs for testing

You can explore the full example product catalog by visiting the Mock Shop Playground and running queries such as the following.

{
  products(first: 10) {
    edges {
      node {
        id
        title
        description
        priceRange {
          minVariantPrice {
            amount
            currencyCode
          }
        }
      }
    }
  }
}

Setting up the project structure

The example implementation follows this directory structure:

src/
├── app/
│   ├── api/
│   │   └── pim/
│   │       └── [id]/
│   │           └── route.ts
│   ├── [locale]/
│   │   └── [slug]/
│   │       ├── page.tsx
│   │       └── page.module.css
│   ├── layout.tsx
│   └── globals.css
├── components/
│   ├── PriceComponent.tsx
│   ├── PriceComponentRegistration.tsx
│   ├── ProductComponent.tsx
│   ├── ProductComponentRegistration.tsx
│   ├── ProductProvider.tsx
│   ├── Experience.tsx
│   └── types.ts
├── services/
│   └── pim.ts
├── utils/
│   ├── products.ts
│   └── format.ts
├── studio-config.ts
├── getExperience.ts
└── i18n.ts

Creating the PIM service layer

The example uses a dedicated service layer to handle all interactions with the external PIM system. This approach provides better separation of concerns and makes the code more maintainable. The typing is based on Mock Shop's response structure, and you'll need typing according to the external system you're connecting to.

// src/services/pim.ts
import { useQuery } from '@tanstack/react-query';

export const convertIdToPimId = (productId: string) =>
  `gid://shopify/Product/${productId}`;

export const convertPimIdToId = (pimId: string) =>
  pimId.replace('gid://shopify/Product/', '');

type ProductPrice = {
  amount: string;
  currencyCode: string;
};

type ProductPriceRange = {
  minVariantPrice: ProductPrice;
  maxVariantPrice: ProductPrice;
};

type ProductImage = {
  id: string;
  url: string;
};

export type ProductData = {
  id: string;
  title: string;
  description: string;
  priceRange: ProductPriceRange;
  featuredImage: ProductImage;
};

type ProductApiResponse = {
  product: ProductData;
};

type FetchResponse<T> = {
  data: T;
};

/** Fetches the product data for a single product ID using GraphQL. */
export const fetchProductData = async (
  productId: string
): Promise<ProductData> => {
  const response = await fetch('https://mock.shop/api', {
    body: JSON.stringify({
      query: /* GraphQL */ `
        {
          product(id: "gid://shopify/Product/${productId}") {
            id
            title
            description
            priceRange {
              minVariantPrice {
                amount
                currencyCode
              }
              maxVariantPrice {
                amount
                currencyCode
              }
            }
            featuredImage {
              id
              url
            }
          }
        }
      `,
    }),
    method: 'POST',
    headers: {
      accept: 'application/json',
      'content-type': 'application/json',
    },
    // Facilitate Next's built-in caching
    cache: 'force-cache',
  });

  const result = (await response.json()) as FetchResponse<ProductApiResponse>;
  return result.data.product;
};

/** Fetches all product data for a list of product IDs in a batch using GraphQL. */
export const fetchProductsData = async (
  productIds: string[]
): Promise<ProductData[]> => {
  const response = await fetch('https://mock.shop/api', {
    body: JSON.stringify({
      query: /* GraphQL */ `
        query searchProducts($ids: [ID!]!) {
          nodes(ids: $ids) {
            ... on Product {
              id
              title
              description
              priceRange {
                minVariantPrice {
                  amount
                  currencyCode
                }
                maxVariantPrice {
                  amount
                  currencyCode
                }
              }
              featuredImage {
                id
                url
              }
            }
          }
        }
      `,
      variables: {
        ids: Array.from(new Set(productIds)).map(convertIdToPimId),
      },
    }),
    method: 'POST',
    headers: {
      accept: 'application/json',
      'content-type': 'application/json',
    },
    // Facilitate Next's built-in caching
    cache: 'force-cache',
  });

  const result = (await response.json()) as FetchResponse<{
    nodes: ProductData[];
  }>;

  return result.data.nodes;
};

/** Fetches the product data for a single product ID via the dedicated API endpoint for client-side use. */
export const fetchProductDataFromApi = async (
  productId: string
): Promise<ProductData> => {
  const response = await fetch(`/api/pim/${productId}`);
  if (!response.ok) {
    throw new Error('Failed to fetch product data');
  }
  return response.json();
};

/** React Query hook for client-side product data fetching. */
export const useClientSideProductData = (
  productId: string,
  enabled?: boolean
) => {
  return useQuery({
    queryKey: ['product', productId],
    queryFn: () => fetchProductDataFromApi(productId),
    staleTime: 5 * 60 * 1000,
    gcTime: 10 * 60 * 1000,
    enabled: !!productId && enabled,
  });
};

Creating the product extraction utility

The product extraction utility analyzes the experience structure and extracts all external system IDs that need to be fetched.

// src/utils/products.ts
import { Experience } from '@contentful/experiences-sdk-react';
import { EntityStore } from '@contentful/experiences-core';
import { ComponentTreeNode } from '@contentful/experiences-core/types';
import {
  convertPimIdToId,
  fetchProductsData,
  ProductData,
} from '@/services/pim';

const getProductNodes = (nodes: ComponentTreeNode[]): ComponentTreeNode[] =>
  nodes.reduce((acc, node) => {
    // Only pay attention to nodes with the right definition IDs of the relevant custom components
    if (['custom-price', 'custom-product'].includes(node.definitionId)) {
      acc.push(node);
    }
    // Recurse into children
    if (node.children) {
      acc.push(...getProductNodes(node.children));
    }
    return acc;
  }, [] as ComponentTreeNode[]);

const isValidProductId = (productId: unknown): productId is string => {
  return typeof productId === 'string' && productId.length > 0;
};

export const extractProductIds = (
  experience: Experience<EntityStore>
): string[] => {
  // Get all nodes with the right definition IDs of the relevant custom components
  // Parsing the tree is a NOT OFFICIALLY SUPPORTED strategy.
  // We’re exploring having a standalone SDK method for this in a future version, so use at your own risk.
  const nodes = getProductNodes(
    experience?.entityStore?.experienceEntryFields?.componentTree.children ?? []
  )
    .map((node) => {
      // Duck-type check for the right type of the custom variable
      if (node.variables.product.type === 'UnboundValue') {
        // Get the effective product ID from the unbound value store
        const productId =
          experience?.entityStore?.unboundValues[node.variables.product.key];
        return productId?.value;
      }

      return undefined;
    })
    // Filter out invalid product IDs (e.g., `undefined` from above due to failed duck-type check or empty strings)
    .filter(isValidProductId);

  // Return unique product IDs to prevent overfetching
  return Array.from(new Set(nodes));
};

export const fetchProducts = async (
  productIds: string[]
): Promise<Record<string, ProductData>> => {
  // Fetch all products from the PIM system in a batch
  const products = await fetchProductsData(productIds);

  // Return a map of product IDs to product data for easy access using product IDs as keys
  return Object.fromEntries(
    await Promise.all(
      products.map((product) => [convertPimIdToId(product.id), product])
    )
  );
};

Building the API route for client-side fetching

In editor view, the experience does not get fetched on the server-side as for end users but gets communicated into the canvas view by Contentful instead. The whole experience will be null and fetching is a no-op function. So for editor mode, we need a client-side API route that can fetch individual product data on the fly. This ensures the editing experience matches the end-user experience.

// src/app/api/pim/[id]/route.ts
import type { NextRequest } from 'next/server';
import { fetchProductData } from '@/services/pim';

export async function GET(
  _req: NextRequest,
  ctx: RouteContext<'/api/pim/[id]'>
) {
  const { id } = await ctx.params;
  const product = await fetchProductData(id);
  return Response.json(product);
}

This simple API route leverages the existing PIM service to fetch product data. The service takes care of getting the required data from the third-party system, including potential authentication, proxying, other security measures, and data transformation, making the API route clean and focused.

Creating the custom components

The example includes two custom components: a PriceComponent for displaying product prices and a ProductComponent for displaying full product information. Both components handle both server-side prefetched data and client-side fetching for editor mode.

The purpose of having a Price custom component and Product custom component is to demonstrate that multiple custom components can be fed by the same external data.

Utility types

In the following components, we're using a small utility type that applies to all Experience components and gets populated by the Experiences SDK automatically.

export type CustomComponentProps<T> = T & {
  className?: string;
  isEditorMode?: boolean;
};

Price component

Rendering

// src/components/PriceComponent.tsx
import { FC } from 'react';
import { useFormatPrice } from '@/utils/format';
import { usePrefetchedProducts } from './ProductProvider';
import { useClientSideProductData } from '@/services/pim';

type PriceComponentProps = {
  productId: string;
  isEditorMode?: boolean;
};

export const PriceComponent: FC<PriceComponentProps> = ({
  productId,
  isEditorMode,
}) => {
  const formatPrice = useFormatPrice();
  // Get all prefetched products from the server side
  const products = usePrefetchedProducts();
  // Get the product data for the current product ID from the prefetched products
  const product = products[productId];
  // When in editor mode or we don't have the product data pre-fetched on the server side, we need to fetch it on the client side
  const { data, isLoading } = useClientSideProductData(
    productId,
    // Enable for editor/canvas mode or if the product data was not prefetched on the server side properly
    isEditorMode || !product
  );
  // Use the prefetched product data if available, otherwise use the client-side fetched data
  const productData = product ?? data;

  // Catch fresh instances of the component without a product ID being set yet (i.e. editor may be working on it still)
  if (!productId) {
    return 'TBD';
  }

  // Loading state in case the product data is being fetched on the client side
  if (isLoading) {
    return 'Fetching…';
  }

  // Last resort placeholder in case no product could be resolved both on the server side and client side for the given ID
  if (!productData) {
    return 'n/a';
  }

  // Get price from product data
  const { priceRange } = productData;
  const minPrice = parseFloat(priceRange.minVariantPrice.amount);
  const maxPrice = parseFloat(priceRange.maxVariantPrice.amount);
  const currency = priceRange.minVariantPrice.currencyCode;

  return (
    <>
      {minPrice === maxPrice ? (
        formatPrice(minPrice, currency)
      ) : (
        <>
          {formatPrice(minPrice, currency)}-{formatPrice(maxPrice, currency)}
        </>
      )}
    </>
  );
};

Custom component registration

// src/components/PriceComponentRegistration.tsx
import { ComponentRegistration } from '@contentful/experiences-sdk-react';
import { Suspense } from 'react';
import clsx from 'clsx';
import { PriceComponent } from './PriceComponent';
import styles from './PriceComponent.module.css';

type PriceComponentProps = CustomComponentProps<{
  product: string;
}>;

export const PriceComponentRegistration: ComponentRegistration = {
  component: ({
    className,
    product,
    isEditorMode,
    // https://www.contentful.com/developers/docs/experiences/custom-components/#component-requirements
    ...rest
  }: PriceComponentProps) => {
    return (
      <div className={clsx(styles.price, className)} {...rest}>
        <Suspense fallback={<div>Loading…</div>}>
          <PriceComponent productId={product} isEditorMode={isEditorMode} />
          {isEditorMode && <div className={styles.info}>{product}</div>}
        </Suspense>
      </div>
    );
  },
  options: {
    enableEditorProperties: {
      isEditorMode: true,
    },
    wrapComponent: false,
  },
  definition: {
    id: 'custom-price',
    name: 'Price',
    category: 'External Content',
    variables: {
      product: {
        displayName: 'Product ID',
        type: 'Text',
        group: 'content',
      },
    },
  },
};

Product component

The product component utilizes the price component internally and displays additional data such as product titles and images.

Rendering

// src/components/ProductComponent.tsx
import { FC } from 'react';
import { Card, Image, Tag, Flex } from 'antd';
import { usePrefetchedProducts } from './ProductProvider';
import { useClientSideProductData } from '@/services/pim';
import { PriceComponent } from './PriceComponent';

type ProductComponentProps = {
  productId: string;
  isEditorMode?: boolean;
};

export const ProductComponent: FC<ProductComponentProps> = ({
  productId,
  isEditorMode,
}) => {
  const products = usePrefetchedProducts();
  const product = products[productId];

  // When in editor mode or we don't have the product data pre-fetched on the server side, we need to fetch it on the client side
  const { data, isLoading } = useClientSideProductData(
    productId,
    isEditorMode || !product
  );
  const productData = product ?? data;

  if (!productId) {
    return <Card>TBD</Card>;
  }

  if (isLoading) {
    return 'Fetching…';
  }

  if (!productData) {
    return 'n/a';
  }

  return (
    <Card
      cover={
        <Image src={productData.featuredImage.url} alt={productData.title} />
      }
      extra={isEditorMode ? <Tag>{productId}</Tag> : null}
    >
      <Flex gap="middle" vertical>
        <Card.Meta
          title={productData.title}
          description={productData.description}
        />
        <PriceComponent productId={productId} isEditorMode={isEditorMode} />
      </Flex>
    </Card>
  );
};

Custom component registration

// src/components/ProductComponentRegistration.tsx
import { ComponentRegistration } from '@contentful/experiences-sdk-react';
import clsx from 'clsx';
import { ProductComponent } from './ProductComponent';

type ProductComponentProps = CustomComponentProps<{
  product: string;
}>;

export const ProductComponentRegistration: ComponentRegistration = {
  component: ({
    className,
    product,
    isEditorMode,
    // https://www.contentful.com/developers/docs/experiences/custom-components/#component-requirements
    ...rest
  }: ProductComponentProps) => {
    return (
      <div className={clsx(className)} {...rest}>
        <ProductComponent productId={product} isEditorMode={isEditorMode} />
      </div>
    );
  },
  options: {
    enableEditorProperties: {
      isEditorMode: true,
    },
    wrapComponent: false,
  },
  definition: {
    id: 'custom-product',
    name: 'Product',
    category: 'External Content',
    variables: {
      product: {
        displayName: 'Product ID',
        type: 'Text',
        group: 'content',
      },
    },
  },
};

Setting up the React context

The React context allows us to pass prefetched data from the server to client components efficiently.

// src/components/ProductProvider.tsx
'use client';

import { createContext, FC, PropsWithChildren, useContext } from 'react';
import { ProductData } from '@/services/pim';

const PrefetchedProductContext = createContext<Record<string, ProductData>>({});

/** Returns a map of all prefetched products based on server-side extracted PIM data. */
export const usePrefetchedProducts = () => useContext(PrefetchedProductContext);

type PrefetchedProductProviderProps = PropsWithChildren<{
  products: Record<string, ProductData>;
}>;

/** Provides a map of all prefetched products from the server side to the client component tree. */
export const PrefetchedProductProvider: FC<PrefetchedProductProviderProps> = ({
  children,
  products,
}: PrefetchedProductProviderProps) => {
  return (
    <PrefetchedProductContext.Provider value={products}>
      {children}
    </PrefetchedProductContext.Provider>
  );
};

Implementing the page component

The page component orchestrates the entire flow: fetching the experience, extracting product IDs, fetching external data, and rendering everything.

// src/app/[locale]/[slug]/page.tsx
import { Layout, LayoutHeader, LayoutContent, LayoutFooter } from 'antd';
import { getExperience } from '@/getExperience';
import { extractProductIds, fetchProducts } from '@/utils/products';
import { detachExperienceStyles } from '@contentful/experiences-sdk-react';
import { Experience } from '@/components/Experience';
import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer';
import styles from './page.module.css';

type Page = {
  params: Promise<{ locale?: string; slug?: string; preview?: string }>;
  searchParams: Promise<{ [key: string]: string | string[] | undefined }>;
};

export default async function ExperiencePage({ params, searchParams }: Page) {
  const { locale = 'en-US', slug = 'home-page' } = (await params) || {};
  const { isPreview, expEditorMode } = (await searchParams) || {};
  const preview = isPreview === 'true';
  const editorMode = expEditorMode === 'true';
  const { experience, error } = await getExperience(
    slug,
    locale,
    preview,
    editorMode
  );

  if (error) {
    return <>{error.message}</>;
  }

  // Extract product IDs from experience and fetch the products on the server side
  const productIds = experience ? extractProductIds(experience) : [];
  const products = await fetchProducts(productIds);

  // Extract the styles from the experience
  const stylesheet = experience ? detachExperienceStyles(experience) : null;

  // Experience currently needs to be stringified manually to be passed to the component
  const experienceJSON = experience ? JSON.stringify(experience) : null;

  return (
    <Layout className={styles.layout}>
      {stylesheet && <style>{stylesheet}</style>}
      <LayoutHeader className={styles.header}>
        <Header />
      </LayoutHeader>
      <LayoutContent className={styles.content}>
        <Experience
          experienceJSON={experienceJSON}
          locale={locale}
          products={products}
        />
      </LayoutContent>
      <LayoutFooter className={styles.footer}>
        <Footer />
      </LayoutFooter>
    </Layout>
  );
}

Update your src/components/Experience.tsx to handle the editor mode:

// src/components/Experience.tsx
import React, { useState } from 'react';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { ExperienceRoot } from '@contentful/experiences-sdk-react';
import { PrefetchedProductProvider } from './ProductProvider';
import { ProductData } from '@/services/pim';

interface ExperienceProps {
  experienceJSON: string | null;
  locale: string;
  /** Pre-fetched products from the server side */
  products: Record<string, ProductData>;
}

const Experience: React.FC<ExperienceProps> = ({
  experienceJSON,
  locale,
  products,
}) => {
  const [queryClient] = useState(() => new QueryClient());

  return (
    <QueryClientProvider client={queryClient}>
      <PrefetchedProductProvider products={products}>
        <ExperienceRoot experience={experienceJSON} locale={locale} />
      </PrefetchedProductProvider>
    </QueryClientProvider>
  );
};

Registering the custom component

Finally, register your custom component with the Experiences SDK.

// src/studio-config.ts
import { defineComponents } from '@contentful/experiences-sdk-react';

defineComponents([PriceComponentRegistration, ProductComponentRegistration]);

Testing the implementation

Testing end-user experience

To test the end-user experience:

1. Create an experience in Contentful Studio.
2. Add the "Price" or "Product" component to your experience from the "External Content" category.
3. Set a valid product ID (see Mock Shop examples).
4. Publish the experience.
5. Visit your site to see the product data rendered on the server-side.

The components will automatically:

• Extract the product ID from the experience structure
• Fetch product data from Mock Shop using GraphQL
• Display the information with proper formatting

In editor mode, you'll see:

• Real product data fetched from Mock Shop
• Editor overlays showing the product ID and title
• Loading states while data is being fetched
• Fallback messages if products aren't found

Caching considerations

The implementation includes several caching strategies:

Server-side caching

• Next.js fetch caching: External API calls use Next.js built-in caching
• API route caching: Client-side API routes include appropriate cache headers (if meaningful depending on the kind of data)

Client-side caching

• React context: Prefetched data is passed via context to avoid re-fetching
• Component state: Components cache fetched data in local state

Security considerations

API security

• Rate limiting: Implement rate limiting on your API routes (to prevent facilitating scrapers accidentally)
• Input validation: Validate all product IDs before making external requests
• Error handling: Don’t expose sensitive error information

Conclusion

This implementation provides a robust solution for integrating external data sources with Contentful Experiences. The hybrid approach ensures optimal performance for end users while maintaining a smooth editing experience. The modular architecture makes it easy to extend and adapt for different external systems and use cases.

Key benefits of this approach

• Performance: Server-side rendering with prefetched data
• Flexibility: Works with any external API
• Editor experience: Seamless editing with real data preview
• Scalability: Efficient caching and parallel data fetching
• Maintainability: Clean separation of concerns

For more advanced use cases, consider implementing additional features like:

• Real-time updates: WebSocket connections for live data
• Batch operations: Optimized bulk data fetching
• Data transformation: Custom data mapping and formatting
• Analytics: Tracking external API usage and performance