Adding dynamic content from an API to a static website at build time
I recently launched a re-write of my brothers Guitar teaching business website: cgguitar.co.uk, during this rewrite I had some guiding principles which I believe are best practices when building any website:
- Limit the number of calls to external services to keep the page load fast.
In this post I’ll describe my approach to getting embedded YouTube playlist content into the website, at build time, reducing the number calls to YouTube client side to only the embedded video and thumbnails, no calls out to the YouTube Data API. In addition to this, i’ll show you how you can keep the site up to date with easy to configure cron jobs (scheduled builds).
The feature that I built, that I will explain, is an embedded YouTube playlist component which fetches all the data and stats for YouTube playlists at build time and renders their video metadata/thumbnails directly into the HTML. You can check out the feature live over at https://www.cgguitar.co.uk/videos/#guitar-lessons.
The problem with client side
Security — if you want to hide your token or keep it secure you either have to:
- Ensure your token only works on your websites domain, but this doesn’t stop people using it from outside of a web browser.
- Add some complex proxy set up where you hide the token on a server you manage, requires having a server or proxy configuration.
Rate limiting/charges — most APIs have limits to the number of API calls you can make, or will start charging you for usage:
- Your website content doesn’t scale, each visitor would be using your token to call the external services for every visit.
- You could end up incurring accidental charges!
Moving your calls to external APIs to build time
This is approach is not a silver bullet, not every feature would support this, e.g. if you want to work with user submitted content. However, if all you are showing is content that changes infrequently, moving the data fetching to build time can be a really great solution.
The next section will assume some knowledge about 11ty, or static site generators in general.
11ty has a plugin called @11ty/eleventy-cache-assets which you can use to fetch any data you like.
The awesome thing about this plugin is that once the data is fetched it is cached so future local builds do not have to re-fetch data, meaning your builds can remain lightning fast which is a common characteristic of any 11ty project.
Embedding YouTube playlists at build time
For my feature I decided I wanted to be able to pick and choose which YouTube playlists that I wanted to show in the website, it is however possible to fetch all YouTube playlists for an account too. I wanted to be able to choose so that I could add, order and describe new playlists in my CMS (Netlify CMS).
The first step to getting my playlists into 11ty is to define them as a collection, to do this inside of the
src/playlists folder I create a playlists.json.
This creates an 11ty collection of all of the playlists, with their “id”, “name” and “descriptions”.
Inside of my videos page I can then work with this collection in my Nunjucks template:
If you are unfamiliar with template languages in 11ty you can read about them over here.
I’ll show what
partials/video-playlist.njk is later on in the article.
fetchYouTubePlaylists is where the magic happens and where we will start to use
@11ty/eleventy-cache-assets. This is an 11ty filter which is defined in my
.eleventy.js config file.
Let’s take a dive a layer deeper:
getPlaylists is making a call to
getPlaylistItem which is where i'm actually doing the data caching.
This function is looping through all of my playlists and fetching the items (videos) in that playlist. It is also adding the name, description and direct link to YouTube for the whole playlist.
The first few things this function does is:
- Set base url for YouTube API: https://www.googleapis.com/youtube/v3/playlistItems
- Set the max number of items in a playlist to return on a page
- Pass in APIKey and build up url in accordance with the API Docs.
You will want to store your API key as an environment variable e.g.
const apiKey = process.env.YT_API_KEY;. For production you can add this environment variable where ever you choose to build/host the site e.g. on Netlify.
Next up it fetches some extra metadata.
fetchMetaInfo fetches things like view count and likes, this is another API call which we would be concerned about if this was client side, but since it's build time, who cares! Implementation available on Github.
Finally i’m looping through all the data and returning an array of
videos for each playlist and a flag
hasMore if the playlist has more than then 20 items shown. In my HTML when I see this flag I add an link out to YouTube to watch the full playlist.
The above code a modified version of the original, where i’m doing a a few extra things you can checkout the full version on Github.
Now I have the website fetching the external data, let’s see how I could approach displaying the content in the HTML.
<a> to the YouTube videos, perhaps the thumbnail could open a tab to YouTube, this needs no JS at all, and is what I did:
You will see that i’m wrapping the whole code in a
I’m not going to go into the implementation of my Web Component in this post but you can check out the source code on Github. The general idea is to consume
<li> list items as child content inside of my
Here is my full Nunjucks template for my html:
Periodically building your website
In order to keep the YouTube playlists up to date I want to be able to build the website every day on schedule.
There are many options when it comes to periodically building a website, I wrote about my approach to doing this in: Scheduling builds on Netlify. In brief, I opted to use Circle CI to call my Netlify build hook every day at 3 PM. I tried Github Actions but there is a major limitation to using an Action for this use case, which I go into in the linked article.
I hope this article was helpful and you can see some of the advantages to moving dynamic content that changes infrequently to be rendered at build time.