I've been spending lots of time lately reading up and trying to better understand the different layers and limitations of caching in Next 14 in an effort to improve some data discrepancies our users have been running into when navigating our app.

This branch aims to explain my current understanding of the Next.js cache strategies and more importantly how best to utilize the cache to provide a great experience for super dynamic, multi page apps (like Pierre).

📝 Quick note I thought it would be fun to share some real code, in a real app (Pierre). I've been seeing so many Next example's solving problems in todo apps or shopping carts with no real users… so below is some real code, for a real product. Hopefully, you find it helpful / interesting. Please no bully haha

Background

We've been building Pierre w/ Next's app directory for the last 11 months. We originally made the decision to use Next's app directory because of some of our constraints around rendering large diffs. Being able to utilize RSC to stream in a rich, syntax highlighted dom + next's general obsession with page speed / performance, seemed like a perfect match for our product.

11 months later and we've worked through many of the growing pains alongside Next, but are overall generally happy, thanks in large part by the Vercel team who have been willing to hop in and give us a hand on multiple occasions ❤.

Our Problems

At Pierre, one big thing we're been trying to change about the way people collaborate on source code today is we're making everything on our platform realtime, from read / unread notifications to teammates online status. As you can imagine, this data is updated from a number of sources (both user instantiated in the client, but also from external server-driven sources like inngest, supabase, etc).

In doing this, what we've noticed is that as users would navigate around Pierre, they'd sometimes get in states where their page data was stale – even immediately following a page navigation . At first this was… surprising.

This is where Next's Router cache comes in. The router cache is a client cache that Next.js manages on your behalf. Anytime a user navigates to a new page, the full route is cached for the next 30s. This is done to make initial navigation events immediate (when revisiting pages or using browser back/forward behavior). However, in our case we were seeing the following behavior:

A user visits their branch list page and it says they have 2 unread messages
This user then navigates to a specific branch page
While visiting the branch page, a different user comes and leaves a new comment on one of the initial user's branches
The initial user then clicks a link to return to their branch list page, but a notification for the new comment isn't displayed
However, if the initial user refreshes, they will see the new notification for the second users comment

This is happening because the initial user is hitting the Next.js 30s router cache. So when they return home, the data isn't revalidated 😢

Possible Solutions

We aren't the only team who has run into similar issues with the cache. And thankfully Tim from the Next team lead a great discussion over on GitHub https://github.com/vercel/next.js/discussions/54075 on possible remedies.

There were a couple of solutions discussed:

Can you just turn off the router cache? No. Well, not yet. The next team is currently considering a proposal around something called staletime, which would allow you to configure the router cache duration on a per page OR per layout basis. So one possible solution would be to nuke the cache by spamming something like export const staletime = 0 on all highly dynamic pages. However, for us that's almost every page, which would mean overall slower navigations in general for our users - even when they didn't need to be slower. So i don't love this solution for us.
Can you just call router.refresh() a bunch? Kinda, although router.refresh has the side effect that calling it means Next throws out your entire router cache. This isn't the end of the world, but ends up being a bit like adding staletime = 0 to everything, with the additional side effect of making a new request to the server, re-fetching data requests, and re-rendering Server Components. As a result router.refresh is a great tool to have at your disposal, but is definitely like the kill -9 of fixing your cache problem.
Can you use revalidatePath or revalidateTag to purge targeted caches? Yes! You can use revalidatePath inside a serverAction or other server component to purge the client-side router cache and revalidate the data cache on subsequent visits. However, the trick here is to know HOW and WHEN, because again, you don't want to over purge your cache (ideally you are only purging it when something has meaningfully changed that your users should see). In the Github discussion from before, Sebastian suggested a clever polling system using Server Actions by using the Server Action to decide if the page should be rerendered with new data.

Our Solution

In Pierre we have a couple really helpful things going for us.

Forgive me for this diagram, i don't have a CS degree 🤓

We have a sidebar rendered in a low level layout which is responsible for tracking and showing notifications relevant to a user across their entire team session.
How it works is after initially rendering, it sets up a subscribe hook with Supabase across a number of different postgres tables, if any of those tables are modified, it does some work to re-render and update activity counts within the layout.
What's great is we can actually rely on that as a hook to intelligently call revalidatePath based on push updates to those subscriptions (which are representative of the types of data we display on our branch list pages – for example new branches to review, comments, approvals, etc).

However, the important part for us is to not stop there (those events don't cover ALL data changes on our platform). So completely fixing this issue means identifying more places in the app where there are user driven state changes (like renaming a repo in the video above) that wouldn't have otherwise been caught by our path revalidation strategy.

Below is my first pass at implementing some of these changes to state in the app. Specifically:
use router.refresh() when updating user settings instead of changing window.location
call router.refresh() if a user updates a repo title
Setup router.refresh() for branch pages on Push (toast user 🥂)
Add a new api route handler for revalidatePath (as described here) – hook this into badge subscribe components
Prefetch all repos in the sidebar