CloudFront Function Powers a Self‑Hosted Prerender Cache on S3 with Batched Puppeteer Lambdas
CloudFront Function-driven prerender cache combines CloudFront, S3, Lambda, and Puppeteer to serve bot-ready HTML and replace a $49/month prerender service.
Why prerendering still matters for modern SPAs
Single‑page applications ship a minimal index.html and rely on client JavaScript to render content. That’s great for real users, but search engines and many social crawlers either don’t run that JavaScript or do so asynchronously and slowly. The result: search snippets that lack meaningful content, pages stuck in “Discovered — currently not indexed,” and blank social previews when links are shared. A prerender cache addresses this gap by serving fully rendered HTML snapshots to bots and social scrapers while regular visitors continue to get the normal SPA experience. In this implementation the CloudFront Function is the gatekeeper that ensures crawlers receive prebuilt HTML stored in S3, preserving the original URL and avoiding redirect penalties.
The site and stack used to validate the approach
The implementation was built for Vairaa, a B2B marketing site with a product catalog, blog, lead CRM, media gallery, and admin panel. The front end is a React 19 + Vite 7 SPA styled with Tailwind CSS 4, animated with Framer Motion, and using Lenis for smooth scroll; Radix UI and TipTap are used for components and rich-text authoring. The backend is a single Node.js 22 Lambda (ARM64) exposing API routes (products, blogs, leads, gallery, uploads, site config) behind API Gateway. Data sits in a single‑table DynamoDB schema. Infrastructure is provisioned with AWS CDK v2: CloudFront serves the SPA from a private S3 bucket via an OAC, and Route53 + ACM handle DNS and TLS.
All of those concrete platform choices shaped architectural trade‑offs — most importantly the decision to reuse the existing CloudFront + S3 origin rather than introduce a separate origin or a Lambda@Edge path.
Architecture in one sentence
A CloudFront Function detects crawlers by User‑Agent, rewrites the request URI to an S3‑stored prerender snapshot under a /prerender/ prefix, and CloudFront serves that snapshot at the original URL with an HTTP 200 response — no redirects, no origin switch.
Why rewrite instead of redirect
Early attempts considered redirecting bots to an API endpoint that returns rendered HTML. Redirects expose a different URL to crawlers and can split signals or raise suspicion in search engines. The pattern implemented here performs a transparent URI rewrite: the crawler requests /products/cocopeat-bricks but CloudFront fetches /prerender/products/cocopeat-bricks/index.html from S3 and returns it at the original URL. The crawler never sees the internal path or a redirect response. That aligns with Google’s dynamic rendering guidance and preserves canonical URLs and indexing signals.
Why keep the prerender cache in the same S3 bucket
A CloudFront Function runs at viewer‑request and can rewrite URI but cannot change the origin. Lambda@Edge could switch origins at origin‑request, but it introduces extra complexity, potential latency, and a heavier deployment model. The chosen path stores prerendered HTML files under a prerender/ prefix inside the existing static hosting bucket (vairaa-static-prod/prerender/…), letting the CloudFront Function simply alter the URI. This keeps infrastructure minimal — one additional Lambda for rendering and some S3 objects — and avoids Lambda@Edge cold‑start costs and more complex deployment mechanics.
The S3 key layout used in this system is hierarchical and mirrors site routes, for example:
- prerender/index.html for /
- prerender/products/index.html for /products
- prerender/products/cocopeat-bricks/index.html for /products/cocopeat-bricks
How the CloudFront Function detects crawlers and performs the rewrite
The CloudFront Function examines the request’s User‑Agent header, matching it against a curated list of crawler and social‑bot identifiers (googlebot, bingbot, slurp, duckduckbot, baiduspider, yandexbot, facebot, twitterbot, linkedinbot, slackbot, telegrambot, discordbot, whatsapp, pinterestbot, redditbot, and others). It skips API paths, product assets, and URIs ending with a static file extension so that only page routes are rewritten. Crucially, the function always returns the modified request object rather than issuing a redirect response; that makes the behavior a rewrite rather than a redirect.
A side note in the implementation: the extension regex avoids rewriting static assets but could conflict with legitimate slugs that contain dots (e.g., /products/version-2.0). In that case an explicit allowlist for file extensions is advised instead of a general dot‑suffix match.
Pre-rendering vs on-demand rendering: trade-offs
There are two valid approaches to deliver prerendered content:
- On‑demand rendering: the first bot request for a URL triggers a rendering Lambda (for example via Lambda@Edge), producing HTML in real time. This works well for very large, rarely changing catalogs because it avoids up‑front work, but the initial request pays the cost and latency of a full Puppeteer render.
- Pre‑rendering (the approach here): an administrative warm‑up process renders all known pages ahead of time and stores HTML files in S3. Bots always hit cached snapshots and receive immediate, zero‑latency responses. The trade‑off is that cache misses return the SPA shell until the next warm‑up; for sites with a manageable number of pages and predictable content changes, pre‑rendering delivers faster bot responses and simpler operations.
Render worker Lambda: Puppeteer, @sparticuz/chromium, and batching
Rendering uses puppeteer-core with @sparticuz/chromium to run headless Chromium inside Lambda. That combination was chosen for deployment simplicity and small package size: Puppeteer + @sparticuz/chromium compresses well for zip-based Lambda deployment (the post notes ~50MB compressed), whereas alternatives like Playwright require much larger container images (~400MB+) and a different deployment model. For the simple requirement — navigate to a URL, wait for rendering, capture page.content() — Puppeteer provides the necessary primitives with a smaller footprint.
To avoid hundreds of cold starts when warming many pages, renders are batched. The implementation chunks the URL manifest into groups of 10 URLs and invokes a Render_Worker Lambda per batch. Each Render_Worker launches Chromium exactly once, opens parallel pages for the batch, navigates each page with waitUntil: ‘networkidle0’, captures the HTML, and writes it to S3 under the prerender prefix. Using Promise.allSettled ensures individual failures don’t abort the whole batch; failed renders are logged for later debugging.
Concrete runtime parameters from the deployment: Render_Worker functions run with ~2048MB memory and a 3‑minute timeout. For the site in question — roughly 50 pages — batching into five render worker invocations and running them in parallel yields a warm‑up that finishes in roughly 20–30 seconds of wall‑clock time.
Warm-up, invalidation, and asynchronous regeneration
Cache lifecycle is driven by a Cache Manager API. The prerender handler builds a manifest consisting of static paths plus product and blog slugs (queried from DynamoDB), chunks the manifest into batches, and fires Render_Worker Lambdas asynchronously using InvocationType: ‘Event’. Asynchronous invocation decouples cache regeneration from the admin API response: when an admin updates content (PUT /api/admin/products/:id, for example), the API updates DynamoDB, deletes stale S3 objects corresponding to affected paths, and fires the Render_Worker asynchronously. The admin sees a synchronous HTTP response immediately while the cache rebuild happens in the background. Lambda’s async retries provide resilience for transient failures.
To prevent concurrent warm‑ups, the system writes a prerender/.warmup-lock object into S3 and checks for it at the start of a warm‑up; if the lock exists the warm‑up endpoint returns 409. The implementation author notes that a DynamoDB item with TTL might be a cleaner lock that automatically expires if a warm‑up crashes.
Handling cache misses and graceful fallback
The CloudFront Function rewrites URIs blindly without checking S3 for the target file. If the prerender HTML for a path is missing, S3 returns a 404 and CloudFront’s configured error response serves the SPA shell (index.html) with HTTP 200. That means a crawler will receive the client shell on a cache miss; it isn’t ideal, but it’s a safe fallback and Google will re‑crawl later. The design assumes warm‑ups are run before launch and automatic invalidation/regeneration keeps misses rare in steady state. If zero‑miss behavior is required, on‑demand rendering via Lambda@Edge could be added to render the page at request time.
No meta tag injection — the app renders its own SEO tags
This system does not rely on post‑capture tag injection. The SPA includes an SEO component that renders canonical, Open Graph, Twitter Card and other meta tags in the DOM before Puppeteer captures page.content(). That means the captured HTML already contains all SEO metadata and canonical URLs pointing to the public site (for example https://vairaa.com/products/cocopeat-bricks), so search engines index the correct URL even though CloudFront retrieved an internal S3 path.
Admin UI and operational visibility
The admin surface provides:
- A manifest vs cache status dashboard showing total cached pages and missing entries
- Per‑page status (Cached or Missing) and last rendered timestamps
- A “Warm Up All Pages” action that triggers the manifest-based warm‑up and shows live progress
- Per‑page Refresh (synchronous re-render) and Delete actions
- A Clear All Cache action with confirmation
Warm‑up progress in the existing UI polls GET /api/admin/prerender/status periodically to compare cached entries against the manifest; the author acknowledges this is functional but could be improved with an SQS‑backed progress counter.
Cost profile for a small site
For a site of about 50 pages the reported resource costs are small:
- S3 storage for ~50 HTML files at ~200KB each (~10MB) is negligible.
- Warm‑up involves a handful of Lambda invocations (about five in this example), and day‑to‑day invalidations trigger a couple of invocations per content change; with Lambda free tier these invocations are effectively free for modest volume.
- CloudFront Function charges are low ($0.10 per 1M invocations) and generally round to zero for low traffic.
- Puppeteer Lambdas using 2048MB memory for short durations were estimated to fit within free or very low billed usage for the cited workload.
The author contrasts this with a $49/month prerender.io subscription and argues the AWS‑native approach can eliminate that recurring cost for sites with manageable page counts.
Concrete implementation notes and safeguards
- Use Promise.allSettled in the Render_Worker so a single page failure doesn’t cancel other renders.
- Batch multiple URLs per Lambda invocation (10 per worker in the posted implementation) so Chromium cold starts are amortized.
- Include a warm‑up lock object (prerender/.warmup-lock) to prevent concurrent warm‑up runs.
- If slugs include dots, replace a general dot‑regex with an extension allowlist to avoid false positives.
- Consider concurrency limits for page tabs if pages are large; a simple semaphore can cap parallel tabs within a single Chromium instance.
Observed results after deployment
After the prerender cache was deployed the site saw a clear change in Search Console metrics: impressions rose noticeably and pages that had been stuck as “Discovered — currently not indexed” began to be indexed. The author points to an inflection around February 10 — the date the solution was put live — and reports improved social link previews where previously previews were blank.
What the author would change next
The implementation lists several pragmatic follow‑ups:
- Add on‑demand rendering for cache misses (via Lambda@Edge) if zero‑miss guarantees become a priority.
- Trigger prerender warm‑up automatically from the frontend deploy pipeline so layout or component changes don’t leave cached HTML stale.
- Replace S3 lock semantics with a DynamoDB lock entry that TTLs automatically in case warm‑ups crash.
- Improve progress tracking with an SQS queue and a centralized counter rather than periodic polling.
Who this pattern is for and when to use it
This design suits teams already hosting a React SPA behind CloudFront and S3 who need reliable bot‑facing HTML — marketing sites, content sites, or product catalogs focused on lead generation — and who prefer to avoid an external prerender SaaS. It is particularly sensible when:
- The total number of pages is manageable to pre‑render.
- You already use CloudFront + S3 and want to minimize new infrastructure.
- Zero‑latency bot responses for cached pages are important and occasional cache misses are acceptable or can be addressed with scheduled warm‑ups.
If a site has thousands of pages that change rarely, or if you require first‑visit rendering guarantees for arbitrary new URLs, an on‑demand approach (Lambda@Edge or a rendering API) may be a better fit.
Broader implications for developers, SEO, and operations
This pattern illustrates how existing CDN and serverless building blocks can be combined to reclaim functionality that many teams outsource to third‑party services. It underscores several platform trends:
- Edge‑level logic (CloudFront Functions) is now powerful enough to perform lightweight request routing and bot detection without origin changes.
- Serverless compute combined with headless browsers is viable for batch rendering workloads when package size and cold starts are managed.
- Pre‑render caches shift complexity from request‑time latency to build‑time or admin‑driven workflows, which can simplify operations while preserving SEO quality.
For teams and businesses, adopting an in‑house prerender cache can reduce recurring SaaS costs and provide tighter integration with content workflows and CI/CD pipelines. For developers, it emphasizes practical trade‑offs: deployment simplicity and low resource footprint (puppeteer-core + @sparticuz/chromium) versus the flexibility and scale of on‑demand rendering.
A few operational caveats deserve attention: prerender caches must be invalidated and rebuilt when site content or layout changes; warm‑up processes and deploy hooks should be integrated into release pipelines to avoid stale snapshots; and monitoring & retry behavior should be in place to surface render failures or bottlenecks.
Forward looking, this approach points to a hybrid future where edge routing and serverless rendering coexist: lightweight edge functions handle detection and routing, while batched or on‑demand serverless renderers produce SEO‑ready HTML. Teams might further refine the pattern with incremental rendering, selective on‑demand fallbacks, or tighter integration into CI/CD to automate warm‑ups on deploys and reduce the surface area for stale renders.
