Mastering Cloudflare Redirects for Substack: A Developer's Guide

Navigating the Complexities of URL redirects in a Cloudflare-Substack Setup

Implementing URL redirects for a Substack blog using Cloudflare can be a complex task, especially when trying to adhere to Substack's recommended DNS settings. This post explores the challenges, potential solutions, and best practices for setting up redirects while maintaining optimal performance and security.

The Cloudflare-Substack Redirect Challenge

I recently faced an interesting challenge when migrating my Retrothusiast WordPress blog to Substack. The main issue arose from the difference in URL structures between WordPress and Substack. While my WordPress posts were accessible at URLs like https://www.retrothusiast.com/awesome-blog-post, Substack appends a /p/ prefix to all post URLs, resulting in https://www.retrothusiast.com/p/awesome-blog-post.

This discrepancy meant that all my existing links would break unless I implemented proper redirects. Naturally, I turned to Cloudflare, a popular content delivery network and DNS provider, to handle these redirects. However, I quickly encountered a significant roadblock.

Substack's DNS Recommendations

Substack's documentation explicitly recommends setting the "www" CNAME record to "DNS Only" when configuring Cloudflare. This setting disables Cloudflare's proxy features, including the ability to use Workers and other redirect features.

Here's what Substack's support AI had to say about using Cloudflare's proxy:

When using Cloudflare for your custom domain on Substack, it's important to set the “Proxy status” of your CNAME record to “DNS only.” This setting ensures that the DNS record is not proxied through Cloudflare's servers, which can interfere with the proper functioning of your Substack publication.

This recommendation posed a significant challenge to my initial plan of using Cloudflare's features for redirects.

Understanding the Risks of Proxying

Before diving into solutions, it's crucial to understand why Substack recommends against proxying through Cloudflare. After extensive research and discussions with Substack support, I identified several potential issues that could arise from using Cloudflare's proxy with a Substack publication:

1. SSL/TLS Certificate Errors: The SSL configuration between Cloudflare and Substack could potentially cause certificate errors, leading to security warnings in browsers.

2. Caching Issues: Cloudflare's caching mechanisms might serve outdated content, causing problems with dynamic content like new posts or updates.

3. Email Deliverability: The proxy could affect how the domain's DNS records are handled, potentially causing issues with email deliverability if SPF, DKIM, and DMARC records are not correctly set.

4. Performance Issues: While Cloudflare generally improves site performance, misconfigurations could lead to increased load times or errors during high traffic periods.

5. Redirection Loops: Incorrectly configured redirects in Cloudflare could cause redirection loops, making the site inaccessible.

These risks, while manageable for an experienced developer, could indeed cause significant issues for the average Substack user. It became clear why Substack recommends the "DNS Only" setting as a safer default option.

Exploring Cloudflare Solutions

Despite Substack's recommendations, I decided to explore Cloudflare's features to see if I could find a workable solution for implementing redirects. My focus landed on Cloudflare Bulk Redirects, a feature available even on the free Cloudflare plan.

Implementing Cloudflare Bulk Redirects

To set up Bulk Redirects, I followed these steps in the Cloudflare UI:

1. Logged into my Cloudflare dashboard.

2. Navigated to the "Rules" section.

3. Clicked on "Bulk Redirects".

4. Created a new Bulk Redirect List.

5. Added my redirect rules, specifying the source URL and the target URL for each redirect.

I made sure to include redirects for all my important posts and categories.

Setting Up the Bulk Redirect Rule

After creating the Bulk Redirect List, I set up a rule to apply these redirects:

1. In the Bulk Redirects section, I clicked on "Create rule".

2. I named the rule "Retrothusiast Redirects".

3. Under "When incoming requests match...", I set:

   • Field: Hostname

   • Operator: equals

   • Value: retrothusiast.com

4. Under "Then, apply matching URL redirects from the following list...", I selected my created redirect list.

5. I reviewed the settings and clicked "Save".

This configuration allows Cloudflare to apply the redirects when requests match my domain, ensuring that old URLs are properly redirected to their new Substack counterparts.

The DNS Only Dilemma

Even with Bulk Redirects set up, I still faced the dilemma of Substack's "DNS Only" recommendation. Cloudflare's redirect features, including Bulk Redirects, require the domain to be proxied through Cloudflare (orange cloud icon). This put me at a crossroads: adhere strictly to Substack's recommendation or implement the necessary redirects.

Weighing the Options

After careful consideration, I found myself facing four distinct options, each with its own set of pros and cons:

1. Use a custom domain with DNS Only: This option would adhere strictly to Substack's recommendations, but it meant living with 404 errors for all my old URLs and losing valuable backlinks.

2. Use a custom domain with a subdomain: By setting up something like substack.retrothusiast.com, I could potentially keep my backlinks and avoid 404 errors. However, this would mean changing my main URL structure, which could confuse regular readers.

3. Don't use a custom domain: Running my blog directly off the Substack domain (retrothusiast.substack.com) would be the simplest solution technically, but it would mean losing my established brand presence and starting from scratch in terms of SEO.

4. Use a custom domain with Cloudflare's Proxy feature: This option would allow me to implement the necessary redirects and maintain my backlinks, but it goes against Substack's recommendations and could potentially lead to issues.

After weighing these options carefully, I ultimately decided to go with option 4: using a custom domain with Cloudflare's Proxy feature enabled. This decision was not made lightly, and I committed to closely monitoring the site for any issues that might arise.

Here's why I chose this path:

1. Preserving SEO and User Experience: By implementing redirects, I could maintain my SEO value and ensure that users following old links would still reach the correct content.

2. Leveraging Cloudflare's Features: The Bulk Redirects feature offered a manageable way to handle the URL structure change without diving into complex server configurations.

3. Balancing Risk and Reward: While this approach goes against Substack's recommendations, the potential benefits of maintaining my site's integrity outweighed the risks in my specific situation.

4. Commitment to Monitoring: I set up thorough monitoring to quickly catch and address any issues that might arise from this configuration.

This decision underscores an important lesson in web development and site migrations: sometimes, you have to make calculated decisions that balance best practices with practical necessities. However, it's crucial to do so with a full understanding of the potential risks and a solid plan for mitigation.

Take Away

This decision underscores an important lesson in web development and site migrations: sometimes, you have to make calculated decisions that balance best practices with practical necessities. However, it's crucial to do so with a full understanding of the potential risks and a solid plan for mitigation.

Conclusion

Implementing redirects for a Substack blog using Cloudflare while adhering to Substack's recommendations is a challenging task that requires careful consideration of various factors. While there's no one-size-fits-all solution, a combination of limited use of Cloudflare Bulk Redirects, strategic use of Substack's import tools, and constant monitoring can achieve a workable balance.

Remember, every website has unique requirements, and what worked for me might not be the perfect solution for everyone. Always test thoroughly and be prepared to adjust your approach as needed.

If you've faced similar challenges or have alternative solutions to share, I'd love to hear about them. Let's continue the conversation and help each other navigate the ever-evolving landscape of web development and content platforms.

Join the Community

Did you find this article helpful? There's plenty more where that came from! Subscribe to DevBrief for regular doses of practical tech insights tailored for busy software engineers and IT professionals. Join our growing community on Substack Chat to discuss this topic further, share your experiences, and connect with like-minded individuals. Let's learn and grow together!