Setting Up & Configuring Content Delivery Network (CDN) Architecture: Experts Share Common Mistakes
More companies are relying on the convenience of content delivery networks (CDNs) to facilitate the delivery of content more seamlessly to website visitors around the world. But setting up and configuring a CDN comes with its own share of potential mishaps if companies don’t fully understand how to properly set up and configure a CDN to maximize the benefits without introducing other issues.
For instance, companies may not fully understand a CDN’s capabilities and thus fail to configure it in the best way to optimize speed and other benefits, rely on default naming conventions or a single domain and subdomain, and a variety of other mistakes that can impact performance yet are easily avoidable with a bit of know-how.
To help you avoid making critical mistakes that can hinder traffic and revenues, we asked a panel of CDN experts to answer this question:
“What’s the biggest mistake companies make in setting up & configuring their content delivery network architecture?”
Find out what our experts had to say below.
Meet Our Panel of CDN Experts:
|· John Leo Weber
|· Alan Crowetz
|· Sarah Miller|
John Leo Weber / @JohnLeoWeber
John Leo Weber is the Director of Search for Geek Powered Studios in Austin TX. John crafts the SEO strategy for Geek Powered clients and writes about digital marketing for a variety of publications across the web.
“One of the biggest mistakes we have seen with setting up CDN’s comes with…”
The serving of duplicate content. Since your website will be serving content from two different domains, it is vital to tell Google that your main URL is the original and the CDN version is a copy. You can do this by adding a rel=“canonical” directive to the http headers of your site. This is an integral step to avoid duplicate content issues with Google and maintaining your rankings.
Matthew Dorman has over 16 years of technology experience implementing successful transitions for high profile digital properties. Recently, Dorman started a new technology services company, Ndevr, Inc.
“The biggest mistake companies often make when setting up their content delivery network architecture is…”
Not continually checking the logs for low cache hit rates or other types of errors is one of the biggest overlooked mistakes I find sites making. The initial set up is generally based on a few assumptions which, under realistic traffic patterns, do not hold up. Periodically checking those logs will help identify both issues as well as changes in your users’ behavior.
Developer, hacker, speaker, community organizer and entrepreneur, Levent is the founder of Movel, a mobile app design and development company in the Washington, DC area. He is actively engaged in several communities on mobile and full-stack development across the Mid-Atlantic region. Levent speaks at conferences, meetup groups, user communities, government, and non-profit associations on topics concerning mobile strategy, user experience, and security.
“The biggest mistake companies make, apart from the fact they wait too long to establish a CDN, is to…”
Not fully understand a CDN’s capabilities and segment their visitors by geographic location. This leads to inefficiencies so that even after the CDN is in place, the users are not receiving the full benefits of geographically collocated CDN servers.
CDN nodes should be deployed at multiple locations, spread across four AWS geographical regions. These nodes will communicate with each other to serve video streaming requests by mobile app users, seamlessly operating, moving content and re-instating a failing instance. Users are unaware of any of this infrastructure and can enjoy an uninterrupted viewing experience.
The main purpose of the CDNs is to deliver content faster, depending on the users’ geographic location. This is the key advantage that we have been seeing and the main reason we recommend CDNs to our customers.
“There are several big mistakes companies make in setting up and configuring their content delivery networks…”
- There are five variables as it refers to CDNs: price, speed, distribution, service level agreement, and expansion capabilities. Focusing on an item like price, for instance, can do you more harm than good as you’ve applied a CDN but added latency and outage issues to your application or site. Alternatively, implementing an expensive solution with maximum expansion and coverage capability can hurt you financially – and be totally unnecessary. There are some incredibly fast CDN providers that are very inexpensive.
- Default naming conventions. Upon implementation, leaving default pathnames can add a lot of extraneous text and unnecessarily large files that load more slowly. Try to implement a CDN domain with a very short name and use concise subdomains for your buckets.
- Don’t limit yourself to a single subdomain and domain for your CDN. Creating multiple aliases for your CDN can be utilized to load assets much faster. Browsers will synchronously load from a single CDN path, but will asynchronously load from multiple CDN paths. In other words, you could have c1.domain.com <http://c1.domain.com/> through c8.domain.com <http://c2.domain.com/> all pointing to the same folder, and then cycle through the subdomains when loading a page.
- Autopublish size and resolution modifications upon uploading the asset, saving yourself from on-the-fly asset production later that delays the user experience. This will ensure retina displays, mobile sizing, and other optimal image sizes are all preloaded so that the first user that visits on a different viewport will experience great performance. Waiting until the file is requested to determine the size and resolution is a huge time-waster.
- Ensure alternative paths can be loaded in the event that assets cannot be loaded due to an outage. The ability to migrate or enable and disable CDNs can save you some embarrassment in the event of a CDN service outage.
Robin Waite / @hostpipe
Robin Waite is a Business Strategy Expert and Co-Founder of Online Strategy Agency, Hostpipe (formerly Coconut). He specializes in creating growth and scalability within established SMEs turning over £500k+. Coconut has helped over 200 organizations to grow over the last 11 years, and he also runs weekly workshops with business owners.
“The biggest mistake we tend to see when companies set up a CDN is…”
Not thoroughly testing the site after switching on the CDN. I’ve seen several examples of sites which have fallen over when the CDN is switched on, primarily due to the compression of the JS and CSS files, which then causes conflicts. Typically business will switch on a CDN, look at the home page and think that everything is okay; however, subpages such as product, checkout, and blog pages typically tend to be the ones that fall over. The simple solution is to have a blueprint of the website architecture (a sitemap) handy so that you can step through every page and check it thoroughly.
I’ve also seen a company sign up to a CDN but never get around to changing their DNS config at domain level…and then wonder why the site’s performance didn’t improve. Oops!
Also, not pausing the CDN during development of the website, especially if you have caching turned on. This is a mistake that I’ve made personally as I’ve asked the client to look at some changes which I made, but they could only see the old site due to the caching and CDN still being turned on. When I paused the CDN everything clicked into place for them, but this took an hour-long telephone conversation to get to the bottom of!
Jeff Huckaby / @rackaid
Jeff Huckaby has worked in IT for nearly 20 years. While studying genetics at Yale University, he became interested in open-source technologies. In 2006, he founded racKAID, a MSP managing open-source hosting solutions. In 2013, MSPMentor named him as one of the world’s top MSP executives. When not working, you will usually spot him on his bicycle, at his favorite coffee shop, or watching his favorite Kung-Fu movies.
“When setting up and configuring a content delivery network architecture, one of the biggest mistakes companies tend to make is…”
Not verifying that your caching is working. Is your CDN really caching? That’s a key question you should be asking, and to find out, you really need to dig into HTTP headers.
Many content management systems send no-cache HTTP directives when responding to a request. As a result, you may suddenly find your site is slower with a CDN than without. The reason is that many CDN providers simply honor whatever cache directives your server sends to them. If your server is sending no caching data or a no-cache directive, they may not cache your files. Your sites will work fine, but you are not getting the benefits of using a CDN.
To avoid this issue, I recommend checking HTTP headers coming both from your origin servers and the CDN. Make sure they are behaving as you intend. I also suggest digging into the logs of your origin severs. You should see a decline in file requests for cached files. By comparing hits before and after you implement the CDN, you can gain a better sense of how well it is working. In many cases, simply tweaking your headers can improve CDN cache hits by ten-fold.
While on the topic of headers, there is one HTTP directive than is a CDN killer – The Vary: User-Agent header. If your CDN provider honors this directive, you will see very low cache hit rates. This directive causes the CDN to store different copies for different User-Agents. Given the large number of User-Agents in use today, this undermines the benefit of the CDN. If at all possible, remove this header or check to ensure that your CDN provider can filter it for you.
Alan Crowetz / @InfoAlan
Alan Crowetz is the CEO of InfoStream, Inc. (named in the Top 100 SMB IT firms worldwide with a 5 Star ranking by Microsoft). Crowetz regularly appears on local news affiliates for NBC, Fox, CBS, and ABC for security, internet, business, and special stories. He holds numerous business and computer degrees and certifications (MBA, MCSE, Series 7, many more) and serves on the Board of Palm Beach Atlantic’s Rinker School of Business. Crowetz is a recognized leader in computer networks, security, and the cloud, has passed the FBI’s InfraGard Records Check, and is a member in the FBI’s InfraGard program.
“There are several painful mistakes firms make but if I had to choose the key mistake it’s…”
Education. Not knowing what questions to ask before moving to the cloud. Not understanding the pros, cons, and pitfalls. They do it because they are enamored with the idea and don’t realize there are other sides to the coin. When done properly, it can be fantastic, but it often is done poorly and with dangerous results.
Colin Jack / @EmboticsCorp
Colin Jack is the Lead Solutions Architect at Embotics, the anti-shackles cloud management platform company with a no sales rep, vendor neutral, easy-to-use, self-service IT solution that can be delivered before lunch. Their vCommander empowers enterprises and service providers to deliver ITaaS across any private, public or hybrid environment without locking them into a platform.
“The biggest mistake companies make in delivering and setting up network architecture is…”
Forgetting that virtual machines and servers are not free.
We have all heard that there is “no such thing as free lunch.” However, many in the workplace truly don¹t understand that virtual resources are not free either. How can you decide the best method for cost modeling?
Three Approaches to Cost Modeling for the Enterprise
It doesn’t matter if you’re doing chargeback, showback, or cost accountability, there are three methodologies that I see being used. The most important factor when choosing a cost model is your motivation for doing so. So much so that when working with customers, this is the first question I ask:
“What do you wish to accomplish by putting the cost model in place?”
Is it to recover costs, name and shame resource abusers, or justify the relevance of your IT Department? This dictates the approach taken to determine the model. In my experience, a cost model is required if you want to change how people behave when consuming virtualized resources. Just because they are virtual doesn’t mean they are free.
It can be a political battle implementing chargeback and showback, but no matter which approach you take it is important to drive home the fact that VMs are not free.
Christian Habermann / @AuctusMarketing
Christian Habermann is the founder of Auctus Marketing. He is a leader and innovator in the online marketing world and start-up scene in Boston. Prior to Auctus Marketing he co-founded Goji.com, which has become one of the most successful online insurance companies, and has made the Inc. 500 List for the past three years. He is the author of SEO for 2014 and holds an MBA in marketing from Vanderbilt.
“This is what I have found to be a very common problem among companies setting up and configuring a content delivery network architecture – from an SEO perspective, and it can really hurt a site’s online traffic…”
Naming your URLS. Once your files are on your new CDN servers they will have a new URL, and lots of people miss this or don’t understand this. So let’s say our old CSS file was at yourcoolsite.com/css/main.css. Well now that you are using a CDN, your CSS file is located at a URL that might look like this: gem garbledeegoohey24374566fth/main.css. Obviously, this URL won’t do much for you in the way of SEO, so make sure you take your time during this crucial step and name your new URLs in an SEO-friendly or appropriate manner.
Chad Holstead / @ChicagolandIT
Chad R. Holstead is the founder of Business Knowledge Systems and is a wealth of vast and diversified experience in the information technology field. Mr. Holstead founded Business Knowledge Systems and focused on providing high end IT consulting services to meet the specific needs and budgets of small and mid-sized businesses. Throughout his tenure with BKS Systems, Chad has personally overseen the planning and implementing of a new network infrastructure for the opening of a new bank, launched a technology support program for community banks, organized the network infrastructure, and provided technology and network consulting services to numerous clients in a variety of industries including professional service firms, manufacturing, distribution, and transportation industries.
“The biggest mistake companies make in setting up and configuring their content delivery network architecture is…”
That they tend to underestimate the bandwidth and data storage space. The consequences of underestimating these issues typically lead to limited backup hardware or bandwidth to effectively backup all their content. The solution is to consider the network architecture in relation to the business goals and plans.
Morgan O’Mara / @morganomara
Morgan O’Mara is the Content Coordinator at Record Nations, a document and cloud management company.
“The biggest mistake companies make when setting up their content delivery network architecture is…”
Not communicating efficiently throughout the office. In some cases, one person, usually IT or the CEO, makes the decision of what cloud service to use. When one person makes the decision, it is often made out of preference and not what is best for everyone at the company. That is why it is most important to see what everyone will be using the cloud for and finding a service that caters to everyone’s needs, rather than the needs of just one person.
It’s also important for the company to be open with all of its employees about the conversion process. Having transparency throughout the transition will save confusion and boost morale.
Daniel Hindi / @BuildFire
Daniel Hindi is a technology veteran with over 16 years of experience building technology stacks for leading companies across a wide variety of industries. In recent years, Daniel has been focusing on start-ups, helping develop a concept and bringing it to fruition. Daniel is currently serving as the CTO of BuildFire, which he joined after his role at Flywheel as Vice president of Engineering. In this role, Daniel headed all development using cloud and mobile technologies. As chief architect of these technologies, Daniel provided his many years of experience in developing scalable systems and managing local and remote development teams.
“When setting up and configuring their content delivery network architecture, companies often make the mistake of…”
Not setting up proper invalidation rules. When you are caching static content that will never change, this is a simple task. However, many companies use CDNs to increase the performance of changeable data. Most data these days come in chunks; if not all chunks invalidate at the same time, your users may experience what seems to be schizophrenic content.
Not analyzing their hit/miss rates. After your CDN has been setup, analyzing the hit rate is crucial. Spikes in miss rates in the beginning are to be expected because the CDN is catching up. However, there will always be some portion of your requests that hit due to new content. If this is just written off, you may have requests that are never cached. If a developer places an unnecessary version buster in the URL, it may go unnoticed and skew your base line readings.
Sarah Miller / @Netrepid
Sarah Miller is a Cisco, Citrix, CompTIA, and Microsoft-certified Solutions Architect at Netrepid. She designs and deploys applications, networks, and products for clients that seamlessly integrate with Netrepid’s data center. Prior to joining Netrepid, she spent over a decade advancing her networking and security skillset in the healthcare sector.
“The biggest mistake companies make in setting up their own CDN architecture is…”
Failing to take the time to understand the process and delivery of the content and HOW their users will use the content. All architects do it. The users ask for ‘eggs’. So they go to work and serve up some AMAZING, perfectly seasoned scrambled eggs…only to find out that their users were expecting sunny side up! Problem is, all they told us they wanted was eggs.
Too often, architects are so worried about designing the system that they forget to connect with the people who will use it. They fail simply for not taking the extra time to fully understand the delivery model.
Mark Rushworth is Head of Digital Marketing at Blue Logic Digital and an experienced website designer, developer, and marketing professional with over 16 years’ experience in deploying and marketing eCommerce solutions for brands including Brooks Brothers, AllSaints, Morphsuits, Aspecto, and TomTom.
“The biggest issue facing CDN architecture setup and configuration is…”
Canonicalization. Bandwidth is wasted when files are duplicated in different folders on a CDN. This is further compounded if the CDN is case sensitive, as most LINUX environments are. An additional canonicalization issue is where files are addressed using appended query-strings. All of this makes centrally managing information difficult, and worse, means content isn’t being cached correctly for users, contributing to reduced performance.
Howie Ross / @howdiz
Howie brings over 10 years of experience developing for the web and over five years leading teams and projects to O3. Born in New York but raised in the Philadelphia area, Howie’s fascination with technology led him to a degree in Computer Science from Rutgers University. In addition to leading one of the most talented teams of developers in the business, Howie sets the technical direction and strategy for the company and assists with requirements gathering and solution architecture for O3’s clients’ projects. In his spare time, Howie likes to balance his time at the computer with time spent outdoors: hiking, biking, camping, rock climbing, snowboarding, and brewing his own beer.
“The biggest mistake that companies make in setting up and configuring their content delivery network architecture is…”
Not considering the level of effort and timeline for migrating their existing content onto the network. This can take days or even weeks depending on the volume of content the company has accumulated.
This can be mitigated by CDNs that will automatically pull from an origin server, but there’s a tradeoff there in that the first user to request a piece of content will incur the load time for a failed CDN request and pull from the origin server.
The other mistake that companies make when setting up and configuring their CDN architecture is not building, updating, and flushing the CDN into their automated deploy process. It’s often left as a manual step that is easily forgotten and can result in an application with display issues and inconsistencies.
Gregory Kent / @ChelseaTech
Gregory Kent in the Operations Coordinator for Chelsea Technologies, a managed IT services firm that provides design, implementation, hosting, support, and security services to the global financial industry. Greg has over seven years of experience in the Information Technology sector.
“The main mistake that companies make when setting up their own content delivery infrastructure is…”
Not making it scalable.
When presented with a project or plan for the initial build-out, most companies will go for the cheapest or most cost effective option. While this makes sense to most businessmen, it’s a definite sore point for any IT specialist who has to deal with such a network. Furthermore, most cost-effective options provide static networks that are not easily expanded or scaled to meet a rising demand. This is especially obvious in the world of content delivery providers. As they are asked to serve more and more content, they will need to expand their network to accommodate such requests.
To allow for scalability, companies should invest more capital into the initial setup of their content delivery network. This would include buys such as managed switches rather than dumb switches, Virtual Machine hosts with more resources than the minimum needed, and server racks that are clean, well planned out, and easy to re-order. In the end, the upfront costs will save much more in the long run.
Geoff Hoesch / @DragonflySEO
Geoff Hoesch is the owner of Dragonfly SEO, a Baltimore-based digital marketing firm focusing on ethical approaches to online marketing solutions.
“While speed of content delivery is important, so is security. Companies often overlook the fact that…”
CDNs are not always 100% compatible with SSL certificates. When setting up a CDN, developers should make sure that it is compatible with SSL certificates, even if they aren’t going to switch to HTTPS now. Some CDNs allow SSL, but only if the certificate is through them. Even for sites without shopping carts, security is important, and search engines are placing extra emphasis on it – so be prepared for the future.
Nathan Milford is the director of infrastructure at Shutterstock, where he is building the foundation for a globally distributed, multi-backend storage system to house and serve hundreds of petabytes of images, music, and video. He has built large-scale distributed systems, large-volume data projects, scalable architectures, and open source.
“When setting up and configuring their content delivery network architecture, companies need to…”
Plan for growth and not focus too much on solving the issue of today, particularly around the cost and sprawl of CDNs. Our infrastructure team meets regularly with technical and product stakeholders to anticipate and prepare to support our future needs. This holds problems at bay while fostering good team work and allowing us to plan and invest for the long-term.