10 Possible Causes for a Drop in Traffic to Your Website
Recently I’ve discovered what caused the amount of visitors to a certain website to drop sharply and for an extended amount of time. I’d noticed myself that the site had started to run slower recently and was loading unpredictably, which are normally bad signs on itself. Then people started to report (amongst other strange things) ‘Google 404 errors‘ on the website. After my initial “screenshots or it didn’t happen“, the screenshots containing an image of Googlebot actually started coming in. It took me days and days of digging and testing to finally discover what caused the problem: incorrect nameservers mistakenly added to the DNS records of the website.
What follows in a checklist of possible causes for a major drop in traffic to your website. May it save you the endless hours of debugging it took me to figure all this out!
1. Analytics tool
Most likely you’re monitoring your site traffic via Google Analytics. While this is a great tool (which we use ourselves as well), it’s not flawless. If you’re contemplating big changes based on your dropped site traffic (as reported by just one tool), you might want to consider looking at another tool (perhaps one based on your server logs) to get a ‘second opinion’ on the statistics provided by the first.
2. Visitor statistics
So, it appears the amount of visitors at your website has dropped, but have they stopped visiting your through all channels? Try opening up your favorite site analytics tool(s) and check out the amount of visitors coming to your website directly, via a referral or via search. Have all these numbers started dropping at the same time? Has just one of them dropped? If for example just the search visitors number have declined, you might want to start paying closer attention to issues related to Google penalties, Google algorithm updates (such as Panda), robots.txt, user agent sniffing, crawler access, etc. Aside from the sources through which your traffic (isn’t) coming in, you might also want to have a look variables such as ‘Bounce rate‘ and ‘Time on site‘. Have they decreased as well?
3. Subversion or FTP
Are you sure the code you’re looking at in your code editor is the same code that is being executed on your server? Perhaps you’ve switched your code to a subversion branch on your localhost and can’t figure out why that one piece of code isn’t being executed online? Maybe your FTP program failed to upload a file, causing your server to still load the older version of it? In short: make absolutely 100% sure that the code you’re looking at is the same code that is being executed on your server.
4. DNS records or Nameservers
Make sure your DNS records contain exactly those values you want them to. It may be that others have access to your DNS settings (such as a system administrator or a web designer) and have edited the records without notifying you. To check your current DNS records and nameservers, head over to MX Toolbox and enter your domain name (without a sub domain) in the input field, like ‘themepartner.com’. The tool will then report back your DNS, nameservers (NS), MX records and the Whois information.
A file that can be placed in the root of your website folder. The robots.txt file can be used to allow or deny access to certain or all user agents. If you have mistakenly blocked access to certain parts of your website, this might be a really tricky problem to track down. Try viewing your robots.txt file by surfing to it directly: www.yourwebsite.com/robots.txt or by checking it via the Google Webmaster Central website.
Both a tremendously powerful file and the cause of many problems with websites. Your .htaccess file (which can only be used on an Apache server) contains logic that is performed by your server before showing your actual website. It can contain code to restrict access to certain users, rewrite URLs or modify PHP settings. If you’re unsure about the contents of your .htaccess file, try disabling it by temporarily renaming the file. Mind you that this renaming might cause a broad range of other problems on your server! Another solution would be to browse your website using another IP address than the one you’re normally using (by utilizing a proxy or your mobile phone with WiFi disabled). If there are any IP related problems in the .htaccess file, they should show up once you’ve changed the IP address you’re using to access the website.
7. Access Control List (ACL)
It could be that you’ve accidentally restricted viewing access for your website to logged-in users or to a certain user group. Because you’ll be logged in as ‘Superadmin‘ most of the time, you might not notice this mistake. Try browsing your website while being logged out to see if you bump into any problems. If you’re still having trouble keeping track of your ACL settings and you’re using Joomla! 1.7, you might want to consider purchasing ACL Manager by Sander Potjer. It’s an amazing tool to give you a graphical overview of your Joomla! ACL settings!
Sometimes the code you’ve committed or uploaded isn’t the same code you see in your web browser. A wide array of caching options is available these days to increase the performance of your website. If problems with caching occur, try looking at opcode caching such as APC, memory object caching like Memcached, the built-in caching system in Joomla! or browser cache itself. Restarting either your server, your computer, or your browser should clear away most of the cache, which should enable you to see the result of what code is actually being ran by the server.
If your website can’t be reached or is responding very slow, then people will simply stop visiting it. Try opening up a new command line prompt (Windows) or a new terminal (Mac, Linux) and run a ‘ping’ command to your server. For example on Windows that command would simply be ‘ping www.example.org‘. Are all packets received? How much is their average response time? If you suspect there might be a problem with your hosting, you could always try to simply contact them and explain the situation. Perhaps there is an outage in one of their server parks or there is another known problem they have ‘forgot’ to mention to their customers. Another helpful thing if you’re hosting your website on a Linux server would be to check your webserver’s error.log file (SSH access is often required) as these often contain valuable information about possible problems on your website.
One slow query can cause your entire website to load at the speed of a turtle walking up a steep hill. Potential causes of this might be the lack of proper indexes on your tables, very large tables, queries selecting for * instead of for specific fields, etc. Slow queries can be viewed by turning on ‘Debug system’ in the ‘Global Configuration’ of your Joomla! Administrator. Setting up the FirePHP add-on for Firefox (combined with the FireBug add-on) might provide additional insight into (AJAX) requests performed by the site. MySQL also has a built-in option to log queries that take a long time. More information about this can be found here.