Categories
Blogroll
Useful Sites
Archives
- September 2010
- August 2010
- July 2010
- June 2010
- May 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- November 2008
Friendly Links!
(Add your link here!)Tags
10-Minute Missing Page Audit
08/18/10
Posted by Dr. Pete
Some of you know that I spend a lot of time behind the scenes here on Pro Q&A. One of the challenges of Q&A is that we often have to tackle complex problems in a very short amount of time – we might have 10-15 minutes to solve an issue like "Why isn’t my page showing up on Google?" with no access to internal data, server-side code, etc.
Of course, I’d never suggest you try to solve your own SEO problems in just 10 minutes, but it’s amazing what you can do when you’re forced to really make your time count. I’d like to share my 10-minute (give or take) process for solving one common SEO problem – finding a "missing" page. You can actually apply it to a number of problems, including:
- Finding out why a page isn’t getting indexed
- Discovering why a page isn’t ranking
- Determining if a page has been penalized
- Spotting duplicate content problems
I’ll break the 10 minutes down, minute by minute (give or take). The mini-clock on each item shows you the elapsed time, for real-time drama.
0:00-0:30 – Confirm the site is indexed
Always start at the beginning – is your page really missing? Although it sometimes gets a bad rap for accuracy (mainly, the total page counts), Google’s site: command is still the best tool for the job. It’s great for deep dives, since you can combine it with keyword searches, "keyword" searches (exact match), and other operators (intitle:, inurl:, etc.). Of course, the most basic format is just:
For this particular job, always use the root domain. You never know when Google is indexing multiple sub-domains (or the wrong sub-domain), and that information could come in handy later. Of course, for now you just want to see that Google knows you exist.
0:30-1:00 – Confirm the page is not indexed
Assuming Google knows your site exists, it’s time to check the specific page in question. You can enter a full path behind the site: command or use a combination of site: and inurl:
If the page doesn’t seem to be on Google’s radar, narrow down the problem by testing out just "/folder" and see if anything on the same level is being indexed. If the page isn’t being indexed at all, you can skip the next step.
1:00-1:30 – Confirm the page is not ranking
If the page is being indexed but you can’t seem to find it in the SERPs, pull out a snippet of the TITLE tag and do an exact-match search (in quotes) on Google. If you still can’t find it, combine a site:example.com with your page TITLE or a portion of it. If the page is indexed but not ranking, you can probably skip the next couple of steps (jump to the 4:00 mark).
1:30-2:00 – Check for bad Robots.txt
For now, let’s assume your site is being partially indexed, but the page in question is missing from the index. Although bad Robots.txt files are, thankfully, getting rarer, it’s still worth taking a quick peek to make sure you’re not accidentally blocking search bots. Luckily, the file is almost always at:
http://www.example.com/robots.txt
What you’re looking for is source code that looks something like this:
It could either be a directive blocking all user agents, or just one, like Googlebot. Likewise, check for any directives that disallow the specific folder or page in question.
2:00-2:30 – Check for META Noindex
Another accidental blocking problem can occur with a bad META Noindex directive. In the header of the HTML source code (between <head> and </head>), you’re looking for something like this:
Although it might seem odd for someone to block a page they clearly want indexed, bad META tags and Rel=Canonical (see below) can easily be created by a bad CMS set-up.
2:30-3:00 – Check for bad Rel=Canonical
This one’s a bit trickier. The Rel=Canonical tag is, by itself, often a good thing, helping to effectively canonicalize pages and remove duplicate content. The tag itself looks like this:
The problem comes when you canonicalize too narrowly. Let’s say for example, that every page on your site had a canonical tag with the URL "www.example.com" – Google would take that as an instruction to collapse your entire search index down to just ONE page.
Why would you do this? You probably wouldn’t, on purpose, but it’s easy for a bad CMS or plug-in to go wrong. Even if it’s not sitewide, it’s easy to canonicalize too narrowly and knock out important pages. This is a problem that seems to be on the rise.
3:00-4:00 – Check for bad header/redirects
In some cases, a page may be returning a bad header, error code (404, for example) or poorly structured redirect (301/302) that’s preventing proper indexation. You’ll need a header checker for this – there are plenty of free ones online (try HTTP Web-Sniffer). You’re looking for a "200 OK" status code. If you receive a string of redirects, a 404, or any error code (4xx or 5xx series), you could have a problem. If you get a redirect (301 or 302), you’re sending the "missing" page to another page. Turns out, it’s not really missing at all.
4:00-5:00 – Check for cross-site duplication
There are basically two potential buckets of duplicate content – duplicate pages within your site and duplicates between sites. The latter may happen due to sharing content with your own properties, legally repurposing contents (like an affiliate marketer might do), or flat-out scraping. The problem is that, once Google detects these duplicates, it’s probably going to pick one and ignore the rest.
If you suspect that content from your "missing" page has been either taken from another site or taken by another site, grab a unique-sounding sentence, and Google it with quotes (to do an exact match). If another site pops up, your page may have been flagged as a duplicate.
5:00-7:00 – Check for internal duplication
Internal duplication usually happens when Google crawls multiple URL variations for the same page, such as CGI parameters in the URL. If Google reaches the same page by two URL paths, it sees two separate pages, and one of them is probably going to get ignored. Sometimes, that’s fine, but other times, Google ignores the wrong one.
For internal duplication, use a focused site: query with some unique title keywords from the page (again, in quotes), either stand-alone or using intitle:. URL-driven duplicates naturally have duplicate titles and META data, so the page title is one of the easiest places to find it. If you see either the same page pop up multiple times with different URLs, or one or two pages followed by this:
…then it’s entirely possible that your missing page was filtered out due to internal duplication.
7:00-8:00 – Review anchor text quality
These last two are a bit tougher and more subjective, but I want to give a few quick tips for where to start if you suspect a page-specific penalty or devaluation. One pretty easy to spot problem is when you have a pattern of suspicious anchor text – usually, an uncommon keyword combination that dominates your inbound links. This could come from a very aggressive (and often low-quality) link-building campaign or from something like a widget that’s dominating your link profile.
Open Site Explorer allows you to pretty easily look at your anchor text in broad strokes. Just enter your URL, click on Anchor Text Distributions (the 4th tab), and select Phrases:
What you’re looking for is a pattern of unnatural repetition. Some repetition is fine – you’re naturally going to have anchor text to your domain name keywords and your exact brand name, for example. Let’s say, though, that 70% of our links pointing back to SEOmoz had the anchor text "Danny Dover Is Awesome." That would be unnatural. If Google thinks this is a sign of manipulative link building, you may see that target page penalized.
8:00-10:00 – Review link profile quality
Link profile quality can be very subjective, and it’s not a task that you can do justice to in two minutes, but if you do have a penalty in play, it’s sometimes easy to spot some shady links quickly. Again, I’m going to use Open Site Explorer, and I’m going to select the following options: Followed + 301, External Pages Only, All Pages on The Root Domain:
You can export the links to Excel if you want to (great for deep analysis), but for now, just spot-check. If there’s something fishy on the first couple of pages, odds are pretty good that the weaker links are a mess. Click through to a few pages, looking out for issues such as:
- Suspicious anchor text (irrelevant, spammy, etc.)
- Sites with wildly irrelevant topics
- Links embedded in an obviously paid or exchanged block
- Links that are part of a multi-link page footer
- Advertising links that are followed (and shouldn’t be)
Also, look for any over-reliance on one kind of low-quality link (blog comments, article marketing, etc.). Although a full link-profile analysis can take hours, it’s often surprisingly easy to spot spammy link-building in just a few minutes. If you can spot it that fast, chances are pretty good that Google can, too.
(10:00) – Time’s Up
Ten minutes may not seem like much (it may have taken you that long just to read this post), but once you put a process in place, you can learn a lot about a site in just a few minutes. Of course, finding a problem and solving it are two entirely different things, but I hope this at least gives you the beginning of a process to try out yourself and refine for your own SEO issues.
| | Posted by randfish
If you’ve been using SEOmoz’s tools for a while, you’ve likely run across the same challenge we have — the data rocks, but you have to run a separate tool and report for each aspect of your SEO. That’s why, for the past year, we’ve been building a new platform to house all of the features and functionality of the SEOmoz toolset, and includes the ability to track metrics over time. We call it the SEOmoz "Web App."
It’s hard for me to resist making this post nothing but a huge collection of screenshots…
Starting TODAY (right now, in fact!), every PRO member has BETA access to the new web app.
We encourage you to create your first campaign(s), start tracking some rankings, crawling some pages and getting those optimization recommendations. The web app is our first application to automatically track data for any site(s) you care about on an ongoing, consistent basis, and make recommendations based on what we find. Thus, the real value comes after a week or two of tracking, and it gets more valuable, useful and reference worthy over time. One of our big goals was to make this app give back some beautiful screenshots and reports showing your progress in reducing the crawl problems, improving optimization and gaining in the rankings over time.
How it Works
The challenge with doing SEO on sites bigger than a few dozen pages is that no human has time to visit every page on a regular basis and check to make sure nothing is wrong, no opportunities are missed, no errors are present, and no search rankings are being unintentionally forfeited through poor optimization. The web app is designed to solve this by:
#1 – Crawling all the pages on your site every week and identifying potential issues in three categories:
- Errors – like 404s, 500s, missing titles, duplicate content, etc.
- Warnings – such as 302s, overly dynamic URL strings, long title elements, meta refreshes, etc.
- Notices – not problems, per se, but issues you might want to be aware of like 301s, rel=canonical tags, meta robots blocking, etc.
While we love what Google’s Webmaster Tools does on this front, we felt there were a lot more issues we wanted to see.
#2 – Tracking Rankings on the keywords you choose and automatically grading the pages that rank in the top 50 for their on-page optimization. It’s a simple concept, but a powerful one. Why? Because, we often found that:
- URLs ranking on page 2, 3 or 4 for a keyword phrase could move up dramatically in rankings and traffic with just a bit of keyword tweaking
- Sometimes, SEOs & marketers might not even realize they had a page that was competitive for a keyword, and showing off that potential automatically could make a huge difference
- Tracking rankings can help identify potential causes of rising/falling traffic and give a key baseline for performance, particularly as we add in data like traffic via web analytics, keyword difficulty scores and estimated search volume.
Basic rank tracking is great, but we think there’s a lot more you can do with the data when it’s integrated with other KPIs and metrics
Tracking competitors simultaneously is helpful for those SEOs who want to keep special tabs on select sites
#3 – On-Page Grading & Recommendations serve as a great companion to rank tracking, enabling easy identification of low hanging fruit (poorly optimized pages that still rank in the top 50 results are often huge opportunities for improved rankings and traffic). We built this feature because:
- It’s frustrating to figure out the best practices for on-page optimziation, but our correlation and ranking models work have made some valuable strides that deserve to be shared.
- No one can visit hundreds or thousands of pages and find all the keywords that are suboptimally targeted. The web app (particularly in concert with tip #1 below) can and it will message these with easy-to-parse grades and simple recommendations.
- You can feel confident that the on-page recommendations are consitently updating with best practices. As we learn more through testing/research or as algorithms change (we re-do our ranking models every few months), that information will quickly make its way into the on-page report cards.
An "A"! That’s good news. Still missing a few recommendations that might be worth checking out, though.
Aha! Using the keyword in the alt attribute might be a good addition (both for normal web rankings and for image SEO)
#4 – Link Analysis for your site and those of key competitors. This tab is still in progress, but in the future, we plan to integrate all of the link analysis abilities of Open Site Explorer and add historical tracking, competitive SERPs analysis and more.
Much more functionality coming soon
#5 – One Comprehensive System to Rule It All. More so than any single feature, we wanted to begin the process of replicating all the functionality of SEOmoz’s PRO tools into a single, integrated platform. You should be able to do all your SEO from one place – the web app. The functionality released today is just a tiny part of what’s to come.
Now I can see and access all the sites I care about from one place in my PRO account.
Here’s a video of me whiteboarding a bit more about the new web app:
How Does this Affect My PRO Account?
If you’re already PRO, think of this as a big, new feature that’s ready to start tracking data for you today. None of the old tools, functionality, guides, Q+A, tips, webinars, etc. are going away. We’re just adding the web app as a key part of the SEOmoz PRO package.
We are, however, making a few shifts that I think you’ll really like.
Previously, PRO membership tiers had limits for how many Linkscape reports they could run. We’ve scaled up our servers to the point where we can handle pretty hefty quantities of requests, so as of today, we’re removing your Linkscape advanced report limits – all PRO members can now run unlimited numbers of Linkscape reports. Q+A remains the same, as does access to all the other tools + content.
Here’s the new PRO membership plans:
You may notice we’ve changed our pricing; however, these prices are for new members only. Existing PRO members will continue to pay what they have in the past, but receive the expanded benefits and privileges we’re launching today. It’s our way of saying thanks for joining us in the early days. PRO members – you’ll get an email soon detailing the improvements/upgrades to your membership.
If you’re not yet PRO, we’re keeping the PRO price of $79/month available until August 25th (14 days from now). After that, the price will go up to $99/month. If you sign up now (at $79/month), you’ll be grandfathered in and won’t be subject to future price increases.
And, as always, as PRO membership gets better, you’ll get immediate access to new features & upgrades and we’ll never raise your price.
5 Quick Tips to Get Started
The web app is certainly designed to be largely self-explanatory and I’m sure many of you have already jumped in and given it a spin. But, for those seeking a few tricks, here’s some that we’ve found in the last few weeks of using it for projects (some in concert with our consulting friends at Distilled).
#1 – Import some of your top referring keywords from your analytics
When I started the SEOmoz campaign, I grabbed our top few hundred keywords that referred traffic and imported them through the manage keywords input box (which kindly accepts CSV, TSV and carriage-return separated inputs).
By exporting my list of top keywords from Google Analytics and pasting them into the web app, I’ve got easy tracking on tons of keywords I care about, and a chance to find some missed opportunities.
#2 – Find pages that rank in the top 10-30 with low grades (Cs, Ds and Fs)
When my rankings and on-page reports came back, I found tons of pages that got Cs or Ds ranking in position 5-50, ripe for improvement.
Those Cs, Ds and Fs are gonna turn into As and send us a lot more traffic!
The best part is the automation – it’s a simple process, but an incredibly tedious one if performed manually. The web app’s a lifesaver on that front. Those Cs, Ds and Fs, if improved to As and Bs, will likely bring up rankings, particularly on the less competitive terms and phrases and those where I may have the links I need to rank, but never took the time to do on-page optimization. Editing a few page titles and adding some keywords to content never felt so rewarding.
#3 – Use the "grade any page" easter egg feature
The web app’s on-page optimization system will soon be replacing our tired and somewhat rickety term target tool. In the meantime, you can run "one-off" reports through an easter egg feature. Just create the keyword you care about and you can edit the URL field manually to grade any page for any term/phrase:
Notice the highlighted portion in blue in the screenshot – easter egg FTW!
#4 – Apply labels to your keyword groups to separately track relevant data/metrics
Nearly every section of the app allows for filtering based on keyword group, error type, etc. By building smart keyword groups (via the "manage keywords" page) to align with the types of terms/phrases I care about, I can separately build reports showing performance in rankings and on-page optimization for those labels.
#5 – Point out crawl diagnostic issues to a friend or potential client
If your friend has a website with some crawl issues, I’m sure they’d appreciate having that pointed out – you can even export the crawl results to CSV and email it over. Likewise, if you’ve got clients or managers you’re trying to convince to do some SEO fixes, the diagnostic visualization can be very handy to show off the problem.
Sending over this report might just bring your buddy a bit more search traffic (or help you win a new client)!
What’s Next?
Oh, believe me – we’re just getting warmed up. 
Our engineering team is committed to regular updates every few weeks, starting with something big a couple weeks from today. Those updates will add features, fix bugs and help grow this web app into the platform professional SEOs deserve – a system that lets you perform all of the essential tasks of SEO from one place – from building, submitting and managing XML Sitemaps to tracking your ROI from organic keywords through integration with your analytics package to tracking search results in verticals like video, maps/local, news, images, et al. to mapping your crawl data from Webmaster Tools against your indexation of pages receiving search traffic (we’ve had some good chats with the WM Tools team about integrating their API in cool ways) and much, much more.
We’ll be publishing a post this weekend inviting you to vote for features you’d most like to see. That feedback will be used to help prioritize our work, so suggest now, vote on the weekend and you’ll see what you want in the app even sooner!
We Need Your Feedback
The web app is the basis for our future plans around providing great SEO software, and we need your help to make it amazing. On every page of the web app, you’ll find a feedback tab on the left-hand side of the page. Please click it often and tell us what you found frustrating, what more you want to see, and even what you liked and want more of. Our team reads every one and build priorities based on the most-requested features.
We also know there’s going to be bugs at launch. Some early ones we’re aware of include:
- Slowness – many of the web app pages take 10-15 seconds or more to load depending on the quantity of keywords, issues or other data involved. We’re working to cache much of this in databases for much faster retrieval in the next few weeks.
- Crawl & Rankings Retrieval – your first data will take 5-20 minutes to populate. We do an initial "speed crawl" of 250 pages, but to be respectful of server load issues, we’re pretty cautious with our requests. That first hour inside the app after creation may feel a bit empty, but just wait, it gets much better
- On-Page Grades – edge cases, including keywords that are out of order or have odd broad matching aren’t perfect yet. We’ll be working to fix this and make this experience as seemless as possible.
- Root Domains – currently, campaigns can only be created with single subdomains (e.g. www.seomoz.org, not *.seomoz.org). We’re working to make root domains an option as well, so you can track all the pages at mydomain.com, www.mydomain.com, blog.mydomain.com, etc. in one campaign.
- Adding more than 10 Keywords Simultaneously – to help with load as we begin public testing, we’ll only collect rankings for the first 10-20 keywords entered immediately (within 10-15 minutes). If you add 50 or 500 keywords at once, we’re limiting rankings and on-page checks until your next update date for now.
This isn’t just another tool – it’s a new approach for SEOmoz – creating an application that uses crawl data, APIs and integrated research to improve your productivity and simplify SEO.
But, we need to be realistic – in this first iteration, we’re not just in beta, we’re at the nascent stages of the app’s potential. What we’re launching today is pretty remarkable, and I feel confident that for nearly everyone reading this post, it can help you earn a considerable amount more traffic from the search engines. That said, the future is what we feel best about, particularly since we hope to have your help in the improvement process.
This is very much a BETA launch, but we think you’ll enjoy playing around with the app and get a lot of value in return. So, what are you waiting for? Go build a campaign!
p.s. If you’d like to learn more, check out our FREE WEBINAR on the Web App coming up this Friday, August 13th at 10am Pacific – Register here. And you can still sign up for the SEOmoz Tools Training at our Seattle Seminar later this month (we’ve just added more seats).
How to Benchmark in Analytics
07/29/10
Posted by JoannaLord
We have a lot of changes going on at SEOmoz (feel free to get excited, we sure are!) and with all of these changes to the site comes the need to focus on tracking. Internally we have spent the last few months redirecting our attention to not only the best practices regarding analytics and data mining, but really pushing ourselves to revisit our analytical processes.
You know what we realized? There sure is a lot of data. While I have always appreciated the reporting features in GA, I find that too often people take the reports at face value and fail to go deeper. It’s unfortunate since it is in those deep dives that you usually discover the data that can change your current course of action. So this post is going to tackle an approach to analytics that is often overlooked and (thanks to Google and their silly naming convention decisions) is rarely used to its fullest capacity. Get excited folks we are going to talk about benchmarking {Woohoo! Insert audience applause here}.
All of you excel spreadsheet lovers out there know plenty of ways to extract data and pinpoint specific red flags or recent successes. In fact, most people use analytics to simply analyze the current state of their account. While this is certainly a priority, it really is one dimensional. Instead of stopping there, why not go further? Why not better understand where your data was, and how you are measuring up? In fact, why not use this data to help inform your internal decisions as a company? It’s like an analytical epiphany—“using past and current data to help guide you moving forward.” Glorious.
While many of the analytics platforms out there have given us a number of ways to compare historical data to current data, we are still limited to two distinct time ranges (for the most part). It’s great to see those two ranges stack up against each other, but that still leaves a lot to be desired. Without going further you miss the "interaction" between those two distinct time ranges.
Benchmarking your data is a great way to discover more about this, often overlooked, gray area. Benchmarking simply means you set a standard at which you compare something else to. When used for data mining, it means you plot two distinct variables (time ranges, metrics, dimensions, etc.) over a period of time and then use these “benchmarks” to infer conclusions when making decisions.
You can then see a more complete picture of your site’s momentum. In my opinion, understanding your site’s momentum is one of the most powerful metrics an analyst can calculate. If you can say with authority that you know how your site is doing and how it will likely be doing in the next week, month, few months, etc., you are in an ideal place. With data like that you can take more calculated risks.
*First, I want to throw out a disclaimer—a little over a year ago Google decided to integrate “Benchmarking” into their Visitors tab in GA. This just made things confusing in my opinion. The GA feature actually shows your site in comparison to a {very very very limited} industry pool of similarly {not really} sized sites. There is a lot wrong with the assumptions of this feature, but for our purposes here, when I say “benchmarking” I mean the act of plotting two distinct variables over time to extract insight…not the {ridiculous-I-can’t-believe-they-took-it-out-of-beta} GA feature.
The "benchmarking" feature in GA on SEOmoz
Okay now that we got that out of the way, let’s talk about how you can benchmark your data to hopefully gather some insight into your site’s performance.
Know your bottom-line (and your "high-line" –yes, I just made that word up)
This is probably the most common approach to benchmarking. It’s a pretty simple way to analyze the current state of your account. You should know your extremes for every metric. For example if you are a company that sells a seasonally successful product, you should know what your lowest conversion rate is for the year, as well as your peak conversion performance. In understanding the extremes you can make better assumptions on how your off season stats are trending. While not the most accurate approach to data mining, benchmarking the extremes of your account enables you to speak intelligently, at any given moment, on how your site is currently performing.
Know your ratios & relationships
Am I the only one that always reads “ratio” as “radio”? I digress. Knowing your metric ratios and how they relate to each other, is a great way to quickly detect when things are headed south. Often, as analysts, we don’t realize something has gone wrong until we see sales are down. While that is an effective method of pinpointing mistakes, it certainly isn’t ideal. Wouldn’t it be nice to quickly identify issues as they actually become issues? Crazy, I know. Well this is exactly what benchmarking the ratios of your site’s metrics can do. At SEOmoz, we use ratio/relationship benchmarking to keep our traffic stats in check. We don’t just plot out how many visitors each section of the site brings in out of the total visitors; we compare those percentages against each other. This gives us a ballpark value to guide us. An example; “the X part of the site brings in roughly twice as much as Y, which brings in about 1/3 of the traffic as Z.”
The great part about this method of benchmarking is you can easily turn it into a visual representation of the different pieces of the pie, and isolate out when things start to shift. Below is an actual example Rand pulled together earlier this week (yes he does that sort of thing for fun! A true data-head!). In this chart we have graphed out the top trafficked pages on our site, and then plotted them against each other to show how they are performing in relation to each other.
Also see a larger, detailed version
You can see the significant drop in the blue segment (our Tools page), which was due to a redirect mistake we made (oops…Rand talks more about that here). By visually representing these sections, we can easily identify shifts in the relationships, which can guide us on where we should focus our attentions (aka fix our silly SEO mistake ASAP!).
Know the norm
Okay I know, I know…I talked a whole lot of trash above on the GA benchmarking feature, and here I am talking about “knowing the norm,” but approaching data analysis this way can be insightful. Knowing and using industry standards in benchmarking can efficiently identify low hanging fruit.
However, the actual GA benchmarking tab is a poor example of this. Keep in mind that sites have to opt into the benchmarking, so (a.) this feature might not even have your industry represented and (b.) you have no way of knowing how many sites these “standards” are calculated on. Also keep in mind there are only three buckets for website “size” in this feature—small, medium, and large. WTF right? Yeah, since when do all websites fit into those three sizes? What am I ordering a latte over here?
With that said, it’s worth knowing the vital metric standards for your industry. If you see that similar sites to your own have a bounce rate of around 40% and you are chilling around 65%, while all the other metrics look closer in range, then you can assume this metric is where you should direct your optimization efforts. This approach isn’t as scalable or as accurate as other benchmarking methods, but it’s definitely worth a mention, if only for peace of mind.
Know the limits
While benchmarking is incredibly effective for things like trending, projecting, and exploring the data, it’s important to know the limits of the process. It is meant to be a discovery process, not a scientific formula. Just like anything else you take away from the data, it is just an insight, not a guarantee. You are making assumptions based on past performances, and performances change. So one word of caution to all of you data-heads out there—benchmarking is a great tool to add to your bag of tricks, but it is only one of many you should be using. Don’t get so caught up in forming relationships between the metrics and dimensions of your site that you lose perspective on the independent variables themselves.
In conclusion
Get in there. I mean it, seriously. I know we are all crazy busy, but that shouldn’t translate into a two minute GA log-in, a quick glance at the vital metrics and a few automated reports. Our analytics are meant to be explored. Benchmarking is one of those processes that may take an extra hour or two, but discoveries made during those few hours can be instrumental in guiding your company’s decisions.
Confession: At SEOmoz we haven’t always been the best with analytics and tracking, but in the past half a year we have refocused our energies on truly knowing what our users are doing, how our site is performing, and finding opportunities within the data. It’s time consuming, and tricky, and what you discover is not always fun to find out, but it has certainly helped us redirect resources where they are needed.
Over the next few months we are rolling out all sorts of good stuff, {the Chrome toolbar launch was just a teaser my friends 
}. We are using processes like benchmarking to better prepare us for these changes. Taking on new challenges as a company is an awesome thing, but doing it with a little data to steer you, makes the ride even more fun.
Posted by Danny Dover
Well folks, this may be the biggest tool introduction since Ryan Seacrest started hosting American Idol. 
Today we are launching our SEO toolbar for Google’s Chrome browser. This sexy beast is full to the brim with SEO insight and time-saving SEO goodness. This free add-on is ripe for picking and available for download right this second.
Ask and Ye Shall Receive
Whether it’s from Twitter, Facebook, email or comments here on the blog, almost every day we get some sort of request for a Chrome Toolbar. We knew there was a high need for it, and wanted to make sure that we didn’t rush and put together something unmozworthy. The new toolbar is pretty baller if I do say so myself (which I just did). It works very similarly to a toolbar in Firefox where it displays across the top of the screen, but with the ability to easily drop it down to the bottom of the page as well.
The new Chrome toolbar has most of the same features as the Firefox edition, but if you want to learn more… please keep reading.
So How Does This Help Me?
1. Search Results Overlay
This new Search Engine Results Page overlay was designed to offer the most relevant link data without getting in the way. You can now use our toolbar to see which search results are getting the most links, and click Explore to run a full analysis in Open Site Explorer. To turn on this overlay, click the settings button on the toolbar, and select SERP Overlay.
.png)
"I get the best ‘feel’ for abstract metrics by seeing them in familiar places. I find it easiest to understand the new metrics by seeing them on search results I’m familiar with; as an added bonus, this is one of the most helpful analyses you can do when looking at a new SERP for the first time." –Will Critchlow
2. Quickly find important SEO information with the Analyze Page Overlay
Our analyze page overlay provides quick access to useful data points which include:
- On-page Optimization Elements – All of the essential SEO on-page elements (title tag, meta description, meta robots, rel canonical) are in one place.
- Location Data – As local search becomes more important, the value of information like server location increase.
- Google cache – Want to see exactly what Google saw when it’s bot crawled a given site. The link to Google’s cache will get you there.
"The overlay is still the most valuable thing for me. I must use it 5+ times every day to get quick info about how many links are on a page, whether it’s using rel="canonical" or whether the keywords are properly included in the right page elements. I hate using ‘view source’ and searching through code; overlay FTW!" –Rand Fishkin
3. Quick Access to Tools from SEOmoz and Third Parties
The tools dropdown has been expanded to include fast access to the latest SEOmoz tools as well as a wealth of other helpful resources, including traffic data, Twitter tools, and domain information.
.gif)
What if I find a bug?
Reporting Bugs
The best way to report bugs is to e-mail us at customerservice(at symbol)seomoz.org. This is the quickest way to get into our development queue.
Known Issues
1. The toolbar overlays some of the page content. We attempted to inject the CSS into the page and push down the page content, but this ended up breaking some useful sites (like Twitter) so we overlay the page instead , but do now offer the ability to display the toolbar at the bottom of the page, which should hopefully help, when you’ve just got to see the Facebook nav.
2. Because of the way the toolbar is rendered as part of the page, it only shows up when the page loads, so if the toolbar is turned on while other windows already have loaded content, you will need to click refresh to see the toolbar. Unfortunately, this is also the case when you open a new chrome window, since chrome shows cached content on open to appear faster.
We hope you enjoy the new toolbar. Please give it a try, and be sure to post feedback in the comments below.
Not yet a Chrome fan? Still plugging away in good ol’ Firefox? Well we haven’t forgotten about you! You were our first love, and can still be downloaded here.
7 Types of SEO Evidence
07/07/10
Posted by Dr. Pete
.jpg)
We’ve had a lot of discussions recently about SEO as a Science. Unfortunately, these discussions sometimes devolve into arguments over semantics or which approach is the "best" in all situations. I’d like to step back for a few moments today and talk about the wider world of SEO evidence. While not all of these types of evidence are "science" in the technical sense, they are all important to our overall understanding. We need to use the best pieces of all of them if we ever hope to develop a mature science of SEO.
The Fundamental Assumption
All science rests on a fundamental assumption, long before any hypothesis is proposed or tested. The fundamental assumption is that the universe is orderly and follows rules, and that through observation and experimentation we can determine those rules. Without an orderly universe, science would be impossible (as would existence, most likely). A related assumption is that these rules are relatively static – if they change, they change very slowly. Our view of the universe may change dramatically, resulting in paradigm shifts, but the underlying rules remain roughly the same.
The advantage we have as SEOs is that we know, for an absolute fact, that our universe is orderly. Like Neo, we have seen The Matrix. The Algorithm consists of lines of code written by humans and running on servers.
The disadvantage for SEO science is that the rules governing our universe are NOT static. The algorithm changes constantly – as often as 400 times per year. This means that any observation, any data, and even any controlled experiment could turn out to be irrelevant. The facts we built our SEO practices on 5 or 10 years ago are not always valid today.
(1) Anecdotal Evidence
All science begins with observation. In SEO, we make changes to sites every day and measure what happens. When rankings rise and fall, we naturally try to figure out why and to tie those changes to something we did in the past. Although it isn’t "science" in the technical sense, the evidence of our own experience is very important. Without observing the universe and creating stories to explain it, we would never learn anything from those experiences.
PROS – Anecdotal evidence is easy to collect and it’s the most abundant form of evidence any of us have. It’s the building block for just about any form of scientific inquiry.
CONS – Our own experiences are easily affected by our own biases. Also, no single experience can ever tell the whole story. Anecdotal evidence is just a starting point.
(2) Prophetic Evidence
SEOs have a unique type of available evidence. Every once in a while, a prophet will descend from the Mountain Top (or Mountain View), shave his head, and speak the words of the Google Gods. Whether or not we choose to believe these prophets, the fact remains that there are people who have seen and written the Algorithm, and those people have access to facts that the rest of us don’t. Their statements (and our ability to critically reconcile those statements) are an important part of the overall puzzle.
PROS – The prophets are as close to objective reality as we’re ever going to get. They have direct insight into the algorithm.
CONS – The prophets don’t have a vested interest in telling us the whole truth. Their messages can be cryptic and even misleading.
(3) Secondhand Evidence
When you hear "secondhand" evidence, you may naturally think of the extreme examples, like hearsay and urban legends:
My cousin’s neighbor’s stylist said that she once changed all of her META tags to "sex poker sex poker sex" and her site immediately jumped to #1 on Google!
To be fair, though, secondhand evidence also includes the legitimate science that came before us and the experiences of our peers. If we were forced to confirm and replicate every single conclusion for ourselves, we would never make any progress. Ultimately, we build on the reliable conclusions of other experts, past and present.
PROS – Secondhand evidence is the foundation for scientific progress.
CONS – Sometimes, experts are wrong, and you have to learn how to tell the difference, especially in a field as young as SEO.
(4) Experimental – "The Wild"
Experimentation is the heart of Capital-S Science. The most basic experiments happen something like this:
- You form a hypothesis ("Adding keywords to my title tag will improve rankings").
- You make a change to test that hypothesis.
- You measure the outcome and find out if you were right.
Most SEO experimentation, by its nature, occurs in the "wild". We have to put our sites out in the world, and we often have to use existing sites that are already complicated and changing.
PROS – By directly forming and testing a hypothesis, we can start to determine causality. We can also repeat the process, helping to validate what we’ve learned.
CONS – Using existing sites in the wild introduces a lot of extra noise. Often, our sites have to keep changing (even during the experiment), and Google is always changing. There’s also a fair amount of risk – if we change our bread-and-butter sites to test SEO theories, mistakes can be costly.
(5) Experimental – Controlled
This is the classic SEO experiment, where we register one or more new domain names and build sites from the ground up. We can even introduce a control group, by building both sites up to Step X and then only changing one of the sites after that point. Even then, it might be best to call these experiments "semi-controlled," since the Google algorithm can still change and we can’t always control outside influences (like someone accidentally linking to one of the sites).
PROS – This approach is about the best we can do, in terms of control, and it separates out a lot of confounding factors.
CONS – The artificial sites we set up in these experiments (often using nonsense words) aren’t always representative of real, complex sites. In addition, these experiments are usually conducted on a sample of just one or very few sites, to save time and money. Statistical significance can be very difficult to achieve.
(6) Correlational Evidence
Sometimes, either we can’t separate out the variables involved in a complex situation (like the 200+ factors Google uses in its ranking model) or direct experimentation would be impossible or unethical. For example, let’s say you want to understand how smoking affects mortality. You can’t take 1000 5-year-olds, force them to smoke for 70 years, and compare them to 1000 non-smoking 5-year-olds. In these cases, you take a very large data set and look at the correlations. In other words, if I look at 1000 smokers and 1000 non-smokers, how likely is each group to die at a certain age? Correlation can help you understood how changes in X (smoking, in this case) co-occur with changes in Y (mortality).
PROS – Correlation can help us mathematically find relationships when direct experimentation is impossible or impractical. These techniques can also help model complex situations where multiple variables are affecting the same outcome.
CONS – Correlation does not imply causation. We don’t know if changes in X cause changes in Y or if they just happen to co-occur (maybe even due to a Factor Z affecting them both).
(7) Large-scale Simulation
If we can collect enough data, we can build a model of the universe and test hypotheses against that model. Now that large-scale indexes are being built to mimic Google (including our own Linkscape and indexes like Majestic), it only stands to reason that we’ll eventually be able to run experiments directly against these models. Although the conclusions we draw from these simulations are only as good as the models themselves, simulation data can help us both improve models and conduct something closer to a laboratory test than is usually possible in SEO.
PROS – Simulations can be controlled. Unlike Google, we know whether we’ve changed the model or not. Experiments can also be run very quickly and on a very large-scale.
CONS – The result of any simulation is only as good as the model it’s built on, and our models are still in their infancy.
Which One Is The Best?
Any type of evidence, including controlled experimentation, has limits. In a field like SEO, where the Google algorithm is constantly changing, relying too much on any one type of evidence can either stall progress or lead us to bad conclusions (or, in some cases, both). Understanding every available source of evidence not only helps us paint a broader, more comprehensive picture, but it also helps us cross-test our hypotheses and prevent mistakes. SEO science is a young and constantly changing field, and, at least for now, SEO scientists will need to adapt quickly.
Patience is an SEO Virtue
06/29/10
Posted by Kate Morris
We have all been there once or twice, maybe a few more than that even. You just launched a site or a project, and a few days pass, you login to analytics and webmaster tools to see how things are going. Nothing is there.
WAIT. What?!?!?!
Scenarios start running through your mind, and you check to make sure everything is working right. How could this be?
It doesn’t even have to be a new project. I’ve realized things on clients’ sites that needed fixing: XML sitemaps, link building efforts, title tag duplication, or even 404 redirection. The right changes are made, and a week later, nothing has changed in rankings or in webmaster consoles across the board. You are left thinking "what did I do wrong?"
A few client sites, major sites mind you, have had issues recently like 404 redirection and toolbar PageRank drops. One even had to change a misplaced setting in Google Webmaster Tools pointing to the wrong version of their site (www vs non-www). We fixed it, and there was a drop in their homepage for their name.
That looks bad. Real bad. Especially to the higher ups. They want answers and the issue fixed now … yesterday really.
Most of these things are being measured for performance and some can even have a major impact on the bottom line. And it is so hard to tell them this, even harder to do, but the changes just take …
Patience
That homepage drop? They called on Friday, as of Saturday night things are back to normal. The drop happened for 2-3 days most likely, but this is a large site. Another client, smaller, had redesigned their entire site. We put all the correct 301 redirects for the old pages and launched the site. It took Google almost 4 weeks to completely remove the old pages from the index. There were edits to URLs that caused 404 errors, fixed within a day, took over a week to reflect in Google Webmaster Tools.
These are just a few examples where changes were made immediately, but the actions had no immediate return. We live in a society that thrives on the present, immediate return. As search marketers, we make c-level executives happy with our ability to show immediate returns on our campaigns. But like the returns on SEO, the reflection of changes in SEO take time.
The recent Mayday and Caffeine updates are sending many sites to the bottom of rankings because of the lack of original content. Many of them are doing everything "right" in terms of onsite SEO, but now that isn’t enough. The can change their site all they want to, but until there is relevant and good content plus traffic, those rankings are not going to return for long tail terms.
There has also been a recent crack down on over optimized local search listings. I have seen a number of accounts suspended or just not ranking well because they are in effect trying too hard. There is a such thing as over optimizing a site, and too many changes at once can raise a flag with the search engines.
One Month Rule
Here is my rule: Make a change, leave it, go do social media/link building, and come back to the issue a month later. It may not take a month, but for smaller sites, 2 weeks is a good time to check on the status of a few things. A month is when things should start returning to normal if there have been no other large changes to the site.
We say this all the time with PPC accounts. It’s like in statistical analysis, you have to have enough data to work with to see results. And when you are waiting for a massive search engine to make some changes, once they do take effect in the system, you then have to give it time to work.
So remember the next time something seems to be not working in Webmaster Tools or SERPs:
- If you must, double check the code (although you’ve probably already done this 15 times) to ensure it’s set up correctly. But then,
- Stop. Breathe. There is always a logical explanation. (And yes, Google being slow is a logical one)
- When did you last change something to do with the issue?
- If it’s less than 2 weeks ago, give it some more time.
- Major changes, give it a month. (Think major site redesigns and URL restructuring)
Posted by MichaelC
First, a quick refresher: URL prettying and 301 redirection can both be done in .htaccess files, or in your 404 handler. If you’re not completely up to speed on how URL rewrites and 301s work in general, this post will definitely help. And if you didn’t read last week’s post on RewriteRule’s split personality, it’s probably helpful background material for understanding today’s post.
 |
"URL prettying" is the process of showing readable, keyword-rich URLs to the end user (and Googlebot) while actually using uglier, often parameterized URLs behind the scenes to generate the content for the page. Here, you do NOT do a 301 redirection. (Unclear on redirection, 301s vs. 302s, etc.? There’s help waiting for you here in the SEOmoz Knowledge Center.) |
|
301s are done when you really have moved the page, and you really do want Googlebot to know where the new page is. You’re admitting to Googlebot that it no longer exists in the old location. You’re also asking Googlebot to give the new page credit for all the link juice the old page had earned in the past. For example, you may have migrated your website to a new content management system, and all of the pages have somewhat different URLs than then had before the move. |
 |
If you’re trigger-happy, you might leap to the conclusion that RewriteRule is the weapon of choice for both URL prettying and 301 redirects. Certainly you CAN use RewriteRule for these tasks, and certainly the regex syntax is a powerful way to accomplish some pretty complex URL transformations. And really, if you’re going to use RewriteRule, you should probably be using it in your httpd.conf file instead.
The Apache docs have a great summary of when not to use .htaccess.
Fear Not the 404 Handler
 |
First, all y’all who tremble at the thought of creating your very own custom 404 handler, take a Valium. It’s not that challenging. If you’ve gotten RewriteRule working and lived to tell the tale, you’re not going to have any difficulty making a custom 404 error handler. It’s just a web page that displays some sort of "not found" message, but it gives you an opportunity to have a look at the page that was requested, and if you can "save it", you redirect the user to the page they’re looking for with just a line or two of code. |
If not, the 404 HTTP status gets returned, along with however you’d like the page to look when you tell them you couldn’t find what they were looking for.
By the way, having your own 404 handler gives you the opportunity to entertain your user, instead of just making them feel sorry for themselves. Check out this post from Smashing Magazine on creative 404 pages.
Having a good sense of humor could inspire love & loyalty from a customer who otherwise might just be miffed at the 404.
Here’s an example of a 404 handler in ASP. Important note: don’t use Response.Redirect – it does a 302, not a 301!
.jpg)
For PHP, you need to add a line to your .htaccess pointing to wherever you’ve put your 404 handler:
- ErrorDocument 404 /my-fabulous-404-handler.php
Then, in that PHP file, you can get the URL that wasn’t found via:
- $request = $_SERVER['REDIRECT_URL'];
Then, use any PHP logic you’d like to analyze the URL and figure out where to send the user.
If you can successfully redirect it, set:
- header("HTTP/1.1 301 Moved Permanently");
- header ("Location: http://www.acmewidgets.com/purple-gadgets.php");
And here’s where it gets a bit hairy in PHP. There’s no real way to transfer control to another webpage behind the scenes–without telling the browser or Googlebot via 301 that you’re handing it off to the other page. But you can use call require() on the fly to pull in the code from the target page. Just make sure to set the HTTP code to 200 first:
- header(‘HTTP/1.1 200 OK’);
And you’ve got to be careful throughout your site to use include_once() instead of include() to make sure you don’t pull a common file in twice. Another option is to use curl to grab the content of the target page as if it were on a remote server, then regurgitate the HTML back in-stream by echoing what you get back. A bit hazardous if you’re trying to drop cookies, though…
And, if you really need to send a 404:
- header(‘HTTP/1.0 404 Not Found’);
Very Important: be careful to make sure you’re returning the right HTTP code from your 404 handler. If you’ve found a good content page you’d like to show, return a 200. If you found a good match, and want Googlebot to know about that pagename instead of what was requested, do a 301. If you really don’t have a good match, be sure you send a 404. And, be sure to test the actual response codes received–I’m a huge fan of the HttpFox Firefox plug-in.
Ease of Debugging
|
This is where the 404 handler really wins my affection. Because it’s just another web page, you can output partial results of your string manipulation to see what’s going on. Don’t actually code the redirection until you’re sure you’ve got everything else working. Instead, just spit out the URL that came in, the URL you’re trying to fabricate and redirect to, and any intermediate strings that help you figure it all out. With RewriteRule, debugging pretty much consists of coding your regex expression, putting in the flags, then seeing if it worked. Is the URL coming in in mixed case? The slashes…forward? Reverse? Did I need to escape that character…or is it not That Special? |
 |
You’re flying blind. It works, or it doesn’t work.
If you’re struggling with RewriteRule regular expressions, Rubular has a nice regex editor/tester.
Programming Flexibility
 |
With RewriteRule, you’ve got to get all the work done in the single line of regex. And while regex is elegant, powerful, and should be worshipped by all, sometimes you’ll want to do more complex URL rewriting logic than just clever substitution. In your 404 handler, you can call functions to do things like convert numeric parameters in your source URL to words and vice versa. |
Access to Your Database
|
If you’re working with a big, database-driven site, you may want to look up elements in your database to convert from parameters to words.
And since the 404 handler is just another webpage, you can do anything with your database that you’d do in any other webpage. |
 |
For example, I had a travel website where destinations, islands, and hotels all were identified in the database by numeric IDs. The raw page that displayed content for a hotel also needed to show the country and island that the hotel was on.
The raw URL for a specific hotel page might have been something like:
/hotel.asp?dest=41&island=3&hotel=572
Whereas the "pretty URL" for this hotel might have been something like:
/hotels/Hawaii/Maui/Grand-Wailea/
When the "pretty URL" above was requested by the client, my 404 handler would break the URL down into sections:
- looking up the 2nd section in the destinations table (Hawaii = 41)
- looking up the 3rd section in the island table (Maui = 3)
- looking up the 4th section in the hotel table (Grand Wailea = 572)
Then, I’d call the ASP function Server.Transfer to transfer execution to /hotel.asp?dest=41&island=3&hotel=572 to generate the content.
Now, keep in mind that you’ll probably want to generate the links to your pretty URLs from the database identifiers, rather than hard-code them. For instance, if you have a page that lists all of the hotels on Maui, you’ll get all of the hotel IDs from the database for hotels where the destination = 41
and island = 3, and want to write out the links like /hotels/Hawaii/Maui/Grand-Wailea/. The functions you write to do this are going to be very, very similar
to the ones you need to decode these URLs in your 404 handler.
Last but not least: you can keep track of 404s that surprise you (i.e. real 404s) by having the page either email you or log the 404′ed URLs to a table
in your database.
Performance
 |
For most people, the performance hit of doing the work in .htaccess is not going to be significant. But if you’re doing URL prettying for a massive site, or have renamed an enormous list of pages on your site, there are a few things you might want to be aware of–especially with Google now using page load speed as one of its ranking factors. |
All requests get evaluated in .htaccess, whether the URLs need manipulation/redirection or not.
That includes your CSS files, your images, etc.
By moving your rewriting/redirecting to your 404 handler, you avoid having your URL pattern-matching code check against every single file requested from your webserver–only URLs that can’t be found as-is will hit the 404 handler.
Having said that, note that you can pattern-match in .htaccess for pages you do NOT want manipulated, and use the L flag to stop processing early in .htaccess for URLs that don’t need special treatment.
Even if you expect nearly every page requested to need URL de-prettying (conversion to parameterized page), don’t forget about the image files, Javascript files, CSS, etc. The 404 handler approach will avoid having the URLs for those page components checked against your conversion patterns every single time they’re fetched.
A Special Case
OK, maybe this case isn’t all that special–it’s pretty common, in fact. Let’s say we’ve moved to a structure of new pretty URLs from old parameterized URLs.
Not only do we have to be able to go from pretty URL –> parameterized URL to generate the page content for the user, we also want to redirect link juice from any old parameterized URL links to the new pretty URLs.
In the actual parameterized web page (e.g. hotel.asp in the above example), we want to do a 301 redirect to the pretty URL. We’ll take each of the numeric parameters, look up the destination, island, and hotel name, and fabricate our pretty URL, and 301 to that. There, link juice all saved…
But we’ve got to be careful not to get into an infinite loop, converting back and forth and back and forth:
.jpg)
When this happens, Firefox offers a message to the effect that you’ve done something so dumb it’s not going even bother trying to get the page. They say it so politely though: "Firefox has detected that the server is redirecting the request for [URL] in a way that will never complete."
By the way, it’s entirely possible to cause this same problem to happen through RewriteRule statements–I know this from personal experience 
It’s actually not that tough to solve this. In ASP, when the 404 handler passes control to the hotel.asp page, the query string now starts with "404;http". So in hotel.asp, we see if the query string starts with 404, and if it does, we just continue displaying the page. If it doesn’t start with 404;http then we 301 to the pretty URL.
Other References
Information on setting up your 404 handler in Apache:
- http://www.plinko.net/404/custom.asp
- http://www.webreference.com/new/011004.html
- http://www.phpriot.com/articles/search-engine-urls/4
Apache documentation on RewriteRule:
ASP.net custom error pages:
Technorati Tags
RewriteRule, 301, htaccess, 404 handler
Posted by richardbaxterseo
On the "So You Want to Test SEO?" panel at this year’s SMX Advanced Seattle, Branko Rihtman from SEO Scientist presented some spiffy looking ranking charts, measuring positioning by keyword, over time. A few people asked me how exactly you make a chart like that. Being something of an Excel fan, I was instantly inspired to share the approach with my fellow SEOmozzers. Here’s a step by step on how to create a rankings chart using Excel.
Collect the data
To be able to produce a chart like my example below, you’re going to need Microsoft Excel, and a rankings checker that will export ranking data, by search engine and by date. For now, I’m using Advanced Web Ranking, but there are lots of other ranking checkers you can use. Start by putting your data in an Excel table named "rankings" just like this:
Create a pivot chart
Pivot tables were designed for exactly this type of application, and making them is heaps of fun. Let’s start by selecting "Insert > PivotTable >PivotChart" in the options along the top of your Excel ribbon.
You should see a window appear. Make sure you’ve named the correct range (our table name: "rankings") and select "New Worksheet", followed by OK.
Drag and drop your legend, axis and value fields
The cool thing about making a pivot table is the drag and drop functionality when you’re creating the row labels and values for the table. Here’s a visual explanation of where to put your keyword, date and position data:
Next, you’ll need to filter for the keywords you’d like to create a chart for. It’s quite inpractical to create a chart with hundreds of keywords, but you can add a good number for comparision purposes. Head to the "Column labels" drop down and filter for the keywords you’d like to build the chart for:
Filter by search engine
If you’ve collected data on multiple search engines, you’ll need to add a filter. Drag the "Search Engine" field down into the "Report Filter" section, and select the search engine you’re interested in using the drop down at the top of your pivot table.
Format your chart nicely
If you’ve followed the instrutions so far, you’ll see a slightly noisy and weird looking bar chart, so next we’ll create a line chart to show the positional changes over time.
For pure charting awesomeness, a simple right mouse click on the chart, followed by "Change chart type > Line", will do the trick. Finally, you’ll need to reverse your Y axis, leaving position 1 at the top and your lower rankings at the bottom. Using your right mouse button, click on the axis and select "Format axis" – you should see a window like this:
The end result
After spending some time having fun with formatting, you can create really nice charts. Here’s mine:
Hope you find these tips useful, and if you’d like some more of this, please shout in the comments!
Posted by randfish
Sometimes, the page you’re trying to rank – the one that visitors will find relevant and useful to their query – isn’t the page the engines have chosen to place first. When this happens, it can be a frustrating experience trying to determine what course of action to take. In this blog post, I’ll walk through some of the root causes of this problem, as well as five potential solutions.
When the wrong page from your site appears prominently in the search results, it can spark a maddening conflict of emotion – yes, it’s great to be ranking well and capturing that traffic, but it sucks to be delivering a sub-optimal experience to searchers who visit, then leave unfulfilled. The first step should be identifying what’s causing this issue and to do that, you’ll need a process.
Below, I’ve listed some of the most common reasons we’ve seen for search engines to rank a less relevant page above a more relevant one.
- Internal Anchor Text
The most common issue we see when digging into these problems is the case of internal anchor text optimization gone awry. Many sites will have the keyword they’re targeting on the intended page linking to another URL (or several) on the site in a way that can mislead search engines. If you want to be sure that the URL yoursite.com/frogs ranks for the keyword "frogs," make sure that anchor text that says "frogs" points to that page. See this post on keyword cannibalization for more on this specific problem.
_ - External Link Bias
The next most common issue we observe is the case of external links preferring a different page than you, the site owner or marketer, might. This often happens when an older page on your site has discussed a topic, but you’ve more recently produced an updated, more useful version. Unfortunately, links on the web tend to still reference the old URL. The anchor text of these links, the context they’re in and the reference to the old page may make it tough for a new page to overcome the prior’s rankings.
_ - Link Authority & Importance Metrics
There are times when a page’s raw link metrics – high PageRank, large numbers of links and linking root domains – will simply overpower other relevance signals and cause it to rank well despite barely targeting (and sometimes barely mentioning) a keyword phrase. In these situations, it’s less about the sources of links, the anchor text or the relevance and more a case of powerful pages winning out through brute force. On Google, this happens less than it once did (at least in our experience), but can still occur in odd cases.
_ - On-Page Optimization
In some cases, a webmaster/marketer may not realize that the on-page optimization of a URL for a particular keyword term/phrase is extremely similar to another. To differentiate and help ensure the right page ranks, it’s often wise to de-emphasize the target keyword on the undesirable page and target it more effectively (without venturing into keyword stuffing or spam) on the desired page. This post on keyword targeting can likely be of assistance.
_ - Improper Redirects
We’ve seen the odd case where an old redirect has pointed a page that heavily targeted a keyword term/phrase (or had earned powerful links around that target) to the wrong URL. These can be very difficult to identify because the content of the 301′ing page no longer exists and it’s hard to know (unless you have the history) why the current page might be ranking despite no effort. If you’ve been through the other scenarios, it’s worth looking to see if 301 redirects from other URLs point to the page in question and running a re-pointing test to see if they could be causing the issue.
_ - Topic Modeling / Content Relevance Issues
This is the toughest to identify and to explain, but that won’t stop us from trying Essentially, you can think of the search engines doing a number of things to determine the degree of relevancy of a page to a keyword. Determining topic areas and identifying related terms/phrases and concepts is almost certainly among these (we actually hope to have some proof of Google’s use of LDA, in particular, in the next few months to share on the blog). Seeing as this is likely the case, the engine may perceive that the page you’re trying to rank isn’t particularly "on-topic" for the target keyword while another page that appears less "targeted" from a purely SEO/keyphrase usage standpoint is more relevant.
Once you’ve gone through this list and determined which issues might be affecting your results, you’ll need to take action to address the problem. If it’s an on-page or content issue, it’s typically pretty easy to fix. However, if you run into external linking imbalances, you may need more dramatic action to solve the mistmatch and get the right page ranking.
Next, we’ll tackle some specific, somewhat advanced, tactics to help get the right page on top:
- The 301 Redirect (or Rel Canonical) & Rebuild
In stubborn cases or those where a newer page is replacing an old page, it may be wise to simply 301 redirect the new page to the old page (or the other way around) and choose the best-converting/performing content for the page that stays. I generally like the strategy of maintaining the older, ranking URL and redirecting the newer one simply because the metrics for that old page may be very powerful and a 301 does cause some loss of link juice (according to the folks at Google). However, if the URL string itself isn’t appropriate, it can make sense to instead 301 to the new page instead.Be aware that if you’re planning to use rel=canonical rather than a 301 (which is perfectly acceptable), you should first ensure that the content is exactly the same on both pages. Trying to maintain two different version of a page with one canonicalizing to another isn’t specifically against the engines’ guidelines, but it’s also not entirely white hat (and it may not work, since the engines do some checking to determine content matches before counting rel=canonical sometimes).
_ - The Content Rewrite
If you need to maintain the old page and have a suspicion that content focus, topic modeling or on-page optimization may be to blame, a strategy of re-authoring the page from scratch and focusing on both relevance and user experience may be a wise path. It’s relatively easy to test and while it will suck away time from other projects, it may be helpful to give the page more focused, relevant, useful and conversion-inducing material.
_ - The Link Juice Funnel
If you’re fairly certain that raw link metrics like PageRank or link quantities are to blame for the issue, you might want to try funnelling some additional internal links to the target page (and possibly away from the currently ranking page). You can use a tool like Open Site Explorer to identify the most important/well-linked-to pages on your site and modify/add links to them to help channel juice into the target page and boost its rankings/prominence.
_ - The Content Swap
If you strongly suspect that the content of the pages rather than the link profiles may be responsible and want to test, this is the strategy to use. Just swap the on-page and meta data (titles, meta description, etc) between the two pages and see how/if it impacts rankings for the keyword. Just be prepared to potentially lose traffic during the test period (this nearly always happens, but sometimes is worth it to confirm your hypothesis). If the less-well-ranked page rises with the new content while the better-ranked page falls, you’re likely onto something.
_ - The Kill ‘Em with External Links
If you can muster a brute force, external link growth strategy, either through widgets/badges, content licensing, a viral campaign to get attention to your page or just a group of friends with websites who want to help you out, go for it. We’ve often seen this precise strategy lift one page over another and while it can be a lot of work, it’s also pretty effective.
While this set of recommendations may not always fix the issue, it can almost always help identify the root cause(s) and give you a framework in which to proceed. If you’ve got other suggestions, I look forward to hearing about them in the comments!
Posted by great scott!
Link building sucks. You know it, I know it, everyone knows it. It can be slow, tedious, and exhausting. It’s also one of the most crucial aspects of complete search engine optimization. So what do you do when faced with the intimidating challenge of building links? Once upon a time, you could’ve just submitted your site to a few hundred cheap directories (or a few thousand like so many of the $99 "SEO" shops offer), arrange for a few dozen reciprocal links from sites with decent PageRank, and maybe even negotiate a nice, keyword-targeted footer link from a reasonably popular blog. Bing-bang-boom, you’ve got several hundred good links with super-optimized anchor text…hellooooo rankings!
Those of you who’ve been playing this game for a while are probably thinking, "ahh, 2004, those were the days!" Everyone else is either looking at the screen incredulously or laughing hysterically, "this stuff doesn’t work at all anymore!" Oh really? Doesn’t it? Sitewide, Reciprocal, and Directory links often have a bad rap because in the last several years they’ve largely become synonymous with cheap, spammy, dishonest, and largely useless scam SEO offers. But here’s the catch: if you’re careful, reasonable, and practical, these oft-maligned practices can still be effective. Don’t go screaming black hat on me, watch this week’s video to learn the how, when, and why of what can make these black sheep of the link building world viable tactics.
As discussed in the video there are times when these strategies can be legitimate. Rand covered these in a lot of detail in our recent PRO Webinar on Advanced Competitive Link Building, so if you’re a PRO Member, be sure to watch the recording. For now, let’s look at some situations where these strategies can still work.
Sitewide Links The early oughties (aka 2000′s) were the like Studio 54 for sitewide links: shady links were snorting coke off of hookers in the dark recesses of footer navigation across the web. Then Google raided the joint looking for manipulative link patterns like the IRS looking for cooked books–the jig was up for footer and sidebar sitewide nav links. To this day you can occasioanlly stumble across a rogue footer containing a few links out to ridiculously unrelated content (one local theater here in Seattle has links out to branded baby care products), but by-and-large this practice is no longer used…except for when it is. Does Disney link to other sites in its content network? Does Lulu link to their SEOmoz and PC magazine awards? Does SEOmoz link to service partners like Distilled and Exact Target? Yes, they/we do and we do so in sitewide footers. These are legitimate and natural relationships. There’s nothing strange or fishy here. In fact, if any of these links were paid, they’d be better off on one or two strong pages rather than on a sitewide navigational element. Basically, you should consider these bad if/when they seem unnatural and/or they’re done alongside other shady stuff.
Reciprocal Links First things first: within niche industries, natural reciprocal links are compeletely natural. In fact they’re often difficult to avoid. Think about the SEO space; SEOmoz, SEOBook, Search Engine Land, Search Engine Journal, and all the others…we’re constantly linking to each other, but do we ever call up Aaron or Loren or Matt and say, "hey, I’ll link to your page if you link back to mine with this exact anchor text"? No, that’d be ridiculous. ‘Reciprocal’ becomes a four-letter word when it becomes clear that your site has an unusually high proportion of 1-to-1 links (you and other sites link to each other only once), often with suspiciously consistent anchor text. Those are the phenomena that start to look shady and draw attention.
Directory Links Here’s the litmus test for a directory: Do they care who you are? Good directories endeavour to actually create a high-value resource by excercising editorial control and restricting listings to sites and businesses that will be of value to their users. Bad directories endeavour to maximize the number of people willing to pay them money to be listed next to Der International Haus of Spamcakes because, hey, it’s a PR3 link! It’s really that simple. Directory links of the good variety can be really solid link sources (they’re often niche or local), but the bad kind (of which you can probably find 20,000 for $99) ain’t gonna do a damn bit of good for you.
When it comes down to it, you simply need to use good judgement with your link efforts. Is this a link someone would not be surprised to find on this site and in this location? Is the link from a site you could or would legitimately link to in a blog post? Would your site or page be a good resource for someone visiting a particularly directory? What about the rest of the content and links, do they seem legitimate? A little honest evaluation and some common sense is really all you need to avoid engaging in bad linking practices.
