Updating our technical Webmaster Guidelines

Webmaster level: All

We recently announced that our indexing system has been rendering web pages more like a typical modern browser, with CSS and JavaScript turned on. Today, we’re updating one of our technical Webmaster Guidelines in light of this announcement.

For optimal rendering and indexing, our new guideline specifies that you should allow Googlebot access to the JavaScript, CSS, and image files that your pages use. This provides you optimal rendering and indexing for your site. Disallowing crawling of Javascript or CSS files in your site’s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings.

Updated advice for optimal indexing

Historically, Google indexing systems resembled old text-only browsers, such as Lynx, and that’s what our Webmaster Guidelines said. Now, with indexing based on page rendering, it’s no longer accurate to see our indexing systems as a text-only browser. Instead, a more accurate approximation is a modern web browser. With that new perspective, keep the following in mind:

  • Just like modern browsers, our rendering engine might not support all of the technologies a page uses. Make sure your web design adheres to the principles of progressive enhancement as this helps our systems (and a wider range of browsers) see usable content and basic functionality when certain web design features are not yet supported.
  • Pages that render quickly not only help users get to your content easier, but make indexing of those pages more efficient too. We advise you follow the best practices for page performance optimization, specifically:
  • Make sure your server can handle the additional load for serving of JavaScript and CSS files to Googlebot.

Testing and troubleshooting

In conjunction with the launch of our rendering-based indexing, we also updated the Fetch and Render as Google feature in Webmaster Tools so webmasters could see how our systems render the page. With it, you’ll be able to identify a number of indexing issues: improper robots.txt restrictions, redirects that Googlebot cannot follow, and more.

And, as always, if you have any comments or questions, please as in our Webmaster Help forum.

Posted by Pierre Far, Webmaster Trends Analyst

How to Recover Lost Pageviews in pushState Experiences

Posted by GeoffKenyon

PushState and AJAX can be used in tandem to deliver content without requiring the entire page to refresh, providing a better user experience. The other week, Richard Baxter dove into the implications of pushState for SEO on Builtvisible. If you’re not familiar with pushState, you should spend some time to read through his post.

If you’re not familiar with delivering content this way, you can check out these sites using pushState and AJAX to deliver content:

Time: When you scroll to the bottom of the article, a new article loads and the URL changes
Halcyon: When you click on a navigation link, the left hand panel doesn’t refresh

While pushState is really cool and great for UX, there are analytics issues presented by this technology.

When the content on a page and URL are updated using AJAX and pushState, in most cases, the 
_trackPageView beacon is not fired and the pageview is not tracked. This artificially increases your bounce rate while reducing your pages per visit, time on site, and total pageviews along with other metrics associated with pageviews. 

How to tell if you’re having tracking problems

If you have a very high bounce rate or are generally curious to check if this is a problem for you, start by installing the GA Debugger extension for Chrome. Then go to the URL you want to investigate and open up the console (windows: control + shift + j, mac: command + option + j). Now, clear the console using the button at the left, and refresh the URL.

Once you refresh the page, you should see GA debugging show up in the console. To check that the initial page view is being tracked, you should see a “sent beacon” for a pageview.

Once you’ve established the initial pageview is tracked, click a link to load another page. If GA is properly tracking pageviews, you should see another pageview beacon being sent. If you don’t see this, then you have a problem.

Capturing these pageviews with GTM

The good news is that even though this is a huge problem, it can easily be fixed with Google Analytics and Google Tag Manager.

Start by creating a new “History Listener” tag. Now set your fire rules to all pages and hit save. This will simply look for changes to the URL.

Now we’ll need to create a separate event to fire a pageview when the URL History Listener fires. To do this, create a new GA tag. 

If you already run Google Analytics from GTM, you’ll simply need to modify your existing tag. This tag should, by default, be set to track pageviews. 

At this point we’ll need to set the firing rules. First, we should make sure the tag is firing on all of our pages for our basic GA installation.

The firing rule for all pages should be a default option.

If you are already running GA via GTM, you’ll already have this set up. You’ll need to create a subsequent firing rule to fire a pageview for this URL History Listener.

To do this, click to add a new firing rule and then select “create new rule.” Name the rule, and then move on to conditions. The default rule should be [url] [contains]; we need to change this to [event] [equals]. Then we’ll set the condition to gtm.historyChange. Now click save.

Now you should be all set to hit publish on your updated tag container. Overnight, you should see a change in your pageviews and related metrics.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Post Pigeon Geo Assessment – How Did Traffic Change by City Part 2

Moses and Rooth Attorneys at Law is a two location criminal defense practice with one office in central Orlando and one in New Smyrna Beach on the coast. I wanted to do a visual mapping comparison of pre and post Pigeon impacts like I did with Barbara Oliver Jewelry & Co. to see the impact of Pigeon both […]

Big Day in SERPs Yesterday, Bigger than Panda 4.1

Post Penguin Update

Our SERP flux tracker records a first serious SERP tremble since Penguin 3.0 roll-out in Australia. Stronger than Panda 4.1. There’s a conversation on Google+ suggesting this could be a Panda refresh:

“I see huuuge increase for domains having issues which feed Panda algorithm.”, Łukasz Rogala

Google suggested that Penguin 3.0 update may take weeks to roll out globally, but we in Australia haven’t seen any data to support a gradual roll-out model.…

The post Big Day in SERPs Yesterday, Bigger than Panda 4.1 appeared first on DEJAN SEO.

Google on Creating a Relevant Second Screen for TV

Google was granted a patent last week that looks like it might have been among one of the earliest patents filed by the company. It involves showing television programs (News Programs to be more exact), and showing web pages that might be relevant to the transcripts of the shows being presented to viewers. The details […]

The post Google on Creating a Relevant Second Screen for TV appeared first on SEO by the Sea.

Video Games Added To Google’s Knowledge Graph

Google is upping its game for gamers, now including video game information in its knowledge graph. Search queries on video games will result in a knowledge graph panel that includes details like the game’s release date, supported platforms, developers, review scores and more. In a report on…

Please visit Search Engine Land for the full article.

Microsoft Execs Lay Out Strategy For Growing Bing’s Market Share: “We’re Putting Search Where People Are”

At a Bing Ads Next event in Redmond, Washington yesterday, Microsoft executives outlined the roadmap ahead for Bing and Bing Ads, which powers paid search advertising on both Bing and Yahoo. In discussing the evolution of Bing and the way Micosoft is t…

Search In Pics: Google Sukkahs, Android Lollipops & Spy Vs Spy Google Glass

In this week’s Search In Pictures, here are the latest images culled from the web, showing what people eat at the search engine companies, how they play, who they meet, where they speak, what toys they have, and more. Google Express Halloween Bags: Source: Google+ Jon Stewart At Twitter:…

Please visit Search Engine Land for the full article.

Bing Ads Delivers An MCC: “Accounts Summary” Now Available For Managing Multiple Accounts

Bing Ads is delivering on a long-requested feature to make managing multiple accounts on the platform easier. Accounts Summary is Bing’s answer to the AdWords My Client Center (MCC). From Accounts Summary agencies and advertisers responsible for multiple Bing Ads accounts can toggle between…

Please visit Search Engine Land for the full article.

Who’s Really Winning The Search War?

Ask any search marketer what Google’s market share is, and they’ll likely say around 66%. That’s the stat comScore has long supplied. Contributor Eli Schwartz asks whether it’s really accurate, and conducts research to prove it.

The post Who’s Really Winning The Search War? appeared first…

Please visit Search Engine Land for the full article.