Disclosure: “I am a customer and an affiliate of SEMRush. This post is opinion. There may be affiliate links on this page where I would make a commission if you were to buy the product. The content is my own and not sponsored. External links to third party sites are moderated by me. Links to internal pages promote my own content and services. I highlight monetised links in an clear way on this site. This page uses cookies. Your privacy is respected. Disclaimer.” Shaun Anderson, Hobo
This useful video from Google (Feb 2017) details exactly why you need a seo audit today BEFORE you do anything important when it comes to SEO.
That video would have been even more useful in 2012 when this sort of thing became a de facto REQUIREMENT to perform for clients before you touched their site, but it is nice to have today to explain some things to clients.
Why audit your site, first?
- Want to build links to your site? You better audit your backlink profile and look to see what stuff should be cleaned up first.
- Want to switch your site your site to https? Or move domain? Perhaps you better audit your current 301 structure and broken backlinks first.
- Want to add new content to the site? You might want to check your website and get rid of the spammy content first. You will want to clean up any outdated or factually incorrect first.
- Want to add more ads to your site? You better check how many ads are currently on your pages, and where they are placed.
- Want SEO to improve your business fortunes? You had better check your strategies past and present meet Google’s current recommendations
You get the picture.
You could call it ‘increased complexity‘.
At this end, Google gave search engine optimisers more pressing problems than building unnatural links or spinning text to beat the competition.
With the frequency of ‘core quality updates’ that we think are based on (basically) machine learning, user click data and human quality rater feedback, to rank in Google for the medium/long term you will need to CLEAN UP AND AT THE SAME TIME UPGRADE your entire web site offering from a technical and content standpoint.
I do not touch a seo project without a full seo audit these days because you can be hired one day and a quality update can wipe out a vulnerable (I mean a site built on spam) the next day, and the seo can get the blame because they ‘touched’ something when they did not.
For the client, the benefits of a professional seo audit on project start are apparent as the challenge facing the site is laid bare in front of them, as are the opportunities and recommendations on how to improve search engine rankings for the future.
Many readers of this blog do seo themselves.
You may be familiar with SEMRush (Disclosure – I am a customer of many years and an affiliate) when it comes to competitor analysis and keyword research, as it is a very popular option amongst SEO professionals.
SEMRush is constantly updating their offering, and this (relatively) new and improved feature is a welcome one.
I rarely use the SEMRush site myself as I am addicted to using their data through API access. The things you can do with the API are wonderful. I use about 20 API from different providers (including SEMrush) to create my in-depth audits.
For beginners on a tight budget, though, and those carrying out the first seo audit of their site, SEMRush is a great option to get an idea of some of the technical issues facing your site.
Table Of Contents
[hoboTOC use=”h2″]
How To Use The SEMRush Site Audit Tool
Introduction to the SEMRush Technical Audit
The SEMRush Site Audit Tool is a fast and painless way to quickly identify simple technical or basic on-page issues your website may have, with minimal fuss and preparation.
Reports are ready within minutes, are easy to understand (even for the most inexperienced users) and include practical solutions for issue resolution and guidance for further investigation/research where required.
The tool automatically crawls your website and checks it against an extensive set of industry expectations and standards when it comes to usability. I won’t list the full set of tests/checks in this article, but some notable features of the audit include:
- Identify Broken Links:- Internal, External & Images are checked
- Page Meta Review:- Titles or Descriptions too long, too short, missing or duplicate?
- Page Headers:- Check 404’s, 301’s and other internal redirects
- On-page:- Canonical, H-tags, Alt tags and other best practice on-page optimisation
- Structural:- Sitemap.xml, robots.txt & AMP page validation
- Page Speed:- Does the site load quickly enough for all devices?
- https / http:- Migrating to https? Are all internal link references consistent and secure?
- hreflang:- International/multi lingual site checks
The results from the site audit can be easily shared with clients or development teams, and because re-auditing is so simple, it’s a great way to track and compare site improvements over time.
How to configure your site audit
It is very easy to run a technical audit using SEMRush. To begin, you first need to create a ‘Project’ on the SEMRush Dashboard
Enter the target URL and assign a ‘Project Name’, then select the ‘Site Audit’ set up:
Crawler Configuration
In most cases, the only ‘default’ parameter that will need to be changed will be the ‘limit of checked pages’, you want to ensure all potential URL’s are included in the crawl. A quick way to determine this to do a quick ‘site:youdomain.com’ search in Google:
For more experienced users, there is a range of options to tailor the crawl, including:
- Custom Checked Page Limit:- Increase this for larger sites
- Select Crawl Source:- Organic site crawl or only target URL’s contained in sitemap.xml
- Select User Agent:- Use default or specify Google Bot
- Crawl Rate:- Specify a low rate via robots.txt to avoid server resource issues (primarily on large sites)
- Include/exclude URL’s:- You can specify sections of the site to crawl or ignore based upon URL patterns/parameters
- Re-crawl Schedule:- Determine when the site should automatically be re-crawled.
Once you are happy, simply hit the ‘Start Site Audit’ button and wait for notification that the crawl is complete.
The Audit Dashboard Explained
Overview
SEMRush generates their own ‘Site Health’ score based upon the total number of ‘checks past’ to ‘checks failed’ ratio. This gives an instant overall idea of the sites current technical set-up.
This is updated after every re-crawl and shows the overall trend, handy for large and regularly updated sites.
The next section summarises the number of site issues found during the audit. The issues fall into 3 categories….
- Errors:- The most severe issues found
- Warnings:- Less severe but still recommended
- Notices:- General best practice elements, which help the overall site quality
The ‘Thematic Score’ info boxes lead through to some useful data on the overall site crawl equity, https and international (or ‘hreflang tags’) implementation.
The most crucial errors and warnings are also brought forward and highlighted:
Issues Tab
Lists all site errors and warnings, with brief but useful explanations of the importance / relevance / potential solution for each element.
Pages with errors are then grouped, so it is extremely useful for identifying shared or page specific issues:
Crawled Pages
Lists all URL’s picked up during site crawl, and you view/filter / summarise the data based upon metrics including:
- Crawl Depth
- Issues Found
- HTTP Status Code
- Re-audit URL’s individually (if required after fixes have been applied)
In addition, you can pull data directly from your Google Analytics account to help with top page identification and prioritization:
Click on site links to see individual page reports listing details/issues to be addressed:
Statistics
Ideal for ongoing reporting and issue resolution tracking. This dashboard provides short-cut links to access sets of URL’s based upon key technical metrics which can then be easily exported or shared.
Compare Crawls
Ideal for tracking site improvements over time, this is especially handy when large teams or different departments are involved.
Progress
The progress tab compares and presents data from each site audit in a graphical format. You can easily overly specific ‘issue’ metrics to get a clear indication of where improvements are being made and more crucially where issues are on the increase.
Summary
The SEMRush Site Audit is a great tool to quickly analyse/review the technical set up of any website. The results can be easily shared or export for further analysis, and should give you a clear indication of where the main problems lie.
A key benefit is the ability to schedule re-crawls / re-fresh data which tracks progress and issue resolution. It is especially useful for clarifying and targeting issues for development teams, and outputs ready made reports for clients.
Resolving underlying technical issues should be the main priority following an SEO Audit. Nothing should be left that will impede the search engines from crawling and indexing your content
Keyword / Ranking Positions Audit:
Introduction to the SEMRush Keyword Tools
SEMRush provides a comprehensive suite of tools relating to keyword research and position tracking. For the purposes of an audit, we minimum we would want to determine is:
- what are the primary terms (most relevant, most desirable) for the target site?
- what terms is the target site actually ranking for?
- what are the top positions (and associated volumes)?
Luckily this is very easy to do.
Using the main search bar, select ‘Organic Research’ from the drop down menu, then input the root domain (without http://, https:// or www.) hit search and once the page reloads, you will be presented with Organic Search Position data for the target URL.
Geographic Options
You can change the Geographic location by country to suit.
Mobile Data
Still in a beta release, the Mobile Search data is available to all users.
Totals Summary / Trend Graph / SERP Features
- Shows the total number of ranking keywords associated with the target site and the total number of users expected to visit the site in the next month (based upon average of previous).
- View either the overall trends, Traffic or Keywords, overlaid with known Google algorithm updates or feature changes.
- Highlights the percentage of the interactive SERP Features the associated site keywords trigger when searched. This gives an insight to the site visibility.
Data Table
Here you will be presented with a breakdown of all ranking terms and URLs and associated data:
Of these, the most pertinent metrics to look for are:
- Search Volume:- The monthly average number of times the term has been searched for (based upon 12 months data)
- Traffic Percentage:- The overall percentage of traffic the term has sent to the site (over the selected period)
Historic Data
Historical keyword data is also available, depending on your current SEMRush plan. In most cases the Fresh data is more than adequate, however in some circumstances being able to look back can be invaluable particularly if it’s an old domain with a potentially ‘checkered’ past.
Summary
As both a keyword research and positioning tool, SEMRush is an excellent all rounder. For site auditing purposes, it is quick and easy to get a good idea of a sites current (and historic) key term & ranking performance, simply by entering the URL.
Competitor Audit
Introduction to the SEMRush Competitor Research Tools
Most companies and organisations will know who their direct / paid search competitors are in regards to the products or services they offer, however, this is not always the same as their ‘organic search’ competitors.
How to identify top organic competitors
Again make sure the target URL is selected via the SEMRush search bar and simply navigate to the ‘Competitors’ Tab
Competitor Positioning Map
You will be presented with a visual representation of the organic competitors for the target URL:
This provides an easy to interpret ‘snapshot’ of how well the target URL performs against competitors in the vertical, based upon the SEMRush ‘Competition Level’ metric which looks primarily at common keywords.
This graph lets you see, at a glance, if the target URL is in direct competition with any big brand or authority sites in the niche.
Organic Competitor Table
This table contains the full set of competitor URLs, and again this can be reordered using the table headings and the data can be easily exported for external analysis/sharing purposes.
For site auditing purposes the most important metrics/columns to look at are:
Competition Level:
“The metric is based on the analysis of the number of keywords of each domain, and the number of the domains’ common keywords. If the domains have a high number of common keywords, they could be considered competitors.”Common Keywords:
“The same keywords the given and queried domains are ranking for in Google’s top 100 organic search results.”SE Keywords:
“Keywords bringing users to the website via Google’s top 100 organic search results.”
As a starting point, you should aim to have a list of approx 5 -10 close ‘organic search’ competitors which can then be analysed in more detail using the full set of SEMRush competitor data features available. You should be able to:
- Evaluate competitor content & ranking performance
- Highlight content gaps on the target site
- Identify new keyword opportunities
Summary
Competitor research and analysis is perhaps the strongest part of the SEMRush offering.
The ability to analyse and (relatively) compare ‘any’ website is invaluable as is being able to really delve into competitor content and link performance. The set of competing URL’s presented often throw up unanticipated insight which can be used to your advantage.
For audit purposes, the results will show a comprehensive overview of the ‘search’ competition, from big brands to small niche sites which you may not be aware of. You will also gain a unique view of where the target site sits in the overall search landscape.
In each example, you are able to further investigate back link, keyword and ranking data which will give you a quick and easy way to look for potential strategy and opportunity.
Backlink Audit:
Introduction to the SEMRush Backlink Analysis Tool
This section provides a detailed synopsis of the target sites back link profile, instantly generating metrics which can be used gauge the current link health, and quickly identify potential issues.
Of all the available reports and data views, the most potentially useful when auditing a site for the first time is…
The total number of backlinks: Quickly determine the quantity and quality of the referring links.
The total number of referring domains: A detailed overview of the linking domains including geographic and TLD authority metrics.
A breakdown of all anchor text: What are the most common phrases used when linking? site URL or brand name? is there any evidence of potential keyword stuffed or ‘spammy’ link building activities? These are easily spotted using this section.
Indexed Pages: Which pages are attracting the most external links, in total and from unique domains. What are the target sites most linkable assests?.
Summary:
The backlinks report can, in a few clicks, result in a useful overview of the target sites backlink profile. Key metrics can help spot potential trouble and you can utilise the inbuilt Disavow file feature to manage bad links moving forward. Quality links are also highighted
While the SEMRush link database is perhaps not as big as other ‘specialist’ back link data providers, it is accurate enough to act upon, to determine where further analysis is required and when turned towards competitor sites, can reveal potential link building opportunities.
Note: SEMrush currently have a new ‘Backlink Audit’ Tool under a beta release, we will be sharing a full review of this feature in the near future.
Content Audit:
In any initial site audit, one of the key questions is… what is the top performing content on this site?
Google Analytics remains the most accurate way to determine top performing content and organic landing pages (other analytical and tracking packages are available, of course), however in the absence of access to site generated statistical data, SEMRush can provide it’s own independent data set from which to draw (relatively accurate, for a third party) insights.
The Organic Research > Pages report lists top content based upon 2 key metrics…
- Traffic Percentage: Estimated percentage of overall site traffic generated by the page.
- Keywords: The number of (top 100) terms the page currently ranks for.
Summary
Although it’s only ‘estimated’ traffic, when this is joined with the ranking and backlink data, a clear indication of the most valuable site content emerges. Conversely, poor performing content is also be highlighted.
With ‘content quality’ an apparent major ranking factor in Google this report is actually one of my favourite reports on SEMRush that is not in the API at the time of writing. To be able to quickly check the terms a page ranks for is invaluable, as is looking at the same reports on competitor websites.
Overall Summary
SEMRush is a great suite of tools for the beginner to do an SEO audit on their site. I also really like the email you get reminding you there’s stuff to be done on your site.
I’ve got my site plugged in and I will keep an eye on how things progress over the next couple of months and update this article with my further thoughts on the tool.
So far, so good.
The amount of data SEMrush has collected is vast and impressive. Part of that data set includes access to historical website keyword ranking data (available to subscribers signed up to the SEMRush GURU package – Disclosure: I am affiliate and customer of SEMrush).
Disclaimer
Disclaimer: “The author does not vouch for third party sites or any third party service. Visit third party sites at your own risk. This article is a personal opinion of research based on my experience of almost 20 years. It is not advice. I am not directly affiliated with Google or any other third party other than via affiliate programme. This website uses cookies only for analytics and basic website functions. This article does not constitute legal advice. The author does not accept any liability that might arise from accessing the data presented on this site.“ Shaun Anderson, Hobo
8,500
followers
2,800
likes
5000+
connections