Today I bring you this Guide of Screaming Frog SEO Spider, an essential SEO tool to be able to do our audits and find what errors we have on our website with the SEO On Page.
We must have controlled the health of our project, even if we are not SEO consultants and what better than with this magnificent tool.
To begin, we downloaded the tool here. Screaming Frog has 2 versions, one free and paid. The free allows you:
- Find broken links and redirects.
- Analyze titles and metadata.
- Meta robots and directives review.
- Create sitemap
- Track up to 500 URLs
The paid version is very complete, but with the free one, we can do a review of our site, to find errors.
To start, we open the program
Review the response codes
Near the top of the screen, click on the gray tab called «Response codes». This will give us all the response codes to all the URLs of the crawled website. To make things easier, click on the “Status Code” column to organize the URLs by status code number.
From here you can check the redirects and make sure that the correct response code is being implemented on each page. Response code 301 is a permanent redirect, while a 302 is applied when it is only temporary (it is rarely used).
After reviewing your 301 redirects twice, it’s time to move on to the addresses that returned 404 error response codes. A 404 response code means that the URL is linking to a non-existent page. Normally, an error message will appear when you click on it. To resolve this problem, you must identify all pages with error codes 404 and 301.
But wait, what about all the pages that are linked to the pages with 404 error? Luckily for you, with the Response Code window open, you can also check for broken internal and external links. Select a URL with a 4XX or 5XX code. In the bottom window, click on the “Inlinks” and “Outlinks” tab to see a list of addresses that link to the damaged URL.
If you want them, Screaming Frog allows you to export all redirects or 401 errors, near the top of the screen there is a filter drop-down menu. Click on the menu of what you would like to export, and then click on the “Export” button to the side to get your data.
Updating the response codes can be quite tedious. However, if you update all pages with redirects and error codes, your user experience will improve considerably (and your rankings will also). It is vital to do this, that’s why I created this Screaming Frog guide.
However, if you update all pages with redirects and error codes, your user experience will improve considerably (and your rankings will also). It is vital to do this, that’s why I created this Screaming Frog guide.
Check URL structures
Website addresses that contain parameters and unusual characters are difficult to track and rank in search results. Also, confusing URLs lead to poor user experience and will make it difficult for people to easily navigate through your website. To make your website easier to track and navigate, you should check the structure of your URLs.
Go to the «URI» tab in Screaming Frog SEO Spider and look at your address list. When going through each address, make sure it is:
- A reasonable length (around four to five words)
- Descriptive (no extraneous characters or punctuation in the parameters)
- Unique (without duplicates)
Correct page titles
After that, we will shift to the «Title of the page» tab. Once you click on the tab, you will see all the titles of your pages, as well as the pixel length of your title and the number of words. As you scroll through the title of each page, you’ll want to look for some things (you can use the filter at the top to easily find the titles of the pages that could be a problem).
There may be pages with the same page title. Make the correct changes so that each page title on your site is unique.
Are the keywords optimized in the title of the page? Make sure that each title of the page contains the most important keywords for your project.
Use the column “Number of characters” to make sure that each title has around 50-65 characters. Too long and it will be cut in the SERPs. Too short and it will not be attractive to your audience.
Analyze Meta Descriptions
In the same way, we need to verify the goals descriptions of the website and make sure they are fully optimized. Check and change goals descriptions that have duplicate content, which is not optimized by keyword or missing. Note that the character count must be around 160 characters. If you want, you can filter and search for specific problems (for example, metadata descriptions are missing), use the filter drop-down menu.
Preview and test fragments of SERP
Did you know that you can preview the appearance of your SERP fragments? Simply click on any link address you want to preview and then in the bottom window near the bottom of the screen, click on “SERP Fragment”. From here you can see the SERP Fragment, as well as a data table that includes the title and length of the description.
If you do not like what you see, you can reset the Meta title and description and preview how it will look. Once you’re satisfied, export your new Meta titles and descriptions in a spreadsheet for quick implementation.
Inspect the H1 titles
In this Screaming Frog guide, I teach you to review the H1. In the next tab labeled “H1” you can quickly see what H1 is missing in your site. Be sure to also check for duplicates and that all your H1 titles are specific and unique.
Optimizing your H1 within your site will greatly improve your keyword rankings. As you progress through your H1, you should ask yourself the following questions:
- Is there only one H1 on each page?
- Do all H1 describe the theme of each page?
- Do your H1 encourage a good user experience?
- Is each keyword H1 optimized?
If you have answered affirmatively to all those questions, you are ready to continue. If not, you have a little more work to do.
You may ask yourself: “Why do I have to review the images?” The answer is that the images can occupy a lot of bandwidth and reduce the load time of your pages. A rule of thumb is to keep your images less than 100kb. With Screaming Frog SEO Spider, you can check which images are slowing down your site.
Now you can review these images and see if you can compress them to a smaller size or replace them with a smaller image.
Review the directives
One way to verify technical problems is to go to the “Policies” tab. Click on the filter drop-down menu in the upper left corner and choose which option you would like to see.
From the filter menu, you can see the web addresses within your site that contain canonical, non-canonical, index, non-index, follow, follow and other properties.
This is an excellent way to verify how the robots.txt file is being implemented on your website. You may have to make adjustments to what is indexed, followed and used as canonical.
Analyze the crawl depth
In this Screaming Frog guide, it’s time to go back to the “Internal” tab (the first tab in Screaming Frog).
First, let’s take a look at the crawling depth of the website. The crawling depth of a URL is the number of clicks that are required to get from the home page to that specific page.
The Depth of tracking will vary and will depend on how large the website is. Keep in mind that the more clicks a crawl depth has, the harder it will be for traffic to visit your webpage and the harder it will be to crawl and rank the page.
If you have an important page on your website (such as a product) that has a crawl depth of 4, you may need to consider changing the organization and the hierarchy of the website so that that page has a crawl depth of 2.
To verify your crawl depth, click on the lower scroll bar in the main window and scroll to the right until you see the column titled “crawl depth”.
Then, you can drag that column closer to the URL so you can see the address and crawl depth data at the same time. You can click on the depth of the tracking column to sort by number.
When you look at your addresses, look for the important pages that have a crawl depth number too high and post it on your project or website audit.
Watch the response time
Next, let’s quickly see the response time of each page on your website.
This is a quick and easy way to identify pages that load too slowly. If you have a slow web page, it will take longer to crawl.
On the same “Internal” tab, use the lower scroll bar and scroll to the right until you see the “Response time” column.
Again, drag it closer to the URL column. Look at the response time of each page to see how fast they are loading.
Find pages with “thin content”
Google has declared that it wants to classify high-quality content. An in-depth page with a lot of descriptive and useful information performs substantially better than a page with only a few paragraphs vaguely.
With the “Internal” tab still open, move the lower scroll bar until you see the “Number of words” column. By clicking on the word count column, you can sort your pages so they have the least amount of words.
As you go through the column, ask yourself the following questions:
- Are there important pages that have less than 300 words?
- Is the content of each keyword on the page optimized?
- Are there pages that may have a too high word count? (An example could be a contact page that has 7000 words).
Generate a report of insecure content
The last feature I want to talk about in this Screaming Frog SEO Spider guide is the unsafe content report. With this report, you can verify how secure your website is against hackers and attacks.
Within Screaming Frog you will also want to make sure that each page does not include HTTP in the actual URL. Instead, make sure that each of the pages on your website is HTTPS, which means Secure Hypertext Protocol.
For not telling you that Google, recommends that websites have the SSL certificate.
Well, no. Screaming Frog has many more tools and capabilities that you can know. You can even connect the Screaming Frog with Google Analytics and the Google Search Console for an even more powerful experience.
I hope you liked the Screaming Frog guide, to check the status of your SEO On Page.
But, hopefully, this helps you have a better base and understanding of how Screaming Frog SEO Spider helps you in your SEO efforts.
When used correctly, Screaming Frog can become a powerful tool for deciphering a website.
What feature of Screaming Frog SEO Spider is your favorite? I would love to read your comments and impressions!