How to Stop Search Engines from Crawling a WordPress Site

Recently, considered one of our customers requested us how they are able to forestall search engines like google from crawling and indexing their WordPress website? There are many situations while you would wish to forestall search engines like google from crawling your site or record it in seek effects. In this text, we will be able to display you the way to forestall search engines like google from crawling a WordPress website.

Why and Who Would Want to Stop Search Engines

For maximum web pages, search engines like google are the most important supply of site visitors. You would possibly ask, why would someone need to block search engines like google?

When beginning out, a lot of other folks don’t understand how to create a native building surroundings or a staging website. If you’re growing your site live to tell the tale a publicly obtainable area identify, you then most likely don’t need Google to index your underneath building or repairs mode web page.

There also are many of us who use WordPress to create non-public blogs, they usually don’t need the ones listed in seek effects as a result of they’re non-public.

Also some other folks use WordPress for mission control or intranet, and also you wouldn’t need your interior paperwork being publicly obtainable.

In the entire above eventualities, you almost certainly don’t need search engines like google to index your site.

A not unusual false impression is if I would not have hyperlinks pointing to my area, then search engines like google will almost certainly by no means to find my site. This isn’t totally true.

There are some ways search engines like google can to find a site related somewhere else. For instance:

  1. Your area identify will have been prior to now owned by means of any individual else, they usually nonetheless have some hyperlinks pointing to your site now.
  2. Some area seek website’s effects may get listed together with your hyperlink on them.
  3. There are actually hundreds of pages with simply checklist of domains, your website can seem on a kind of.

There are a lot of items taking place on the net and maximum of them aren’t underneath your regulate. However, your site continues to be underneath your regulate, and you’ll instruct search engines like google to now not index or practice your site.

Video Tutorial

If you don’t just like the video or want extra directions, then proceed studying.

Blocking Search Engines from Crawling and Indexing Your WordPress Site

WordPress comes with a integrated characteristic that permits you to instruct search engines like google now not to index your website. All you wish to have to do is discuss with Settings » Reading and take a look at the field subsequent to Search Engine Visibility possibility.

Search engine visibility setting in WordPress

When this field is checked, WordPress provides this line to your site’s header:

<meta identify="robots" content material="noindex,practice" />

WordPress additionally modifies your website’s robots.txt document and upload those strains to it:


User-agent: *
Disallow: /

These strains ask robots (internet crawlers) now not to index your pages. However, it’s completely up to search engines like google to settle for this request or forget about it. Even despite the fact that maximum search engines like google appreciate this, some web page or random symbol from your website would possibly get listed.

How to Make Sure Your Site Doesn’t Appear in Search Results?

The default WordPress visibility settings instruct search engines like google now not to index your site. However, search engines like google would possibly nonetheless finally end up crawling and indexing a web page, document, or symbol from your site.

The handiest approach to to make certain that search engines like google don’t index or move slowly your site in any respect is by means of password protective your whole WordPress website at the server stage.

This method when someone tries to get admission to your site, they’re requested to supply a username and password even prior to they succeed in WordPress. This comprises search engines like google as neatly. Upon login failure, they’re proven 401 error and the bots flip away. Here is how to password give protection to a whole WordPress website.

Method 1: Password Protecting an Entire Site the usage of cPanel

If your WordPress webhosting supplier provides cPanel get admission to to organize your webhosting account, then you’ll give protection to your whole website the usage of cPanel. All you’ve gotten to do is log in to your cPanel dashboard after which click on on ‘Password Protect Directories’ icon underneath the ‘Security’ segment.

Password protect directories

Next, you’re going to want to make a selection the folder the place you’ve gotten put in WordPress. Usually, it’s the public_html folder.

Select the root folder

If you’ve gotten a couple of WordPress websites put in underneath public_html listing, then you wish to have to click on at the folder icon to browse and make a selection the folder for the site you need to password give protection to.

On the following display, you wish to have to input a identify for the secure listing and click on at the ‘Save’ button.

Enter a name for protected directory

Your knowledge shall be stored and cPanel will reload the web page. After that, you wish to have to upload a certified person by means of coming into a username and password.

Add authorized user

That’s all, you’ve gotten effectively added password coverage to your WordPress website.

Now, each time a person or seek engine visits your site they’ll be induced to input the username and password you created previous to view the website.

Password protected website

Method 2: Password Protecting WordPress with a Plugin

If you might be the usage of a controlled WordPress webhosting answer, on occasion you don’t have get admission to to cPanel.

In that case, you’ve gotten the choice to use other WordPress plugins to password give protection to your site. Below are the 2 most well liked answers:

  1. SeedProd – it’s the number one coming quickly and upkeep mode plugin for WordPress, used on over 800,000+ web pages. It comes with entire get admission to regulate and permissions options that you’ll use to cover your site from everybody together with search engines like google. We have a step-by-step how-to information for SeedProd.
  2. Password Protected – it’s a quite simple approach to password give protection to your WordPress website with a unmarried password (no person introduction wanted). See our step-by-step information on how to password give protection to a WordPress website.

We hope this text helped you forestall search engines like google from crawling or indexing your WordPress website. You may additionally need to see our final step-by-step WordPress safety information for newbies.

If you loved this text, then please subscribe to our YouTube Channel for WordPress video tutorials. You too can to find us on Twitter and Facebook.

Recent Articles

How to Automatically Skip Netflix Intros in your Chrome Browser

It is the winter season and we know how we wish to spend it; wrapped around in...

4 Amazon Prime Benefits that You Never Knew Existed

Amazon Prime can do a lot more than giving you access to Prime video streaming and giving...

5 Best Android apps to Sideload on Your TV

Smart TVs are the newest necessity for every tech friendly household. With the popularity of OTT platform...

Unique Team Video Chat Apps that will Make Work From Home Fun

How much time have you spent on video calls over the past few months? For every student and working professional, the answer...

Upgrade Your iCloud Storage Plan on all Your Devices

No matter how much money you spend on your device, you never seem to have enough space. The problem is particularly prevalent...

Related Stories

Leave A Reply

Please enter your comment!
Please enter your name here