Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to Localise per Region (Europe, America, APAC, EMEI) and not per country as best SEO practise?
Hi SEO expertises! I am currently working with a client that initially have an English website targeting UK users but want to expand their market into four new regions (Europe, America, APAC and EMEI) keeping English as a main language. I would like to request your help here as I told the client ISO location and hreflang it will be just possible per language and they must need to localise each English region with local keywords, however I would like to double check if it will be any way (Sitemap, Hreflang) we can tell Google we are targeting per region and not per country? Thanks a lot!
International SEO | | Atalig20 -
Worldwide and Europe hreflang implementation.
Hi Moz ! We're having quite a discussion here and I'd like to have some inputs. Let me explain the situation and what we plan to do so far. One of our client has two separate markets : World and Europe. Both pages versions will be mostly the same, except for the fact that they will have their own products. So basically, we'd want to show only the European EN version to Europe and the standard EN version to the rest of the world, same goes for FR and ES. As far as IT, DE, CS and SK, they will only be present within the european version. Since we cannot target all Europe with a single hreflang tag, we might have to do it for every single european countries. Regarding this subject, SMX Munich recently had quite an interesting session about this topic with a confirmation coming from John Mueller saying that we can target a single URL more than once with different hreflang tags. You can read more here : http://www.rebelytics.com/multiple-hreflang-tags-one-url/ So having all this in mind, here's the implementation we plan to do : www.example.com/en/ Self canonical www.example.com/fr/ - hreflang = fr www.example.com/es/ - hreflang = es www.example.eu/it/ - hreflang = it www.example.eu/de/ - hreflang = de www.example.eu/cs/ - hreflang = cs www.example.eu/sk/ - hreflang = sk www.example.eu/fr/ - hreflang = be-fr www.example.eu/fr/ - hreflang = ch-fr www.example.eu/fr/ - hreflang = cz-fr www.example.eu/fr/ - hreflang = de-fr www.example.eu/fr/ - hreflang = es-fr www.example.eu/fr/ - hreflang = fr-fr www.example.eu/fr/ - hreflang = uk-fr www.example.eu/fr/ - hreflang = gr-fr www.example.eu/fr/ - hreflang = hr-fr etc… . This will be done for all european countries (FR, EN and ES). www.example.com/en/ - x-default Let me know what you guys think. Thanks!
International SEO | | Netleaf.ca0 -
Can you target the same site with multiple country HREFlang entries?
Hi, I have a question regarding the country targeting aspect of HREFLANG. Can the same site be targeted with multiple country HREFlang entries? Example: A global company has an English South African site (geotargeted in webmaster tools to South Africa), with a hreflang entry targeted to "en-za", to signify English language and South Africa as the country. Could you add entries to the same site to target other English speaking South African countries? Entries would look something like this: (cd = Congo, a completely random example) etc... Since you can only geo-target a site to one country in WMT would this be a viable option? Thanks in advance for any help! Vince
International SEO | | SimonByrneIFS0 -
How To Rank A UK Website On Google.com (US)
Hi, I've done some research on this but couldn't find any definitive answer I can trust! We have a client who resides in the UK. They have '.com' domain, hosted on a UK server, using UK spelling. Their business objective for this year is to expand in the USA, including the opening of a warehouse over there. They are wanting us to rank their website on both Google.co.uk and Google.com (North America); besides changing the geolocation settings in GWT's, and building links from .com websites is there anything else we can do to increase their visibility on Google.com? Many thanks in advance, appreciated!
International SEO | | Webpresence
Lee.0 -
URL Structure - Homepage, Country and State Pages
Hello, I am creating a website (or websites if best format) that will have state-specific boating license courses for every state in the US, Canada and Australia. I would like the content to be available on the website in English, French and Spanish. I want to be the global leader in providing boat test courses. For the (1) homepage, (2) country pages, and (3) state pages, what is best SEO format I should use for:
International SEO | | Monologix
(a) URL structure
(b) "href lang" code
(c) rel canonical code
(d) will meta content with non-English pages need to also be in the non-English language of that page? Also, what server company do you recommend I host my website with? I am a non-programmer and learning SEO, so any and all help will be greatly appreciated! Thank you very much in advance!!!0 -
Does Google's algorithm work the same in every country?
I can't help but feel this is a silly question! but does Google algorithm work exactly the same throughout all countries? I run a few sites in the UK and a couple in Spain but can't help but feel that my Spanish sites are harder to rank for. The sites that rank the best are business directories in Spain... whereas here in the UK you'd be lucky to find one on page one..
International SEO | | david.smith.segarra0 -
Sub-domains or sub-directories for country-specific versions of the site?
What approach do you think would be better from an SEO perspective when creating country-targeted versions for an eCommerce site (all in the same language with slight regional changes) - sub-domains or sub-directories? Is any of the approaches more cost effective, web development-wise? I know this topic's been under much debate and I would really like to hear your opinion. Many thanks!
International SEO | | ramarketing0 -
SEO Company in Asia
Hi All. I have a client looking to expand their industrial services to southeast Asia (Vietnam and Indonesia specifically right now). Does anyone know of an SEO/Online Marketing firm local to that region that may be able to help them network with businesses and industries there? I've gone through the SEOmoz member database and reached out to a couple people with agencies in that area but never heard back from them. I personally thought a local firm would be more beneficial to the client but I'm also open to suggestions on ways that we might be able to help them market their services online from the US. Thanks so much! Megan
International SEO | | ILM_Marketing0