Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Screaming Frog returning both HTTP and HTTPS results...
-
Hi,
About 10 months I switched from HTTP to HTTPS. I then switched back (long story).
I noticed that Screaming Frog is picking up the HTTP and HTTPS version of the site. Maybe this doesn't matter, but I'd like to know why SF is doing that. The URL is: www.aerlawgroup.com
Any feedback, including how to remove the HTTPS version, is greatly appreciated.
Thanks.
-
No problem! I'm not too familiar with secure migrations in WP, but I bet there is some kind of plugin for that.
-
Hi,
Thank you! Fixing that link prevents SF from finding the HTTPS pages.
I use WP for this site. Is there a quick way to redirect HTTPS to HTTP? As opposed to forwarding page by page? Ideally, there is a way in WP (either in WP directly or via some plugin) that I can tell WP to automatically 301 redirect all HTTPS to HTTP. I use WPEngine for my hosting, if that's at all relevant.
Thank you again for your excellent answer.
-
Hi,
Found the source of your issue: this URL (https://www.aerlawgroup.com/sex-crimes.html) has a link pointing to it from a HTTP URL (http://www.aerlawgroup.com/sex-crimes/new-allegations-emerge-in-sexual-misconduct-case.html), which is probably how SF found all the other secure URLs.
All of your HTTPS page return 200. If you want to go back to non-secure, those secure URLs will need to be redirected back to HTTP, the same as you probably did when you went to secure.
Also, looks like you've still got a couple HTTPS pages indexed: https://www.google.com/search?q=site%3Aaerlawgroup.com+inurl%3Ahttps&oq=site%3Aaerlawgroup.com+inurl%3Ahttps&gs_l=serp.3...5577.15088.0.15655.24.24.0.0.0.0.150.1864.19j5.24.0....0...1.1.64.serp..3.0.0.bf5ZiosfHMI
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help with force redirect HTTP to HTTPS
Hi, I'm unsure of where I should be putting the following code for one of my Wordpress websites so that they redirect all HTTP requests to HTTPS. RewriteEngine On RewriteCond %{HTTPS} !=on RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301] This is my current htaccess file: *missing
Intermediate & Advanced SEO | | Easigrass0 -
How long after https migration that google shows in search console new sitemap being indexed?
We migrated 4 days ago to https and followed best practices..
Intermediate & Advanced SEO | | lcourse
In search console now still 80% of our sitemaps appear as "pending" and among those sitemaps that were processed only less than 1% of submitted pages appear as indexed? Is this normal ?
How long does it take for google to index pages from sitemap?
Before https migration nearly all our pages were indexed and I see in the crawler stats that google has crawled a number of pages each day after migration that corresponds to number of submitted pages in sitemap. Sitemap and crawler stats show no errors.0 -
Strange 404s in Screaming Frog
I just ran a website (Drupal) through screaming frog and the only 404s I found related to web pages which were the same as URLs already used on the website plus the company phone number so... www.company.com/[their phone number] - www.company.com/services[their phone number] - any ideas what might be causing this problem?
Intermediate & Advanced SEO | | McTaggart0 -
Why would our server return a 301 status code when Googlebot visits from one IP, but a 200 from a different IP?
I have begun a daily process of analyzing a site's Web server log files and have noticed something that seems odd. There are several IP addresses from which Googlebot crawls that our server returns a 301 status code for every request, consistently, day after day. In nearly all cases, these are not URLs that should 301. When Googlebot visits from other IP addresses, the exact same pages are returned with a 200 status code. Is this normal? If so, why? If not, why not? I am concerned that our server returning an inaccurate status code is interfering with the site being effectively crawled as quickly and as often as it might be if this weren't happening. Thanks guys!
Intermediate & Advanced SEO | | danatanseo0 -
Multiple 301 redirects for a HTTPS URL. Good or bad?
I'm working on an ecommerce website that has a few snags and issues with it's coding. They're using https, and when you access the website through domain.com, theres a 301 redirect to http://www.domain.com and then this, in turn, redirected to https://www.domain.com. Would this have a deterimental effect or is that considered the best way to do it. Have the website redirect to http and then all http access is redirected to the https URL? Thanks
Intermediate & Advanced SEO | | jasondexter0 -
My website (non-adult) is not appearing in Google search results when i have safe search settings on. How can i fix this?
Hi, I have this issue where my website does not appear in Google search results when i have the safe search settings on. If i turn the safe search settings off, my site appears no problem. I'm guessing Google is categorizing my website as adult, which it definitely is not. Has anyone had this issue before? Or does anyone know how to resolve this issue? Any help would be much appreciated. Thanks
Intermediate & Advanced SEO | | CupidTeam0 -
HTTPS moz.org untrusted - invalid cert
https://www.moz.com/ has an invalid cert guys This Connection is Untrusted You have asked Firefox to connect
Intermediate & Advanced SEO | | irvingw
securely to www.moz.com, but we can't confirm that your connection is secure.
Normally, when you try to connect securely,
sites will present trusted identification to prove that you are
going to the right place. However, this site's identity can't be verified. What Should I Do? If you usually connect to
this site without problems, this error could mean that someone is
trying to impersonate the site, and you shouldn't continue. www.moz.com uses an invalid security certificate. The certificate is only valid for moz.com (Error code: ssl_error_bad_cert_domain) If you understand what's going on, you
can tell Firefox to start trusting this site's identification.
Even if you trust the site, this error could mean that someone is
tampering with your connection.
Don't add an exception unless
you know there's a good reason why this site doesn't use trusted identification.1 -
How long does google take to show the results in SERP once the pages are indexed ?
Hi...I am a newbie & trying to optimize the website www.peprismine.com. I have 3 questions - A little background about this : Initially, close to 150 pages were indexed by google. However, we decided to remove close to 100 URLs (as they were quite similar). After the changes, we submitted the NEW sitemap (with close to 50 pages) & google has indexed those URLs in sitemap. 1. My pages were indexed by google few days back. How long does google take to display the URL in SERP once the pages get indexed ? 2. Does google give more preference to websites with more number of pages than those with lesser number of pages to display results in SERP (I have just 50 pages). Does the NUMBER of pages really matter ? 3. Does removal / change of URLs have any negative effect on ranking ? (Many of these URLs were not shown on the 1st page) An answer from SEO experts will be highly appreciated. Thnx !
Intermediate & Advanced SEO | | PepMozBot0