sitemap.xml for auth sites

The reason that I ask is that I found one page out there in the wild that were talking about creating a user account for crawlers with read-only permissions.

This feels a little like a social engineering exploit. Actually, it feels a lot like one.

Well, it was in a foreign language to me and I just read i briefly, so I might have misunderstood the concept.

---

You can't make an user/password, that would be a security hole. And they don't even pick up on that.
What you need to to is look for a googlebot user agent, and verify it's origin. And let that bot inside without username/password.

https://support.google.com/webmasters/an...0553?hl=en

So, it will be semi-public.

If Google starts crawling those pages (you got Google Search Console right?) you should see those urls failing. You should return a correct header for Google and other bots if they find a page that require login. Like 403. If you return 200, they will think it's the correct content showing up.

OK, this is way ahead of what I have been doing before. I will try to learn more about the concept before I give it a try. Thanks for your effort!