Do search engines crawl links with query strings?

Do search engines crawl links with query strings? - .htaccess files are extremely useful in many cases for users who either do not have root permissions or for users who simply aren't comfortable in making changes in their web server's configuration file. Trying to debug .htaccess not working isn't always the easiest thing to do, however, hopefully by checking the discuss below mentioned about seo, htaccess, web-crawlers, search-engine-indexing, .htaccess common problems as well as the troubleshooting tips, you'll have a better grasp on what you may have to modify to get your .htaccess file running smoothly.Problem :

If I were to have a url like https://example.com/books, that contained several links to urls such as:

https://example.com/books?id=foo

https://example.com/books?id=bar

https://example.com/books?id=baz

Would search engines crawl and index the content returned by these URLs? If not, what would happen if I were to use a rewrite rule in my .htaccess file to rewrite pretty urls to ugly ones with query strings:

https://example.com/books/foo -> https://example.com/books?id=foo

https://example.com/books/bar -> https://example.com/books?id=bar

https://example.com/books/baz -> https://example.com/books?id=baz

Would there be any difference/preference? Or would a search engine crawl both?

Solution :

Yes. Query strings are part of the URL and as such are viewed as directing the robots to different pages.

There may be a preference to using a path because then Google can test the parent. So if you have:

https://example.com/books/baz

Google will eventually test

https://example.com/books

on its own (it could be that they don't do it anymore though, but I'm sure I read that somewhere on a page owned by Google.)

One thing for sure, if you allow both, make sure to add a canonical meta tag with your preferred URL otherwise you could get penalized because of the duplication.

Additionally, if you would like to do some further testing, give the htaccess tester tool a try. It allows you to specify a certain URL as well as the rules you would like to include and then shows which rules were tested, which ones met the criteria, and which ones were executed.