I had the below issue in our SharePoint 2016 farm , for some reason it was working fine until a patch was applied and the server was rebooted . The patch had nothing to do with it though
Issue
1)Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled.
2)Crawling of this item failed, HTTP 504: Gateway Timeout. Try accessing the item using a browser on the crawl machine. If URL is accessible through the browser, it is possible that the crawl targets for that host are not configured correctly. Please contact the Host Administrator for assistance
Error refers to disabling the loop back but this has already been done , but we can remove the registry entry and try again though , also we could try applying the latest patch . But all this would require change in registry and reboot of server .
First error message was seen for the HTTP site and the second error message was seen for the HTTPS site
1)We did try to add the crawl rule and assign certificates to crawl the HTTPS URL , but nothing resolved the issue
2)But as a standard procedure , you will need to follow the steps in below link to make sure you have the right configuration for search
https://social.technet.microsoft.com/wiki/contents/articles/25863.access-is-denied-verify-that-either-the-default-content-access-account-has-access-to-this-repository-or-add-a-crawl-rule-to-crawl-this-repository.aspx
3)I had all the configuration set correctly , but it still would not work
Finally by clearing the proxy did the magic
below are the script to be run as admin on the SharePoint server which is defined as crawler
Issue
1)Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled.
2)Crawling of this item failed, HTTP 504: Gateway Timeout. Try accessing the item using a browser on the crawl machine. If URL is accessible through the browser, it is possible that the crawl targets for that host are not configured correctly. Please contact the Host Administrator for assistance
Error refers to disabling the loop back but this has already been done , but we can remove the registry entry and try again though , also we could try applying the latest patch . But all this would require change in registry and reboot of server .
First error message was seen for the HTTP site and the second error message was seen for the HTTPS site
1)We did try to add the crawl rule and assign certificates to crawl the HTTPS URL , but nothing resolved the issue
2)But as a standard procedure , you will need to follow the steps in below link to make sure you have the right configuration for search
https://social.technet.microsoft.com/wiki/contents/articles/25863.access-is-denied-verify-that-either-the-default-content-access-account-has-access-to-this-repository-or-add-a-crawl-rule-to-crawl-this-repository.aspx
3)I had all the configuration set correctly , but it still would not work
Finally by clearing the proxy did the magic
below are the script to be run as admin on the SharePoint server which is defined as crawler
netsh winhttp show proxy
netsh winhttp reset proxy
Net stop Osearch16
Net start Osearch16
netsh winhttp reset proxy
Net stop Osearch16
Net start Osearch16
