GitHub - spatie/robots-txt: Determine if a page may be crawled from robots. txt, robots meta tags and robot headers
![Volacci on Twitter: "Are you a #marketer that wants to easily change or update your #Drupal website's robots.txt file? Here's a handy #module to solve that problem. https://t.co/SlpE1WUHGe https://t.co/Wt3EGSkC3k" / Twitter Volacci on Twitter: "Are you a #marketer that wants to easily change or update your #Drupal website's robots.txt file? Here's a handy #module to solve that problem. https://t.co/SlpE1WUHGe https://t.co/Wt3EGSkC3k" / Twitter](https://pbs.twimg.com/media/E5Eymw7WUAIJPcq.jpg)
Volacci on Twitter: "Are you a #marketer that wants to easily change or update your #Drupal website's robots.txt file? Here's a handy #module to solve that problem. https://t.co/SlpE1WUHGe https://t.co/Wt3EGSkC3k" / Twitter
![I am facing this issue Host had problems last week, robots.txt, robots.txt not available - Google Search Central Community I am facing this issue Host had problems last week, robots.txt, robots.txt not available - Google Search Central Community](https://storage.googleapis.com/support-forums-api/attachment/thread-112659113-4232086646474327055.jpg)