standardebooks/web
1
0
Fork 0
mirror of https://github.com/standardebooks/web.git synced 2025-07-05 06:10:36 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update robots.txt to disallow downloads by more SEO bots

Browse source
This commit is contained in:
Alex Cabal 2025-05-31 11:58:45 -05:00
parent 865b7db10d
commit ed3c4aec89
1 changed files with 3 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
www/robots.txt
View file

@ -9,4 +9,7 @@ User-agent: SemrushBot
User-agent: DotBot User-agent: DotBot
User-agent: AhrefsBot User-agent: AhrefsBot
User-agent: SEOkicks User-agent: SEOkicks
User-agent: DataForSeoBot
User-agent: proximic
Disallow: /ebooks/*/downloads/* Disallow: /ebooks/*/downloads/*