Added allowlist.txt and robots.txt
This commit is contained in:
parent
eebee18ccc
commit
61921ae7fc
3 changed files with 73 additions and 2 deletions
|
@ -1,2 +1 @@
|
||||||
# BarkPackList
|
# Bark Pack Lists
|
||||||
|
|
44
allowlist.txt
Normal file
44
allowlist.txt
Normal file
|
@ -0,0 +1,44 @@
|
||||||
|
# Bark Pack
|
||||||
|
tonybark.com
|
||||||
|
tonybark.art
|
||||||
|
tpaw.social
|
||||||
|
furryblue.com
|
||||||
|
|
||||||
|
# Stefen Auris
|
||||||
|
stefenauris.com
|
||||||
|
theheart.land
|
||||||
|
|
||||||
|
# Fediverse
|
||||||
|
## Mastodon
|
||||||
|
meow.social
|
||||||
|
furry.engineer
|
||||||
|
framapiaf.org
|
||||||
|
dook.social
|
||||||
|
mastodon.social
|
||||||
|
pixelfed.social
|
||||||
|
mas.to
|
||||||
|
bark.lgbt
|
||||||
|
twit.social
|
||||||
|
|
||||||
|
## PeerTube
|
||||||
|
fair.tube
|
||||||
|
fedi.video
|
||||||
|
cpy.re
|
||||||
|
flooftube.net
|
||||||
|
|
||||||
|
# Art Communities
|
||||||
|
sheezy.art
|
||||||
|
furaffinity.net
|
||||||
|
sofurry.com
|
||||||
|
inkbunny.net
|
||||||
|
e621.net
|
||||||
|
e6ai.net
|
||||||
|
|
||||||
|
# Non Profits
|
||||||
|
neocities.org
|
||||||
|
wikipedia.org
|
||||||
|
torproject.org
|
||||||
|
archive.org
|
||||||
|
|
||||||
|
# Political Sites
|
||||||
|
nationalzero.com
|
28
robots.txt
Normal file
28
robots.txt
Normal file
|
@ -0,0 +1,28 @@
|
||||||
|
User-Agent: GPTBot
|
||||||
|
User-Agent: ChatGPT-User
|
||||||
|
User-Agent: ClaudeBot
|
||||||
|
User-Agent: anthropic-ai
|
||||||
|
User-Agent: CCBot
|
||||||
|
User-agent: Bytespider
|
||||||
|
User-Agent: FacebookBot
|
||||||
|
User-Agent: Twitterbot
|
||||||
|
User-Agent: PerplexityBot
|
||||||
|
User-Agent: OmigiliBot
|
||||||
|
User-Agent: AISearchBot
|
||||||
|
User-Agent: mlbot
|
||||||
|
User-Agent: PetalBot
|
||||||
|
User-Agent: Amazonbot
|
||||||
|
User-Agent: DuckAssistBot
|
||||||
|
User-agent: Diffbot
|
||||||
|
User-Agent: OAI-SearchBot
|
||||||
|
User-Agent: OAI-SearchBot
|
||||||
|
User-Agent: omgili
|
||||||
|
User-Agent: omgilibot
|
||||||
|
User-Agent: PerplexityBot
|
||||||
|
User-Agent: Webzio-Extended
|
||||||
|
User-Agent: Meta-ExternalAgent
|
||||||
|
User-Agent: Meta-ExternalFetcher
|
||||||
|
User-Agent: cohere-ai
|
||||||
|
User-agent: Google-Extended
|
||||||
|
User-agent: PerplexityBot
|
||||||
|
Disallow: /
|
Loading…
Add table
Reference in a new issue