Add initial readme
This commit is contained in:
42
readme.md
Normal file
42
readme.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Django AI Blocker
|
||||
|
||||
This is a very small Django app that's purpose is to block requests to the site where the User Agent header contains a
|
||||
string that is associated with known AI Scrapers and indexers that will scrape the site for the purpose of training
|
||||
LLMs. If you don't want the content of your website to be used to train LLMs, then this app will help you block AI
|
||||
scraping bots.
|
||||
|
||||
This app isn't foolproof as at some point in the future, the AI scraping bots may change their User Agent headers. That
|
||||
said, this app is intended to be easily configurable so that if you discover a user agent string that is not currently
|
||||
in the list, you can easily add it.
|
||||
|
||||
## How does it work?
|
||||
|
||||
This app adds a simple middleware file that checks the user agent header of every request to the site. If the header
|
||||
contains a string known to be associated with an AI scraper, then the middleware will immediately return an HTTP 403
|
||||
status code.
|
||||
|
||||
## Installation
|
||||
|
||||
TBD
|
||||
|
||||
## Usage
|
||||
|
||||
```py
|
||||
# settings.py
|
||||
|
||||
INSTALLED_APPS = [
|
||||
# Other installed apps...
|
||||
'ai_blocker',
|
||||
]
|
||||
|
||||
MIDDLEWARE = [
|
||||
# Other middleware...
|
||||
'ai_blocker.middleware.AIBlockerMiddleware',
|
||||
]
|
||||
```
|
||||
|
||||
The precise location in the middleware chain where you add this middleware is up to you.
|
||||
|
||||
## Configuration
|
||||
|
||||
TBD
|
||||
Reference in New Issue
Block a user