NAME

    robotex - automated bot protection and robots.txt generator

SYNOPSIS

    npx robotex [COMMAND] [OPTIONS]

COMMANDS

    (no command) : Generates the robots.txt file based on blocklists.

    init : Generates a starter robotex.json config file.

    --dry-run : Previews your robots.txt in the terminal without saving.

MIDDLEWARE USAGE (Next.js / Node)

To actively block bots that ignore robots.txt files, use the exported function:

import { isBot } from 'robotex';
import { NextResponse } from 'next/server';

export async function middleware(request) {
    const userAgent = request.headers.get('user-agent');
    if (await isBot(userAgent)) {
        return new NextResponse('Access Denied', { status: 403 });
    }
    return NextResponse.next();
}

[ RETURN TO MAIN MAINFRAME ]