{"kind":"tools","id":"robots","slug":"robots","name":"Robots.txt Tester","description":"Read a site's robots.txt, see the rules that apply to each crawler, and test whether a specific URL would be allowed or blocked — the file where an accidental line can remove a whole site from search.","website":"https://openvibe.tools/tool/robots","sources":[{"url":"https://openvibe.tools/api/v1/tools","checked_at":"2026-11-13T05:45:05.242Z"}],"last_checked":"2026-11-13T05:45:05.242Z","group":"net"}