What RankedJobBot does, how it behaves, and how to stop it.
What RankedJobBot does, how it behaves, and how to stop it.
Ranked publishes a job board for software engineers and adjacent roles in Japan. RankedJobBot visits public careers pages and job boards, reads the postings, and republishes AI-written summaries of them so people can search across employers in one place. We link every listing back to your original posting, and we send applicants to you.
Requests to job-board APIs carry this User-Agent:
RankedJobBot/1.0 (+https://ranked.jp; jobs@ranked.jp)
Requests that fetch listing and job pages currently send an ordinary browser User-Agent instead. That is a leftover we are changing rather than a way of hiding: this page, the addresses below, and the blocking instructions all cover those requests too.
Blocking works either way. We matchRankedJobBot against your robots.txt on our side before falling back to User-agent: *, so the rule below is honoured whatever header a given request happens to carry.
It runs once a day, from AWS ap-northeast-1 (Tokyo). It fetches plain HTML first and only renders a page in a headless browser when the listing is JavaScript-rendered and the plain response comes back empty. It submits no forms, creates no accounts, and never attempts to reach anything behind a login.
Requests to the same host are queued and spaced at least 3 seconds apart, and no more than 3 of our workers run at once. So the most any single site sees from us is about one request per second, and in practice usually less. We treat that as a ceiling rather than a target.
If we are still causing you problems, tell us and we will slow down or stop — we would much rather hear from you than have you find out from a graph.
We read and obey robots.txt. To stop us specifically, without affecting search engines:
User-agent: RankedJobBotDisallow: /
We check it at most every few hours, so a rule takes effect on the next crawl at the latest. A Disallow under User-agent: *applies to us too.
Blocking the crawler stops us collecting anything new, but it does not by itself pull down what is already published. To have listings removed or corrected, use the form.
You can also just email admin@ranked.jp. We do not ask for a reason and we do not argue about it.
Anything about the crawler: jobs@ranked.jp. Anything else, including legal: admin@ranked.jp.