MatcherNews-Reader

What it does

It only reads pages linked from the RSS/Atom feeds our users subscribe to: one article per request, when an agent asks for that specific article. It doesn't crawl, follow links or read pages nobody asked for.

It extracts the article's text and hands it only to the agent that asked, in that one response, with the article's link and a note to attribute it to the site. Matcher News doesn't publish it, show it on any page or store it: neither the page nor the extracted text is kept anywhere.

The rules it follows

• It honors robots.txt (RFC 9309) for the MatcherNews-Reader token, or the * group when there is none. If robots.txt can't be read because of a server error, it reads nothing from that site.
• It doesn't read a page whose X-Robots-Tag header or robots meta tag says noai, none, noindex or noarchive, unless the directive is addressed only to another crawler (googlebot: noindex). When it can't tell, it honors the directive.
• It checks robots.txt and our block list again on every redirect.
• It is rate limited per user, per host and in total: at most 30 page requests per minute for each host that feeds link to (a page reached through a redirect counts towards the host of the link). robots.txt is normally read once a day per site.

How to block it

Add this to your site's robots.txt:

User-agent: MatcherNews-Reader
Disallow: /

Or send X-Robots-Tag: noai (or <meta name="robots" content="noai">) on the pages you want to keep out. Changes to robots.txt apply within 24 hours.

Contact

To have your whole site excluded, or for any question about MatcherNews-Reader, write to hello@matcher.news from an address at your domain. We add your site to our block list.