Disable search indexing on interdiffs and some datagrids.
Review Request #14363 — Created March 5, 2025 and submitted
There are a number of pages in Review Board that aren't worth having
search engines crawl. While review requests and diffs can be useful,
interdiffs are not. While the All Review Requests page, Users page, and
Groups page are useful, the lists of review requests from a user or
group are not, since they're in All Review Requests.The Djblets datagrid code has been updated to enable greater control
over search indexing of datagrids, enabling us to disable some search
indexing.We now include
rel="nofollow noindex"
in links to interdiffs for both
diffs and file attachments, and we include a<meta>
tag for disabling
indexing on the diff viewer when showing interdiffs and on the User's
and Group's review requests datagrid pages.This of course is all optional for search engines, and the most
difficult ones (particularly AI training bots) won't respect most or any
of this, but it's at least helpful for the bots who are well-behaved.
Tested the affected pages and links and verified that they contained
the correctnofollow
andnoindex
hints.
Summary | ID |
---|---|
41f42b56bcfca28f894a895e9e6e074c27d6ae50 |