# MaltaMotors — robots.txt
#
# Public site: allow everything except the API and the health endpoint.
# Sitemap pointer at the bottom is the standard way to advertise
# the sitemap location to crawlers that don't yet know the domain.

User-agent: *
Allow: /
Disallow: /api/
Disallow: /status

# Crawler hygiene: don't index error pages or the lead-form overlay
# state. Search crawlers can spend their budget on the actual VDPs.
Disallow: /*?error=
Disallow: /*?lead=

# NOTE — auth.html, sell.html, my-leads.html, my-searches.html and
# admin.html are deliberately NOT listed above, though they used to be.
# Every one of them carries its own <meta name="robots" content="noindex">,
# and Disallow and noindex work against each other: a crawler that obeys
# the Disallow never fetches the page, so it never reads the noindex. The
# URL then stays eligible for indexing from any external link, listed
# without a snippet — the opposite of what the Disallow was for. Letting
# them be fetched is what makes the noindex effective.
#
# /api/ stays disallowed because a JSON response cannot carry a meta tag;
# the vhost sends X-Robots-Tag on those paths instead. /status is a health
# endpoint with no content worth a crawl.

Sitemap: https://maltamotors.com/sitemap.xml