User-agent: Scrapy Allow: / User-agent: * Disallow: /wp-admin/ Sitemap: http://www.brisanet.pt/i18n-index-sitemap.xml Sitemap: http://www.brisanet.pt/en/i18n-index-sitemap.xml