User-agent: * Crawl-delay: 5 Disallow: /wp-admin/ Sitemap: http://saudishift.com/sitemap_index.xml User-agent: Twitterbot Crawl-delay: 1 Allow: /