User-agent: gsa-crawler Allow: /Portals/2/Departments/ Allow: /Portals/3/library_files/ Allow: /Portals/4/SiteContent/ Allow: /Portals/5/SiteContent/ Allow: /Portals/6/SiteContent/ Crawl-delay: 2 User-agent: sogou spider Disallow: / User-agent: Baiduspider User-agent: Baiduspider-video User-agent: Baiduspider-image Disallow: / User-agent: Yandex Disallow: / User-agent: MJ12bot Disallow: / User-agent: * Disallow: /*/ctl/ Disallow: /admin/ Disallow: /App_Browsers/ Disallow: /App_Code/ Disallow: /App_Data/ Disallow: /App_GlobalResources/ Disallow: /bin/ Disallow: /Components/ Disallow: /Config/ Disallow: /contest/ Disallow: /controls/ Disallow: /DesktopModules/ Disallow: /Documentation/ Disallow: /HttpModules/ Disallow: /images/ Disallow: /Install/ Disallow: /js/ Disallow: /Portals/ Disallow: /Providers/ Disallow: /Resources/ContentRotator/ Disallow: /Resources/ControlPanel/ Disallow: /Resources/Dashboard/ Disallow: /Resources/FeedBrowser/ Disallow: /Resources/OpenForceAd/ Disallow: /Resources/Search/ Disallow: /Resources/Shared/ Disallow: /Resources/SkinWidgets/ Disallow: /Resources/TabStrip/ Disallow: /Resources/Widgets/ Disallow: /User-Profile/ Disallow: /Default.aspx?tabid=1177&EntryID=974 Disallow: /Departments/Police/PoliceBlog/tabid/1177/EntryId/974/ Disallow: /Departments/Police/PoliceBlog/tabid/1177/BlogDate/2009-12-31/DateType/month/ Disallow: /Departments/Police/PoliceBlog/tabid/1177/EntryId/974/Default.aspx Disallow: /Departments/Police/PoliceBlog/tabid/1177/BlogDate/2009-12-31/DateType/month/Default.aspx Disallow: /mctl/EventDetails/ Disallow: /BlogDate/ Disallow: /Employee-Login/ Disallow: /Employee-Login Disallow: /openenrollment/ Crawl-delay: 2