Little Things
Published by Elana December 17th, 2006 in Random Ramblings, Tech.I went and removed the wiki from Google and other search engines. I forced Google to do a crawl across the site with their removal tool. As long as a search engine behaves well and reads my robots.txt files, we should be fine. I also spent the time to import the robots.txt I had created for clampcampus.com so I could easily eliminate some known bad crawlers. Of course, this doesn’t help if the crawler is so bad it ignores the robots.txt, but it’s the best I can do.
Once the wiki is locked down, the robots won’t be able to get to it to index the content anyhow. The wiki is a full robot exclude while the main site is not, although I don’t let the robots index images. I also removed the dead links to the old wiki URLs at elanaandcarl.com. This will get rid of some of the wiki content previously indexed as well.
I hope people like the design of the elanaandcarl.com now. I got the design through the Open Source Web Design Project. I highly recommend fellow web developers who can’t design their way out of a paper bag to utilize that site and those of you who can design, please contribute! ![]()

0 Responses to “Little Things”
Please Wait
Leave a Reply