Posts Tagged ‘robots.txt’

Is Your Robots.txt File Hurting Traffic?

Earlier this year, I made the mistake of taking the advice of an seo guru, and modified my robots.txt file per his instructions. The issue he brought up was one of duplicate content, and how you don’t want the spiders coming across duplicate content through your archives, tag pages, author pages, etc. So to deal […]