sweh
Joined: 13 Apr 2004
Posts: 205
|
| Posted: Wed Jun 09, 2004 4:08 pm Post subject: Let that be a lesson to me! |
|
|
Earlier this month I was shocked and astounded to see that I'd used 1% of my bandwidth in 1 day. WTF? I don't get that much email and don't serve that many pages! Hmm... Now it's the 9th and I've only used 2%. OK... so what ahappened?
Wow, my personal web site has 1500 hits already. I migrated the site to the linode on 16th May and for that half of the month there was 800 hits total.
More research... Ah hah... 3 or 4 search engines decided to reindex my site at the beginning of the month, and I have 3 cgi scripts that generate 2300 links to data views...
I think I need a robots.txt file to prevent those cgi's from being indexed!
Let that be a lesson to me :-) |
|
