| View previous topic :: View next topic |
| Author |
Message |
trev
President
Joined: 15 Oct 2002
Posts: 3011
Location: NW England, UK
4037 ants
|
 Posted: Sat Jun 14, 2008 11:47 pm Post subject: AVG8 skews all web stats badly...! |
 |
|
I've been trying to track down why some of our sites are getting stupidly busy recently and thought maybe it was just either good rankings or lots of people wanted to see stuff. However it seems there might be another reason - AVG Anti Virus.
Essentially say a user searches for Buckingham Palace on Google. AVG will load up each of those usually 10 results as if they came from the users browser, and scan them for problems. This results in every link getting a page view and very possibly a "unique visitor" counted.
That's fine if the user actually visits the page, but most likely they'll click on say result 3, go off to that and never look at any others. The other sites have wasted bandwidth, server power on a non-human user, and they think they've got another unique to count. Multiply this up across larger popular sites with lots of pages and you start to really eat resources.
Considering there are around 70 million AVG users in the world and only 20 million of them have upgraded so far, this isn't looking good. The only way currently to spot this is to look for the user agent:
| Quote: | | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813 |
It's the 1813 bit that gives the game away currently, but that could change. Assuming this isn't being used by other malware yet, we have a site getting around 80,000 uniques a month. I checked the logs and since the 8th June, we've been hit by 95322 pre-fetch checks from that user agent. Over 12000 just today...!
The register on AVG8
For anyone with root access to the server, you can check the logs by:
(replace example.com with the web site domain, no www.)
| Code: | cd /usr/local/apache/domlogs/
grep -c "Windows NT 5.1;1813" example.com |
| Code: | cd /usr/local/apache/domlogs/
grep -Pc "14/Jun/2008.+Windows NT 5.1;1813" example.com |
First one checks all the entries in the log, the second checks the 14th's entries. This probably only works on Apache, but should be tweakable for other log formats.
Trev
_________________
Mad Theories - Ambleside - Coniston - Grange |
|
| Back to top |
|
 |
pinyo
Executive PA
Joined: 05 Aug 2004
Posts: 77
94 ants
|
|
| Back to top |
|
|
trev
President
Joined: 15 Oct 2002
Posts: 3011
Location: NW England, UK
4037 ants
|
| Posted: Mon Jun 16, 2008 8:24 pm Post subject: |
|
|
Nope as the IP address is that of the user running the AVG software and downloading the search engine results. It's all run from each users copy of AVG and since it doesn't report back to a central place there's no whitelisting which would at least improve things a little.
Even worse it appears it doesn't do any caching, so for example a user clicks a link then goes back to Google, the software rescans all the links. At least those don't appear as uniques, but they still take up bandwidth and processor time.
Trev
_________________
Mad Theories - Ambleside - Coniston - Grange |
|
| Back to top |
|
|
trev
President
Joined: 15 Oct 2002
Posts: 3011
Location: NW England, UK
4037 ants
|
| Posted: Wed Jul 09, 2008 3:56 pm Post subject: |
|
|
From what I've heard, AVG have now amended this system so it'll stop scanning everything and only scan the page you're looking at - like everyone told them to. Still don't know why they can't run it as a proxy as has been suggested a few times.
Trev
_________________
Mad Theories - Ambleside - Coniston - Grange |
|
| Back to top |
|
|
SeoAddict
Secretary
Joined: 10 Aug 2008
Posts: 7
10 ants
|
| Posted: Sun Aug 10, 2008 3:06 pm Post subject: avg 8 |
|
|
I think AVG 8 checks up if a site is safe to look at or not.
it definitely uses up some system memory so it's up to you if you want
to use AVG 8.0 or not...
_________________
Make Money Online
Free Small Businesses Network Tools || Squidoo Lens |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
