Zombie Robots Are Eating My Site

December 13, 2013  I’ve posted an update about the zombie bot issue – the short version is – it went away for a year and a half, and now it’s back. Check it out.

One of my sites has been under mass attack by bots for a month now, without cease.  It’s cost me (and my developer partner) time, money, and an undue amount of stress.  It’s undermined my analytics and stats.

When Bots Attack

It Just. Keeps. Coming.

And while there are a couple things we’ve been able to do to minimize the damage, essentially there’s no way to stop it. It just. keeps. coming. And frankly, if it continues, and spreads, there could be big repercussions across the web on ad revenue and analytics.

What I Know

The attack started on February 21, 2012, around noon.  I keep an OCD-level eye on my traffic, and I noticed a big jump in direct traffic.  This is unusual, because this particular site is less than a year old, and has not had a chance to develop a lot of branding yet.  It’s pretty well situated in the search engines for its niche, but not that many people know it by name.  Anything more than twenty or thirty percent direct traffic would definitely be odd.  And then I started noticing some other strange behaviors:

  • All the traffic was reported as Internet Explorer (versions 6 through 9)
  • All the traffic was reported as Windows (XP through Win 7)
  • The traffic was coming from all over the world (and the site is focused on ONE state in the US) and from thousands of IP numbers & ISPs.
  • It was all hitting the home page and leaving immediately.  My bounce rate quickly soared to about 99%
  • There was nothing – no one thing – that I could pinpoint to block this traffic from coming in. No commonality.
  • It was executing javascript – because it showed up in Google Analytics, Statcounter & Woopra.

Strangest of all, the traffic was *slow* – drip drip drip.  Never so much to come anywhere near a DDOS, or have an effect on the server, but at any given point, there would be six to ten “visitors” on the site at a time.  While it looked very much like actual human browser traffic, it wasn’t difficult to conclude that this was something automated.

What I Don’t Know

The first thing I did was turn off my AdSense.  Any kind of automated traffic like this would (quite reasonably) be seen as a risk to advertisers, and I had no idea what this was or what it could do. I turned ads off on the entire site. After several days when it was obvious it was only hitting the home page, I could turn the ads back on for everything BUT the home page.

That same afternoon, Roger Dooley  posted a thread on Webmaster World about the same sort of bot attack on one of his own sites. Over on the Google Analytics help forum, a discussion was forming with more and more reports of this same strange traffic pattern.  Most (but not all) started on February 21.

The first thought that went through my head was that I was somehow being targeted (paranoia!)  But if someone really wanted to attack me, they probably wouldn’t have done it on this site.  Comparing notes with as many people as I could find, there seemed to be no commonality on the receiving end – some sites had AdSense, some did not. Some sites were WordPress, some were static HTML.

Then I thought, maybe it was some kind of a probe, looking for a WordPress exploit.  But other (non-WP) sites were being hit.  Harvesting email addresses? I don’t list any email addresses on that site, and besides, all this was doing was loading the home page, over and over and over.

More paranoia set in – maybe someone who got Pandalyzed and wrote something to trash Google Analytics, and I was unwittingly part of the beta test?  After all, Google gets a lot of aggregate information from GA; trashing the stats like this would definitely damage trust in the product.

But whether it was a targeted attack, a coding error, or collateral damage didn’t matter. What mattered was that I might have to shut down my site. It’s a community service event site, and it’s supported by ad revenue, both Google’s and (hopefully) direct local ads.  I couldn’t run AdSense on a site with bogus traffic, and it would be fairly difficult to sell direct advertising without decent stats to show potential advertisers.  And this particular site sucks up a LOT of resources when it’s at peak; it needs to be able to pay for itself.

Zombie Bot Traffic

Like I wouldn’t notice that.

I asked my host, TigerTech, to take a look, and they said as far as they could tell, it looked just like human traffic. There were no User Agents or anything else by which we could block this traffic (without blocking real live users)

After a week or so, Roger posted a theory that perhaps Compuware’s Gomez Peer program (which pays users to install a screen saver that tests site and network performance, and collects benchmarking information) might be behind it.  Some people had reported contacting Compuware and the traffic mysteriously stopped.  But I contacted them, they opened up an investigation and determined that my site was not in their database, and the IP numbers I sent them were not part of their peer program. They also told me that Gomez identifies itself in the User Agent.  I have no reason not to believe them.  (And the people who said their traffic had mysteriously dropped jumped the gun – it came back.) So that was a dead end too.

I did a lot of frantic Googling for other people having the same problem, and we tried a lot of things, none of which ended up panning out.  Bill Atchison (@IncrediBILL) of CrawlWall put a lot of time in as well, making suggestions, writing scripts to collect data, and so forth.  In the end, he came to the same conclusion as everyone else – it was browser traffic. There was no way to 100% block it without blocking real human users.

Where I Am Now

As of today, March 20, 2012 it will be four weeks since this attack started.  It’s still going on.  It’s still a slow drip, and it’s gotten a lot slower on my site, although other people are reporting being hit much harder.  After a peak of around 10k visits per day, it’s now settled down to a steady 1k per day, give or take a hundred visits.  It’s still hitting the home page only. We’ve taken some steps to block ads and analytics, as much as we can; it means we are not showing ads or analytics to some real users too, but that’s our collateral damage.  We’re also not allowing non-English browsers, because this site is targeted only to a region of the United States.  If it is a virus attack, maybe some of the infected Windows machines were cleaned up, I dunno.  As I get closer to my peak season on this site, I’ll have to evaluate what effect this will have on my earnings (the home page being the best earner) and whether or not I’ll be able to keep it going.  Fortunately, I don’t rely on this site for my income; if I did I would be in trouble.  Just waiting to see if this will end one day, as mysteriously as it began, or if it will be scaled up, or …?  I just have no idea. All it appears to do is come hit the page.

What It Means

This is the hard one.  Maybe nothing.  Not that I’m an alarmist or anything (who’m I kidding, of course I’m an alarmist) if this spreads, or is the pre-cursor to some larger attack, it could seriously screw up the web.  It could affect ads. It could affect every type of analytics – if you aren’t tracking for conversions, how are you ever gonna know how much of your traffic is real and how much is fake? It could affect end user trust in analytics.

Or it just might go away.

If you’ve seen or experienced anything like this, please chime in below.

  • http://twitter.com/rogerdooley @rogerdooley

    I did hear from Compuware, who said they checked my URL, site name, etc. and could not find it in their system – Compuware/Gomez: Bot Attacks Aren’t Us

    Meanwhile, the bot traffic on my site has leveled out at a high level. One page that would normally get single-digit views daily is seeing about 4,000 visits every day. Very weird, because it seems pointless.

  • http://twitter.com/mosquitohawk @mosquitohawk

    Consider using a service like Cloudflare. It does a great job of dealing with this kind of crap and it has a free plan & a WordPress plugin. :)

    • netmeg

      We're looking at that, but there may be reasons why we can't use it on this site. I really doubt they'd be able to do anything on this; there are no discernable differences between the bot traffic and a real browser used by a human.

      • netmeg

        Update – I did try it in Cloudflare for a few days, but they only ended up detecting around 300 bot visits out of thousands and thousands.

  • http://www.theistudio.com/muse/ Judith

    Had the same *exact* thing happened to one of my WordPress sites this week. No ads or AdSense — pure content. My host, HostGator, took the site off-line being the site is on a shared server claiming it was effecting other sites. What was odd is I have a bunch of sites on that server and didn't see any issues with any other sites , including the one HG shut off….

    Installed W3 Total Cache WordPress plugin and MaxCDN and that resolved the resource issues HostGator was concerned about.

    Being the site gets hefty traffic at first I didn't think much about it until I started digging into this. The same exact thing as you describe — all hitting the home page, index.php. HG as well claimed to not see anything out of sorts or to be concerned about and just blamed it on increased traffic.

    Will let you know if I find out anything more…. ;-)

    • netmeg

      Were they all direct traffic (with no referrer) and all Internet Explorer? That seems to be a constant, and that's what I would look at first.

  • Pingback: SearchCap: The Day In Search, March 20, 2012 | Market 7()

  • http://twitter.com/lookadoo @lookadoo

    I have no suggestions but wanted to comment that just reading your (and Roger & Bill's) Sherlock-Holmes approach has been impressive. Nice investigative work.

    Most curious is the drop from 10K to 1K. hmmm

    Meg, you'll update this post when you find out more? Please do.

    • http://twitter.com/HunterSatter @HunterSatter

      Thanks Dana,

      I'm the developer partner mentioned in the beginning. Bill, Meg, and I slaved away an entire weekend trying to find a unique foot print for all of this. The traffic drop is attributed to a filtering mechanism we put in place and while it does bottleneck the traffic for the most part it still does not provide a permanent solution.

      Bogus traffic = Roll coaster ride of unfun

    • netmeg

      Thanks. Yea the drop is strange – of course, it might not be quite that much. We did take some measures to filter out traffic that definitely didn't belong, like non-english browsers. If it *is* a Windows virus botnet (which is what it looks like to me) then possibly some of the infected machines were cleaned up by an antivirus update. That's all I got.

  • http://www.joydeepdeb.com/ Joydeep

    We can block Bots from crawling via robots.txt file, but a SPAM Bots will not honor the robot.txt file.
    And as there were no User Agents, even redirecting this somewhere else is not possible.

    I would like to know more, and what future steps you will take.

    • http://twitter.com/HunterSatter @HunterSatter

      You're right in the sense that something as crude as robots.txt will not work. The issue here is that the bogus traffic is simulating MSIE traffic to the point that you cannot distinguish it from real people using MSIE. The only noticeable pattern is the instant bounce. It's as if they're touching the page and immediately dropping the connection once they get the OK that the page has loaded.

      • https://www.facebook.com/mark.r.robertson Mark Robertson

        are you guys still seeing quick bounces? I dont know if the attack switched strategy or now, but what was once a very low time on site at the start of the attack, we're now seeing average time on site…

    • netmeg

      At the moment, I am reacting and not preventing. I have some friends thinking about it. But honestly, I don't have a solution yet.

  • Matt

    Hey guys — you might be interested in my very similar post from March 2 — the large media site I run is experiencing this same problem.

    The only technical solution that I've seen that could possibly work (see the last update to my post, above) is to conditionally load analytics/ads when a mouse or key movement is detected — this assumes that the bad traffic does not involve any real user interaction … this might be a good first question to answer. Are there any DOM events associated with this traffic? If not, then simply don't load any analytics or ads on direct traffic from IE browsers with no events. Easier said than done for sure — but so far it's the only idea that I've heard that would actually work.

    Google does not seem to be in the mood to talk about this — we're still waiting for a response.

    We are hosted by wordpress.com VIP. WP.com engineers are now aware of the problem. It affects their internal analytics as well, and probably many of their millions of sites.

    • http://netmeg.com/official-bio/ netmeg

      Yah; that sounds like the same thing. Do you know when yours started? I've started wondering if there was anything special about February 21, because a lot of reports I've seen have used the same date (but not all)

    • http://twitter.com/HunterSatter @HunterSatter

      Great idea Matt! Currently we are conditionally loading ads and GA. Right now it looks like there is an immediate exist as soon as the DOM is finished loading. Detecting for mouse movement sounds like a great idea!

      • Matt

        Cool :-) We're doing this now. Here's how if you're interested. http://stkywll.com/2012/04/27/annoying-robots-a-s

        • https://www.facebook.com/mark.r.robertson Mark Robertson

          Are you guys still seeing this… We've started to get attacked much worse in the past week. Wondering if anyone has any updates? BTW – I talked directly with quality team at Adsense. They're aware of the situation and believe it or not – told me that they're seeing it everywhere and dont know themselves what it is. That was last week ;-(

          • netmeg

            Interesting. No, mine have stayed off since the 19th (oddly, the date of the last Panda refresh, though I doubt it's related) and several other people on WMW mentioned theirs stopped the same day. If it was a test, I dunno if it stopped because I passed or because I failed.

            I hope someone finds a solution. I wouldn't fall over dead with surprise if they came back at some point but I sure hope not.

          • Matt

            I'm not sure there's going to be much of a solution to this — we took things into our own hands and just started not loading analytics for the bad traffic. (see link above) Presumably you could do the same thing for adSense. It is worrying that they don't know the nature of this — it seems pretty damn pointless.

            The most plausible theories I've heard so far:

            – this is actually an attack on AdSense and/or Google itself b/c it renders these products unreliable or unusable, especially for small sites.
            – this is an attempt to bruteForce disqus, facebook or some other application that is loaded client side. This seems a bit implausible as there would almost certainly be better ways to do something like that.

    • netmeg

      Do you know when your attack started?

  • pmi

    Could this be a trojan hijacking computers in succession slowly? This substantiates the drip. We're experiencing the same thing as everybody – with the exception that our issue dates back to last May. Direct traffic is up 700% this Jan over last. We reached out to Danny Sullivan via Twitter and he's unaware of this issue too (yea. He and Google?). We suspected the attack on web hosts, but have discovered that it's happening at the website level. Anybody try changing web hosts? Does if follow?
    We're trying to determine if it's IP based or domain based. Rrrrrr…

    • netmeg

      I have other sites on the same server, same IP. They're not affected.

  • http://twitter.com/lenwood @lenwood

    This is pretty scary! Most of the comments here are about handling the extra traffic, which misses the point. Sure CDN's & caching will make sure that this doesn't bring the site down, but this won't help you get advertising back on the site. I'll be very curious to learn how you ultimately address this. Here are a couple of ideas.

    – Only show ads to referral & search visitors.
    – Only show ads to repeat visitors.
    – Restrict ads to visitors within your city or state.

  • http://questio.co.uk Chris Porter

    I have the same issue, though I'd forgot about it until reading this post. It's on an older site that only make a couple hundred per month in aff. sales so it's unaffected from a monetisation point of view (no adsense, no direct ad sales).

    But this got me pondering on why someone would use their resources (bot network) to load random websites. It's not enough to be a DoS and there is no obvious monetary gain.

    But, if you were to control a network of tens of thousands of bots and your control of this network was under constant attack (anti-spyware program updating, computers being formatted, rival malware, rival developers and anti-malware professionals) then you'd need a way to constantly monitor the health of your network.

    I can imagine sending out an "are you alive?" command every so often wouldn't be very effective due to the decentralized nature these networks. But, if you just coded your bot from the beginning to make a simple http request to a random website, once every few seconds and then report the http code back to HQ. Well, it might be one way of monitoring the health of your bot network.

    Just a thought. Obviously many webmasters get caught in the crossfire here but we must focus on the motive to figure out the cause.

  • PeterA

    We've been seeing the same thing since Jan 26th. It peaked at 33K visits a day dropped to near zero then came back up again, then down again. Its at about 3-5K visits a day now. We put a roadblock up to intercept traffic that matched the footprint to protect traffic quality for our ads. Here is my write up on it: http://researchferret.blogspot.com/2012/02/strang

  • PeterA

    I've tried to think on motive for the zombie traffic. These are the options that I keep coming back to (I've thought up a few more and shot them down):

    1) It is a massive botnet in waiting. The whole thing is assembled for some major attack yet to come. It is distributing its energy on a variety of random sites while it waits. If this botnet ever did decide to use its power for a denial of service attack it would be very hard for whoever it targets to actually stop it.

    2) It is the early phase of an ad fraud scheme. Right now the traffic does nothing but load the front page. What if it clicked ads or racked up ad impressions in mass though and once it was honed they directed the traffic to sites that were part of a fraud ring? What if the few ways we can detect the traffic were eliminated in a newer version of the attack and ad networks / site owners couldn't filter out the traffic?

    3) It is the "phone home" mechanism of a botnet. The operators gave orders to check multiple sites for its next set of commands. They either hoped to compromise the sites to post the commands or there is some sort of exploit in a common 3rd party javascript all of the sites use that ties in to this (I am grasping at straws here.)

    At one point I entertained the idea that it was someone with malicious intent trying to negatively influence SERPs by manipulating bounce rates (this would only work IF Google used Google Analytics data in its algorithm–which I am pretty sure it doesn't) or get the site pulled from AdSense. The fact that a variety of sites of different sizes are experiencing this makes this scenario unlikely.

    What other ideas have the rest of you had? Who would benefit from this? How does someone benefit from this?

    • netmeg

      Hi Peter – thanks for commenting. All of your scenarios have run through my head as well. The issue of the bounce rate – I think it's probably unlikely that that would affect Google SERPs; we've had a lot of discussions on WebmasterWorld about this. The general consensus is that bounce rate is a fairly noisy signal – there's lots of perfectly valid reasons for a high bounce rate that don't infer poor quality – and it'd be far more likely that Google would pay more attention to whether someone hit a page once, and then went back to the SERPs to try something else.

      We've experienced no loss of regular traffic or ranking – just the opposite, as we get nearer to our peak season. Mostly this is just a distraction that's keeping me from running ads on the affected pages, and borking my analytics.

      Google has reached out and asked for some information, and I've given them what I've got, along with a few other sites that have been hit for "data points" so they are aware of it. I don't know that there's anything they can do either, but they certainly have a lot more data (and a lot more resources) than I do.

  • http://jesperastrom.com Jesper Åström

    We are experiencing this as well. Could you please confirm if you are seing these visits as unique visitors as well? We are experiencing unique visitors increase, actually almost double for a site in Sweden. The increase is site wide, thus not only to the index page. Traffic seems to be coming from all over Sweden with a variety of 53-97% increase per commune. Bounce rate is extremely high. All change in traffic is explained in direct traffic.

    • netmeg

      They started out as unique visits of course, but I’m getting lots of return visits now, which supports the idea that it’s some kind of botnet of infected machines. Mine are coming from IPs all over the world.

      • http://jesperastrom.com Jesper Åström

        Thnx. Ours are virtually only from Swedish IPs for the Swedish site.

      • http://www.seo-ranking-tools.de Twitch

        Mine are coming from IPs all over the world, too. In Feb. there was only Traffic from US, but March and April are really bad :-(…

        I have not that much return visits at the moment.

  • http://www.seo-ranking-tools.de Twitch

    I have the same Problem since 26th January. But at this weekend my traffic explode! I tried to filter the bad IPs and I found a way to minimize it up to 40%. The rest is hard to filter and, my opinion… impossible!!
    The only filter i analyzed is, that the user IP and the user host has the same value. If that happen I forwarded the user to himself.

    Has someone else a solution?

  • http://www.seo-ranking-tools.de Twitch

    Update: I forgot one thing to tell: The Bot visited every time the startpage of the domain and is following the external links. My Banner Partner are not amused, because their traffic grows as well with my referrer.

  • jean

    Hi all. I'm receiving similar, but all the traffic is coming from facebook. Here's how my situation started:
    I generated some tracking code for my web analytics program and then inserted it into my Adcenter ads. What happened is this: The tracking url must of got crawled by facebook or something.
    In my analytics, it shows that my ad got clicked and is being served via facebook!
    I've turned off my ads, but the traffic is still coming! I've contacted microsoft Adcenter in relation to this problem and they are currently investigating. The next step will be for me to send my raw server access log to them for further investigation… I'll keep you posted
    Cheers, Jean

    • jean

      Update: Microsoft Adcenter have not found any problems; But here's the weired thing – 1) My ads are paused but i'm getting free clicks!
      2) I don't think this is bot related because, I got 30 subscribers from it – I usually get 1 subscriber every 2 weeks to a month, but this weired traffic netted me 30 in just 10 hours!
      3) My site is getting bookmarked by; Pintrest, stumblr, stumbleupon… I'm noticing some patterns and discovering some new things. Could I of possibly stumbled across a traffic generating method by accident? I am a bit lost at the moment, but will investigate further and will also SEE if I could monetize this traffic somehow!
      I'll keep you all posted of my findings.
      Cheers, jean

      • netmeg

        This does sound like some other phenomenon, Jean. These bots that are coming here have no referrer or way to backtrack them.

        • jean

          Have you tried checking the ip address on the projecthoneypot ? This is a site that keeps records of ips that have been missbehaving. You can access it here :http://www.projecthoneypot.org/search_ip.php

          If you have Cpanel to access your server, then you can view visits to your site along with the ip address.

          Whenever I get a direct entry onto my page, I always check the ip with projecthoneypot. If it shows up as bad, then I log into Cpanel and then deny that ip address, access to my website. I also check my error logs to see if the offending ip shows up there as well.

          Anyhow, since blocking some ips, I've noticed the traffic die down a bit. Some of the bad ips tracked were termed: dictionary attackers, email spammers, rule breakers etc…

          I'll research some more into tracking these hard to track bots. But for now, tracking the ip address may be the only way to make some sense of what's going on.

          I'll keep you posted of my findings.
          Cheers Jean

          • netmeg

            I had tens of thousands of IPs, wouldn't be practical. So far, my bots are still way down, fortunately. Others have not been so lucky.

          • jean

            I understand FULLY. I probably don't have as much as you and I still found it a tedius task. Anyhow, my traffic has dropped considerably, but don't know when, or if it will rise again. Experimenting and researching working with server side scripts. What I want to know is; how can you work from the user agent side of things? For example, I was and still am recieving traffic from fb. The user agent is something like :facebook hit counter. I don't know, but what I have noticed is; Groups of "bad" ips are using the same "agent". So maybe traffic from that level can be redirected "somewhere else".
            Not sure, but hope my findings can be helpful to your readers.
            Thanks again for this post, it's been a stepping stone to endevour further and gain some NEW knowledge.
            Cheers, Jean

  • Kevin

    Thanks for this post Netmeg,

    I came under this attack starting April 9th and it continues at about 25000 more direct visitors per day than the usual 2000.

    One difference is, the user agent is not limited to a version of Internet Explorer like most experiencing this.

    Are you still under attack? Or have you found any other good solutions?

    • netmeg

      Yep, still going on. Sometimes more, sometimes less, but still happening.

  • netmeg

    Update – 4/20/2012. As of yesterday, my bot traffic dropped way down, to about 1-3 visits per hour. It's even less today. Furthermore, other people on WMW have reported a sudden drop-off as well. But still other people report their bot traffic has actually increased. I have no explanation for any of this. It's pretty darn weird. I'm keeping my various blocks handy though, in case it starts up again.

    I never heard back from Google or any of my other people who were "looking into it" but I never really expected to.

    I got nothing. But I'm happy it's leveled off, at least for now.

    • PeterA

      I hope it stays down for you. It picked back up on our site on April 18th. It seems to be going in a cycle with several peak days followed by a gradual drop off and a lull. Then three weeks later another surge hits. It is discouraging that you haven't heard back from Google.

      In one of your comments you noted that you are starting to see some repeat visits from IPs or machines matching the fingerprint. Can you confirm this? We haven't noticed repeats yet.

      • netmeg

        Yea; I can't say I'd be surprised if it started up again. The fact that it stopped is as mysterious as why it started. Was it a test? Did I pass? Did I fail?

        Yes, we definitely have a lot of repeats after a month or so; as detected by StatCounter. I'm still getting a visit or two an hour. Several people on WebmasterWorld have also mentioned their traffic has all but stopped, while others say theirs have increased.

        Don't expect anything from Google. Even if they found something, they're not set up to respond on this small a scale. Moreover, even if they figured it out, I dunno that there's anything they could do about it.

  • https://www.facebook.com/mark.r.robertson Mark Robertson

    Im trhilled I found this thread but scared as it seems that no one can find a solution to the problem. Are there any udpdates to what you've found? our site is 100% alive due to paid advertising, which I cant in good conscious continue to charge for. Which means, we're going out of business if we cant find any solution. i would love to find out if anyone has any updates or knows of any way to stop the attack? I'd really appreciate anything input.

    • netmeg

      Unfortunately I have not yet seen any updates as to what is causing or what can stop the attack. Mine is down to about 20-30 visits per day, and has been since April 19. Others are still getting it, and some have experienced an increase. I guess the first thing I would do if I were you would be to see if what's happening to your site fits the same profiles – ie. is it all direct traffic? Are they all showing as IE users with no referrer? Coming from all over the world? And are they hitting all pages? In my case, they were only hitting the home page, which was a PITA, but at least I could disable ads easily on one page.

      • http://twitter.com/MarkRRobertson @MarkRRobertson

        Thanks Netmeg. I just came back to this post to see if there were any updates and didnt get notification of your reply. So, much appreciated. We're still seeing about 200K visits to our homepage each day (also just homepage, though I dont know what PITA means). As far as I can tell, the IPs are well distributed and are all direct traffic so we have no idea at this point how to stop it. I'm dynamically loading analytics and ad javascripts so as to block them from loading anytime someone hits our homepage using IE, but it's just a short term solution and still is causing us to upgrade servers to a silly level, and we are blocking some legitimate traffic to our homepage as a result. Im raking my brain to think of ways to stop the attack and working with some security experts but we're unaware of any tactic as of yet to actually stop it. If you have any updated information, would very much appreciate it. I find it interesting that yours seemed to have gone away. Im wondering if you can recall making any changes at that time that could have done something? Thanks again so much for your input.

        • http://netmeg.com netmeg

          Hi Mark – I don't believe my attack ever came back, but I got busy (and my busy season for the site has come and gone – and it was wayyyy busy so I wouldn't have noticed the bot traffic anyway) I'm pretty sure I didn't do anything or change anything to make it stop. One day, almost two months the day after it started, it just went down to fewer than half a dozen visits per day. A few other people reported the same thing, while still others are still having the visits. It's just the damndest thing all the way around. (PITA = "pain in the ass")

          Nope, just checked Google Analytics, and as far as I can tell, it's still gone. I dunno if I will ever know. But people need to keep banging the drum on this. I still get a lot of daily visits on this post, so it must still be happening to a number of people.

  • http://tech4idiots.org tech84

    Maybe you can block them using the robots.txt? I had some experience with spam attacks and I tell you it sucks big time, it increases server loads and bandwith and even took up a lot of disk space on my account.

    • netmeg

      Problem with that is #1 there's no signature to block them – they look like normal human visitors, and #2 robots.txt only works on bots that choose to obey it. Bad bots don't give a shit about robots.txt.

  • http://twitter.com/ashnallawalla @ashnallawalla

    I haven't checked the timing of the negative SEO activity in case this is a competitor either targeting you or has merely picked you for a test run.

    • http://netmeg.com netmeg

      Well it happened pretty much before all the negative SEO frou frou spiked up. And if it were (there aren't that many competitors in this particular niche) this wasn't my most crucial site.

      I don't know what it was, and because normal traffic on the site has grown so much, I haven't really been able to tell if it's come back, but I don't think it has. Other people are still seeing it though.

  • Elizabeth

    Hi.The same thing has happened to my website in the u.k.Up to over 4000 unique visitors a day.It seems a pointless exercise.I was paranoid when it first happened but after reading all these other comments i do feel a little better.

  • Magyver

    24 hour a day peace of mind is simple, you just never heard of it. 99% of the "bad guys" who attack websites are bots. They do their damage without even registering so even using a plugin with the database resources of StopForumSpam may not be good enough.

    I was lucky very early with my first website and found not only StopForumSpam, but a partner of theirs SpambotSecurity. The latter has a free product called ZBblock. ZBblock uses the database from StopForumSpam for registration and commenting but also has very sophisticated preemptive anti-bot script.

    ZBblock is bot proof, period. You should not only Google all 3 names but you should become a member of both websites. I know the owners of both websites well, their free products have kept my sites safe for 2 years now.

  • JC1

    are you posting you new links to Twitter? I think, at least in my case, when my new links hit my Twitter feed I get a rush of bots monitoring that feed that immediately go to my site. Immediately after posting a new article link on my Twitter page I'll get about 50 visits within the first minute, GA live visit beta will only show 1 or 2 live on site though.

    • http://netmeg.com netmeg

      At the time I had the bot attack, it was off season for this particular site, so I was not tweeting anything for it. I do know that as soon as you tweet a link the bots come out, but this is a different type of thing, I'm pretty sure.

  • http://supernaturalresearch.com Milton Thomas X

    Same here… I unfortunately allowed the attack to go on for a week prior to dismantling it because an article had just gotten mentioned on radio and I was receiving a rush (4,000 + extra visits daily) from legitmate visits and Google wiped out just about every ad revenue due to the 400+ bot visits…. Most of my attacks are from England and Canada. I am in the united states. The article that got attacked is posted below. About 400 fradulent visits daily and up to 100 clicks. Ive taken your advice and replaced google ads with Amazon. curious if the bots will click on these?

    • http://netmeg.com netmeg

      (I removed your link) I have no idea that my bots were clicking on links because the minute I saw it, I removed AdSense. Since Amazon is affiliate, it won't matter if they click on those ads, unless they have credit cards and can place orders. But in general if you are getting fraudulent traffic, you're going to have an issue with ad monetization. Even if it just shows up one day, it doesn't matter because it's considered "a significant risk to the advertisers"

  • wheel

    Don't have any informative reply netmeg, but did want to share your distress at this. Getting hammered by the bad guys is frustrating. Too many people think it doesn't matter, but as you noted it costs individual businesses thousands of dollars in hard costs plus endless hours of time. It's not different than throwing a rock through the front window of a retail store, with the same effect.

  • Antony

    Just a thought but have you considered that this could be stumbleupon, I have known the direct traffic to spike when my site gets excessively stumbled – the stumble frame doesn't always seem to send referral data correctly if used in IE.

    This would explain why you are only seeing IE on Windows machines from thousands of IP addresses and that they stay for only seconds.

    • http://netmeg.com netmeg

      Hi Antony; thanks for chiming in. No, it's not likely that it was Stumbleupon. I never get much SU traffic even on my sites that have been around a while, and the one that was hit was probably way too new for StumbleUpon.

  • Nick

    Could this be cross-site request forgery? Having unsuspecting people's computers visit sites unknowingly will alter their personalized search results and, theoretically, if you send people to the right results you can create your own #1 rankings. While this wouldn't be an easy task it would involve much of what you are seeing, Netmeg. Have we really entered that age already?

  • ixc

    This same thing just started happening to my site yesterday. It seems to come in waves; I monitor my site traffic casually throughout the day and noticed page hits would get up to about 240 at a time. My site that normally gets from 300-500 visitors daily had almost 1,800 yesterday. As I’m typing (was searching for an answer) it just went up to about 250 then trickled back down to 0. When it happens it’s not during the time when I’m running adwords or bing ads, and I don’t have any ads within the site. My site stays pretty much the same all the time, the only thing I’ve added recently was a winter snowfall script. At first I was kinda happy, thinking the surge in traffic might increase page rank, but I guess since it’s bots it could also damage it. Since it just started it will take a few days to see any kind of trend. I have no known competitors for my site, so I don’t believe it’s targeted against me. Just gets tons of visitors adding up quickly, peaks and then drops.

    This is truly scary though, for the basic fact that millions of bots like this could likely disable any public website’s servers, including banks and a lot more. I wonder if 12/21/12 will be the end of a working internet lol. If anyone else has some idea what’s going on pls respond!

  • http://instantxboxcodes.com ixc

    Happening again (3rd day). Seems to be getting stronger, as today it peaked at over 700 visitors. The 2 things I've done lately were adding a snowfall effect script, and I also added a mobile website script through dudamobile. I'm going to try removing the dudamobile script to see if that could be related , though the bots are coming into the regular site page, not the m. site. Noticed that the ip's are mostly u.s. and london. I just can't figure out what the motive behind this could be. There are no ads to click in the site, and there is apparently no harm being done. Just strange and annoying.

  • IXC

    I think I've discovered the problem. I had a mobile website set up through dudamobile, with the script installed on my webpage. I removed the script on Monday and have not seen the same problem for almost 3 days. I'm not blaming dudamobile for the attack, but maybe something in the script made the bots come. That was the only change I made, so I think I can conclude that was the problem.

    • http://netmeg.com netmeg

      Glad you got it figured out!

  • http://www.seoarcher.com tom little

    Hey guys sorry about your problems. I do not yet have that but fear that I may from someone that is i competition with my site. the only thing I can figure out to do is to place a popup captcha on my pages, this may deter the robot, but will also deter possible visitors. In any event I will keep digging ans see if I can come up with something, since with all the easy to get proxys out there a single ip may not track and trace..

  • Dusty

    Did you ever resolve it? I have something similar going on but its hitting thousands of dynamic url’s on a site I manage The traffic is coming from all over the US.

  • Erick Roberts

    Everything happens for a reason. You are getting clickbombed because you have encountered some blackhat competitors. The best way to take down a website is by jacking the ads, getting your adsense banned or dilute your ctr.

    There is no effective way to block this attacks, you could try filtering the traffic… and my best bet would be filtering the traffic by country. Can make the work of your enemy harder.

    Good luck!

    • http://netmeg.com netmeg

      Doubtful. The affected site had no competitors, and the attack never ended up actually clicking on ads, I just took them off to be sure.

  • http://netmeg.com netmeg

    Here's an interesting post. I wonder if this had anything to do with it. http://www.theregister.co.uk/2013/03/19/chameleon

  • teodormarin

    any news? I'm also having this problem, going crazy soon

  • zzzzzzzz

    just saying, i take all the bot hits, redirect them and sell "web hits" on ebay. so i dont see why you are trippin, i want more webots hitting my server

    • http://netmeg.com netmeg

      Well first of all, I'm no longer tripping because it stopped after two months. Second of all – I'm working towards a more long term strategy than trying to offload zombie bots as genuine traffic on eBay. Tomato, Tomahto.

  • ktmrad

    Have been experiencing this for a year to one of our URLs. We get about 15k requests per day from 1000s of IP addresses from mostly end user ISPs all over the world ( mostly us ). User agent is a mix of real windows agents. This really does seem like end user PCs being used for some automated task. Why are they hitting one of our URLs so often and for a year now beats me.
    I checked out if gomez was behind this by first calling them and seeing if they have our domain in their system. They said they did not and also told me that their "last mile" service would have a "gomez" identifier in the agent. I wanted to verify this so i downloaded and installed their client program which is called gomezpeerzone, activated it and then tcpdumped the traffic. Indeed it was used by their control to hit lots of sites, but at the tcp level all their requests did have "gomez" in user agent.
    So I'm still stumped.

  • AppleTom21

    I want to see the bots as well as the human activity. What software should I use to capture ALL of this? Thanks.

  • netmeg

    For anyone who’s still following this thread, I posted an update here -> http://netmeg.com/more-zombie-bots/ about the return of these bots – they’re back!

  • http://kitokid.com/tag/%D8%A7%D9%84%D8%B9%D8%A7%D8%A8-%D9%85%D8%A7%D9%87%D8%B1/ kitokid

    Thanks for your marvelous posting! I quite enjoyed reading it