Anyone having trouble with Bowfishing Country moving slow tonight. I can't get files to upload to my albums to put in post or nothing. Barley getting around the forum??? Any Ideals?:headbang:
NorthWind99
mines very slowwwww, just rebooted and saw this tread, nothing else to do tonight, hope it gets fixed
So we were attacked last night?? Is that what heard?? Where did the attack come from??
You know your doing something right when people are trying take you down :td:
You know your doing something right when people are trying take you down :td:
Yes we are aware. 3 people working on this nonstop. It appears to be an outside influence but finding and stopping these things is ridiculously difficult. That's why these have the ability to bring down sites like mastercard, amazon, ebay... etc.
Working on it.
Working on it.
ahhh.... back to normal.
Server uptime: 3 hours 1 minute 22 seconds
Total accesses: 194,443
Total Traffic: 738.1 MB
CPU Usage: 1.8% CPU load
17.9 requests/sec
Speed should be getting better hopefully through the evening as we clean up more things.
Server uptime: 3 hours 1 minute 22 seconds
Total accesses: 194,443
Total Traffic: 738.1 MB
CPU Usage: 1.8% CPU load
17.9 requests/sec
Speed should be getting better hopefully through the evening as we clean up more things.
Sort of. I believe it was just a script kiddie. Essentially one of the support files for the forums was being access directly and in rapid fashion. This in turn was generating errors but more importantly eating up server threads. The thread would climb so high that they would max the CPU at 100% and stay there. Secondarily because the thread count was maxing out it was causing Apache to orphan these threads and not follow through to the close state. This would in turn leave less threads for everything else.
rebooting the server would result in about 20 seconds of peace and then it would jump to 100% and hold.
Running through the logs files was brutal. at 18 requests per second and troublesome requests only happening 3-4 times a minute that meant there were A LOT of lines to dig through for clues.
So, changes to the file location, changes to the thread management for apache, changes to the access restrictions and a few other tweaks later and we are running at a good clip with only 1.5% of CPU on average use.
Problem is that these scripts shift targets and the only thing you gain is the experience next time as to what to look for.
The next thing will be to move the entire directory to a new one which breaks the scripts for a while but causes hell with search engines.
Hopefully it just wont come back. We may do a complete reboot later but dont like doing that remotely.
Just to improve any lingering speed issues.
and yes i need a drink.
rebooting the server would result in about 20 seconds of peace and then it would jump to 100% and hold.
Running through the logs files was brutal. at 18 requests per second and troublesome requests only happening 3-4 times a minute that meant there were A LOT of lines to dig through for clues.
So, changes to the file location, changes to the thread management for apache, changes to the access restrictions and a few other tweaks later and we are running at a good clip with only 1.5% of CPU on average use.
Problem is that these scripts shift targets and the only thing you gain is the experience next time as to what to look for.
The next thing will be to move the entire directory to a new one which breaks the scripts for a while but causes hell with search engines.
Hopefully it just wont come back. We may do a complete reboot later but dont like doing that remotely.
Just to improve any lingering speed issues.
and yes i need a drink.
Good stuff Alex! Glad your good at what you do! Sucks on the attack part though, they do a good job of using multiple servers or were you able to track anything down?
It was multiple IPs - mostly overseas but a couple domestic. Not surprising though with zombie clients that probably started out as malware.
Someone here got maleware. The script sees VB as an exploit and finds it in the history. Kicks the url back to other machines that hit the files from multiple locations.
At first i thought it was a SQL injection type of attack but if that was it they got nowhere.
truth is we may never know. When this stuff happens the sole focus is uptime. not educating ourselves about the exploit.
I doubt it is truly over. Will be watching for the zombie hour tonight. after 11pm eastcoast time as scripts are designed to hit their exploits at the time of most vulnerability... when the admins sleep.
Fortunately I dont
Maybe we will get lucky and wont have to deal with it again.
Have backup plans if needed. They are just a pain to deal with while a bunch of guys are busy counting to 1 million on live forum
Someone here got maleware. The script sees VB as an exploit and finds it in the history. Kicks the url back to other machines that hit the files from multiple locations.
At first i thought it was a SQL injection type of attack but if that was it they got nowhere.
truth is we may never know. When this stuff happens the sole focus is uptime. not educating ourselves about the exploit.
I doubt it is truly over. Will be watching for the zombie hour tonight. after 11pm eastcoast time as scripts are designed to hit their exploits at the time of most vulnerability... when the admins sleep.
Fortunately I dont
Maybe we will get lucky and wont have to deal with it again.
Have backup plans if needed. They are just a pain to deal with while a bunch of guys are busy counting to 1 million on live forum
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
-
?
