System files disappearing! Installation security help, please?

[eluser]Jay Turley[/eluser]
Hi-

We are hosting several sites with rackspace cloud hosting, and everything has been just fine up until a couple days ago. Suddenly a client started complaining that their sites are no longer sending them emails from things like contact forms.

So I ftp into one of their sites to take a look, and what I find is an empty system folder. Nothing in it.

Strangely, the site was still working, with the exception of the application blank-screening whenever the line

Code:

$this->load->library('email');

was encountered.

I reuploaded the entire system folder, and things are working again.

Now another client is complaining that the same thing has happened.

So, I'm wondering-- well, first, I'm wondering if we've been hacked. But second, I'm wondering what the permissions should be on the system directory and application directory. Also, what kind of access should be enabled through .htaccess.

Any help would be greatly appreciated.

[eluser]markup2go[/eluser]
Check your CI logs and FTP logs to see what's going on. Enable error reporting using ini_set() or in your php.ini might help as well.

I myself have cause a virus before that stole all the saved ftp accounts on my PC and routinely injected XSS malware in various files. I'm pretty sure you probably are seeing the white space due to syntax errors from a similar cause.

I should also note that I recently purchased a cloud server from Rackspace, and they neglected to tell me that the IP was blacklisted for spam. This could cause emails to be blocked on the receiving end.

Let me know what happens this is interesting

I should also add that if you did suffer the FTP intrusion that you should reset all your passwords and clean the infected PC.

Thanks,
Jason

[eluser]Jay Turley[/eluser]
Thanks, I'll try this. I doubt it's a virus; I recently switched to Linux, and the malware there seems to be a much smaller ecosystem ;-) But it's certainly a possibility. And yes, I'll keep you posted.

[eluser]markup2go[/eluser]
Are you running a ftp server? What I meant is this nasty bug will infect YOUR PC, if you are using a PC to develop on. Be careful.

[eluser]BrianDHall[/eluser]
Well in the words of Gidget from Wow Wow Wubbzy, "that wasn't supposed to happen."

For CI files you should be ok to set everything to 0555 (read and execute only), and depending on your configuration you might be ok with 0444 (read only) but you'd have to play with it to make sure.

Note that you'd probably best run this recursively on CI system AFTER moving out your application folder

You will need to go in and manually set permissions for your logger though, or you will be unintentionally disabling logging.

Security lock down procedures basically state you need to assume total password compromise across the board and reset all usernames and passwords for anyone with access to your system, including database passwords. It's usually what causes this sort of things, compromised accounts.

I'd also check all your code and make certain to check any unset, unlink, eval, and exec commands and make sure you don't have something that could be injected to cause such deletions.

Also if CI still runs with no System, there is something wrong there, it should error out. Check for symlinks and your index.php and make sure you know what CI system folder it is really using.

[eluser]Jay Turley[/eluser]
Okay, I got you.

Our host, of course, is running an FTP server.

I am personally running Ubuntu 9.04 on my laptop (at least until tonight when I upgrade to 9.10) and am NOT running an FTP server on it. I am however running Apache and MySQL with nice strong firewall rules to block everything in or out except for port 80. My router is firewalled too.

[eluser]BrianDHall[/eluser]
oh yeah, it should be pointed out that as markup2go notes, most compromises are not of servers, but far less secure workstations with admin access, especially ones with stored passwords.

[eluser]Jay Turley[/eluser]
[quote author="BrianDHall" date="1256878519"]Well in the words of Gidget from Wow Wow Wubbzy, "that wasn't supposed to happen."

For CI files you should be ok to set everything to 0555 (read and execute only), and depending on your configuration you might be ok with 0444 (read only) but you'd have to play with it to make sure.

Note that you'd probably best run this recursively on CI system AFTER moving out your application folder

You will need to go in and manually set permissions for your logger though, or you will be unintentionally disabling logging.

Security lock down procedures basically state you need to assume total password compromise across the board and reset all usernames and passwords for anyone with access to your system, including database passwords. It's usually what causes this sort of things, compromised accounts.

I'd also check all your code and make certain to check any unset, unlink, eval, and exec commands and make sure you don't have something that could be injected to cause such deletions.

Also if CI still runs with no System, there is something wrong there, it should error out. Check for symlinks and your index.php and make sure you know what CI system folder it is really using.[/quote]

Okay, thanks. Running through this point by point:

0555 is what I am using, except on cache and log directories. Applications reside outside of the system directory. No unset/unlink/eval/exec statements used as far as I can tell. XSS and SQL injection in place. index.php DOES point to correct system folder

The "still running" is - I am thinking - having something to do with the "cloud" persisting the application somehow, perhaps through caching. This is unclear to me, and was never a problem in the past on previous hosts.

Now that I think about it though, the previous site was infested with malware at one point, and I am pretty sure it came from our client who uses a mini-CMS I built to edit site content. I wiped it from the site and did a complete password change across the board, and that took care of it. That was a few months ago.

Now that it's on three of our client sites in the cloud, and I haven't even accessed one of them with my new linux system, I am thinking it's probably not malware on my end.

Thanks for the help!

[eluser]Ben Hirsch[/eluser]
We are also on rackspace cloud and 2 days ago have been noticing that porn links are being appended to the root level index.php file for multiple domains.

I am still trying to sort out where the compromise happened. Was it XSS, or does somebody have our master password? Freaking out a little bit. In security lockdown right now. I spent the entire day yesterday changing passwords only to awake to a fresh batch of new porn links today.

If you discover anything new, please let me know. Starting to think this could be a rackspace issue as only one person has our master password.

[eluser]Ben Hirsch[/eluser]
BTW also... your "still running" issue may be that the attacker placed a static version of your home page in the web root directory. One of our domains had that.