Deleting folders from public_html without crashing sites

Hello All,

I have hosted some sites in the past that are no longer on my Linode. Their folders still remain in;

/home/sites/public_html

I want to delete those folders for unused sites, but when I go into webmin or cyberduck to delete them, after deleting a folder of an old site, all my other sites stop working.

Is there a proper way to delete this folders without crashing my other sites?

I hope that is enough information, if not, please ask. I really appreciate everyone's help.

Thanks