Security at Work · Forensics

How to diagnose whether a server was hacked: a technical guide to forensics and incident response

Quick answer

Diagnosing a compromised server requires collecting volatile evidence before any corrective action, then checking processes, network connections, accounts and persistence in a standardized order. A hacked server rarely shows a single indicator: confirmation comes from correlating CPU anomalies, suspicious network traffic, new accounts or SSH keys, altered binaries and deleted logs. According to NIST SP 800-61r2, preserving evidence before containing the incident is decisive for attribution, effective remediation and legal compliance.

Decripte is a cybersecurity company serving businesses from 1 to 100,000+ employees — from assessment to 24x7 incident response.

Warning signs

  • Abnormal CPU or memory usage. Persistent CPU usage above 80% with no justified workload is a classic sign of cryptojacking (unauthorized cryptocurrency mining). Run 'top' or 'htop' on Linux and check which process is consuming resources. On Windows, Task Manager > Details. Cryptominers frequently disguise themselves with names like 'kworker', 'sysupdate', 'java' or random strings.
  • Network connections to unknown external IPs. Established connections to external IPs on non-standard ports (4444, 5555, 8888, 31337, or 443 to suspicious IPs) indicate communication with C2 infrastructure. Use 'ss -tulnp' or 'netstat -antp' on Linux and 'netstat -anob' on Windows. Confirm the IPs against reputation databases such as AbuseIPDB, VirusTotal or Shodan.
  • User accounts or SSH keys created without authorization. New local accounts, especially with UID 0 (root) or added to the sudo/administrators group without justification, confirm attacker persistence (MITRE ATT&CK T1136 — Create Account). On Linux, check /etc/passwd and /etc/shadow; on Windows, Event ID 4720 in the Security Log. SSH keys added to ~/.ssh/authorized_keys are especially dangerous because they allow passwordless access.
  • Unknown processes or services running. Processes running from unusual directories such as /tmp, /var/tmp, /dev/shm (Linux) or %TEMP%, %APPDATA% (Windows) are strong indicators of malware. Webshells appear as child processes of the web server (apache2, nginx, php-fpm, w3wp.exe). Backdoors frequently create persistent services with names that mimic legitimate system services.
  • Deleted logs or time gaps. Zeroed-out log files, missing entries for specific periods, or Event ID 1102 on Windows (audit log cleared) are evidence of active anti-forensics — technique T1070 in MITRE ATT&CK. Experienced attackers erase their tracks immediately after initial access. If the server uses a centralized SIEM (Splunk, Wazuh, Elastic), the remote logs may reveal what was deleted locally.
  • Suspicious files in temporary or web server directories. Webshells (PHP, ASPX or JSP files with eval, exec, system, passthru functions) in /var/www or public IIS folders allow the attacker to run commands remotely. Binary files in /tmp with execute permission, especially with recent modification dates outside business hours, indicate uploaded post-exploitation tools (technique T1505.003 in MITRE ATT&CK).

Step by step

  1. 1

    1. Preserve volatile evidence immediately

    Before any other action, capture the state of RAM (on Linux: 'sudo avml /mnt/evidencias/mem.lime' or 'sudo dd if=/proc/kcore'; on Windows: WinPmem or Magnet RAM Capture). Record the exact date/time (UTC), hostname, uptime and the list of active processes. Never power off or reboot the server without this capture — volatile data is lost permanently and may contain encryption keys, active attacker sessions and critical IOCs.

  2. 2

    2. Isolate the machine from the network without shutting it down

    Remove network cables or apply firewall rules to block all outbound traffic except the management connection you are using. On Linux: 'iptables -I OUTPUT -j DROP' (except the active SSH session). On Windows: disable network adapters via Device Manager or apply a firewall policy. Isolation prevents further data exfiltration and attacker communication with C2 (Command & Control) infrastructure, technique T1071 in MITRE ATT&CK.

  3. 3

    3. Identify anomalous processes and connections

    On Linux, run: 'ps auxf' (process tree), 'ss -tulnp' (open ports with PID), 'lsof -i' (open network files), 'netstat -antp' (established connections). Look for processes with generic names (kworker, sysupdate, java) that lack a legitimate path, connections to external IPs on unusual ports (4444, 8888, 31337) and processes running from /tmp, /var/tmp or /dev/shm. On Windows: use 'netstat -anob' in CMD as administrator and Sysinternals Process Explorer to see full paths.

  4. 4

    4. Audit user accounts and SSH keys

    On Linux: 'cat /etc/passwd | grep -v nologin' (accounts with a shell), 'lastb' (failed login attempts), 'last' (recent logins), 'who' (active sessions), 'find /home /root -name authorized_keys -exec cat {} \;' (authorized SSH keys). Look for recently created accounts, duplicate UID 0 entries and unknown SSH keys — technique T1098 (Account Manipulation) in MITRE ATT&CK. On Windows: 'net user', 'net localgroup administrators' and check Event ID 4720 (account created) in Event Viewer.

  5. 5

    5. Check for persistence (cron, services, registry)

    On Linux, check every persistence mechanism: 'crontab -l -u root', 'cat /etc/crontab', 'ls /etc/cron.*', 'systemctl list-units --state=failed', 'find /etc/systemd /usr/lib/systemd -name "*.service" -newer /boot' (new services), 'ls -la /etc/rc.d/ /etc/init.d/' and shell profiles ('/etc/profile.d/', '~/.bashrc', '~/.profile'). On Windows: use Autoruns (Sysinternals) to inspect every startup point, 'schtasks /query /fo LIST /v' for scheduled tasks and check HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run in regedit.

  6. 6

    6. Verify the integrity of binaries and suspicious files

    On Linux: 'rpm -Va' (RPM-based) or 'dpkg --verify' (Debian-based) to detect altered binaries. Run 'rkhunter --check' and 'chkrootkit' for known rootkits. Look for webshells with: 'find /var/www -name "*.php" -newer /etc/passwd -ls' and 'grep -r "eval(base64_decode" /var/www/'. Check for suspicious SUID files: 'find / -perm -4000 -type f 2>/dev/null'. On Windows, use Defender Offline Scan and check recently modified files in the IIS or Apache folders.

  7. 7

    7. Analyze logs and detect tampering

    On Linux: 'tail -500 /var/log/auth.log' (authentication), '/var/log/secure' (Red Hat), '/var/log/syslog', 'ausearch -k authentication' (auditd). Check whether logs were truncated ('ls -la /var/log/'; 0-byte files are a critical alert) and look for time gaps in the logs — technique T1070 (Indicator Removal) in MITRE ATT&CK. On Windows: Event Viewer > Security (Event ID 1102 = audit log cleared, 4624/4625 = logins, 7045 = new service installed, 4698 = scheduled task created).

  8. 8

    8. Confirm the compromise and trigger a formal response

    If two or more indicators are confirmed, treat this as a confirmed incident. Do not try to clean the server manually — the correct approach, per NIST SP 800-61r2 and ISO/IEC 27035, is to reprovision from a clean snapshot or a verified baseline image. Document every IOC found (IPs, hashes, process names, C2 domains), notify the incident owner and, if personal data is involved, assess notification to the ANPD under LGPD Art. 48. Engage an incident response firm for independent forensics and specialized containment.

What NOT to do

  • Do not reboot or shut down the server immediately. Rebooting wipes all of RAM — where the attacker's active processes, encryption keys, open sessions and volatile artifacts essential to forensics live. Shutting down can also trigger malware self-destruct mechanisms. Capture memory and the full system state before taking any action.
  • Do not try to 'clean' the server manually. Removing malware without reprovisioning the server is a false sense of security. Advanced rootkits and backdoors survive superficial removals, hiding in kernel modules, firmware or non-obvious persistence points. The correct practice is to reprovision from a clean, verified image, as recommended by NIST SP 800-61r2.
  • Do not keep operating the compromised server in production. A compromised server is an active attack vector against other systems on the network, customers and partners. Keeping the server in production during the investigation violates the duty of care and can aggravate legal liability — especially if third parties' personal data is exfiltrated during that period.
  • Do not alter log files or system metadata. Modifying any file during the investigation — even to 'tidy things up' — contaminates the evidence chain of custody. If the incident escalates into a judicial or regulatory proceeding (LGPD, PCI-DSS, SOC 2), tampered evidence can invalidate the entire forensic process and create additional liability for the company.
  • Do not tip off the attacker about the detection prematurely. Avoid blocking the attacker's access before collecting enough evidence, as this can trigger remote cleanup of their tracks. In advanced scenarios (APT — Advanced Persistent Threat), attackers actively monitor whether they have been detected. Work quietly: isolate on the network before blocking credentials or closing backdoors.

Why servers get hacked: the most common vectors

Most server intrusions start from a small set of vectors: exploitation of known vulnerabilities in exposed services (CVEs in Apache, Nginx, PHP, OpenSSH, RDP), weak or reused credentials, SQL injection and malicious file uploads in vulnerable web applications, and supply-chain compromise via software dependencies.

According to the MITRE ATT&CK framework, the most commonly observed initial-access techniques on servers include T1190 (Exploit Public-Facing Application), T1078 (Valid Accounts — use of compromised legitimate credentials) and T1133 (External Remote Services — exposed VPN, RDP, SSH). Linux servers are frequently targeted by cryptojacking and botnet campaigns; Windows servers are preferred targets for ransomware and lateral movement via Active Directory.

The average time between compromise and detection (MTTD — Mean Time to Detect) is weeks to months in environments without active monitoring. An IDS/IPS, a SIEM and behavioral baselines dramatically reduce this time and are the difference between a contained incident and a large-scale data breach.

Diagnosis on Linux: essential commands for forensics

Forensic diagnosis on Linux follows an order of decreasing volatility, as recommended by the SANS Institute. Start with the most volatile data: RAM, running processes, active network connections and open sockets. Never start with on-disk logs — they are the least volatile and the first to be tampered with by attackers.

Core commands in the triage phase: 'ps auxf' (full process tree with arguments), 'ss -tulnp' (open TCP/UDP ports with PID), 'lsof -i' (all open network files), 'netstat -antp' (connections with processes), 'who -a' and 'last -50' (sessions and recent logins), 'cat /etc/passwd | awk -F: "$3==0"' (accounts with UID 0), 'find / -mtime -3 -type f -not -path "/proc/*" 2>/dev/null' (files modified in the last 3 days).

For persistence analysis: 'crontab -l', 'cat /etc/crontab', 'ls /etc/cron.{d,daily,hourly,monthly,weekly}/', 'systemctl list-units --type=service', 'find /etc/systemd -name "*.service" -newer /etc/os-release'. For integrity: 'rpm -Va 2>/dev/null | grep "^..5"' (altered MD5 hashes on RPM-based) or 'debsums -c 2>/dev/null' (Debian-based). Run 'rkhunter --update && rkhunter --check' and 'chkrootkit' for known rootkits.

Assess your company for free

See in minutes what is already exposed from your business.

Decripte's free Threat Management plan maps vulnerabilities, monitors threats and shows leaked credentials — no credit card and no technical team.

Start free now

Diagnosis on Windows: native tools and Sysinternals

On Windows Server, forensic diagnosis begins with Event Viewer (eventvwr.msc), focusing on the Security Log. The critical Event IDs for intrusion investigation are: 4624 (successful login — check Logon Type 3 for network and Type 10 for RemoteInteractive/RDP), 4625 (login failure — identify brute force), 4720 (account created), 4732/4728 (added to a privileged group), 7045 (service installed), 4698 (scheduled task created), 1102 (audit log cleared) and 4688 (process created, requires audit policy enabled).

Essential tools: 'netstat -anob' in CMD as administrator (connections with PID and executable name), Sysinternals Autoruns (all system persistence points on one screen), Sysinternals Process Explorer (processes with full paths, digital signatures and integrated VirusTotal lookup), 'schtasks /query /fo LIST /v' (detailed scheduled tasks), 'net user' and 'net localgroup Administrators' (local accounts and administrators).

For malware analysis on Windows: check services with suspicious paths ('sc query type= all state= all'), inspect the registry under HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run and its variants, look for malicious DLLs with Process Explorer (DLL path column) and run Windows Defender Offline Scan for detection without interference from active malware. The Microsoft Safety Scanner and MSERT are useful for additional scanning in compromised environments.

The malware types most often found on servers

Webshells are script files (PHP, ASPX, JSP, Python) planted on web servers that allow remote command execution via an HTTP request. They are practically invisible to end users and often go unnoticed for years. Identify them by searching for dangerous functions in recently modified files: 'grep -rn "eval(base64_decode\|system(\|exec(\|passthru(\|shell_exec(" /var/www/ --include="*.php"'. Webshells are classified as technique T1505.003 in MITRE ATT&CK.

Cryptominers (technique T1496 — Resource Hijacking) are deployed to mine cryptocurrency using the victim's computing power, generating infrastructure costs and performance degradation without destroying data. They are detected by high CPU usage, processes with random names or names that mimic system processes, and connections to known mining pools (pool.minexmr.com, xmrpool.eu, supportxmr.com).

Backdoors and RATs (Remote Access Trojans) establish persistent communication channels with the attacker's C2 infrastructure. They can operate in reverse mode (the server initiates the outbound connection to evade firewall blocking), use legitimate protocols such as HTTPS, DNS or ICMP for tunneling, and survive reboots via multiple simultaneous persistence mechanisms. On Linux, Diamorphine and Reptile are common rootkits; on Windows, Cobalt Strike Beacon and Meterpreter are tools frequently found in forensic investigations.

Evidence preservation and chain of custody

Proper evidence preservation is what separates a professional incident response from a simple cleanup. According to NIST SP 800-86 (Guide to Integrating Forensic Techniques into Incident Response), evidence should be collected in order of volatility (RAM > processes > connections > disk), with a cryptographic hash (SHA-256) calculated immediately after each collection to guarantee integrity.

On Linux, create a memory dump with 'sudo avml /mnt/evidencias/mem.lime' or use the LiME module (Linux Memory Extractor). Take disk snapshots before any intervention: 'sudo dd if=/dev/sda bs=64K conv=noerror,sync | gzip > /mnt/evidencias/disk_$(date +%Y%m%d_%H%M%S).img.gz'. Record the hash: 'sha256sum /mnt/evidencias/disk_*.img.gz > /mnt/evidencias/hashes.txt'. On Windows, use FTK Imager (free) or Magnet ACQUIRE for bit-for-bit forensic images.

The chain-of-custody documentation must record: who collected it, when (date/time in UTC), from which system (hostname, IP, serial number), which tool was used, where the evidence was stored and who had access to it. Without a proper chain of custody, the evidence has no legal validity and cannot be used in judicial proceedings, police reports or regulatory notifications to the ANPD.

What to do after confirming the compromise

After confirming the incident, follow the structured process in NIST SP 800-61r2: Containment (isolate the system), Eradication (eliminate the root cause), Recovery (restore service with assurances) and Lessons Learned (post-incident). Containment was already done with the network isolation. The correct eradication for compromised servers is a full reprovisioning — never just removing the visible malware.

Secure reprovisioning: provision a new server from a verified golden image, restore data from validated backups (predating the compromise), apply all available security patches before going into production, rotate every credential (passwords, SSH keys, certificates, API tokens) that had access to the compromised server, and enable active monitoring (EDR, SIEM, auditd) before going live.

If data subjects' personal data was or may have been compromised, the LGPD (Law 13.709/2018), in its Art. 48, requires notification to the ANPD and to the affected data subjects within a reasonable time — interpreted by the regulator as 72 hours for the initial notification. For companies subject to PCI-DSS (card data), SOC 2 or ISO 27001, notification and documentation obligations are even more specific. Engage privacy-specialized legal counsel alongside the IR team.

Key terms

IoC (Indicator of Compromise)
A digital artifact that indicates, with high probability, that a system has been compromised. IoCs include IP addresses and domains associated with attacker infrastructure, cryptographic hashes (MD5, SHA-256) of malicious files, Windows registry keys created by malware, names of processes and services associated with known attack tools, and network traffic patterns characteristic of C2 communication. IoCs are shared between organizations via Threat Intelligence platforms such as MISP, OTX (AlienVault) and STIX/TAXII feeds for proactive detection in other environments.
Backdoor
A hidden mechanism installed by the attacker to maintain persistent access to the compromised system, independent of the original credentials and of any fixes to the vulnerabilities that were exploited. Backdoors can be webshells, tampered system services, hidden user accounts, kernel modules (rootkits), unauthorized SSH keys or modifications to operating-system binaries. The goal is to ensure the attacker can return to the system even after the victim detects and tries to remediate the incident, without needing to re-exploit the initial vulnerability.
Persistence
The set of techniques attackers use to ensure their access to a system survives reboots, credential changes and removal attempts. In MITRE ATT&CK, the tactic TA0003 (Persistence) groups dozens of specific techniques for Linux and Windows. Common mechanisms include: scheduled tasks (cron on Linux, Scheduled Tasks on Windows), malicious system services, modification of startup scripts (rc.local, shell profiles), Windows registry keys (Run, RunOnce), DLL hijacking, and deployment of kernel-level rootkits. Identifying and eliminating ALL persistence mechanisms is fundamental to effective eradication — leaving a single backdoor active makes the server vulnerable to the same attacker.
Cryptojacking
An attack in which the victim's server is secretly used to mine cryptocurrency (usually Monero/XMR, because it resists tracing) for the attacker's benefit. The most obvious sign is abnormally high CPU usage (often 70-100%) with no justified workload. Beyond the processing consumption, cryptojacking increases electricity and cloud infrastructure costs, degrades the performance of legitimate applications and can cause hardware failures from overheating. It is classified as technique T1496 (Resource Hijacking) in MITRE ATT&CK and is one of the most prevalent attack types on Linux servers exposed to the internet without active monitoring.

Frequently asked questions

What is the first step when you suspect a server was hacked?

The first step is to capture volatile evidence — especially RAM — before any other action. Rebooting or shutting down the server wipes data critical to the investigation. On Linux, use 'avml' or 'dd' for a memory dump; on Windows, use Magnet RAM Capture or WinPmem. Next, isolate the server from the network to prevent further exfiltration and communication with the attacker's C2 infrastructure.

How can I tell if a Linux server was compromised without external tools?

Using the operating system's own native tools: 'ps auxf' to see all processes and their hierarchy, 'ss -tulnp' for open ports with PID, 'last' and 'lastb' for login history, 'cat /etc/passwd' to check accounts with a shell, 'find /tmp /var/tmp /dev/shm -type f -executable' for suspicious binaries, and 'crontab -l' for unauthorized scheduled tasks. Any anomaly in this data is grounds for a deeper investigation.

Should I reinstall the server or can I just remove the malware?

Always reprovision the server from a clean, verified image. Removing only the visible malware is unsafe because professional attackers deploy multiple persistence mechanisms, backdoors hidden in kernel modules (rootkits) and may have altered system binaries imperceptibly. NIST SP 800-61r2 is explicit: reliable eradication on compromised systems requires a full reprovisioning, not manual cleanup.

How do I identify a webshell on a web server?

Look for recently modified script files in the web server's public folders that contain command-execution functions. On Linux with PHP: 'find /var/www -name "*.php" -newer /etc/passwd -ls' for recent files, and 'grep -rn "eval(base64_decode\|system(\|exec(\|passthru(" /var/www/' for dangerous functions. Webshells often have names that mimic legitimate files (wp-config.php.bak, .htaccess.php, admin_old.php) and may be obfuscated with base64 or XOR to evade detection by simple strings.

What is MITRE ATT&CK and why is it relevant to server diagnosis?

MITRE ATT&CK is a globally recognized knowledge framework that catalogs the tactics, techniques and procedures (TTPs) used by real attackers in production environments. For server diagnosis, it is relevant because it lets you map each indicator found to a specific technique — for example, T1136 (Create Account), T1053 (Scheduled Task/Job), T1070 (Indicator Removal), T1505.003 (Web Shell). This mapping guides the investigation, helps predict what else the attacker may have done, and standardizes communication with technical and executive stakeholders.

When should I notify the ANPD about a hacked server?

The LGPD (Art. 48) requires notification to the ANPD whenever there is a security incident that may cause relevant risk or harm to personal data subjects. The regulatory deadline interpreted by the ANPD is up to 72 hours for the initial notification after becoming aware of the incident. The notification must state: the type of data affected, the estimated number of data subjects, the containment measures adopted and the DPO (Data Protection Officer) contact. Servers that process the personal data of customers, employees or partners are subject to this obligation.

What is the difference between an incident and an investigation in the context of incident response?

An incident is a confirmed, isolated security event — a compromised server, a breached account, detected malware. An investigation is a cross-correlation process that seeks the full scope of the compromise: which other systems were affected, what the initial vector was, how long the attacker has been on the network, which data was accessed or exfiltrated. Forensic investigation turns data from multiple incidents into actionable intelligence, identifying the attacker's full campaign rather than just a single point of compromise.

Security for businesses

Decripte protects companies of every size — from solo founders to Enterprise.

A full platform and services: threat management, 24x7 SOC, incident response, pentest and compliance. Start free and see what has already leaked from your business.