When MikroTik released an emergency patch for an SSH authentication bypass vulnerability already being exploited, security teams faced a dilemma: do you apply the patch immediately, or investigate first to see if attackers have already breached your systems?
This isn't just theoretical. Attackers have been adding new accounts to compromised MikroTik devices, meaning patching alone won't remove them. The patch can detect compromise and set a "Flagged" status, but that's detection, not remediation. For vCISO practitioners, guiding organizations through such incidents, the sequence of actions is critical.
The Case for Patching Immediately
The argument for patching first is simple: every moment the vulnerability is open is an opportunity for attackers. Delaying patching to hunt for evidence leaves your systems exposed.
This aligns with the core principle of vulnerability management: reduce attack surface quickly. When exploitation is confirmed, speed is key. Delaying patching for forensics can lead to further compromise.
Operationally, immediate patching simplifies response. Your team applies the update, monitors the "Flagged" status, and investigates flagged systems. You're dealing with a closed vulnerability, reducing complexity and focusing resources on potentially compromised systems.
Resource constraints also support this approach. Most organizations lack dedicated forensics teams. Requiring investigation before patching delays updates or burdens IT staff with tasks they're not trained for, neither of which improves security.
The Case for Investigating Before You Patch
The investigate-first approach focuses on evidence preservation. Patching changes the system state. Attackers might notice the patch, accelerate activities, or destroy evidence. Capturing the pre-patch state might be your only chance to understand the breach.
This treats the incident as a crime scene. You need to document before cleaning up. The MikroTik vulnerability involves attackers adding accounts, so identifying these before they notice your actions is crucial. Patching first might alert them, leading to data loss or further breaches.
Advocates of this approach also highlight the limitations of automated detection. The patch's "Flagged" status is a basic indicator, not a full forensic analysis. It may miss sophisticated attackers. Investigating first means you're not solely relying on vendor detection.
There's also a learning component. Patching immediately prevents your team from developing skills to investigate active compromises. Without this experience, you're dependent on vendor indicators rather than building internal detection capabilities.
Where Practitioners Actually Land
In practice, experienced teams don't universally choose one approach. They base their response on asset criticality and resources.
For internet-facing MikroTik devices with administrative access, the usual answer is to patch immediately. These are high-value targets under attack. The risk of further compromise during investigation outweighs the forensic value of the pre-patch state. You patch, check flagged statuses, and investigate systems identified as potentially compromised.
For internal devices or systems with strong network monitoring, investigating before patching is more viable. You can isolate the device, conduct forensics, and patch once you understand the scope. This works when you have time and expertise.
A hybrid approach involves snapshot forensics. Before patching, capture memory dumps, configuration files, user account lists, and log files. This takes minutes, not hours. You're not doing a full investigation pre-patch, but preserving evidence. Then patch immediately and investigate using the captured data.
Our Take
Patch first, but don't stop there.
The MikroTik scenario is clear: when a vulnerability is actively exploited and the vendor's patch includes compromise detection, close the vulnerability first. The investigation shifts to post-patch verification.
Instead of examining every device for compromise before patching, patch everything and focus on systems flagged by the patch. You're targeting a smaller, higher-probability set.
The critical mistake isn't patching first; it's treating the patch as the incident's end. Organizations that apply the update and move on without ensuring attackers haven't maintained persistence remain compromised.
Your post-patch verification should include reviewing all user accounts created recently, examining SSH access logs for unusual patterns, checking for configuration changes that enable persistence, and monitoring outbound connections from patched devices. These steps don't delay the patch; they ensure the investigation isn't skipped.
For vCISO practitioners, this means ensuring clients understand that patch management and incident response are sequential phases of the same process. You patch to stop the bleeding, then verify the wound is closed.
