Skip to content
Training ⑤

Community Anomaly Detection Monitor History in Serious Life?


As I allude right here, my extensive-held impression is that no accurate anomaly-based community IDS (NIDS) has at any time been effective commercially and/or operationally. There have been some bits of results, to be guaranteed (“OMG WE CAN DETECT PORTSCANS!!!”), but in whole, they (IMHO) really do not really measure up to Good results of the strategy.

In light-weight of this viewpoint, in this article is a exciting issue: do you assume the present-day generation of machine learning training (ML) – and “AI”-dependent (why is AI in offers?) methods will perform improved? Be aware that I am aiming at a genuinely, seriously lower bar: will they get the job done superior than – for each the previously mentioned statement – not at all? But my definition of “work” features “work in today’s messy and evolving genuine everyday living networks.”

This is truly a more durable issue than it appears. Of course, ML and “AI” aficionados (who, as I am hearing, are frequently saner in contrast to the blockchain training sorts … these are extra akin to clowns, definitely) would claim that of course “now with ML, points are thoroughly different”,because cyber AI” and “subsequent upcoming up coming technology deep learning training just performs.”

On the other hand, some of the rumors we are listening to mention that in noisy, flat, poorly managed networks anomaly detection devolves to … no, truly! … to signatures and fixed activity thresholds wherever human beings write regulations about what is lousy and/or not good.

Before we delve into this, let us consider about the which means of the term ANOMALY. In the previous, “anomaly-based” was about foolish TCP stack protocol anomalies and other “broken packets.” Nowadays it appears that the expression “anomaly” applies to mathematical anomalies in extended-term action styles – and not simply packets like in the 1990s.

So, will it work? This can’t really be answered devoid of inquiring “work to detect what?”

Let’s go as a result of a few illustrations we are listening to about:

  • C2/C&C relationship from malware to an Unidentified [for known, signatures and TI work well, no need to ML it] piece of attacker infrastructure – this was reported to operate by some individuals, and it is not a stretch to think about that anomaly detection can do the job below, at the very least some of the time
  • Connection to some malicious area [UNKNOWN to be bad at detection time, see above] – DGA area detection is now baby’s 1st ML, so it does function [with some “false positives”, but then again, this is a separate question]
  • Inner recon these as a port scan – it will work, but then again, this is probably the only detail in which the old programs also worked [but with false alarms too]
  • Stolen info exfiltration by an attacker – we’ve listened to some noises that it may perhaps perform, but then once more – we’ve listened to the similar about DLP. IMHO, the jury is nevertheless out on this one… Let us say I feel anomaly detection could detect some exfiltration some of the time with some quantity of “false positives” and other “non-actionables”
  • Lateral movement by the attacker – the identical as higher than, IMHO, the jury is continue to out on this one and how successful it can be in genuine existence. I’d say we’ve read illustrations where it labored, and some where by it was as well noisy to be beneficial or failed outright.

Aside from that, I have seem to be some naïve makes an attempt to use supervised ML to prepare devices to master superior/undesirable visitors in normal. IMHO, this is a total missing trigger. It worked brilliantly for binaries (pioneered by “Vendor C”, for case in point), but IMHO this is 100% hopeless for general network targeted visitors.

Ultimately, if the earlier mentioned detection rewards do not materialize for you, we are back again in the “dead packet storage” land (albeit with metadata, not packets).

Posts relevant to this investigate:

Group: detection  monitoring  network  network-forensics  nta  security