Archive for the ‘non-PE threats’ Category

Machine learning vs. social engineering

Machine learning is a key driver in the constant evolution of security technologies at Microsoft. Machine learning allows Microsoft 365 to scale next-gen protection capabilities and enhance cloud-based, real-time blocking of new and unknown threats. Just in the last few months, machine learning has helped us to protect hundreds of thousands of customers against ransomware, banking Trojan, and coin miner malware outbreaks.

But how does machine learning stack up against social engineering attacks?

Social engineering gives cybercriminals a way to get into systems and slip through defenses. Security investments, including the integration of advanced threat protection services in Windows, Office 365, and Enterprise Mobility + Security into Microsoft 365, have significantly raised the cost of attacks. The hardening of Windows 10 and Windows 10 in S mode, the advancement of browser security in Microsoft Edge, and the integrated stack of endpoint protection platform (EPP) and endpoint detection and response (EDR) capabilities in Windows Defender Advanced Threat Protection (Windows Defender ATP) further raise the bar in security. Attackers intent on overcoming these defenses to compromise devices are increasingly reliant on social engineering, banking on the susceptibility of users to open the gate to their devices.

Modern social engineering attacks use non-portable executable (PE) files like malicious scripts and macro-laced documents, typically in combination with social engineering lures. Every month, Windows Defender AV detects non-PE threats on over 10 million machines. These threats may be delivered as email attachments, through drive-by web downloads, removable drives, browser exploits, etc. The most common non-PE threat file types are JavaScript and VBScript.

Figure 1. Ten most prevalent non-PE threat file types encountered by Windows Defender AV

Non-PE threats are typically used as intermediary downloaders designed to deliver more dangerous executable malware payloads. Due to their flexibility, non-PE files are also used in various stages of the attack chain, including lateral movement and establishing fileless persistence. Machine learning allows us to scale protection against these threats in real-time, often protecting the first victim (patient zero).

Catching social engineering campaigns big and small

In mid-May, a small-scale, targeted spam campaign started distributing spear phishing emails that spoofed a landscaping business in Calgary, Canada. The attack was observed targeting less than 100 machines, mostly located in Canada. The spear phishing emails asked target victims to review an attached PDF document.

When opened, the PDF document presents itself as a secure document that requires action a very common social engineering technique used in enterprise phishing attacks. To view the supposed secure document, the target victim is instructed to click a link within the PDF, which opens a malicious website with a sign-in screen that asks for enterprise credentials.

Phished credentials can then be used for further attacks, including CEO fraud, additional spam campaigns, or remote access to the network for data theft or ransomware. Our machine learning blocked the PDF file as malware (Trojan:Script/Cloxer.A!cl) from the get-go, helping prevent the attack from succeeding.

Figure 2. Phishing email campaign with PDF attachment

Beyond targeted credential phishing attacks, we commonly see large-scale malware campaigns that use emails with archive attachments containing malicious VBScript or JavaScript files. These emails typically masquerade as an outstanding invoice, package delivery, or parking ticket, and instruct targets of the attack to refer to the attachment for more details. If the target opens the archive and runs the script, the malware typically downloads and runs further threats like ransomware or coin miners.

Figure 3. Typical social engineering email campaign with an archive attachment containing a malicious script

Malware campaigns like these, whether limited and targeted or large-scale and random, occur frequently. Attackers go to great lengths to avoid detection by heavily obfuscating code and modifying their attack code for each spam wave. Traditional methods of manually writing signatures identifying patterns in malware cannot effectively stop these attacks. The power of machine learning is that it is scalable and can be powerful enough to detect noisy, massive campaigns, but also specific enough to detect targeted attacks with very few signals. This flexibility means that we can stop a wide range of modern attacks automatically at the onset.

Machine learning models zero in on non-executable file types

To fight social engineering attacks, we build and train specialized machine learning models that are designed for specific file types.

Building high-quality specialized models requires good features for describing each file. For each file type, the full contents of hundreds of thousands of files are analyzed using large-scale distributed computing. Using machine learning, the best features that describe the content of each file type are selected. These features are deployed to the Windows Defender AV client to assist in describing the content of each file to machine learning models.

In addition to these ML-learned features, the models leverage expert researcher-created features and other useful file metadata to describe content. Because these ML models are trained for specific file types, they can zone in on the metadata of these file types.

Figure 4. Specialized file type-specific client ML models are paired with heavier cloud ML models to classify and protect against malicious script files in real-time

When the Windows Defender AV client encounters an unknown file, lightweight local ML models search for suspicious characteristics in the files features. Metadata for suspicious files are sent to the cloud protection service, where an array of bigger ML classifiers evaluate the file in real-time.

In both the client and the cloud, specialized file-type ML classifiers add to generic ML models to create multiple layers of classifiers that detect a wide range of malicious behavior. In the backend, deep-learning neural network models identify malicious scripts based on their full file content and behavior during detonation in a controlled sandbox. If a file is determined malicious, it is not allowed to run, preventing infection at the onset.

File type-specific ML classifiers are part of metadata-based ML models in the Windows Defender AV cloud protection service, which can make a verdict on suspicious files within a fraction of a second.

Figure 5. Layered machine learning models in Windows Defender ATP

File type-specific ML classifiers are also leveraged by ensemble models that learn and combine results from the whole array of cloud classifiers. This produces a comprehensive cloud-based machine learning stack that can protect against script-based attacks, including zero-day malware and highly targeted attacks. For example, the targeted phishing attack in mid-May was caught by a specialized PDF client-side machine learning model, as well as several cloud-based machine learning models, protecting customers in real-time.

Microsoft 365 threat protection powered by artificial intelligence and data sharing

Social engineering attacks that use non-portable executable (PE) threats are pervasive in todays threat landscape; the impact of combating these threats through machine learning is far-reaching.

Windows Defender AV combines local machine learning models, behavior-based detection algorithms, generics, and heuristics with a detonation system and powerful ML models in the cloud to provide real-time protection against polymorphic malware. Expert input from researchers, advanced technologies like Antimalware Scan Interface (AMSI), and rich intelligence from the Microsoft Intelligent Security Graph continue to enhance next-generation endpoint protection platform (EPP) capabilities in Windows Defender Advanced Threat Protection.

In addition to antivirus, components of Windows Defender ATPs interconnected security technologies defend against the multiple elements of social engineering attacks. Windows Defender SmartScreen in Microsoft Edge (also now available as a Google Chrome extension) blocks access to malicious URLs, such as those found in social engineering emails and documents. Network protection blocks malicious network communications, including those made by malicious scripts to download payloads. Attack surface reduction rules in Windows Defender Exploit Guard block Office-, script-, and email-based threats used in social engineering attacks. On the other hand, Windows Defender Application Control can block the installation of untrusted applications, including malware payloads of intermediary downloaders. These security solutions protect Windows 10 and Windows 10 in S mode from social engineering attacks.

Further, Windows Defender ATP endpoint detection and response (EDR) uses the power of machine learning and AMSI to unearth script-based attacks that live off the land. Windows Defender ATP allows security operations teams to detect and mitigate breaches and cyberattacks using advanced analytics and a rich detection library. With the April 2018 Update, automated investigation and advance hunting capabilities further enhance Windows Defender ATP. Sign up for a free trial.

Machine learning also powers Office 365 Advanced Threat Protection to detect non-PE attachments in social engineering spam campaigns that distribute malware or steal user credentials. This enhances the Office 365 ATP comprehensive and multi-layered solution to protect mailboxes, files, online storage, and applications against threats.

These and other technologies power Microsoft 365 threat protection to defend the modern workplace. In Windows 10 April 2018 Update, we enhanced signal sharing across advanced threat protection services in Windows, Office 365, and Enterprise Mobility + Security through the Microsoft Intelligent Security Graph. This integration enables these technologies to automatically update protection and detection and orchestrate remediation across Microsoft 365.


Gregory Ellison and Geoff McDonald
Windows Defender Research





Talk to us

Questions, concerns, or insights on this story? Join discussions at the Microsoft community and Windows Defender Security Intelligence.

Follow us on Twitter @WDSecurity and Facebook Windows Defender Security Intelligence.

Enhancing Office 365 Advanced Threat Protection with detonation-based heuristics and machine learning

Email, coupled with reliable social engineering techniques, continues to be one of the primary entry points for credential phishing, targeted attacks, and commodity malware like ransomware and, increasingly in the last few months, cryptocurrency miners.

Office 365 Advanced Threat Protection (ATP) uses a comprehensive and multi-layered solution to protect mailboxes, files, online storage, and applications against a wide range of threats. Machine learning technologies, powered by expert input from security researchers, automated systems, and threat intelligence, enable us to build and scale defenses that protect customers against threats in real-time.

Modern email attacks combine sophisticated social engineering techniques with malicious links or non-portable executable (PE) attachments like HTML or document files to distribute malware or steal user credentials. Attackers use non-PE file formats because these can be easily modified, obfuscated, and made polymorphic. These file types allow attackers to constantly tweak email campaigns to try slipping past security defenses. Every month, Office 365 ATP blocks more than 500,000 email messages that use malicious HTML and document files that open a website with malicious content.

Figure 1. Typical email attack chain

Detonation-based heuristics and machine learning

Attackers employ several techniques to evade file-based detection of attachments and blocking of malicious URLs. These techniques include multiple redirections, large dynamic and obfuscated scripts, HTML for tag manipulation, and others.

Office 365 ATP protects customers from unknown email threats in real-time by using intelligent systems that inspect attachments and links for malicious content. These automated systems include a robust detonation platform, heuristics, and machine learning models.

Detonation in controlled environments exposes thousands of signals about a file, including behaviors like dropped and downloaded files, registry manipulation for persistence and storing stolen information, outbound network connections, etc. The volume of detonated threats translate to millions of signals that need to be inspected. To scale protection, we employ machine learning technologies to sort through this massive amount of information and determine a verdict for analyzed files.

Machine learning models examine detonation artifacts along with various signals from the following:

  • Static code analysis
  • File structure anomaly
  • Phish brand impersonation
  • Threat intelligence
  • Anomaly-based heuristic detections from security researchers

Figure 2. Classifying unknown threats using detonation, heuristics, and machine learning

Our machine learning models are trained to find malicious content using hundreds of thousands of samples. These models use raw signals as features with small modifications to allow for grouping signals even when they occur in slightly different contexts. To further enhance detection, some models are built using three-gram models that use raw signals sorted by timestamps recorded during detonation. The three-gram models tend to be more sparse than raw signals, but they can act as mini-signatures that can then be scored. These types of models fill in some of the gaps, resulting in better coverage, with little impact to false positives.

Machine learning can capture and expose even uncommon threat behavior by using several technologies and dynamic featurization. Features like image similarity matching, domain reputation, web content extraction, and others enable machine learning to effectively separate malicious or suspicious behavior from the benign.

Figure 3. Machine learning expands on traditional detection capabilities

Over time, as our systems automatically process and make a verdict on millions of threats, these machine learning models will continue to improve. In the succeeding sections, well describe some interesting malware and phishing campaigns detected recently by Office 365 ATP machine learning models.

Phishing campaigns: Online banking credentials

One of the most common types of phishing attacks use HTML and document files to steal online banking credentials. Gaining access to online bank accounts is one of the easiest ways that attackers can profit from illicit activities.

The email messages typically mimic official correspondence from banks. Phishers have become very good at crafting phishing emails. They can target global banks but also localize email content for local banks.
The HTML or document attachment are designed to look like legitimate sign-in pages or forms. Online banking credentials and other sensitive information entered into these files or websites are sent to attackers. Office 365s machine learning models detect this behavior, among other signals, to determine that such attachments are malicious and block offending email messages.

Figure 4. Sample HTML files that mimic online banking sign in pages. (Click to enlarge)

Phishing campaigns: Cloud storage accounts

Another popular example of phishing campaigns uses HTML or document attachments to steal cloud storage or email account details. The email messages imply that the recipient has received a document hosted in a cloud storage service. In order to supposedly open the said document, the recipient has to enter the cloud storage or email user name and password.

This type of phishing is very rampant because gaining access to either email or cloud storage opens a lot of opportunities for attackers to access sensitive documents or compromise the victims other accounts.

Figure 5. Sample HTML files that pose as cloud storage sign in pages. (Click to enlarge)

Tax-themed phishing and malware attacks

Tax-themed social engineering attacks circulate year-round as cybercriminals take advantage of the different country and region tax schedules. These campaigns use various messages related to tax filing to convincer users to click a link or open an attachment. The social engineering messages may say the recipient is eligible for tax refund, confirm that tax payment has been completed, or declare that payments are overdue, among others.

For example, one campaign intercepted by Office 365 ATP using machine learning implied that the recipient has not completed tax filing and is due for penalty. The campaign targeted taxpayers in Colombia, where tax filing ended in October. The email message aimed to alarm taxpayers by suggesting that they have not filed their taxes.

Figure 6. Tax-themed email campaign targeting taxpayers in Colombia. The subject line translates to: You have been fined for not filing your income tax returns

The attachment is a .rar file containing an HTML file. The HTML file contains the logo of Direccin de Impuestos y Aduanas Nacionales (DIAN), the Colombianes tax and customs organization, and a link to download a file.

Figure 7. Social engineering document with a malicious link

The link points to a shortened URL hxxps://bit[.]ly/2IuYkcv that redirects to hxxp://dianmuiscaingreso[.]com/css/sanci%C3%B3n%20declaracion%20de%20renta.doc, which downloads a malicious document.

Figure 8: Malicious URL information

The malicious document carries a downloader macro code. When opened, Microsoft Word issues a security warning. In the document are instructions to Enable content, which executes the embedded malicious VBA code.

Figure 9: Malicious document with malicious macro code

If the victim falls for this social engineering attack, the macro code downloads and executes a file from hxxp:// The downloaded executable file (despite the file name) is a file injector and password-stealing malware detected by Windows Defender AV as Trojan:Win32/Tiggre!rfn.

Because Office 365 ATP machine learning detects the malicious attachment and blocks the email, the rest of the attack chain is stopped, protecting customers at the onset.

Artificial intelligence in Office 365 ATP

As threats rapidly evolve and become increasingly complex, we continuously invest in expanding capabilities in Office 365 Advanced Threat Protection to secure mailboxes from attacks. Using artificial intelligence and machine learning, Office 365 ATP can constantly scale coverage for unknown and emerging threats in-real time.

Office 365 ATPs machine learning models leverage Microsofts wide network of threat intelligence, as well as seasoned threat experts who have deep understanding of malware, cyberattacks, and attacker motivation, to combat a wide range of attacks.

This enhanced protection from Office 365 ATP contributes to and enriches the integrated Microsoft 365 threat protection, which provides intelligent, integrated, and secure solution for the modern workplace. Microsoft 365 combines the benefits and security technologies of Office 365, Windows, and Enterprise Mobility Suite (EMS) platforms.

Office 365 ATP also shares threat signals to the Microsoft Intelligent Security Graph, which uses advanced analytics to link threat intelligence and security signals across Office 365, the Windows Defender ATP stack of defenses, and other sensors. For example, when a malicious file is detected by Office 365 ATP, that threat can also be blocked on endpoints protected by Windows Defender ATP and vice versa. Connecting security data and systems allows Microsoft security technologies like Office 365 ATP to continuously improve threat protection, detection, and response.



Office 365 Threat Research