Sigma Rules 102: Writing Detections (Search Identifiers)

Posted by:

If you are reading this blog post via a 3rd party source it is very likely that many parts of it will not render correctly. Please view the post on signalscorps.com for the full interactive viewing experience.

In this post I will dive into the details of Sigma detection logic and how to start writing more advanced rules.

The Sigma Rule format is very flexible and can be used for detections on any type of log file.

Inside a Sigma Rules, the detection attribute is where the actual logic for when the rule will be triggered.

Here is the detection specification defined by Sigma, in this post I will try and illustrate the main concepts with clear examples.

The detection section contains a set of sub-attributes that represent searches on log data and how they should be evaluated:

Search Identifier (aka selections): What you actually wish to select/search from the log data
Conditions (condition): How should the Selections or filters are to be evaluated

Let me show you how these work using a few examples.

Search Identifiers

A Search Identifier definition can consist of two different data structures - Lists and/or Maps.

Lists are the simplest way to define a Search Identifier. They contain strings that are applied to the full log message and are linked with a logical OR statement.

detection:
  search_identifier_1:
    - EVILSERVICE
    - svchost.exe -n evil
  condition: search_identifier_1

In this example, search_identifier_1 matches on EVILSERVICE OR svchost.exe -n evil in a log line.

Maps consist of key:value pairs, in which the key is a field in the log data and the value a string or integer value.

They are different to Lists, because Lists do not allow you to define the key (log field) on which you want to match.

detection:
  search_identifier_1:
    - EventLog: Security
  condition: search_identifier_1

In this example I am matching on any events where the EventLog EQUALS Security.

This is a very simplistic example, in reality I would want to use more than one set of key:value pairs in the rule to avoid false positives.

To support this use-case, a Map can contain multiple elements. All elements of a Map are joined with a logical AND.

detection:
  search_identifier_1:
    - EventLog: Security
      EventID: 4769
      TicketOptions: '0x40810000'
      TicketEncryption: '0x17'
  condition: search_identifier_1

Logically this example reads; (Eventlog EQUALS Security) AND (EventID EQUALS 4679) AND (TicketOptions EQUALS 0x40810000) AND (TicketEncryption EQUALS 0x17).

Like Lists, you can also create Lists of Maps. Lists of Maps are joined with a logical OR.

detection:
  search_identifier_1:
    - EventLog: Security
      EventID:
      - 517
      - 1102
  condition: search_identifier_1

In this example, the elements of the Map are EventLog and EventID. Inside the EventID field there is a List of Maps.

Logically this translates to; (EventLog EQUALS Security) AND (EventID EQUALS 4679 OR EventID EQUALS 1102)

There are special field values that can be used inside both Lists and Maps;

An empty value is defined with ''
A null value is defined with null

For example, if I wanted to search on log lines where the EventID field value was null I could use:

detection:
  search_identifier_1:
    - EventID: null
  condition: search_identifier_1

Map field names

I have jumped ahead slightly in this tutorial. When you sit down to write detection rules, you might be wondering how to get the field names used inside the Search Identifiers including EventLog, EventID, TicketEncryption and TicketOptions.

For new fields you should use them as they appear in the log source and strip all spaces (be careful not to use localised field, i.e. non default, names if they have been set).

Let me use an example to demonstrate.

Windows Security Log Events

For those that are not familiar with Windows logs, an event ID is assigned to each log to describe what action is being logged. Here is a nice reference of Windows Audit log IDs.

Take this sample Windows Audit log (Event ID = 4688);

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
 <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" /> 
 <EventID>4688</EventID> 
 <Version>2</Version> 
 <Level>0</Level> 
 <Task>13312</Task> 
 <Opcode>0</Opcode> 
 <Keywords>0x8020000000000000</Keywords> 
 <TimeCreated SystemTime="2015-11-12T02:24:52.377352500Z" /> 
 <EventRecordID>2814</EventRecordID> 
 <Correlation /> 
 <Execution ProcessID="4" ThreadID="400" /> 
 <Channel>Security</Channel> 
 <Computer>WIN-GG82ULGC9GO.contoso.local</Computer> 
 <Security /> 
 </System>
- <EventData>
 <Data Name="SubjectUserSid">S-1-5-18</Data> 
 <Data Name="SubjectUserName">WIN-GG82ULGC9GO$</Data> 
 <Data Name="SubjectDomainName">CONTOSO</Data> 
 <Data Name="SubjectLogonId">0x3e7</Data> 
 <Data Name="NewProcessId">0x2bc</Data> 
 <Data Name="NewProcessName">C:\\Windows\\System32\\rundll32.exe</Data> 
 <Data Name="TokenElevationType">%%1938</Data> 
 <Data Name="ProcessId">0xe74</Data> 
 <Data Name="CommandLine" /> 
 <Data Name="TargetUserSid">S-1-5-21-1377283216-344919071-3415362939-1104</Data> 
 <Data Name="TargetUserName">dadmin</Data> 
 <Data Name="TargetDomainName">CONTOSO</Data> 
 <Data Name="TargetLogonId">0x4a5af0</Data> 
 <Data Name="ParentProcessName">C:\\Windows\\explorer.exe</Data> 
 <Data Name="MandatoryLabel">S-1-16-8192</Data> 
 </EventData>
</Event>

Here some fields are defined as attributes of the XML tags (in the <System> header of the events).

The EventID is captured in this section;

 <EventID>4688</EventID>

Here I can use the field names as shown in the Sigma Rule (EventID:)

 <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" /> 

Should be used as fields Provider_Name: and Provider_Guid:.

Inside the <EventData> section, you will see fields defined like so;

 <Data Name="TargetUserName">dadmin</Data> 

Here the field name used in the Sigma Rule should be TargetUserName:.

To illustrate this, here is a Sigma Rule that would match to this log line;

detection:
  search_identifier_1:
    - EventID: 4688
      Provider_Name: 'Microsoft-Windows-Security-Auditing'
      Provider_Guid: '{54849625-5478-4994-A5BA-3E3B0328C30D}'
      TargetUserName: 'dadmin'
  condition: search_identifier_1

Some fields are already mapped and defined in generic configuration files under fieldmappings. These mappings are useful because in many cases the same field might be named in different ways in a log for the same product but on different versions (for example, Windows log field names can change depending on version of Windows in use).

Take the bottom of the windows-audit.yml configuration file.

fieldmappings:
  Image: NewProcessName
  ParentImage: ParentProcessName
  Details: NewValue
  #CommandLine: ProcessCommandLine  # No need to map, as real name of ProcessCommandLine is already CommandLine
  LogonId: SubjectLogonId

This predefined mapping instructs me to use certain field names. For example, if my log contains the field name NewProcessName I should use Image in the Sigma Rule (not NewProcessName).

In the sample Windows Log above you can see the NewProcessName field defined;

 <Data Name="NewProcessName">C:\\Windows\\System32\\rundll32.exe</Data> 

However, to create a Sigma Rule that matches to this event I should use the field name Image as defined by the fieldmappings. Here is an example rule to demonstate;

detection:
  search_identifier_1:
    - image: 'C:\\Windows\\System32\\rundll32.exe'
  condition: search_identifier_1

Value Modifiers

Maps can be made more advanced through the use of Value Modifiers.

There are two types of Value Modifiers:

Transformation Modifier: transform values into different values. Furthermore, this type of modifier is also able to change the logical operation between values.
Type Modifier: change the type of a value. The value itself might also be changed by such a modifier, but the main purpose is to tell the backend that a value should be handled differently by the backend (e.g. it should be treated as regular expression when the re modifier is used). More on backends in a later tutorial.

All Value Modifiers are appended after the field name with a | character.

1. Transformation Modifier

For example, I could use the Transformation Modifier contains which puts * wildcards around the values, such that the value is matched anywhere in the field. Here is an example;

detection:
  search_identifier_1:
    CommandLine|contains:
      - DumpCreds
      - invoke-mimikatz
  condition: search_identifier_1

Here, the Search Identifier matches the CommandLine field in the log data and uses the Transformation Modifier contains in order to check if the keywords DumpCreds OR invoke-mimikatz are present in the field.

For example this detection would match on CommandLine="Has detected DumpCreds" OR CommandLine="DumpCreds" OR CommandLine="now invoke-mimikatz"

If I did not use the contains Value Modifier like so;

detection:
  search_identifier_1:
    CommandLine:
      - DumpCreds
      - invoke-mimikatz
  condition: search_identifier_1

Now only an exact match for the CommandLine field would match. This would be either CommandLine="DumpCreds" OR CommandLine="invoke-mimikatz". CommandLine="Has detected DumpCreds" would not match.

You might want to use a more specific Transformation Modifier, like startswith OR endswith.

detection:
  search_identifier_1:
    CommandLine|startswith:
      - DumpCreds
      - invoke-mimikatz
  condition: search_identifier_1

Here the CommandLine field in the log line must start with either DumpCreds or invoke-mimikatz.

The all Transformation Modifier can also prove very useful on occasion. As noted, Lists of values are treated by default using the logical OR statement. This modifier changes this to AND.

detection:
  search_identifier_1:
    CommandLine|all:
      - DumpCreds
      - invoke-mimikatz
  condition: search_identifier_1

In this example, I am now saying the CommandLine field in the log line must have both DumpCreds AND invoke-mimikatz in its value.

Transformation Modifiers can can also be chained, (e.g. fieldname|mod1|mod2: value. The value modifiers are applied in the given order to the value.

Here is an example to demonstrate;

detection:
  search_identifier_1:
    CommandLine|all|startswith:
      - DumpCreds
      - invoke-mimikatz
  condition: search_identifier_1

This example logically reads the CommandLine field in the log line must have both DumpCreds AND invoke-mimikatz at the start of the field value to trigger a match.

There are a few other Transformation Modifiers in the current Sigma specification not covered here. You can find them here.

2. Type Modifier

Sometimes Transformation Modifiers do not quite suit what you are trying to achieve, particularly with more complex detections.

Currently only one type of Type Modifier exists for Regular Expressions (re). Here is an example of it being used;

detection:
  search_identifier_1:
    - CommandLine|re: '\$PSHome\[\s*\d{1,3}\s*\]\s*\+\s*\$PSHome\['
    - CommandLine|re: '\$ShellId\[\s*\d{1,3}\s*\]\s*\+\s*\$ShellId\['
    - CommandLine|re: '\$env:Public\[\s*\d{1,3}\s*\]\s*\+\s*\$env:Public\['
  condition: search_identifier_1

Here the log line CommandLine field values must match at least one of the Regular Expressions defined.

You might be tempted to use the Regular Expressions Type Modifier a lot, though avoid it where possible. At the time of writing it is only supported by one backend (the Elasticsearch query string backend (es-qs)) for conversion.

In many cases a Transformation Modifier can achieve the same thing (and is better supported during rule conversion);

detection:
  search_identifier_1:
    - CommandLine|re: '\\payload.*\skeyset'

Can be achieved with the all Transformation Modifier;

detection:
  search_identifier_1:
    CommandLine|contains|all:
      - \payload
      - keyset
  condition: search_identifier_1

A note on backslashes

In my previous two example I used a backslash \.

Backslashes have two functions in Sigma:

Backslash as plain value
Backslash as prefix to escape characters with special meanings: the backslash \ itself, as well as the wildcards * and ?.

Handling the backslash in this way has the advantage that values that contain single backslashes (like the previous examples) can be expressed in a plain way without the need for escaping.

On the other hand, corner cases do require additional escaping.

Naming Search Identifiers

Throughout this tutorial I defined each Search Identifier as search_identifier_N. This was purely for demonstration purposes.

The actual name you give to each Search Identifier does not particularly matter. For example, the previous filter example could be written like so;

detection:
  log_selection:
     EventID: 4738
  log_filter:
     PasswordLastSet: null
  condition: log_selection and not log_filter

More descriptive names can be useful when using multiple Search Identifiers.

Multiple Search Identifiers

In many cases you will also want to use more than one Search Identifier to create a more logical detection.

For example;

detection:
  search_identifier_1:
    CommandLine|contains:
      - DumpCreds
      - invoke-mimikatz
  search_identifier_2:
    CommandLine|contains:
      - rpc
      - token
      - crypto
  search_identifier_3:
    CommandLine|contains:
      - bitcoin
  condition: 1 of them

Here I use three Search Identifiers. The condition sub-attribute defines how they should be treated to trigger a detection.

In this case, one of the three Search Identifiers (1 of them) must be true.

Conditions can be defined using a variety of Operators, all of which I will cover in the next tutorial.

Sigma Certification (Virtual and In Person)

The content used in this post is a small subset of our full training material used in our Sigma training.

If you want to join a select group of certified Sigma professionals, subscribe to our newsletter below to be notified of new course dates.

Discuss this post

JOIN US