Sigma Rules 101: Metadata and Logsources

Posted by:

If you are reading this blog post via a 3rd party source it is very likely that many parts of it will not render correctly. Please view the post on signalscorps.com for the full interactive viewing experience.

In this post I will introduce the structure of Sigma Rules and how to write attributes to support detection content

I would be willing to bet that for every hundred readers of this post, there will be more than 10 SIEM or XDR tools being used among you all.

This is problematic because it add a large amount of friction to the sharing detection content.

What is needed is a detection rule standard. A way to write a query once and use it everywhere, on any system.

That is the ambitious aim of the Sigma project.

Sigma has been around for about three years. Though in the last year it has seen a marked increase in adoption.

Sigma is for log files what Snort is for network traffic and YARA is for files.

Sigma Github Repo

The Sigma Rule format has been designed to accommodate for conversion into other query languages to match the systems on which they will be used. For example, a Sigma Rule could be translated into Splunk’s query language, SPL, or Google Chronicle’s YARA-L 2.0 queries.

Before jumping into the detail, let me first explain the foundations of Sigma Rules.

The YAML Structure

Sigma Rules are structured in a YAML format.

The Sigma Rule specification documents and describes the available attributes and the values they support.

As I go through this tutorial, I recommend having some example rules open to cross-reference this theory to real world implementations.

The Sigma project maintains a body of public rules in the core Sigma repository here.

When writing Sigma Rules, I find it helpful to think of them as five core sections:

General metadata: Descriptive information about the rule
Versioning information: How to handle and track changes to the rule
Log information: describes the log data on which the detection is meant to be applied to
Detection:
Detection handling

1. General metadata

`title`

A brief title for the rule that should contain what the rules is supposed to detect.

Example;

title: Google Cloud Storage Buckets Modified or Deleted

Required: true
Validation: min. 1 character, max. 256 characters
Full specification

`description`

A short description of the rule and the malicious activity that can be detected

Example;

description: Detects when storage bucket is modified or deleted in Google Cloud.

Required: false
Validation: max. 65,535 characters
Full specification

`author`

Creator of the rule.

Example;

author: Austin Songer @austinsonger`

Required: false
Validation: none
Full specification

`references`

References to the source that the rule was derived from. These could be blog articles, technical papers, presentations or even tweets.

Example;

references:
  - https://cloud.google.com/storage/docs/json_api/v1/buckets

Required: false
Validation: none
Full specification

`status`

Declares the status of the rule from a defined list supported by Sigma.

Example; status: experimental

Required: false
Validation: Either;
- stable: the rule is considered as stable and may be used in production systems or dashboards.
- test: an almost stable rule that possibly could require some fine tuning.
- experimental: an experimental rule that could lead to false results or be noisy, but could also identify interesting events.
- deprecated: the rule is replace or cover by another one. The link is made by the related field.
- unsupported: the rule can not be use in its current state (special correlation log, home-made fields)
Full specification

2. Versioning information

All versioning fields are optional, but I strongly recommend including them all when writing your own rules.

`id`

Sigma rules should be identified by a globally unique identifier in the id attribute. For this purpose random generated UUIDs (version 4) are recommended.

Example;

id: 4d9f2ee2-c903-48ab-b9c1-8c0f474913d0

Required: false
Validation: None (recommended UUID version 4)
Full specification

`date`

The date the rule was created.

Example;

date: 2021/08/14

Required: false
Validation: YYYY/MM/DD
Full specification (currently missing)

`related`

Rule identifiers should change for the following reasons:

Major changes in the rule. E.g. a different rule logic.
Derivation of a new rule from an existing or refinement of a rule in a way that both are kept active.
Merge of rules.

To being able to keep track on relationships between detections, Sigma rules may also contain references to related rule identifiers and the description of the relationships.

Example;

related:
  - id: 08fbc97d-0a2f-491c-ae21-8ffcfd3174e9
  type: derived
  - id: 929a690e-bef0-4204-a928-ef5e620d6fcc
  type: obsoletes

Here the current rule is derived from another rule (id = 08fbc97d-0a2f-491c-ae21-8ffcfd3174e9) and replaced (obsoletes) a rule (id = 929a690e-bef0-4204-a928-ef5e620d6fcc).

Required: false
Validation: Either;
- derived: Rule was derived from the referred rule or rules, which may remain active.
- obsoletes: Rule obsoletes the referred rule or rules, which aren’t used anymore.
- merged: Rule was merged from the referred rules. The rules may be still existing and in use.
- renamed: The rule had previously the referred identifier or identifiers but was renamed
- similar: Use to relate similar rules to each other (e.g. same detection content applied to different log sources, rule that is a modified version of another rule with a different level)
Full specification

`tags`

A Sigma rule can be categorised with tags. Tags can be anything, but I would personally recommend using the predefined Sigma tags (or sending a pull request / creating an issue to the Sigma repo with proposals for new tags).

A tag ultimatley provides more contextual information about the rule.

Tags are namespaced, the dot is used as separator. The three predefined Sigma tags have the following namespaces; attack. (MITRE ATT&CK), car. (MITRE Cyber Analytics Repository), and tlp. (Traffic Light Protocol)

Example;

tags:
  - attack.defense_evasion
  - attack.t1027
  - attack.execution
  - attack.t1059.001
  - car.2016-04-005
  - tlp.amber

Required: false
Validation: lower-case letters, underscores and hyphens only. Dots used to defined the namespace.
Full specification

`level`

The level attribute describes the criticality of a triggered rule which can be useful to a SOC team in prioritising how they handle the detection.

Example;

level: critical

Required: false
Validation: Either;
- informational: Rule is intended for enrichment of events, e.g. by tagging them. No case or alerting should be triggered by such rules because it is expected that a huge amount of events will match these rules.
- low: Notable event but rarely an incident. Low rated events can be relevant in high numbers or combination with others. Immediate reaction shouldn’t be necessary, but a regular review is recommended.
- medium: Relevant event that should be reviewed manually on a more frequent basis.
- high: Relevant event that should trigger an internal alert and requires a prompt review.
- critical: Highly relevant event that indicates an incident. Critical events should be reviewed immediately.
Full specification

`x_date_modified`

In my own Sigma Rules I use our own custom attribute x_date_modified, where a rule is changed, but does not require the rule id to be changed (see; related Property)

Example;

date: 2021/08/14

Required: custom
Validation: YYYY/MM/DD

`x_spec version`

As Sigma is a quickly evolving standard, I also use our own custom attribute x_spec version to track the version of the specification used using the GitHub commit ID (found here).

Required: custom
Validation: 6 characters [a-z1-9]

3. Log information

Attributes

The logsource attribute describes the log data on which the detection is meant to be applied to and has a number of sub-attributes to allow it to be very specific;

category:
- used to select all log files written by a certain group of products, like firewalls or web server logs.
  - e.g. firewall, web, antivirus
product:
- used to select all log outputs of a certain product, e.g. all Windows Eventlog types including “Security”, “System”, “Application” and the new log types like “AppLocker” and “Windows Defender”.
  - e.g. windows, apache
service:
- used to select only a subset of a product’s logs, like the “sshd” on Linux or the “Security” Eventlog on Windows systems.
  - e.g. sshd, applocker
definition
- used to describe the logsource, including some information on the log verbosity level or configurations that have to be applied.
  - e.g. INFO, DEBUG

Read the Log Source specification here.

Let me share an example for Windows Powershell;

logsource:
  product: windows
  service: powershell
  definition: Write-Information

Instead of referring to particular services explicitly, I could point to a specific category of logs (to capture all Windows products):

logsource:
  category: process_creation
  product: windows
  definition: Write-Information

Or I could be very specific, and use all the sub-attributes;

logsource:
  category: process_creation
  product: windows
  service: powershell
  definition: Write-Information

Sigma compiles the logsource sub-attributes using AND statements, so for the last example I am saying the logsource must be; category: process_creation AND product: windows AND service: powershell AND definition: Write-Information.

The more detailed the definition, the more efficient the log search will be (SIEM admins rejoice).

You can also only pass one logsource attribute per rule, and thus by definition Sigma Rules are specific to logsources.

`logsources`

As you start to consider writing your own rules, you will want to know where I obtained the values for these attributes from (e.g. category: process_creation). In order to understand that its important to know how these fields are used downstream.

For fields with existing field-mappings, you can use the mapped field name.

[These are defined in logsources configuration files found in the main Sigma repository, under ./tools/config.

Take the generic windows-audit.yml that covers;

title: Conversion for Windows Native Auditing Events

Inside which there is a list of logsources for Windows auditing events;

logsources:
  process_creation:
    category: process_creation
    product: windows
    conditions:
      EventID: 4688
      rewrite:
      product: windows
      service: security

The first logsource covers category: process_creation on product: windows.

The configuration also maps logsources to additional conditions and maps Sigma field names to environment-specific ones.

If you need to create a new logsources entry, it is good practice to submit a pull request to the repository with the appropriate mappings in all the current config files for the different backends (QRadar, Splunk etc.).

For example, you can see in the splunk-windows config, Powershell sources;

  windows-powershell:
    product: windows
    service: powershell
    conditions:
      source: 'WinEventLog:Microsoft-Windows-PowerShell/Operational'

And in the Elastic winlogbeat config;

  windows-powershell:
    product: windows
    service: powershell
    conditions:
      winlog.channel: 'Microsoft-Windows-PowerShell/Operational'

The config files are used by Sigmac and pySigma to map a specific log source field in the downstream product (e.g. Splunk source= or Winlogbeat winlog.channel=) to the attributes that are used in your rule.

I will talk in detail about these tools and config files in a future tutorial, but what this means for creation of new rules is that; you should first refer to these config files to check if the logsource you intend to use already exists.

A shortcut is also to take a look at existing Sigma Rules to see if a rule exists with category/product/service defined for which you are writing a rule for. Keep in mind that an author can add any strings in these sub-attributes values, therefore so make sure you double check the author has added existing config entries too.

In our opinion this is one of the areas that really needs further standardisation with a wider list of defined categories, products, and their services.

4 and 5. Detections

Now the metadata, versioning, and logsource sections of the rule are understood and can be written, in the next tutorial I will cover the core section of the rule – the detection and detection information.

Sigma Certification (Virtual and In Person)

The content used in this post is a small subset of our full training material used in our Sigma training.

If you want to join a select group of certified Sigma professionals, subscribe to our newsletter below to be notified of new course dates.

Discuss this post

JOIN US