If you are reading this blog post via a 3rd party source it is very likely that many parts of it will not render correctly. Please view the post on signalscorps.com for the full interactive viewing experience.
In this post I will show an open-source TAXII Server allowing you to quickly get up and running to share intelligence.
Note: this post is written for OASIS TAXII version 2.1. The concepts discussed are not always correct for earlier versions of OASIS TAXII. This post also talks about some specifics related to the Signal Corps implementation of a TAXII 2.1 Server, namely the exclusive use of STIX 2.1 content.
The last two tutorials showed you how to interact with Collections on TAXII Servers via the TAXII API. If you are an intelligence producer, you will want access to a TAXII Server to start distributing your work.
Many products, especially threat intelligence platforms, typically have built in TAXII Server functionality. However, in this post I wanted to explore options for running your own TAXII Server. For this I found medallion, an open-source TAXII implementation build by Oasis.
medallion has been designed to be a simple front-end REST server providing access to the endpoints defined in the TAXII 2.1 specification.
It is important to note that medallion was designed as a prototype and a reference implementation of TAXII 2.1 – it was not intended for production use.
medallion can be installed using pip.
git clone https://github.com/oasis-open/cti-taxii-server
cd cti-taxii-server
git checkout v3.0.0
python3 -m venv tutorial_env
source tutorial_env/bin/activate
pip3 install medallion
medallion -h
If you are wondering why I cloned the cti-taxii-server repository, it is to have the demo config and data files locally which will be used in this tutorial.
The first thing needed is to define the configurations in the medallion config file. The config file contains:
- configuration information for the backend plugin (including an initial data file)
- a simple user name/password dictionary
Two back-end plugins are provided with medallion,
- The Memory back-end: persists data “in memory”. It is initialized using a json file that contains TAXII data and metadata. It is possible to save the current state of the in memory store, but this back-end is really intended only for testing purposes
- The MongoDB backend is somewhat more robust and makes use of a MongoDB server, installed independently.
For simplicity in this tutorial, I will demonstrate with the Memory back-end.
The repository also ships with a sample data file under /medallion/test/data/default_data.json that will back fill the TAXII server with initial data (you can of course add more / remove it once the server is running, as shown in the last post).
In this sample data file you will see the Discovery URL, API Roots (e.g. api1), Collections (and their STIX 2.1 Objects) and Status messages. If it is unclear how these are structured in the file at first glance it will become much clearer as you move through this post.
As required by the TAXII specification, medallion supports HTTP Basic authorization. The user names and passwords for this are stored in the config file in plain text (remember, it is not production ready).
First, create a new config file;
vi tutorialconf.json
Then add the following content to cover all the considerations detailed above;
{
"backend": {
"module": "medallion.backends.memory_backend",
"module_class": "MemoryBackend",
"filename": "/medallion/test/data/default_data.json"
},
"users": {
"admin": "Password0",
"user1": "Password1",
"user2": "Password2"
},
"taxii": {
"max_page_size": 100
}
}
You will also notice a taxii property not mentioned earlier. No values need to be supplied (in which case default values will be used). max_page_size defines the maximum number of Objects returned per page.
Now the config file is complete, I can run medallion like so;
medallion tutorialconf.json --host localhost --port 5000
If successful, you should see the server start;
* Serving Flask app 'medallion'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
Now that the server is running, I can use what has been shown in the last three posts to interact with the server.
Note, I first need to encode a user:password to base64 to pass in the Authorization header. You can use one of the users in the config file to do this (e.g. admin:Password0 in base64 = YWRtaW46UGFzc3dvcmQw).
Let me start by navigating to the discovery endpoint.
curl -i http://localhost:5000/taxii2/ \
-H "Accept: application/taxii+json;version=2.1" \
-H "Authorization: Basic YWRtaW46UGFzc3dvcmQw" \
-A "Signals Corps Curl Demo Client/1.0" \
Which returns;
HTTP/1.1 200 OK
Content-Type: application/taxii+json;version=2.1
Content-Length: 304
Connection: close
{
"api_roots" : [
"http://localhost:5000/api1/",
"http://localhost:5000/api2/",
"http://localhost:5000/trustgroup1/"
],
"contact" : "string containing contact information",
"default" : "http://localhost:5000/trustgroup1/",
"description" : "This TAXII Server contains a listing of",
"title" : "Some TAXII Server"
}
Now I can view the Collections inside one of the API Roots;
curl -i http://localhost:5000/trustgroup1/collections/ \
-H "Accept: application/taxii+json;version=2.1" \
-H "Authorization: Basic YWRtaW46UGFzc3dvcmQw" \
-A "Signals Corps Curl Demo Client/1.0"
The response shows 5 available Collections, some I can_read only, others I can can_read and can_write;
HTTP/1.1 200 OK
Content-Type: application/taxii+json;version=2.1
Content-Length: 1188
Connection: close
{
"collections": [
{
"can_read": false,
"can_write": true,
"id": "472c94ae-3113-4e3e-a4dd-a9f4ac7471d4",
"media_types": [
"application/stix+json;version=2.1"
],
"title": "This data collection is for testing querying across collections"
},
{
"can_read": true,
"can_write": true,
"id": "365fed99-08fa-fdcd-a1b3-fb247eb41d01",
"media_types": [
"application/stix+json;version=2.1"
],
"title": "This data collection is for testing adding objects"
},
{
"can_read": true,
"can_write": true,
"description": "This data collection is for collecting high value IOCs",
"id": "91a7b528-80eb-42ed-a74d-c6fbd5a26116",
"media_types": [
"application/stix+json;version=2.0",
"application/stix+json;version=2.1"
],
"title": "High Value Indicator Collection"
},
{
"can_read": true,
"can_write": false,
"description": "This data collection is for collecting current IOCs",
"id": "52892447-4d7e-4f70-b94d-d7f22742ff63",
"media_types": [
"application/stix+json;version=2.1"
],
"title": "Indicators from the past 24-hours"
},
{
"can_read": false,
"can_write": false,
"description": "Non accessible",
"id": "64993447-4d7e-4f70-b94d-d7f33742ee63",
"media_types": [
"application/stix+json;version=2.1"
],
"title": "Secret Indicators"
}
]
}
Selecting the Collection “High Value Indicator Collection” (91a7b528-80eb-42ed-a74d-c6fbd5a26116), I can query the Objects.
curl -i http://localhost:5000/trustgroup1/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/objects/ \
-H "Accept: application/taxii+json;version=2.1" \
-H "Authorization: Basic YWRtaW46UGFzc3dvcmQw" \
-A "Signals Corps Curl Demo Client/1.0"
Below I have cut some of the response for brevity;
HTTP/1.1 200 OK
X-TAXII-Date-Added-First: 2014-05-08T09:00:00.000000Z
X-TAXII-Date-Added-Last: 2017-12-31T13:49:53.935000Z
Content-Type: application/taxii+json;version=2.1
Content-Length: 1849
Connection: close
{
"more": false,
"objects": [
{
"created": "2014-05-08T09:00:00.000Z",
"id": "relationship--2f9a9aa9-108a-4333-83e2-4fb25add0463",
"modified": "2014-05-08T09:00:00.000Z",
"relationship_type": "indicates",
"source_ref": "indicator--cd981c25-8042-4166-8945-51178443bdac",
"spec_version": "2.1",
"target_ref": "malware--c0931cc6-c75e-47e5-9036-78fabc95d4ec",
"type": "relationship"
},
{
"created": "2014-05-08T09:00:00.000Z",
"id": "indicator--cd981c25-8042-4166-8945-51178443bdac",
"indicator_types": [
"file-hash-watchlist"
],
"modified": "2014-05-08T09:00:00.000Z",
"name": "File hash for Poison Ivy variant",
"pattern": "[file:hashes.'SHA-256' = 'ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c']",
"pattern_type": "stix",
"spec_version": "2.1",
"type": "indicator",
"valid_from": "2014-05-08T09:00:00.000000Z"
},
...
]
}
Finally I can add data to the TAXII Server, in this case a new Indicator Object (indicator--9ef5baeb-ad55-4c2c-aa89-ea5515dacf61);
curl -i -X POST http://localhost:5000/trustgroup1/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/objects/ \
-H "Accept: application/taxii+json;version=2.1" \
-H "Content-Type: application/taxii+json;version=2.1" \
-H "Authorization: Basic YWRtaW46UGFzc3dvcmQw" \
-A "Signals Corps Curl Demo Client/1.0" \
-d "{\"objects\":[{\"type\":\"indicator\",\"spec_version\":\"2.1\",\"id\":\"indicator--9ef5baeb-ad55-4c2c-aa89-ea5515dacf61\",\"created\":\"2022-01-01T01:00:00.000Z\",\"modified\":\"2022-01-01T01:00:00.000Z\",\"name\":\"Malicious site hosting downloader\",\"description\":\"This organized threat actor group operates to create profit from all types of crime.\",\"indicator_types\":[\"malicious-activity\"],\"pattern\":\"[url:value = 'http:\/\/x4z9arb.cn\/4712\/']\",\"pattern_type\":\"stix\",\"valid_from\":\"2022-01-01T01:00:00.000Z\"}]}"
Which is successful, as shown by the status response;
HTTP/1.1 202 ACCEPTED
Server: Werkzeug/2.2.2 Python/3.9.13
Date: Mon, 15 Aug 2022 06:44:40 GMT
Content-Type: application/taxii+json;version=2.1
Content-Length: 311
Connection: close
{
"failure_count": 0,
"id": "365b393f-3094-44f4-b7c3-3c1532523da6",
"pending_count": 0,
"request_timestamp": "2022-08-15T06:44:40.121597Z",
"status": "complete",
"success_count": 1,
"successes": [
{
"id": "indicator--9ef5baeb-ad55-4c2c-aa89-ea5515dacf61",
"version": "2022-01-01T01:00:00.000Z"
}
],
"total_count": 1
}
I can also delete it too;
curl -i -X "DELETE" http://localhost:5000/trustgroup1/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/objects/indicator--9ef5baeb-ad55-4c2c-aa89-ea5515dacf61/ \
-H "Accept: application/taxii+json;version=2.1" \
-H "Authorization: Basic YWRtaW46UGFzc3dvcmQw" \
-A "Signals Corps Curl Demo Client/1.0"
Which successfully deletes it;
HTTP/1.1 200 OK
Content-Type: application/taxii+json;version=2.1
Content-Length: 0
Connection: close
Other TAXII Server options
medallion is a well feature TAXII 2.1 server. With a few tweaks the backend to improve scalability and security, it is in my opinion the best option to build your own TAXII server from.
There are some other open-source options out there (uncovered using a GitHub search);
- OpenTAXII: TAXII server implementation in Python from EclecticIQ. Beware, the docs are outdated noting it supports only TAXII specification version 1.0 and 1.1, however, it does indeed have support for 2.1
- FreeTAXII: A cyber threat intelligence server based on TAXII 2 and written in Golang. Does support 2.1.
- TAXII-Server: A TAXII Server (2.1) designed for maximum flexibility and connectivity options with backend data sources.
If you know of any others, please do share them with me and I can include in this post.
Next Up: Consuming Data using TAXII Clients
Polling TAXII Servers can be done manually, as shown in the last two posts. However, a number of TAXII Clients exist with the logic to do this on an ongoing basis, pulling in new intelligence as it is published to a Collection.
In the final part of this series I will cover a few open-source TAXII Clients that exist, and how to get started with them.
TAXII 2.1 Certification (Virtual and In Person)
The content used in this post is a small subset of our full training material used in our TAXII 2.1 training.
If you want to join a select group of certified TAXII 2.1 professionals, subscribe to our newsletter below to be notified of new course dates.
Discuss this post
Never miss an update
Sign up to receive new articles in your inbox as they published.