MHN's MongoDB Collections

Note: The _id field has been omitted from each of the examples below.

nmemosyne database

This mongo database is populated by Pwnlandia's fork of mnemosyne. Here are the collections it uses.

counts

Precomputed summaries of counts over time ranges designed to speed up some aggregation queries. This was added by Pwnlandia specifically for MHN.

Examples:

{
  "date": "20150423",
  "identifier": "glastopf.events",
  "event_count": 13
}

{
  "date": "20150423",
  "identifier": "55df459c-f8a6-11e3-9dcd-040114afa401",
  "event_count": 487
}

daily_stats

Daily summaries stats across various dimensions of the data. This was included with mnemosyne by default.

Example:

{
  "channel": "dionaea.connections",
  "date": "20140514",
  "hourly": {
    "0": 99,
    "1": 72,
    "2": 68,
    "3": 162,
    "4": 106,
    "5": 81,
    "6": 82,
    "7": 88,
    "8": 145,
    "9": 105,
    "10": 112,
    "11": 70,
    "12": 88,
    "13": 65,
    "14": 125,
    "15": 202,
    "16": 227,
    "17": 157,
    "18": 106,
    "19": 210,
    "20": 144,
    "21": 88,
    "22": 97,
    "23": 66
  }
}

dork

Searches, file paths, and associated metadata collected from various honeypots, usually as it relates to web application honeypots. This was included with mnemosyne by default.

Example:

{
  "content": "/proxychecker/check.cgi",
  "type": "inurl",
  "lasttime": "2015-05-09T20:44:50.599Z",
  "count": 3
}

file

Files and associated metadata collected from various honeypots. This was included with mnemosyne by default.

Example:

{ 
	"hashes" : { 
		"sha1" : "7aa47c77d45530b5cc382d51dde93a9f226cc3f5",
		"sha512" : "b0522284d6cbffe02d75e1d7d2b7b23c9d4de359938d60464b3aad7349f5516579f048eae40a60f29741bf5efcbfb2ed2b21a3f37daac6a4faeb0cd7df05a290",
		"md5" : "5a83a4c473702f24e5ee4cf39db1b9f1" 
	},
	"data" : "00000...",
	"content_guess" : "data",
	"encoding" : "hex",
	"hpfeed_ids" : [ ObjectId("YYYYYYYYYYYY") ] 
}

hpfeed

The raw message sent back from a honeypot stored in whatever form it was sent in (usually JSON). This was included with mnemosyne by default.

Example:

{
	"ident" : "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX",
	"timestamp" : ISODate("2015-04-23T01:27:38.322Z"),
	"normalized" : true,
	"payload" : "{\"timestamp\": \"2015/04/23 01:12:08\", \"server_ip\": \"1.2.3.4\", \"client_port\": 39593, \"mod\": \"mtu\", \"client_ip\": \"5.6.7.8\", \"server_port\": 22, \"subject\": \"cli\", \"link\": \"Ethernet or modem\", \"raw_mtu\": \"1500\"}",
	"channel" : "p0f.events" 
}

metadata

Meta stored around IP addresses. This is used as a pre-computed collection of observations around an attacker IP. Mainly used for p0f data. This was added by Pwnlandia specifically for MHN.

{
	"honeypot" : "p0f",
	"ip" : "1.2.3.4",
	"timestamp" : ISODate("2015-05-08T21:58:56.444Z"),
	"link" : "Ethernet or modem",
	"os" : "Windows XP",
	"uptime" : "13 days 4 hrs 46 min (modulo 198 days)" 
}

session

A normalized 5-tuple like record that summaries a honeypot interaction. This was included with mnemosyne by default.

{ 
	"destination_ip" : "1.2.3.4", 
	"protocol" : "TCP", 
	"hpfeed_id" : ObjectId("XXXXXXXXXXXXXXXXXXXX"), 
	"timestamp" : ISODate("2015-04-29T06:25:55.712Z"), 
	"source_ip" : "5.6.7.8", 
	"snort" : { 
		"priority" : 2, 
		"header" : "1:2001219:19", 
		"classification" : 4, 
		"signature" : "ET SCAN Potential SSH Scan" 
	}, 
	"source_port" : 54015, 
	"honeypot" : "snort", 
	"identifier" : "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX", 
	"sensor" : "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX", 
	"destination_port" : 22 
}

url

URLs and associated metadata collected from various honeypots. This was included with mnemosyne by default.

hpfeeds database

This database is used by the hpfeeds broker.

auth_key

This collection is populated as new hpfeeds users are added using this script: https://github.com/Pwnlandia/hpfeeds/blob/master/broker/add_user.py.

Example record:

{
  "identifier": "dionaea-sensor-0001",
  "secret": "XXXXXX",
  "publish": [
    "mwbinary.dionaea.sensorunique",
    "dionaea.capture",
    "dionaea.capture.anon",
    "dionaea.caputres",
    "dionaea.connections",
    "dionaea.shellcodeprofiles",
    "dionaea.dcerpcrequests"
  ],
  "subscribe": []
}