Skip to content

Commit

Permalink
Initial release to GitHub for open sourcing effort.
Browse files Browse the repository at this point in the history
Parsing and recovery of SQLite database and WAL files
Started documentation of classes in README.md files with Mermaid
Added PyInstaller scripts and builds for windows and linux
Incorporated output options for SQLite, XLSX, and CSV
Added initial beta carving of journal files
  • Loading branch information
DC3-DCCI authored and DC3-DCCI committed Jul 30, 2021
0 parents commit 5862fd9
Show file tree
Hide file tree
Showing 69 changed files with 20,313 additions and 0 deletions.
23 changes: 23 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# This .gitignore file specified the files to exclude from the git project.
#

# Pycharm Files
.idea

# VS Code Files
.vscode

# Python Files
*.pyc
*.pyo

# Pyinstaller Files
/build
/dist

# Packing files
/sqlite_dissect.egg-info

# Other
/output
/log
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Change Log

v0.0.6 (2021-07-29)
------------------

- Initial external release of application and source code
- Parsing and recovery of SQLite database and WAL files
- Started documentation of classes in README.md files with Mermaid
- Added PyInstaller scripts and builds for windows and linux
- Incorporated output options for SQLite, XLSX, and CSV
- Added initial beta carving of journal files
23 changes: 23 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
DC3 SQLite Dissect Open Source License

DC3 SQLite Dissect software was developed by the Department of Defense Cyber
Crime Center (DC3). By delegated authority pursuant to Section 801(b) of Public Law
113-66, DC3 grants the following license for this software:

Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following condition:

The above permission notice and the below warranty notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE DEVELOPERS, OR LICENSORS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
333 changes: 333 additions & 0 deletions README.md

Large diffs are not rendered by default.

10 changes: 10 additions & 0 deletions _version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@

"""
_version.py
This script identifies the version of the sqlite dissect library.
"""

__version__ = "0.0.6"
81 changes: 81 additions & 0 deletions api_usage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import logging
import os
import sqlite_dissect.constants as sqlite_constants
import sqlite_dissect.interface as sqlite_interface

"""
api-usage.py
This script shows an example of the api usage for a specific test file.
"""

# Setup logging
logging_level = logging.ERROR
logging_format = '%(levelname)s %(asctime)s [%(pathname)s] %(funcName)s at line %(lineno)d: %(message)s'
logging_date_format = '%d %b %Y %H:%M:%S'
logging.basicConfig(level=logging_level, format=logging_format, datefmt=logging_date_format)

# Setup console logging
console_logger = logging.StreamHandler()
console_logger.setLevel(logging_level)
console_logger.setFormatter(logging.Formatter(logging_format, logging_date_format))
logging.getLogger(sqlite_constants.LOGGER_NAME).addHandler(console_logger)

"""
API Usage
The three fields below need to be filled in and are currently hardcoded:
file_name: The SQLite file to investigate (and associated WAL file if it exists in the same directory)
table_name: The table in the file to create a signature of and carve against the SQLite file with.
column_names: The columns in the table we are interested in printing out carved data from.
Note: Below will carve entries from the b-tree page of the table and the freelists. The use case of cross b-tree
carving is not yet implemented yet in SQLite Dissect.
"""

# Specify the file details
file_name = "FILE_NAME"
table_name = "TABLE_NAME"
column_names = ["COLUMN_ONE", "COLUMN_TWO"]

# Create the database
database = sqlite_interface.create_database(file_name)

# Create the write ahead log
wal_file_name = file_name + sqlite_constants.WAL_FILE_POSTFIX
write_ahead_log = sqlite_interface.create_write_ahead_log(wal_file_name) if os.path.exists(wal_file_name) else None

# Create the version history
version_history = sqlite_interface.create_version_history(database, write_ahead_log)

# Create the signature we are interested in carving
table_signature = sqlite_interface.create_table_signature(table_name, database, version_history)

# Account for "without rowid"/virtual table signatures until supported
if not table_signature:
print("Table signature not supported (\"without rowid\" table or virtual table)")
exit(0)

# Get the column indices of the columns we are interested in
column_name_indices = {}
for column_name in column_names:
column_name_indices[column_name] = sqlite_interface.get_column_index(column_name, table_name, version_history)

# Get a version history iterator for the table
carve_freelists = True
table_history_iterator = sqlite_interface.get_version_history_iterator(table_name, version_history,
table_signature, carve_freelists)
# Iterate through the commits in the history for this table
for commit in table_history_iterator:
# The table was only modified if the commit was updated for this table and make sure there were carved cells
if commit.updated and commit.carved_cells:
carved_cells = commit.carved_cells
for carved_cell in carved_cells.itervalues():
for column_name in column_name_indices.keys():
record_column = carved_cell.payload.record_columns[column_name_indices.get(column_name)]
print("Commit version: %s table record column: %s has serial type: %s with value of: \"%s\"." %\
(commit.version_number, column_name, record_column.serial_type, record_column.value))
Loading

0 comments on commit 5862fd9

Please sign in to comment.