Merge pull request #670 from BC-SECURITY/release/5.4.2

v5.4.2 into main
BC-SECURITY · Jun 12, 2023 · 0722f44 · 0722f44
2 parents ce3fdec + b1d9a3b
commit 0722f44
Show file tree

Hide file tree

Showing 51 changed files with 306 additions and 262 deletions.
diff --git a/.github/actions/update-starkiller/action.yml b/.github/actions/update-starkiller/action.yml
@@ -1,25 +1,21 @@
 name: 'Update Starkiller'
-description: 'Updates the config.yaml and starkiller submodule to the specified version and commits the change.'
+description: 'Updates the config.yaml to the specified version and commits the change.'
 inputs:
   starkiller-version:
-    description: 'The branch to merge from'
+    description: 'The starkiller ref to use'
     required: true
   app-version:
     description: 'The changelog section to update'
     required: true
   repo:
     description: "The starkiller repo to use. If not set, doesn't update the field."
     required: false
+  changelog-path:
+    description: "The changelog path"
+    default: CHANGELOG.md
 runs:
   using: "composite"
   steps:
-    - name: Install Python
-      uses: actions/setup-python@v2
-      with:
-        python-version: '3.9'
-    - name: Install minimal python packages
-      shell: bash
-      run: python -m pip install pyyaml
     # This is technically only needed for the first time we update the public repo, as a bootstrap.
     # After that, the submodule is already initialized to the right repo.
     - name: Update config.yaml repo
@@ -29,28 +25,15 @@ runs:
         sed -i "s|repo:.*|repo: ${{ inputs.repo }}|" empire/server/config.yaml
     - name: Update config.yaml ref
       shell: bash
-      # this depends on there not being any other keys in teh config named ref
+      # this depends on there not being any other keys in the config named ref
       # if there are, this will break. We could be safer to load/dump the yaml,
       # but would lose the comments.
       run: |
         sed -i "s/ref: .*/ref: ${{ inputs.starkiller-version }}/" empire/server/config.yaml
-    # If use_temp_dir is true, Starkiller is cloned into a temp directory, the CI will fail,
-    # and the submodule will not be updated. So set it to false, make the changes, then set it back.
-    - name: Update config.yaml use_temp_dir
-      shell: bash
-      run: |
-        sed -i'.bak' "s/use_temp_dir: .*/use_temp_dir: false/" empire/server/config.yaml
-    - name: Run starkiller update script
-      shell: bash
-      run: python empire.py sync-starkiller
-    - name: Reset use_tmp_dir
-      shell: bash
-      run: |
-        mv empire/server/config.yaml.bak empire/server/config.yaml
     - name: Update changelog
       shell: bash
       run: |
-        sed -i '/## \[${{ inputs.app-version }}\]/a -   Updated Starkiller to ${{ inputs.starkiller-version }}' CHANGELOG.md
+        sed -i '/## \[${{ inputs.app-version }}\]/a -   Updated Starkiller to ${{ inputs.starkiller-version }}' ${{ inputs.changelog-path }}
     - name: Commit changes
       shell: bash
       run: |

diff --git a/.github/ci-and-release.md b/.github/ci-and-release.md
@@ -121,23 +121,24 @@ the `private-main` branch.
 Once the `release/` pull request is merged, the `Private - Tag Release` workflow will automatically run.
 The workflow will create a tag and release on the `HEAD` of `private-main` using the release notes from `CHANGELOG.md` for the body of the release.
 
-### 6. Start Sponsor/Kali Release
+### 6. Repeat Step 2 - Prerelease Merge
+Repeat step 2 to merge `private-main` into `sponsors-main` and `kali-main`.
+
+### 7. Start Sponsor/Kali Release
 Start the release by running the `Sponsors & Kali - Create Release` manual workflow.
 If starkiller needs to be updated, provide a `starkillerVersion` input. The value provided should be a git tag minus the `-kali` or `-sponsors` suffix.
 
-This will first attempt to merge the `private-main` branch into `sponsors-main` and `kali-main` with the new release changes. Most likely, if there is a merge conflict here it is caused by `CHANGELOG.md` and should be minor. If that occurs, the merge conflict can be resolved in the pull request via the GitHub editor, or locally. 
-
-If a Starkiller tag was provided, it will update the Starkiller submodule and the changelog accordingly. It does this on the `sponsors-main` and `kali-main` release branches separately.
+If a Starkiller tag was provided, it will update the Starkiller config and the changelog accordingly.
 
 A release PR will then be opened for each branch and the test suite will run.
 
 
-#### 7. Manual Step - Merge sponsor/kali release PRs
+#### 8. Manual Step - Merge sponsor/kali release PRs
 Once the workflow runs, it will open two pull requests from the `release/v{version}-sponsors` and `release/v{version}-kali` branches to `sponsors-main` and `kali-main` respectively.
 
 Check the changelog on these branches, this will be the changelog that is used for the release notes.
 
-If there are sponsor/kali specific changelog entries that need to be added, add them to the `CHANGELOG.md` file on the release branch.
+If there are sponsor specific changelog entries that need to be added, add them to the `CHANGELOG-SPONSORS.md` file on the release branch.
 
 You can get a list of the new commits that are in this release by using the following command. Replace `v4.9.0-sponsors` with whatever the previous release was.
 ```
@@ -151,7 +152,7 @@ the `sponsors-main` branch or `kali-main` branch.
 
 **Potential Enhancement** We could add automation that copies the `unreleased` section from the target branch to the version section in the `head` branch.
 
-### 7. Tag and Release
+### 9. Tag and Release
 Once the pull requests are merged, the `Sponsors - Tag Release` and `Kali - Tag Release` workflows will automatically run.
 The workflows will create a tag and release on the `HEAD` of `sponsors-main` and `kali-main`, using the release notes from `CHANGELOG.md` for the body of the release.
 

diff --git a/.github/workflows/lint-and-test.yml b/.github/workflows/lint-and-test.yml
@@ -20,14 +20,28 @@ jobs:
         run: |
           pip install ruff
           ruff .
+  matrix-prep-config:
+    runs-on: ubuntu-latest
+    steps:
+      - id: release
+        if: ${{ startsWith(github.head_ref, 'release/') || contains( github.event.pull_request.labels.*.name, 'run-all-versions') }}
+        run: |
+          echo "config={\"python-version\": [\"3.8\", \"3.9\", \"3.10\", \"3.11\"]}" >> $GITHUB_OUTPUT
+      - id: not-release
+        if: ${{ !startsWith(github.head_ref, 'release/') }}
+        run: |
+          echo "config={\"python-version\": [\"3.8\", \"3.11\"]}" >> $GITHUB_OUTPUT
+    outputs:
+        config: ${{ steps.release.outputs.config || steps.not-release.outputs.config }}
   test:
-    needs: lint
+    needs:
+      - matrix-prep-config
+      - lint
     timeout-minutes: 15
     runs-on: ubuntu-latest
     name: Test Python ${{ matrix.python-version }}
     strategy:
-      matrix:
-        python-version: [ '3.8', '3.9', '3.10', '3.11' ]
+      matrix: ${{ fromJson(needs.matrix-prep-config.outputs.config) }}
     steps:
       - uses: actions/checkout@v3
         if: ${{ github.repository == 'BC-SECURITY/Empire' }}
@@ -64,6 +78,7 @@ jobs:
         run: |
           DATABASE_USE=mysql poetry run pytest . -v
       - name: Run test suite - sqlite
+        if: ${{ startsWith(github.head_ref, 'release/') || contains(github.event.pull_request.labels.*.name, 'test-sqlite') }}
         run: |
           DATABASE_USE=sqlite poetry run pytest . -v
 
@@ -181,4 +196,4 @@ jobs:
         uses: plexsystems/[email protected]
         with:
           image: docker.io/bcsecurity/empire-test-parrotrolling:latest
-          config: .github/install_tests/cst-config-parrotrolling.yaml
+          config: .github/install_tests/cst-config-parrotrolling.yaml
diff --git a/.github/workflows/release-sponsor-kali-start.yml b/.github/workflows/release-sponsor-kali-start.yml
@@ -19,7 +19,7 @@ jobs:
       - name: Check out code
         uses: actions/checkout@v3
         with:
-          ref: private-main
+          ref: sponsors-main
           submodules: 'recursive'
           fetch-depth: 0
           token: ${{ secrets.RELEASE_TOKEN }}
@@ -37,61 +37,22 @@ jobs:
         with:
           starkiller-version: ${{ env.STARKILLER_TAG }}
           app-version: ${{ env.APP_VERSION }}
-      - name: Get release branch name
-        run: |
-            echo "RELEASE_BRANCH=release/$APP_VERSION-sponsors" >> $GITHUB_ENV
-      - name: Create release branch
-        run: git checkout -b ${{ env.RELEASE_BRANCH }}
-      - name: Push new branch
-        run: git push origin ${{ env.RELEASE_BRANCH }}
-      - name: Create pull request into ${{ env.TARGET_BRANCH }}
-        uses: thomaseizinger/[email protected]
+          changelog-path: CHANGELOG-SPONSORS.md
+      - name: Update Sponsors Changelog
+        uses: vinnybod/[email protected]
         with:
-          GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}
-          head: ${{ env.RELEASE_BRANCH }}
-          base: ${{ env.TARGET_BRANCH }}
-          title: v${{ env.APP_VERSION }} into ${{ env.TARGET_BRANCH }}
-          reviewers: ${{ github.event.issue.user.login }}
-          body: |
-            This PR was automatically generated by the `release-sponsor-kali-start` workflow.
-            I've updated the version name and code commit: ${{ steps.make-commit.outputs.commit }}.
-            This PR should be merged with a merge commit, **not a squash commit.**
-            Merging this PR will trigger a tag and release automatically.
-  # this is basically copy/paste from the above.
-  create_kali:
-    # disable for now.
-    if: false
-#    if: ${{ github.repository == 'BC-Security/Empire-Sponsors' }}
-    runs-on: ubuntu-latest
-    steps:
-      - name: Set env vars
+          tag: v${{ env.APP_VERSION }}
+          changelogPath: CHANGELOG-SPONSORS.md
+          skipDiff: false
+      - name: Commit changelog and manifest files
+        id: make-commit
         run: |
-          echo "TARGET_BRANCH=kali-main" >> $GITHUB_ENV
-          echo "STARKILLER_TAG=${{ github.event.inputs.starkillerVersion }}-kali" >> $GITHUB_ENV
-      - name: Check out code
-        uses: actions/checkout@v3
-        with:
-          ref: private-main
-          submodules: 'recursive'
-          fetch-depth: 0
-          token: ${{ secrets.RELEASE_TOKEN }}
-      - name: Initialize mandatory git config
-        run: |
-          git config user.name "GitHub Actions"
-          git config user.email [email protected]
-      - name: Get app version
-        run: |
-          # Get the version from pyproject.toml using sed
-          echo "APP_VERSION=$(sed -n 's/^version = "\(.*\)"/\1/p' pyproject.toml)" >> $GITHUB_ENV
-      - name: Update Starkiller Version
-        if: ${{ github.event.inputs.starkillerVersion }}
-        uses: ./.github/actions/update-starkiller
-        with:
-          starkiller-version: ${{ env.STARKILLER_TAG }}
-          app-version: ${{ env.APP_VERSION }}
+          git add -A
+          git commit --message "Prepare release ${{ env.APP_VERSION }} sponsors"
+          echo "commit=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
       - name: Get release branch name
         run: |
-            echo "RELEASE_BRANCH=release/$APP_VERSION-kali" >> $GITHUB_ENV
+            echo "RELEASE_BRANCH=release/$APP_VERSION-sponsors" >> $GITHUB_ENV
       - name: Create release branch
         run: git checkout -b ${{ env.RELEASE_BRANCH }}
       - name: Push new branch

diff --git a/.github/workflows/release-sponsor-kali-tag.yml b/.github/workflows/release-sponsor-kali-tag.yml
@@ -5,7 +5,6 @@ on:
       - closed
     branches:
       - sponsors-main
-#      - kali-main
 
 jobs:
   tag_release:
@@ -27,6 +26,11 @@ jobs:
       - name: Extract release notes
         id: extract_release_notes
         uses: vinnybod/extract-release-notes@v2
+      - name: Extract release notes - sponsors
+        id: extract_release_notes_sponsors
+        uses: vinnybod/extract-release-notes@v2
+        with:
+          changelog_file: CHANGELOG-SPONSORS.md
       - name: Create Release
         id: create_release
         uses: actions/create-release@v1
@@ -35,6 +39,10 @@ jobs:
         with:
           tag_name: v${{ env.TAG_NAME }}
           release_name: v${{ env.TAG_NAME }}
-          body: ${{ steps.extract_release_notes.outputs.release_notes }}
+          body: |
+            ${{ steps.extract_release_notes.outputs.release_notes }}
+            
+            ### Sponsors Only
+            ${{ steps.extract_release_notes_sponsors.outputs.release_notes }}
           draft: false
           prerelease: false
diff --git a/.gitignore b/.gitignore
@@ -14,6 +14,7 @@ empire/server/data/generated-stagers/*
 empire/server/downloads/*
 empire/server/api/static/*
 empire/server/api/v2/starkiller-temp
+empire/server/api/v2/starkiller
 
 # client
 empire/client/generated-stagers/*

diff --git a/.gitmodules b/.gitmodules
@@ -46,9 +46,6 @@
 [submodule "empire/server/csharp/Covenant/Data/ReferenceSourceLibraries/DotNetStratumMiner"]
 	path = empire/server/csharp/Covenant/Data/ReferenceSourceLibraries/DotNetStratumMiner
 	url = https://github.com/BC-SECURITY/DotNetStratumMiner.git
-[submodule "empire/server/api/v2/starkiller"]
-	path = empire/server/api/v2/starkiller
-	url = https://github.com/BC-SECURITY/Starkiller.git
 [submodule "empire/server/csharp/Covenant/Data/ReferenceSourceLibraries/RunOF"]
 	path = empire/server/csharp/Covenant/Data/ReferenceSourceLibraries/RunOF
 	url = https://github.com/BC-SECURITY/RunOF.git
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [5.4.2] - 2023-06-07
+
+-   Updated Starkiller to v2.3.2
+-   Fixed python modules not running properly (Cx01N)
+-   Updated python multi_socks to run with Python 3 (Cx01N)
+
+## [5.4.1] - 2023-06-02
+
+-   Fix database reset issue with MySQL (@Vinnybod)
+-   Add a message to the client recommending the use of the Starkiller (@Vinnybod)
+-   Fixed issue with Invoke-wmi not returning a success message (@Cx01N)
+-   Fixed dynamic function issue with Powerview (@Cx01N)
+-   Pair down the amount of minutes needed to run pull request builds (@Vinnybod)
+
+## [5.4.0] - 2023-05-22
+
+-   Remove Starkiller as a submodule, treat it as a normal directory (@Vinnybod)
+    -   Everything should 'just work', but if you have issues after pulling these latest changes, try deleting the Starkiller directory before running the server `rm -r empire/server/api/v2/starkiller`. 
+-   Some improvements to the release flow after starkiller submodule removal (@Vinnybod)
+
+## [5.3.0] - 2023-05-17
+
+-   Add the ability to specify a module option as a file (@Vinnybod)
+
 ## [5.2.2] - 2023-04-30
 
 -   Updated Starkiller to v2.2.0
@@ -467,7 +491,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 -   Updated shellcoderdi to newest version (@Cx01N)
 -   Added a Nim launcher (@Hubbl3)
 
-[Unreleased]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.2.2...HEAD
+[Unreleased]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.4.2...HEAD
+
+[5.4.2]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.4.1...v5.4.2
+
+[5.4.1]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.4.0...v5.4.1
+
+[5.4.0]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.3.0...v5.4.0
+
+[5.3.0]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.2.2...v5.3.0
 
 [5.2.2]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.2.1...v5.2.2
 

diff --git a/empire/client/src/EmpireCliState.py b/empire/client/src/EmpireCliState.py
@@ -78,9 +78,11 @@ def notify_disconnected(self):
     def connect(self, host, port, socketport, username, password):
         self.host = host
         self.port = port
+        server = f"{host}:{port}"
+
         try:
             response = requests.post(
-                url=f"{host}:{port}/token",
+                url=f"{server}/token",
                 data={"username": username, "password": password},
                 verify=False,
             )
@@ -92,7 +94,7 @@ def connect(self, host, port, socketport, username, password):
             self.connected = True
 
             self.sio = socketio.Client(ssl_verify=False, reconnection_attempts=3)
-            self.sio.connect(f"{host}:{port}/socket.io/", auth={"token": self.token})
+            self.sio.connect(f"{server}/socket.io/", auth={"token": self.token})
 
             # Wait for version to be returned
             self.empire_version = self.get_version()["version"]
@@ -102,6 +104,7 @@ def connect(self, host, port, socketport, username, password):
             self.notify_connected()
             print_util.title(
                 self.empire_version,
+                server,
                 len(self.modules),
                 len(self.listeners),
                 len(self.active_agents),

diff --git a/empire/client/src/menus/InteractMenu.py b/empire/client/src/menus/InteractMenu.py
@@ -503,14 +503,14 @@ def execute_shortcut(self, command_name: str, params: List[str]):
                         try:
                             file_directory = post_body.get("options")["File"]
                             filename = file_directory.split("/")[-1]
-                            post_body.get("options")["File"] = filename
                             data = get_data_from_file(file_directory)
                         except Exception:
                             log.error("Invalid filename or file does not exist")
                             return
                         response = state.upload_file(filename, data)
                         if "id" in response.keys():
                             log.info("File uploaded to server successfully")
+                            post_body.get("options")["File"] = response["id"]
                         elif "detail" in response.keys():
                             if response["detail"].startswith("[!]"):
                                 msg = response["detail"]

diff --git a/empire/client/src/menus/UseModuleMenu.py b/empire/client/src/menus/UseModuleMenu.py
@@ -83,14 +83,14 @@ def execute(self):
                 try:
                     file_directory = self.record_options["File"]["value"]
                     filename = file_directory.split("/")[-1]
-                    self.record_options["File"]["value"] = filename
                     data = get_data_from_file(file_directory)
                 except Exception:
                     log.error("Invalid filename or file does not exist")
                     return
                 response = state.upload_file(filename, data)
                 if "id" in response.keys():
                     log.info("File uploaded to server successfully")
+                    self.record_options["File"]["value"] = response["id"]
 
                 elif "detail" in response.keys():
                     if response["detail"].startswith("[!]"):