Teamserver/teamclient Sliver model

[maxlandon]

This PR aims at consistently and systematically taking advantage of the Cobra CLI development API so as to:

• Reach full interfacing of the Sliver Framework with arbitrary system shell environments.
• Generalize and strenghen the Sliver client-server model into a thiner but more efficient and
  versatile component (a teamclient/teamserver management system/library).
• Adapt the existing code to take full advantage of the aforementionned changes, and add some new
  small features/commands to that effect.

Table of Contents

• Table of Contents
  • I - Summary of Changes
  • II - New features
  • III - Example worflows
  • IV - Codebase notes
    - client/
    - server/
    - protobuf/
    - vendor/
  • V - Security
  • VI - Other related topics
  • VII - Remaining areas of work
    • Asciicast & history
    • Settle the dust
  • VIII - New paths
    • Quickly restructuring some command trees
    • Toward a single Sliver framework binary
  • IX - Related issues

I - Summary of Changes

• All code related to client and server connections/RPC has been refactored and published under a
  new team library for teamclient/teamserver style software. This has numerous advantages, including
  a much better inclusion into the cobra Command API, more efficient pre-run/pre-connect workflows,
  and many benefits unravelling from this.
• Full support for arbitrary CLI-style command execution in Sliver, with completion support for 9
  different major shell environments.
• Error unwrapping for all gRPC-returned errors (eg. get rid of rpc error code = 2 and no message)
• Automatic filtering and blocking on commands non-compatible for a given target context,
  at execution-time and in the interactive aspects. Custom generated errors (handled by dependencies)
  indicate why a given command cannot be executed.
• Enhancing the syntax of many commands (ex. all --id flags have been replaced with positional
  arguments). As well and related, all non-implant commands have completion for pretty much
  everything, from file flags all the way to client-side network tools, extension commands, etc.
• Various display enhancements, for things like automatic table wrapping, rigorous beacon task
  status display synchronization.

II - New features

Additionally, there a few new side-features going with this:

• Everything related to connecting clients to servers, managing servers, users management is now
  encapsulated both at the API level and at the CLI level with the teamserver and teamclient
  commands.
• sliver-client implant history command, for diplaying command history per-implant, with various
  filters and display options.
• Systematic and coherent handling of blocking/non-blocking commands in both the console and CLI.
  Examples include network tools, beacon task spawning, management and cancelling, and much more.
• Multi-client synchronization of beacon task cancelling: executing a beacon task in a terminal
  without & will block until task is completed, or if another terminal receives a tasks cancel
  command: the terminal 1 will exit.
• Various completion helpers everywhere in the command tree, and automatic command filtering in
  the completion context based on the target context (the implant type, capabilities and host).

III - Example worflows

Some very small examples of possible usage now, between many others. The ones below aim to be as
representative as possible of the new typical base workflows, at various points in operation.

The only dependency for optimal CLI usage is the carapace completion binary, sourced in your shell.
Various installation methods available, and a single one-liner in your ".shell_rc" will make the trick.
https://github.com/rsteube/carapace-bin.

Aliases:

sls=sliver-server
slc=sliver-client
sli=sliver-client implant --use <implant_id> 

Sliver teamserver deployment:

# Import a pre-existing user file, add a new user to the teamserver, 
# start some persistent listeners, and setup a basic systemd service 
# with everything ready to work.
sls teamserver import former_sliver_team_ca.pem
sls teamserver user --name michel --host localhost
sls listen --listener grpc/TSNet --host 10.10.10.10 --port 1789 --persistent
sls teamserver systemd --host localhost --user > /etc/systemd/user/sliver.service
systemctl start sliver.service

# Import our new remote teamserver to use it locally.
slc teamclient import michel_localhost.teamclient.cfg 

# Then, query the teamserver aspects either from the 
# sliver-binary itself, or from a remote client.
sls teamserver status       # Server-only command
sls teamserver client users # Server version of ...
slc teamclient users        # ...this command.

Sliver server basic usage:
(Note that all these commands work identically both in slc and sls binaries, although
the consistency in command behavior acrosss makes this more relatively useful than one thinks).

# Generate an implant and a job
slc generate --os linux --skip-symbols --mtls localhost:8888
slc mtls --lhost locahost --persistent
slc jobs

# Or decide to start a new server instance in builder mode.
# Full completion for all flags, platform, teamserver configs, etc.
sls builder --config michel_localhost.teamclient.cfg --enable-target windows/arm64 

# Manage the armory and use it on an implant (if platform compatible)
# Note the `sli` alias indicating that we are targeting a given implant,
# for which there is an entire subtree of commands in the CLI.
# Actually... the full subtree.
slc armory install tgtdelegation          # Completion for all armory contents on all client shells.
sli tgtdelegation type domain.com --flags # Full support for both Sliver flags and extension ones.

Sliver implant interactions:

# Browse basic information for an implant with builtins 
# and simple extensions. We assume a beacon running on Windows.
# These commands are blocking...
sli info; sli pwd;

# But our shell has jobs right ? spawn a task, manage it and cancel it.
sli sa-whoami &             # Yup son, not back in the charts but we're back in your shell.
sli tasks                   # Show our tasks, with their command-lines as their descriptions.
sli tasks cancel 2dsd328ds  # Cancel it. There's more about that, but not now.

# Print the history for this implant, with timestamps and only for 
# our user. 5 lines of code can wrap this in an fzf stuff for your shell.
sli history --user --time

# Now let's use a session and work with network tools.
sli=sliver-client implant --use <session_id>
sli ifconfig                                           # Find your target.
sli portfwd add -l localhost:1324 -r 10.120.23.23:1499 # This blocks in your shell...
^Z                                                     # ... so just background it.

# Our use any of your favorite commands. Have fun.
sli execute-assembly --amsi <completed-file> --args 'super complicated args strings'
sli bof-roast 
slc extensions install <extension-manifest>
sli custom-extension --args

IV - Codebase notes

• My code formatter adds points and stuff like wherever I open files... therefore you will see
  I avoided the implant/ directory like the plague.
• Dependencies introduced in Continuing CLI changes #1353 have been updated to their latest versions, with many
  code/feature improvements, added options and bug fixes.

client/

The client code is obviously the one that has seen the most changes, where multiple packages
have been entirely refactored, strenghened and sometimes trimmed down to not much. Some things:

• The assets/ directory is now free of any teamclient/server config related stuff, logging, etc.
• The console/ directory has been split into different files per main area (core, shell,
  commands, teamclient connections/loops, etc). It ensures to the best extent possible to
  provide coherent backend functions for either cli/ packages, or any importer of a
  console.SliverClient in general.
• The transport/ dir has been refactored in the same fashion as its server counterparts.
• The cli/ directory is now mostly for last-minute assembly of components in other packages.
• All the exported code has been commented for external users. Also many comments in the code
  (remove if wanted/needed).
• The command/ directory has naturally gone through many changes, thickening and refactoring.
  It includes a new flags/ package with utility functions to register completions, command
  target filters, flagsets, etc.

server/

The server code is however the one that has the highest quantity of removed code, since everything
teamclient/server related has been refactored out as a dependency module. Some packages have thus
entirely disappeared while a new one has been introduced for server-binary only commands.

• The daemon/ package has been removed and now handled by reeflective/team.
• The cli/ package has been trimmed down as much and in a same fashion as the client/cli.
• configs/ and assets have been cleaned from everything everything teamserver-related.
  Assets unpacking is now called through a dedicated function called as cobra pre-run instead of init.
• The certs package has simply been stripped out of all user cryptographic material functions.
  The teamserver library handles this (the code was simply ported/copied), while still using
  sliver's DB. Same thing for exporting those files. There are now separate MTLS CAs: one for mTLS
  implants, managed through the Sliver API, and the users' teamserver one, which is managed through
  the teamserver API.
• The core/ package has seen a small but important change: the eventBufSize channel size is now
  0, making it blocking. If it is not, the server will crash because of a race condition I could
  not really pinpoint. I have not seen any notable difference in performance, but this might want
  to be investigated a little further.
• The msf/ package has new functions that automatically cache some calls to msfvenom for a list
  of available encoders/payload/platforms when Sliver's assets are unpacked. Used by completions.
• The transport/ package has been refactored to fit reeflective/team client/server interfaces.
  Overall, neither the size or the contents of the package or that of the API don't change much,
  it's just a complete rewrite. Functionality stays strictly the same.
• The builder/ package has seen just as many changes as it needed to fit within the new code,
  including for handling CLI-style interrupts.
• The rpc/ package has new files for teamclient-related functions, and for implant history ones.
  Logging has also been enlarged for these purposes. Stuff for beacon tasks canceling and cmdline
  descriptions, etc. Added code for MSF compilers data.
• Finally, the command/ directory is the equivalent of the client/command directory, with the
  same structure and exported functions, to be used identically by entrypoints.

protobuf/

• Some message types have been added for 1) implant command history saving and fetching, 2)
  Metasploit modules data (used by stager commands), 3) attached command-lines to beacon tasks.
• Some protobuf RPC/common objects have been refactored into a different file, because I suspect
  that you might want to reuse them with the teamclient/teamserver model down the road (that is,
  write tools which will leverage teamclients, the first of which being sliver-crackstation).

vendor/

• Always very scared about that one. But looking at it, it seems that changes are quite restrained.
  Usual golang.org/x/sys things and some filesystem stuff (the teamclients/teamserver have support
  for full in-memory operation mode, not used in Sliver obviously).
• As mentionned above, introduced reeflective/team dependency for teamclient/teamserver code.
  As also mentionned, it is still possible to just copy the repository code and use it with your
  own import paths. A few sed directives should be enough to handle this.

V - Security

The default PR/issue template for Sliver includes a security section so as a matter of good habit,
several key things should be mentioned and reminded:

• The code of this PR nor any of its new dependencies demand full control over the backends storing
  the data, that is they accept pre-connected databases, standard and audit loggers and will work
  with them. These libraries and related code changes ultimately provide tools to manipulate user
  and team data, enforcing and ensuring strong security defaults, and only this.
• No code execution has been added in any way in the server/ directory (nor any in the client/ one
  in fact). The msfvenom command is ran once at assets unpacking (if available) for caching some
  data server-side, about payload modules.
• No behavioral changes should have been introduced in the C2 code nor in any of its aspects, apart
  from the tcp and named-pipe commands being moved in the tree.
• All client requests are still logged and audit-logged regardless of their exec/completion end
  purpose. See following notes on verbosity and log cluttering.

VI - Other related topics

• The recently added taskmany command is not functional anymore... It was clearly not leveraging
  our yielder functions the way they should/can be, anyway. It seems to me that this command is
  not really needed anymore, nor would take a sound and consistent approach to multi-target command
  execution, but I can be wrong obviously.
• Given the new possibilities in CLI flags/completion/UI aspects, you might want to change the
  --os and --arch flags by a single --platform flag with classic Go platform arguments os/arch.
  Multiparts completion is dead easy for this, and it will ensure that only valid combinations are
  proposed at any given time.
• I noticed (I think this is not due to me) that only grpc.Unary requests are sliver.logged, not
  grpc.Stream ones. Maybe this is normal, and those requests are audit-logged anyway.
• Some logging streams can now be very verbose: for every CLI connection, we have 3-5 streams
  opened, plus at least 1 request for each command core domain. This also applies to some extent
  to the completions, when they need server-fetched data. Performance is not an issue here, just
  that the logs get cluttered quickly, especially by gRPC logging middleware.
• Lots of things could be said on the new completion library features, among which caching is
  extremely useful to us for minimizing the heaviest data passed around (compilers info, modules,
  etc). Caching is handled/managed per binary, so sliver-server, sliver-client, sliver-crackstation,
  etc, will all cache their own stuff, and everything will work smoothly.

VII - Remaining areas of work

Asciicast & history

The recording system is the least functional/mature part of the work, because the effects of the
latter put into question which scope should the recording process consider, both at record time as
well as in the replay one.

Good news first:

• All console sessions are recorded just like since the functionality was introduced.
• Most client commands are also logged server-side anyway, and their output as well.

Less good news or things I've not checked/solved:

• Since the I/O is very much open because everything is going to be ran in system shells.
  This means there are choices to make between recording asciicast output of straight out
  command output bytes.
• There should be a way to quickly browse and replay some time frame of recordings, for a given
  user or implant. Plus there can be concurrent work/commands on a given implant, with several users
  generating session data. Not a problem in itself, just need to get the replay thing right.
• There are currently bugs due to how Go io.MultiWriter handles stream closings: given a stream
  A and a stream B, if A calls Close() or is closed, the io.MultiWriter will close all the other
  streams at once. This is not good as it messes with various things in various scenarios (one time
  it will be the readline blocking, the other the completion system, etc, etc.)
  Again, no big deal or serious dead-end here, just details to take care of.

Settle the dust

The entire migration to the cobra CLI is not trivial as it means full interfacing with the user OS.
As much as this present work has emphasized defining and ensuring a consistent and obvious behavior
for everything related, and despite its pretention to have encapsulated most of the utility and
boilerplate behind simple functions, there are still some things that will see some change:

• Some boilerplate code might be able to be removed in the future.
• More efficient teamserver selection strategies might be used.
• Log verbosity is quite high as mentionned before.
• How "plugin/extensions" can interface with Sliver clients/servers, and what can the latters do
  for easy and safe development of the formers.
• As described in the previous section about recording, ensure that everything aligns with this.
• Inspect which aspects of Sliver can be better interfaced/served via the CLI, as opposed to new
  features to implement in the framework itself.
• One or two documentation pages about the CLI aspects will probably be welcome for many.

VIII - New paths

Quickly restructuring some command trees

It's easier than ever to play with the command tree, and it's probably why it should be done
now so as to provide a solid and clear one for the foreseeable future. Easier to extend, maintain
and to explore. Below are some examples, which I'll probably include in a separate issue/PR:

sli msf-inject
sli msf          # Msf currently is a command, either make subcommands optional
        inject   # like this one
        generate # Or move the current msf impl as a subc and use msf as root namespace

sli wg-socks
sli wg
        socks
        portfwd

slc generate traffic-encoders   # This command ...
sli reconfigure                 # ... and this one 
sli transports              # The transports root is for generic transport management (swap/load/etc)
               reconfigure  # Some commands are available only in implant context, with a filter.
               encoders     # While others are available both in slc and in sli context.

Same for C2 related commands:

slc mtls
     listen # reverse handler, currently what we have
     dial   # bind handler

     serve # serve a stage. The verb fits in that it indicates some content to be given somewhere.
     stage # send a stage to a remote stager ready to execute the payload.

Toward a single Sliver framework binary

The teamclient/server architecture and its management of directories emphasize the correct
segregation of command trees, APIs, but also of the working directories used by the binaries.

This in turn might enable to work from a single binary that is able to act as a server or
as client, at various points of execution or for specific tasks. A representative example is
the way Metasploit uses its own .msf/ working directory to store user modules and such.
In short:

• It solves the problem of how fast users can develop/modify their implant/client code and use it.
• Partially related, it solves how fast and easily users can expose their custom implants behind
  their server.
• Also related but client-side: same conclusions apply to extensions: write your extension locally,
  and expose its implant code behind your personal builder, then use it with your CLI (which
  automatically loads extension CLIs)
• It solves how fast you can spin up the entire framework with custom changes.

A significant advantage for Sliver is that the CLI is now efficient enought to provide analog
functionality, but with much more efficiency and UI confort. All of the following illustrate this:

• When you msfconsole, or when you msfvenom, you use different entrypoints, but they all points to
  the same framework ruby core. As a result, the slightest invocation to it for even just the list
  of available modules takes years.
• Serving metasploit as a true teamserver is a nightmare, let alone with good performance.
• Having it fully and correctly interfaced with shells and programming languages is a lofty dream.
• As a consequence of all the above, "turning your local Metasploit framework around" and exposing
  it to a Metasploit teamserver is impossible.

There are lots of things to say about this that I won't list here, but I will humbly suggest to keep
this possibility at the back of the head.

You could imagine some commands that are similar to the chezmoi tool (dotfiles management):

# Example commands
sliver dev             # Manage the framework source code, for implants, servers, clients, etc.
        cd             # Open a shell in the git root ~/.sliver/framework/
        install <rule> # Install the artifacts we produce when developing the tool.
        code <path>    # Alias to edit files in the source tree.

# Example workflow
sliver dev cd
git checkout master
git pull BishopFox
sliver dev install client # Alternatively, since we're in root, `make client`
sliver builder --flags    # Use our new implant code.

IX - Related issues

Below is a list of issues that might be solved/solvable thanks to the present PR.

• ASCII Cinema-like Recording #26: See previous section.
• Feature request: autoruns and resource file #193: CLI execution and scripting should have solved this.
• Customizable prompt #244: Not solved by the current PR, no new prompt stuff in console, but quite related.
• Add window resizing support to PTY shell #442: Maybe readline can help here, not with code but with what to do and what not.
• Feature Request: Catch Ctrl-Z When Inside Implant Shell #515: The current PR does not handle CtrlZ in any particular way in readline. There are however
  some methods offered by the console and its readline to bind custom handlers.
• Client Logs Not Written to Disk #570: See previous section.
• Psexec Support in Beacon Mode #583: This will fail with more explicit errors until when/if this is supported on beacons.
• Can't spawn Shell and can't install Armory on Win mTLS #662: Solved since the changes in the PR Continuing CLI changes #1353.
• Interactive Shell Backgrounding #879: Either solved with CtrlD in sliver menu, or with the new CLI system itself.
• Cancel Running Task Command #940: Solved by current PR, even supports automatic canceling of foreign CLI-invoked commands.
• improve execute-assembly cli parsing ux #944: Much likely this is solved by the current PR and better yet, but someone should check.
• Unable to establish C2 Session DNS implant #975: Probably this PR can help solving/enhancing what is already being done here.
• Alias argument support #979: Same here.
• Saves not changed? #1058: Since the settings command has been kept available in the CLI, maybe an auto-save call
  should be added (even as an option itself ?).
• No --save flag when using nanodump BOF extension #1163: Currently no extensions allow for specifying the file to which to save the ouput, and
  the --save flag is a boolean. Cobra affords us to make it an optarg --flag=optvalue, with
  no drawback on completion and help clarity (those are handled great with the new libs).
  As well, the mere fact of having now full CLI support widens the number of possible approaches.
• Please create cheat sheet on wiki or add example command with "-h" option #1205: The cobra API has great support for various sections of help per-command, with multiple
  output formats possible. Maybe you will want to triage your help package a bit to fit within
  this better.

[maxlandon]

I have to hand-pick a few things related to builder/crackstation permissions in the RPC middleware code, and this should be good.

Lots of things have went in the last commits; the dependencies have now gotten rid of any replace directives,
many fixes in the console/readline, and the carapace completion engine now has an amazing support for default cobra completions (and vice-versa), meaning that anyone not having the carapace binary installed on their systems will still profit from all completers through the classic cobra sliver completion bash script.

[maxlandon]

Tagging a few persons on this as it's the last message I'll be able to send for a long time. End of my journey in computer science.

You guys might be interested (or find out sooner or later that you should have been) by the work that is in there.
You might want to require more eyes and hands, if you feel this is too much for you (spoiler alert, it's not: a bit of work in exchange for years of profit).

Long story short: the most efficient and versatile command-line interface that you will get to see for a C2 program, nothing sort of that. It goes way beyond a closed-loop console. You might even get rid of the console itself.
Everything in here is tailored to that only end.

Video presentation of the interface (the 2nd part is the most interesting here):

https://youtu.be/4rb82RDmmFg?si=y0INhdNySwMOX7Px

Inspecting/discovering/understanding the code (videos done to go through the code, progressively):

https://youtu.be/lUSTPFIfZmU (first part of code introduction)
https://youtu.be/emM96uCD-3s (second part)
https://youtu.be/xOKnZAXMoPY (usage and demos, but probably not useful if you look at the 1.6.0 video above).

The entrypoint packages (client/console, client/cli, server/cli) are heavily commented: that's for until the maintainers won't need any of those anymore.

Teamclient/teamserver code/commands

Last note (sorry for this one): the team library dependency is an external repo of mine. If you don't want it externally, just copy-paste it in your sliver codebase. Some packages in team/internal will need to be client-side.
Also, and since unfortunately no one checked this before introducing permissions in the middleware, there is an adjustment to do somewhere here regarding the --permissions flag: those are not checked in the code here (although present but commented out, in server/transport/middleware.teamserver.initAuthMiddleware()).

Notes to each:

@C-Sto : your extension code has been merged and should normally work like a charm.
@lesnuages: sorry for bothering with lots of shitty PRs in the past. Glad you guys refused them.
@moloch-- : I've learned tons with your tools, was cool for me to try to improve the stuff I could improve.
@RafBishopFox : never interacted with you, but maybe you'll enjoy trying this out with friends or work mates.
@cmprmsd: I saw you had some business with command-line parsing, you might be interested to give help to check how awesome this is working now.

Merge conflicts at this very time (26/12/2023):

• client/extensions: much likely the SliverConsoleClient type that I renamed SliverClient.
• docs/install: don't know, because I literally did git checkout BishopFox/master docs.
• client/command/{server.go,sliver.go}: just keep the files in this PR, not the master branch.
  Also, make sure to check the list of issues that I've pinged in the original description: many of them should be solved through this.
• go.mod: not sure as well. To be on the same page I did git checkout BishopFox/master vendor go.mod go.sum then updated my own console-related dependencies to their latest versions. Normally everything should be identical.

Be sure to check at least the first video. I bet a glass that you won't waste your time.
Godspeed !

[cmprmsd]

hey @maxlandon
I watched your video and certainly I like the improvements to the console (in terms of customization), but I'm not a huge fan of the non-interactive way as you lose many advantages of the interactive shell (notifications etc.)

I can see that some people will like it a lot, but it looks complicated to me and the interactive console is cross platform already so you wouldn't have to mess around with zsh,bash,powershell etc.