v0.25.6

Release 0.25.6

• Moving from Go 1.17 to 1.18
• Bug fix: Fixes an issue in quickstart "Host it anywhere" where an EXTERNAL_DNS was not added to the PKI causing failures when attempting to use a router from outside the hosted environment.

v0.25.5

Release 0.25.5

• Bug fix: Fixes an issue where dial could fail if the terminator router didn't response to routing last
• Enhancement: Updated Control Channel to use new heartbeat logging mirroring Links in Release 0.25.0
• Enhancement: Added Circuit Creation Timespan which denotes how long the fabric took to construct a requested circuit.

{
    "namespace": "namespace",
    "event_type": "event_type",
    "circuit_id": "circuit_id",
    "timestamp": "2022-04-07T14:00:52.0500632-05:00",
    "client_id": "client_id",
    "service_id": "service_id",
    "creation_timespan": 5000000, //Timespan in nanoseconds
    "path": "path"
}

• Bug fix: Fixes an issue where Edge administrator checks would not take default admin flag into account
• Bug fix: Fix an issue with docker-compose quickstart not properly loading env vars
• Enhancement: Add support for Apple M1 using the ziti quickstart CLI script
• Enhancement: Use an env file for docker-compose quickstart for easier version changes and other duplicated field values
• Enhancement: Allow for version override using the ziti quickstart CLI script
• Change: Renamed pushDevBuild.sh to buildLocalDev.sh, the script used for building a local dev version of the docker quickstart image
• Bug fix: Fixes an issues where isAdmin would always default to false on updates (put/patch)
• Bug fix: Identity property externalId was not properly rendering on GET and not handled consistently on PUT and PATCH
• Enhancement: External JWT Signer Issuer & Audience Validation
• Enhancement: Add ability to define local interface binding for link and controller dial
• Bug fix: Edge Management REST API Doc shows Edge Client REST API Doc
• Enhancement: ziti db explore <ctrl.db> command has been added to explore offline database files
• Enhancement: The mgmt API is now available via websocket. The stream commands are now available on ziti fabric
• Enhancement: Most list commands have been updated with tabular output
• Enhancement: ziti edge show is now available with subcommands config and config-type
  • ziti edge list configs no longer shows the associated json. It can be viewed using ziti edge show config <config name or id>
• Enhancement: ziti edge update config-type is now available
• Enhancement: ziti edge create|update identity now supports --external-id
• Bug fix: Fixes an issue where the router config would use hostname instead of the DNS name
• Bug fix: When establishing links, a link could be closed while being registered, leading the controlller and router to get out of sync
• Enhancement: Add min router cost. Helps to minimize unnecessary hops.
  • Defaults to 10, configurable in the controller config with the minRouterCost value under network:
• Enhancement: Can now see xgress instance and link send buffer pointer values in circuit inspections. This allows correlating to stackdumps
• Enhancement: Can now see xgress related goroutines by using ziti fabric inspect '.*' circuitAndStacks:<circuitId>
• Enhancement: If a router connects to the controller but is already connected, the new connection now takes precedence
  • There is a configurable churn limit, which limits how often this can happen.
  • The default is 1 minute and is settable via routerConnectChurnLimit under network
• Enhancement: Flow control changes
  • Duplicate acks won't shrink window. Duplicate acks imply retransmits and the retransmits already affect the window size
  • Drop min round trip time scaling to 1.5 as will get scaled up as needed by duplicate ack detection
  • Drop round trip time addition to 0 from 100ms and rely purely on scaling
  • Avoid potential stall by always allowing at least one payload into sender side, even when receiver is full.
    • This way if receiver signal to sender is lost, we'll still having something trying to send
• Enhancement: When router reconnects to controller, re-establish any embedded tunneler hosting on that router to ensure router and controller are in sync

External JWT Signer Issuer & Audience Validation

External JWT Signers (endpoint /external-jwt-signers) now support issuer and audience optional string fields.
These fields may be set to null on POST/PUT/PATCH or omitted; which will result in no validation of incoming
JWT's aud and iss fields. If issuer is defined, JWT iss fields will be validated. If audience is defined, JWT
aud fields will be validated. If a JWT contains multiple audience values as an array of strings and will be validated,
validation will check if the External JWT Signer's audience value is present as one of the values.

Add ability to define local interface binding for link and controller dial

The network interface used to dial the controller and router links can be provided in the router configuration file. The interface can be provided as either a name or an IP address.

ctrl:
  endpoint:             tls:127.0.0.1:6262
  bind:                 wlp5s0

link:
  dialers:
    - binding:          transport
      bind:            192.168.1.11

v0.25.4

Release 0.25.4

NOTE: Link management is undergoing some restructuring to support better link costing and multiple interfaces. The link types introduced in 0.25 should not be used. A more complete replacement is coming soon.

• Enhancement: Add additional logging information to tunnel edge routers. Now adds the local address to the router/link chain.
• Enhancement: Add additional metrics for terminator errors.
  • service.dial.terminator.timeout: Raised when the terminator times out when connecting with it's configured endpoint
  • service.dial.terminator.connection_refused: Raised when the terminator cannot connect to it's configured endpoint
  • service.dial.terminator.invalid: Raised when the edge router is unable to get or access the terminator
  • service.dial.terminator.misconfigured: Raised when the fabric is unable to find or create the terminator
• Enhancement: Authentication Policies
• Enhancement: JWT Primary/Secondary Authentication
• Enhancement: Required TOTP (fka MFA) Enrollment
• Bug fix: Fix router panic which can happen on link bind
• Bug fix: Fix router panic which can happen if the router shuts down before it's fully up an running
• Enhancement: Avoid router warning like destination exists for [p57a] by not sending egress in route, since egress will always already be established
• Enhancement: Change default dial retries to 2 from 3
• Enhancement: Add circuit inspect. ziti fabric inspect .* circuit:<circuit-id> will now return information about the circuit from the routers. This will include routing information as well as flow control data from the initiator and terminator.
• Change: Support for link types removed

Authentication Policies

Authentication policies are configuration that allows administrators to enforce authentication requirements. A single
authentication policy is assigned to each identity in the system. This assignment is controlled on the Identity
entities within the Ziti Edge Management API. If an authentication policy is not specified, a system default policy is
applied that. The default policy represents the behavior of Ziti v0.25.3 and earlier and may be updated to the network's
requirements.

Assignment

The Identity entity now supports a new field authPolicyId. In the REST Edge API this field is optional during create
and existing calls to POST /identities will succeed. Every identity must have exactly one authentication policy
assigned to it. If one is not assigned, the default authentication policy will be used (authPolicyId == default)

Example w/o authPolicyId:

POST /edge/v1/management/identities

{
    "name": "zde",
    "type": "User",
    "isAdmin": false,
    "enrollment": {
        "ott": "true"
    },
    "roleAttributes": [
        "dial"
    ]
}

Example w/ authPolicyId:

POST /edge/v1/management/identities

{
    "name": "zde",
    "type": "User",
    "isAdmin": false,
    "enrollment": {
        "ott": "true"
    },
    "roleAttributes": [
        "dial"
    ],
    "authPolicyId": "xyak1."
}

Default Authentication Policy

Ziti contains a single default authentication policy that is marked as a "system" definition. It cannot be deleted,
but it can be updated. This authentication policy has a well known id of default. It can be viewed according to the
following example:

GET /edge/v1/management/auth-policies/default

{
  "data": {
    "_links": {
      "self": {
        "href": "./auth-policies/default"
      }
    },
    "createdAt": "2022-03-30T17:54:55.785Z",
    "id": "default",
    "tags": {},
    "updatedAt": "2022-03-30T17:54:55.785Z",
    "name": "Default",
    "primary": {
      "cert": {
        "allowExpiredCerts": true,
        "allowed": true
      },
      "extJwt": {
        "allowed": true,
        "allowedSigners": null
      },
      "updb": {
        "allowed": true,
        "lockoutDurationMinutes": 0,
        "maxAttempts": 0,
        "minPasswordLength": 5,
        "requireMixedCase": false,
        "requireNumberChar": false,
        "requireSpecialChar": false
      }
    },
    "secondary": {
      "requireExtJwtSigner": null,
      "requireTotp": false
    }
  },
  "meta": {}
}

AuthPolicy Endpoints

The following endpoints were added to support CRUD operations:

• List GET /edge/v1/management/auth-policies
• Create POST /edge/v1/management/auth-policies
• Detail GET /edge/v1/management/auth-policies/{id}
• Replace PUT /edge/v1/management/auth-policies/{id}
• Patch PATCH /edge/v1/management/auth-policies/{id}
• Delete Delete /edge/v1/management/auth-policies/{id}

And have the following properties:

• name: a unique name for the policy
• primary.cert.allowed - allow certificate based authentication
• primary.cert.allowExpiredCerts - allows clients with expired certificates to authenticate
• primary.extJwt.allowed - allow external JWT authentication
• primary.extJwt.allowedSigners - a specific set of external jwt signers that are allowed, if not set all enabled signers are allowed
• primary.updb.allowed - allow username/password authentication
• primary.updb.lockoutDurationMinutes - the number of minutes to lock an identity after exceeding maxAttempts, 0 = indefinite
• primary.updb.minPasswordLength - the minimum lengths passwords must be, currently a placeholder
• primary.updb.requireMixedCase - requires passwords to include mixed cases, currently a placeholder
• primary.updb.requireNumberChar - requires passwords to include at least 1 number, currently a placeholder
• primary.updb.requireSpecialChar - requires passwords to include at least 1 special character, currently a placeholder
• secondary.requireExtJwtSigner - requires an additional JWT bearer token be provided on all API requests, null is disabled
• secondary.requireTotp - requires TOTP (fka MFA enrollment) enrollment to be completed and in use
  Example Create:

{
    "name": "Original Name 1",
    "primary": {
        "cert": {
            "allowExpiredCerts": true,
            "allowed": true
        },
        "extJwt": {
            "allowed": true,
            "allowedSigners": [
                "2BurseGARW"
            ]
        },
        "updb": {
            "allowed": true,
            "lockoutDurationMinutes": 0,
            "maxAttempts": 5,
            "minPasswordLength": 5,
            "requireMixedCase": true,
            "requireNumberChar": true,
            "requireSpecialChar": true
        }
    },
    "secondary": {
        "requireExtJwtSigner": null,
        "requireTotp": false
    },
    "tags": {
        "originalTag1Name": "originalTag1Value"
    }
}

JWT Primary/Secondary Authentication

A new primary authentication mechanism is available in addition to cert and passsword (UPDB). The internal
method name is ext-jwt and it allows authentication by providing a bearer token by a known external JWT signer.
A new entity External JWT Singer has been introduced and is defined in subsequent sections.

Successful primary authentication requires:

1. The target identity must have an authentication policy that allows primary external JWT signer authentication
2. The JWT provided must include a kid that matches the kid defined on an external JWT signer
3. The JWT provided must include a sub (or configured claim) that matches the identity's id or externalId (see below)
4. The JWT provided must be properly signed by the signer defined by kid
5. The JWT provided must be unexpired
6. The encoded JWT must be provided during the initial authentication in the Authorization header with the prefix Bearer and subsequent API calls

A new secondary factor authentication mechanism is available in addition to TOTP (fka MFA). Both TOTP and ext-jwt
secondary authentication factors can be enabled at the same time for a "nFA" setup.

Successful secondary authentication requires all the same JWT token validation items, but as a secondary
factor, not providing a valid JWT bearer token on API requests will drop the request's access to
"partially authenticated" - which has reduced access. Access can be restored by providing a valid JWT bearer token.
Additionally, to turn on the functionality, an authentication policy that has the requireExtJwtSigner field must be
set to a valid external JWT signer and assigned to the target identity(ies).

External JWT Signers

External JWT Signers can be managed on the following new REST Edge Management API endpoints:

• List GET /edge/v1/management/ext-jwt-signers
• Create POST /edge/v1/management/ext-jwt-signers
• Detail GET /edge/v1/management/ext-jwt-signers/{id}
• Replace PUT /edge/v1/management/ext-jwt-signers/{id}
• Patch PATCH /edge/v1/management/ext-jwt-signers/{id}
• Delete Delete /edge/v1/management/ext-jwt-signers/{id}

And support the following properties:

• name - a unique name for the signer
• certPem - a unique PEM x509 certificate for the signer
• enabled - whether the signer is currently enabled or disabled
• externalAuthUrl - the URL clients should use to obtain a JWT
• claimsProperty - the property to alternatively use for the target identity's id or externalId
• useExternalId - whether to match the claimsProperty to id (false) or externalId (true)
• kid - a unique kid value that will be present in a valid JWT's kid header

Example Create:

POST /edge/v1/management/ext-jwt-signers

{
    "certPem": "-----BEGIN CERTIFICATE-----\nMIIBizC ...",
    "enabled": true,
    "kid": "c7e2081d-b8f0-44b1-80fa-d73872692fd6",
    "name": "Test JWT Signer Pre-Patch Kid",
    "externalAuthUrl" : "https://my-jwt-provide/auth",
    "claimsProperty": "email",
    "useExternalId": "true"
}

The above example creates a new signer that is enabled and that will instruct clients that they can attempt to obtain
a JWT from https://my-jwt-provide/auth. The JWT that is returned from `https://my-jwt-prov...

v0.25.3

Release 0.25.3

• Enhancement: Add cost and precedence to host.v1 and host.v2 config types. This allows router-embedded tunnelers the ability to handle HA failover scenarios.
• Bug fix: Router link listener type was only inferred from the adverise address, not the bind address

v0.25.2

Release 0.25.2

Deprecations

The Ziti Edge management REST /database and /terminators endpoints are being deprecated. They belong in the
fabric management API, but there was no fabric REST api at the time when they were added. Now that they are
available under fabric, they will be removed from the edge APIs in a future release, v0.26 or later.

What's New

• Enhancement: Only translate router ids -> names in ziti edge traceroute when requested to with flag
• Enhancement: Add the /database rest API from edge to fabric, where they below
  • ziti fabric db now as the same commands as ziti edge db
• Enhancement: Add ziti agent command for sending IPC commands. Contains copy of what was under ziti ps.
• Enhancement: Add ziti agent controller snapshot-db <name or pid> IPC command

v0.25.1

Release 0.25.1

• Bug fix: Fix panic caused by race condition at router start up
  • Regression since 0.25.0

v0.25.0

Release 0.25.0

Breaking Changes

Routers with version 0.25.0 or greater must be used with a controller that is also v0.25 or greater.
Controllers will continue to work with older routers. Router of this version should also continue to interoperate with older routers.

NOTE: You may be used to seeing two links between routers, if they both have link listeners. Starting with v0.25 expect to see only
a single link between routers, unless you use the new link types feature.

What's New

• Bug fix: Fixed an issue with the ziti CLI quickstart routine which also affected router and controller config generation leaving many config fields blank or incorrect.
  • Note: This fix was previously reported to have been fixed in 0.24.13 but the fix was actually applied to this release.
• Enhancement: Router Link Refactor
  • Support for multiple link types
  • Existing link notifications
  • Link heartbeats/latency have changed
  • Inspect and ps upport for links
  • Router version dissemination
  • Distributed control preparation
• Enhancement: ziti fabric list routers now includes the link listener types and advertise addresses

Router Link Refactor

Multiple Link Types

Routers can now configure multiple link listeners. Listeners now support an option 'type' attribute. If no type is provided, the link type will be derived from the address. For example, given the following configuration:

link:
  dialers:
    - binding:          transport
  listeners:
    - binding:          transport
      bind:             tls:127.0.0.1:7878
      advertise:        tls:127.0.0.1:7878

    - binding:          transport
      bind:             tls:127.0.0.1:5876
      advertise:        tls:127.0.0.1:5876
      type: cellular

The first listener will have a type of tls and the second listener will have a type of cellular.

Routers will now try to maintain one link of each type available on the target router.

When using ziti fabric list links the link type will now be shown.

Existing link notifications

As the controller doesn't persist links, when the controller restarts or loses connection it loses all information about router links. Routers can now notify the controller about existing links when they reconnect. If they receive a link dial request for a link that they already have (based on the target router and link type), they can now report back the existing link. This should prevent the number of links to remain relatively constant.

Link Heartbeats

Because we are now limiting the number of links it is even more vital to ensure that links are healthy, and to respond quickly when links become unresponsive. To that end links now use heartbeats. As data flows across the link, heartbeat headers will be added periodically. Heartbeat responses will be added to return messages. If the link is currently quiet, explicit heartbeat messages will be sent. Heartbeats will also be used to measure latency. If heartbeats are unreturned for a certain amount of time, the link will be considered bad and torn down, so a new one can be established.

The link.latency metric now is calculated starting when the message is about to be sent. It may have a few extra milliseconds time, as the response waits briefly to see if there's an existing message that the response can piggyback on.

Previously link.latency include both queue and network time. Now that it only has network time, there's a new metrics, link.queue_time which tracks how long it takes messages to get from send requested to just before send.

Inspect and ps support for links

ziti fabric inspect .* links can now be used to see what links each router knows about. This can be useful to determine if/how the controller and routers may have gotten out of sync.

Router can also be interrogated directly for their links via IPC, using ziti ps.

$ ziti ps router dump-links 275061
id: 4sYO18tZ1Fz4HByXuIp1Dq dest: o.oVU2Qm. type: tls
id: 19V7yhjBpHAc2prTDiTihQ dest: hBjIP2wmxj type: tls

Router version dissemination

Routers now get the version of the router they are dialing a link to, and pass their own version to that router as part of the dial. This allows routers to only enable specific features if both sides of the link support it.

Distributed Control preparation

Giving the routers have more control over the links prepares us for a time when routers may be connected to multiple controllers. Routers will be able to notify controllers of existing links and will be prepared to resolve duplicate link dial requests from multiple sources.

v0.24.13

Release 0.24.13

• Bug fix: Fixed an issue with the ziti CLI quickstart routine which also affected router and controller config generation leaving many config fields blank or incorrect.
• Enhancement: Added new noTraversal field to routers. Configures if a router should allow/disallow traversal. Required on create/update commands.
• Enhancement: ziti edge update edge-router now supports either --no-traversal flag which will allow/disallow a given router from being used to traverse.
• Enhancement: ziti fabric list routers and ziti edge list routers will now display the noTraversal flag of associated routers.
• Feature: 1st Party Certificate Extension

1st Party Certificate Extension

Ziti Edge Client and Management API both support certificate extension for Ziti provisioned certificates. Before a
client certificate expires, the client can elect to generate a new client certificate that will extend its valid period
and allows the client to optionally utilize a new private key.

Process Outline:

1. The client enrolls, obtaining a client certificate that is signed by the Ziti Controller
2. The client authenticates
3. The client provides a CSR
4. The client receives a new public certificate
5. The client verifies with the controller the new public certificate has been obtained

Detailed Outline

The client enrolls and authenticates with the controller as normal. If the client wishes to extend its client certificate,
it can request that at any time by doing:

POST /edge/{client|management}/current-identity/authenticators/{id}/extend

{
  "clientCertCsr": "-----BEGIN NEW CERTIFICATE REQUEST-----\n..."
}

If the authenticator specified by {id} is a certificate based authenticator and provisioned by Ziti, it will be allowed.
If not, 4xx HTTP status code errors will be returned outlining the issue. If ok, a 200 OK will be returned in the format of:

{
  "clientCert": "-----BEGIN CERTIFICATE-----\n....",
  "ca": ""-----BEGIN CERTIFICATE-----\n...."
}

At this point the controller will have stored the new certificate, but it is not usable for authentication until the client
proves that is has properly stored the client certificate. This verification is done by sending the client certificate
back to the controller:

POST /edge/{client|management}/current-identity/authenticators{id}/extend-verify
{
  "clientCert": "-----BEGIN CERTIFICATE-----\n...."
}

On success, 200 OK is returned and the new client certificate should be used for all future authentication requests.

v0.24.12

Release 0.24.12

• Enhancement: Allow xgress proxy configuration in router config to accept service id or service name
• Build: Docker build process fixes

v0.24.11

Release 0.24.11

• Bug fix: Fix ziti CLI env. Config was getting set to current directory, instead of defaulting to under $HOME
• Enhancement: Go tunneler support for service-side resolution of SRV, MX, TXT records for wildcard domains