Skip to content

Commit

Permalink
Corrected env docs; generate env var docs based on opts-specs (#346)
Browse files Browse the repository at this point in the history 
* Fixed docs

* Generate env var docs from opts-specs (single source of truth)

* STORE_HTTP_REQUESTS should use opts-specs

* Review feedback
  • Loading branch information
@mdemare
mdemare authored Dec 10, 2024
1 parent 01e9615 commit e3357b6
Show file tree
Hide file tree
Showing 5 changed files with 27 additions and 20 deletions.
11 changes: 6 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,8 @@ must be set:
```
API_HOSTNAME Hostname for listing web API
API_PORT HTTP port for serving web API
WORKER_HOSTNAME Hostname for listing worker web API
WORKER_PORT HTTP port for serving werker web API
CLIENTS_INFO_PATH CLients info config file
CLIENTS_INFO_PATH Clients info config file
CONNECTION_TIMEOUT_MILLIS HTTP connection timeout in milliseconds
GATEWAY_PASSWORD OOAPI Gateway Password
GATEWAY_ROOT_URL OOAPI Gateway Root URL
GATEWAY_USER OOAPI Gateway Username
Expand All @@ -151,17 +150,19 @@ KEYSTORE_ALIAS Key alias in keystore
KEYSTORE_PASSWORD Keystore password
REDIS_KEY_PREFIX Prefix for redis keys
REDIS_URI URI to redis
RIO_RETRY_ATTEMPTS_SECONDS Comma-separated list of number of seconds to wait after each RIO retry.
RIO_READ_URL RIO Services Read URL
RIO_RECIPIENT_OIN Recipient OIN for RIO SOAP calls
RIO_RETRY_ATTEMPTS_SECONDS Number of seconds to wait for first, second, etc. retry of RIO command, comma separated
RIO_UPDATE_URL RIO Services Update URL
STATUS_TTL_SEC Number of seconds hours to keep job status
STATUS_TTL_SEC Number of seconds to keep job status
STORE_HTTP_REQUESTS Boolean; should all http traffic be logged? Defaults to true.
SURF_CONEXT_CLIENT_ID SurfCONEXT client id for Mapper service
SURF_CONEXT_CLIENT_SECRET SurfCONEXT client secret for Mapper service
SURF_CONEXT_INTROSPECTION_ENDPOINT SurfCONEXT introspection endpoint
TRUSTSTORE Path to trust-store
TRUSTSTORE_PASSWORD Trust-store password
WORKER_API_HOSTNAME Hostname for listing worker web API
WORKER_API_PORT HTTP port for serving worker web API
```

The `CLIENTS_INFO_PATH` should specify a json file with settings for client-id, schac-home and oin:
Expand Down
7 changes: 6 additions & 1 deletion src/nl/surf/eduhub_rio_mapper/cli_commands.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
(:require [clojure.data.json :as json]
[clojure.java.io :as io]
[clojure.string :as str]
[nl.jomco.envopts :as envopts]
[nl.surf.eduhub-rio-mapper.clients-info :as clients-info]
[nl.surf.eduhub-rio-mapper.config :as config]
[nl.surf.eduhub-rio-mapper.endpoints.api :as api]
[nl.surf.eduhub-rio-mapper.endpoints.worker-api :as worker-api]
[nl.surf.eduhub-rio-mapper.job :as job]
Expand Down Expand Up @@ -119,6 +121,9 @@
(let [[client-info [type id]] (parse-client-info-args args clients)]
(resolver type id (:institution-oin client-info)))

"document-env-vars"
(envopts/specs-description config/opts-spec)

("upsert" "delete" "delete-by-code")
(let [[client-info [type id rest-args]] (parse-client-info-args args clients)
job (merge (assoc client-info
Expand All @@ -132,4 +137,4 @@
name-id id})
{:action command
::ooapi/id id}))]
(job/run! handlers job (= (System/getenv "STORE_HTTP_REQUESTS") "true")))))
(job/run! handlers job (= "true" (:store-http-requests config))))))
17 changes: 10 additions & 7 deletions src/nl/surf/eduhub_rio_mapper/config.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,16 +66,16 @@
:api-hostname ["Hostname for listing web API" :str
:default "localhost"
:in [:api-config :host]]
:worker-api-port ["HTTP port for serving web API" :int
:worker-api-port ["HTTP port for serving worker web API" :int
:default 8080
:in [:worker-api-config :port]]
:worker-api-hostname ["Hostname for listing web API" :str
:worker-api-hostname ["Hostname for listing worker web API" :str
:default "localhost"
:in [:worker-api-config :host]]
:job-retry-wait-ms ["Number of ms to wait before retrying job" :int
:job-retry-wait-ms ["Number of milliseconds to wait before retrying a failed job" :int
:default 5000
:in [:worker :retry-wait-ms]]
:job-max-retries ["Max number of retries of a job" :int
:job-max-retries ["Max number of retries of a failed job" :int
:default 3
:in [:worker :max-retries]]
:redis-uri ["URI to redis" :str
Expand All @@ -88,9 +88,12 @@
:default [5,30,120,600]
:parser parse-int-list
:in [:rio-config :rio-retry-attempts-seconds]]
:status-ttl-sec ["Number of seconds hours to keep job status" :int
:status-ttl-sec ["Number of seconds to keep job status" :int
:default (* 60 60 24 7) ;; one week
:in [:status-ttl-sec]]})
:in [:status-ttl-sec]]
:store-http-requests ["Boolean; should all http traffic be logged? Defaults to true." :str
:default "true"
:in [:store-http-requests]]})

(defn help []
(envopts/specs-description opts-spec))
Expand Down Expand Up @@ -164,7 +167,7 @@
config (update cfg :worker merge
{:queues (clients-info/institution-schac-homes clients)
:queue-fn :institution-schac-home
:run-job-fn #(job/run! handlers % (= (System/getenv "STORE_HTTP_REQUESTS") "true"))
:run-job-fn #(job/run! handlers % (= "true" (:store-http-requests cfg)))
:set-status-fn (status/make-set-status-fn cfg)
:retryable-fn status/retryable?
:error-fn status/errors?})]
Expand Down
2 changes: 1 addition & 1 deletion src/nl/surf/eduhub_rio_mapper/endpoints/status.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@
;; xml response converted to edn.
(assoc :attributes {:opleidingseenheidcode opleidingseenheidcode})

(and (= (System/getenv "STORE_HTTP_REQUESTS") "true")
(and (= "true" (:store-http-requests config))
(#{:done :error :time-out} status)
(-> data :http-messages))
(assoc :http-messages (-> data :http-messages))
Expand Down
10 changes: 4 additions & 6 deletions src/nl/surf/eduhub_rio_mapper/main.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@
(:gen-class))

(def commands
#{"upsert" "delete" "delete-by-code" "get" "show" "resolve" "serve-api" "worker" "help" "dry-run-upsert" "link" "test-rio"})
#{"upsert" "delete" "delete-by-code" "get" "show" "resolve" "serve-api" "worker" "help" "dry-run-upsert" "link"
"document-env-vars" "test-rio"})

(defn -main
[command & args]
Expand All @@ -43,7 +44,7 @@

(let [result (cli-commands/process-command command args (config/make-config-and-handlers))]
(case command
("serve-api" "worker")
("serve-api" "worker" "show" "test-rio")
nil

"get"
Expand All @@ -53,10 +54,7 @@
("dry-run-upsert" "link")
(pprint/pprint result)

("show" "test-rio")
nil

"resolve"
("resolve" "document-env-vars")
(println result)

("upsert" "delete" "delete-by-code")
Expand Down

0 comments on commit e3357b6

Please sign in to comment.