diff --git a/docs/build/credentials.md b/docs/build/credentials.md index df94f9825c7b..c5eba989f7c4 100644 --- a/docs/build/credentials.md +++ b/docs/build/credentials.md @@ -5,30 +5,33 @@ title: Credentials ## Credentials Credentials are used to authorize connections to destination systems. In the -future, our adaptors will use credentials to fetch meta-data from source and +future, our Adaptors will use credentials to fetch meta-data from source and destination applications and make the job writing process easier. -Some systems (Salesforce, OpenMRS, DHIS2) require an instanceUrl, host, or -ApiUrl. Leave off the final "/" in these Urls: `https://login.salesforce.com` or -`http://demo.openmrs.org/openmrs` or `https://play.dhis2.org`. - Credentials can only be viewed, or edited by a single user — their "owner" (or the person that created that credential). All the collaborators on a particular -project can choose those credentials for use when defining a job. +Project can choose those credentials for use when defining a job. + +![Credentials Page](/img/settings_credentials.png) -There are two special types of credentials, in addition to the myriad standard -application-specific and authentication protocol-specific credentials. +### Create a new Credential -### Raw Credentials +You can create a new Credential while configuring a new Step in your Workflow, +or via the Settings > Credentials page. +[Read this](/documentation/next/manage-projects/manage-credentials) for more on +managing credentials. -Raw credentials are valid JSON documents which are passed into a job's runtime -state. Note that owners of these credentials will be able to view them, in their -entirety, in the clear. +### Understand the app-specific credentials -### Keychain Credentials +Check out the dedicated [Adaptor docs](/adaptors) page for your app to inspect +the `configuration schema` and see what credential details will be required to +authenticate with your app (e.g., `username`, `api_key`). -Keychain credentials allow for a single job to make use of multiple credentials. -They work by inspecting the data in the job's runtime state (i.e., `state.data`) -and checking for the value of a predetermined identifier. Based on that value, -present in the data for a given source message, for example, _another_ -credential will be selected and applied for that particular job run. +If your app is not listed in the Adaptors section, then inspect your app's API +documentation to see what is required for "authentication". You can then create +a `Raw JSON` Credential in OpenFn to define whatever credential inputs are +require (e.g., `{"api_key": "ADD-your-let", "url": "add-url"}`). + +Some systems (Salesforce, OpenMRS, DHIS2) require an instanceUrl, host, or +ApiUrl. Leave off the final "/" in these Urls: `https://login.salesforce.com` or +`http://demo.openmrs.org/openmrs` or `https://play.dhis2.org`. diff --git a/docs/build/paths.md b/docs/build/paths.md new file mode 100644 index 000000000000..b76638a93598 --- /dev/null +++ b/docs/build/paths.md @@ -0,0 +1,42 @@ +--- +title: Paths & Path Conditions +sidebar_label: Paths +--- + +A Path is both a visual and functional indication defining the sequence of Steps +the Workflow follows when executed. Read on more on the different types of Paths +and configuration tips. + +## Path Conditions + +There are 4 types of Path Conditions that define whether the Workflow will +proceed to the next Step when executed: + +1. Always (the next Step will always run after the execution of the prior Step + is completed) +2. On Success (the next Step will run only if the execution of the prior Step + _succeeded_) +3. On Failure (the next Step will run only if the execution of the prior Step + _failed_) +4. Matches a JavaScript Expression (the next Step will only run if the condition + or custom expression evaluates to be true) + +![Path Conditions](/img/path_conditions.png) + +## Writing JavaScript Expressions for Custom Path Conditions + +Write your own JavaScript expression if you want to define a **custom +condition** that evaluates the initial state of the step. + +The workflow will only continue to the next step if if the JavaScript expression +evaluates to be true. + +![Custom Conditions](/img/path_js_expression.png) + +## Disable Paths to deactivate + +To "deactivate" part of your Workflow and the Steps that follow a specific Path +sequence: + +1. Click on the `Path` you want to deactive +2. Select the `Disable this path` checkbox diff --git a/docs/build/steps/job-examples.md b/docs/build/steps/job-examples.md new file mode 100644 index 000000000000..8a9e9fa3635d --- /dev/null +++ b/docs/build/steps/job-examples.md @@ -0,0 +1,534 @@ +--- +title: Job Code Examples +sidebar_label: Job Code Snippets & Examples +--- + +## Snippets and samples + +Below you can find some code block for different functions and data +handling contexts to use in your Jobs. **Also see the [Library Examples](/adaptors/library) for more Job examples for other Adaptors.** + +:::info Questions? + +If you have any job-writing questions, ask on [Community](https://community.openfn.org) to seek assistance from the OpenFn core team and other implementers. + +::: + +### Job expression (for CommCare to SF) + +The following job expression will take a matching receipt and use data from that +receipt to upsert a `Patient__c` record in Salesforce and create multiple new +`Patient_Visit__c` (child to Patient) records. + +```js +upsert( + 'Patient__c', + 'Patient_Id__c', + fields( + field('Patient_Id__c', dataValue('form.patient_ID')), + relationship('Nurse__r', 'Nurse_ID_code__c', dataValue('form.staff_id')), + field('Phone_Number__c', dataValue('form.mobile_phone')) + ) +), + each( + join('$.data.form.visits[*]', '$.references[0].id', 'Id'), + create( + 'Visit__c', + fields( + field('Patient__c', dataValue('Id')), + field('Date__c', dataValue('date')), + field('Reason__c', dataValue('why_did_they_see_doctor')) + ) + ) + ); +``` + +### Accessing the "data array" in Open Data Kit submissions + +Notice how we use "each" to get data from each item inside the "data array" in +ODK. + +```js +each( + '$.data.data[*]', + create( + 'ODK_Submission__c', + fields( + field('Site_School_ID_Number__c', dataValue('school')), + field('Date_Completed__c', dataValue('date')), + field('comments__c', dataValue('comments')), + field('ODK_Key__c', dataValue('*meta-instance-id*')) + ) + ) +); +``` + +### ODK to Salesforce: create parent record with many children from parent data + +Here, the user brings `time_end` and `parentId` onto the line items from the +parent object. + +```js +each( + dataPath('data[*]'), + combine( + create( + 'transaction__c', + fields( + field('Transaction_Date__c', dataValue('today')), + relationship( + 'Person_Responsible__r', + 'Staff_ID_Code__c', + dataValue('person_code') + ), + field('metainstanceid__c', dataValue('*meta-instance-id*')) + ) + ), + each( + merge( + dataPath('line_items[*]'), + fields( + field('end', dataValue('time_end')), + field('parentId', lastReferenceValue('id')) + ) + ), + create( + 'line_item__c', + fields( + field('transaction__c', dataValue('parentId')), + field('Barcode__c', dataValue('product_barcode')), + field('ODK_Form_Completed__c', dataValue('end')) + ) + ) + ) + ) +); +``` + +> **NB - there was a known bug with the `combine` function which has been +> resolved. `combine` can be used to combine two operations into one and is +> commonly used to run multiple `create`'s inside an `each(path, operation)`. +> The source code for combine can be found here: +> [language-common: combine](https://github.com/OpenFn/language-common/blob/master/src/index.js#L204-L222)** + +### Create many child records WITHOUT a repeat group in ODK + +```js +beta.each( + '$.data.data[*]', + upsert( + 'Outlet__c', + 'Outlet_Code__c', + fields( + field('Outlet_Code__c', dataValue('outlet_code')), + field('Location__Latitude__s', dataValue('gps:Latitude')), + field('Location__Longitude__s', dataValue('gps:Longitude')) + ) + ) +), + beta.each( + '$.data.data[*]', + upsert( + 'Outlet_Call__c', + 'Invoice_Number__c', + fields( + field('Invoice_Number__c', dataValue('invoice_number')), + relationship('Outlet__r', 'Outlet_Code__c', dataValue('outlet_code')), + relationship('RecordType', 'name', 'No Call Card'), + field('Trip__c', 'a0FN0000008jPue'), + relationship( + 'Sales_Person__r', + 'Sales_Rep_Code__c', + dataValue('sales_rep_code') + ), + field('Date__c', dataValue('date')), + field('Comments__c', dataValue('comments')) + ) + ) + ); +``` + +### Salesforce: perform an update + +```js +update("Patient__c", fields( + field("Id", dataValue("pathToSalesforceId")), + field("Name__c", dataValue("patient.first_name")), + field(...) +)); +``` + +### Salesforce: Set record type using 'relationship(...)' + +```js +create( + 'custom_obj__c', + fields( + relationship( + 'RecordType', + 'name', + dataValue('submission_type'), + field('name', dataValue('Name')) + ) + ) +); +``` + +### Salesforce: Set record type using record Type ID + +```js +each( + '$.data.data[*]', + create( + 'fancy_object__c', + fields( + field('RecordTypeId', '012110000008s19'), + field('site_size', dataValue('size')) + ) + ) +); +``` + +### Telerivet: Send SMS based on Salesforce workflow alert + +```js +send( + fields( + field( + 'to_number', + dataValue( + 'Envelope.Body.notifications.Notification.sObject.phone_number__c' + ) + ), + field('message_type', 'sms'), + field('route_id', ''), + field('content', function (state) { + return 'Hey there. Your name is '.concat( + dataValue('Envelope.Body.notifications.Notification.sObject.name__c')( + state + ), + '.' + ); + }) + ) +); +``` + +### HTTP: fetch but don't fail! + +```js +// ============= +// We use "fetchWithErrors(...)" so that when the +// SMS gateway returns an error the run does not "fail". +// It "succeeds" and then delivers that error message +// back to Salesforce with the "Update SMS Status" job. +// ============= +fetchWithErrors({ + getEndpoint: 'send_to_contact', + query: function (state) { + return { + msisdn: + state.data.Envelope.Body.notifications.Notification.sObject + .SMS__Phone_Number__c, + message: + state.data.Envelope.Body.notifications.Notification.sObject + .SMS__Message__c, + api_key: 'some-secret-key', + }; + }, + externalId: state.data.Envelope.Body.notifications.Notification.sObject.Id, + postUrl: 'https://www.openfn.org/inbox/another-secret-key', +}); +``` + +### Sample DHIS2 events API job: + +```js +event( + fields( + field('program', 'eBAyeGv0exc'), + field('orgUnit', 'DiszpKrYNg8'), + field('eventDate', dataValue('properties.date')), + field('status', 'COMPLETED'), + field('storedBy', 'admin'), + field('coordinate', { + latitude: '59.8', + longitude: '10.9', + }), + field('dataValues', function (state) { + return [ + { + dataElement: 'qrur9Dvnyt5', + value: dataValue('properties.prop_a')(state), + }, + { + dataElement: 'oZg33kd9taw', + value: dataValue('properties.prop_b')(state), + }, + { + dataElement: 'msodh3rEMJa', + value: dataValue('properties.prop_c')(state), + }, + ]; + }) + ) +); +``` + +### Sample DHIS2 data value sets API job: + +```js +dataValueSet( + fields( + field('dataSet', 'pBOMPrpg1QX'), + field('orgUnit', 'DiszpKrYNg8'), + field('period', '201401'), + field('completeData', dataValue('date')), + field('dataValues', function (state) { + return [ + { dataElement: 'f7n9E0hX8qk', value: dataValue('prop_a')(state) }, + { dataElement: 'Ix2HsbDMLea', value: dataValue('prop_b')(state) }, + { dataElement: 'eY5ehpbEsB7', value: dataValue('prop_c')(state) }, + ]; + }) + ) +); +``` + +### sample openMRS expression, creates a person and then a patient + +```js +person( + fields( + field('gender', 'F'), + field('names', function (state) { + return [ + { + givenName: dataValue('form.first_name')(state), + familyName: dataValue('form.last_name')(state), + }, + ]; + }) + ) +), + patient( + fields( + field('person', lastReferenceValue('uuid')), + field('identifiers', function (state) { + return [ + { + identifier: '1234', + identifierType: '8d79403a-c2cc-11de-8d13-0010c6dffd0f', + location: '8d6c993e-c2cc-11de-8d13-0010c6dffd0f', + preferred: true, + }, + ]; + }) + ) + ); +``` + +### merge many values into a child path + +```js +each( + merge( + dataPath("CHILD_ARRAY[*]"), + fields( + field("metaId", dataValue("*meta-instance-id*")), + field("parentId", lastReferenceValue("id")) + ) + ), + create(...) +) +``` + +### arrayToString + +```js +arrayToString(arr, separator_string); +``` + +### access an image URL from an ODK submission + +```js +// In ODK the image URL is inside an image object... +field("Photo_URL_text__c", dataValue("image.url")), +``` + +### alterState (alter state) to make sure data is in an array + +```js +// Here, we make sure CommCare gives us an array to use in each(merge(...), ...) +fn(state => { + const idCards = state.data.form.ID_cards_given_to_vendor; + if (!Array.isArray(idCards)) { + state.data.form.ID_cards_given_to_vendor = [idCards]; + } + return state; +}); + +// Now state has been changed, and we carry on... +each( + merge( + dataPath('form.ID_cards_given_to_vendor[*]'), + fields( + field('Vendor_Id', dataValue('form.ID_vendor')), + field('form_finished_time', dataValue('form.meta.timeEnd')) + ) + ), + upsert( + 'Small_Packet__c', + 'sp_id__c', + fields( + field('sp_id__c', dataValue('ID_cards_given_to_vendor')), + relationship('Vendor__r', 'Badge_Code__c', dataValue('Vendor_Id')), + field( + 'Small_Packet_Distribution_Date__c', + dataValue('form_finished_time') + ) + ) + ) +); +``` + +### Login in to a server with a custom SSL Certificate + +This snippet describes how you would connect to a secure server ignoring SSL +certificate verification. Set `strictSSL: false` in the options argument of the +`post` function in `language-http`. + +```js +post( + `${state.configuration.url}/${path}`, + { + headers: { 'content-type': 'application/json' }, + body: { + email: 'Luka', + password: 'somethingSecret', + }, + strictSSL: false, + }, + callback +); +``` + +## Anonymous Functions + +Different to [Named Functions](#examples-of-adaptor-specific-functions), +Anonymous functions are generic pieces of javascript which you can write to suit +your needs. Here are some examples of these custom functions: + +### Custom replacer + +```js +field('destination__c', state => { + console.log(something); + return dataValue('path_to_data')(state).toString().replace('cats', 'dogs'); +}); +``` + +This will replace all "cats" with "dogs" in the string that lives at +`path_to_data`. + +> **NOTE:** The JavaScript `replace()` function only replaces the first instance +> of whatever argument you specify. If you're looking for a way to replace all +> instances, we suggest you use a regex like we did in the +> [example](#custom-concatenation-of-null-values) below. + +### Custom arrayToString + +```js +field("target_specie_list__c", function(state) { + return Array.apply( + null, sourceValue("$.data.target_specie_list")(state) + ).join(', ') +}), +``` + +It will take an array, and concatenate each item into a string with a ", " +separator. + +### Custom concatenation + +```js +field('ODK_Key__c', function (state) { + return dataValue('metaId')(state).concat('(', dataValue('index')(state), ')'); +}); +``` + +This will concatenate two values. + +### Concatenation of null values + +This will concatenate many values, even if one or more are null, writing them to +a field called Main_Office_City_c. + +```js +... + field("Main_Office_City__c", function(state) { + return arrayToString([ + dataValue("Main_Office_City_a")(state) === null ? "" : dataValue("Main_Office_City_a")(state).toString().replace(/-/g, " "), + dataValue("Main_Office_City_b")(state) === null ? "" : dataValue("Main_Office_City_b")(state).toString().replace(/-/g, " "), + dataValue("Main_Office_City_c")(state) === null ? "" : dataValue("Main_Office_City_c")(state).toString().replace(/-/g, " "), + dataValue("Main_Office_City_d")(state) === null ? "" : dataValue("Main_Office_City_d")(state).toString().replace(/-/g, " "), + ].filter(Boolean), ',') + }) +``` + +> Notice how this custom function makes use of the **regex** `/-/g` to ensure +> that all instances are accounted for (g = global search). + +### Custom Nth reference ID + +If you ever want to retrieve the FIRST object you created, or the SECOND, or the +Nth, for that matter, a function like this will do the trick. + +```js +field('parent__c', function (state) { + return state.references[state.references.length - 1].id; +}); +``` + +See how instead of taking the id of the "last" thing that was created in +Salesforce, you're taking the id of the 1st thing, or 2nd thing if you replace +"length-1" with "length-2". + +### Convert date string to standard ISO date for Salesforce + +```js +field('Payment_Date__c', function (state) { + return new Date(dataValue('payment_date')(state)).toISOString(); +}); +``` + +> **NOTE**: The output of this function will always be formatted according to +> GMT time-zone. + +### Use external ID fields for relationships during a bulk load in Salesforce + +```js +array.map(item => { + return { + Patient_Name__c: item.fullName, + 'Account.Account_External_ID__c': item.account + 'Clinic__r.Unique_Clinic_Identifier__c': item.clinicId, + 'RecordType.Name': item.type, + }; +}); +``` + +### Bulk upsert with an external ID in salesforce + +```js +bulk( + 'Visit_new__c', + 'upsert', + { + extIdField: 'commcare_case_id__c', + failOnError: true, + allowNoOp: true, + }, + dataValue('patients') +); +``` \ No newline at end of file diff --git a/docs/build/steps/job-expressions.md b/docs/build/steps/job-expressions.md index 7b2440f095b8..3b3bafdc7f8f 100644 --- a/docs/build/steps/job-expressions.md +++ b/docs/build/steps/job-expressions.md @@ -1,12 +1,21 @@ --- title: Write Job expressions +sidebar_label: Write Jobs --- To define the business logic and data transformation rules or logic for individual `Steps` in your workflow, you will need to write a `Job`. This article will provide a basic overview of Job expressions & writing tips. -## About Job expressions +:::tip + +For example Jobs written by the OpenFn core team and other users, check out the +[Library](/adaptors/library) or other project repositories under +[Github.com/OpenFn](https://github.com/OpenFn). + +::: + +## About Jobs A `Job` is evaluated as a JavaScript expression and primarily defines the specific series of [Operations](/docs/build/steps/operations.md) (think: tasks, @@ -67,8 +76,19 @@ time that the operation (`create` in the above expression) is executed. ### A Job with custom JavaScript -See below for an example Job with custom JavaScript code for data -transformation. +To write your own custom JavaScript functions, simply add an `fn(...)` block to +your code as below. + +```js +fn(state => { + //write your own function to manipulate/transform state + return state; +}); +``` + +Alternatively, you can add custom JavaScript code in-line any Adaptor-specific +functions. See example job below where JavaScript was added to transform the +data value outputted for `Name`. ```js create( diff --git a/docs/build/steps/multiple-operations.md b/docs/build/steps/multiple-operations.md index dfd2e7a0ff22..cad33a9576c6 100644 --- a/docs/build/steps/multiple-operations.md +++ b/docs/build/steps/multiple-operations.md @@ -1,5 +1,5 @@ --- -title: Chaining multiple operations in one Job +title: Chaining operations in 1 Step sidebar_label: Chaining operations --- @@ -51,7 +51,4 @@ This page describes why you might want to chain multiple operations in a single ) ); ``` - -``` - -``` +Check out the Job [Library Examples](/adaptors/library/) for more examples. diff --git a/docs/build/steps/step-editor.md b/docs/build/steps/step-editor.md index 313f8602b088..794e9d53bddb 100644 --- a/docs/build/steps/step-editor.md +++ b/docs/build/steps/step-editor.md @@ -1,9 +1,14 @@ --- title: Edit Steps via the Inspector -sidebar_label: Edit Steps +sidebar_label: Edit & Test Steps --- -Use the "Inspector" interface on the platform to create, edit, and test Steps. +This page outlines how to edit and test Steps in your Workflow using the +Inspector interface. + +## Edit & Test Steps via the Inspector + +Use the `Inspector` interface on the platform to create, edit, and test Steps. (If you built your Workflow locally using the CLI, you can edit your Jobs on the app via this interface.) @@ -15,4 +20,27 @@ To access this interface: For more on how to write custom business logic and data transformation rules in the `Editor`, see the docs on -[writing Job expressions](/docs/build/steps/job-expressions.md). +[writing Jobs](/documentation/next/build/steps/job-expressions) and check out +the below video. + + + +## Run & Test Steps + +When running Steps to test the configuration, every Run will have an initial +state (which may contain an `Input`) and results in a final state that will +include `Logs` and an `Output`. + +- `Input` - Data (JSON) that is used as the starting Input for a Step to utilise + in its run. An Input can exist for a Work Order and individual Steps within a + Run, though it is possible for either to exist without an Input. +- `Output` - Data (JSON) that is created as the Output of a Step's execution. An + Output can exist for a Work Order and individual jobs within a run, and + typically contains the data sent to the target app. +- `Logs` - A record generated by the workflow execution engine that details the + activities performed when running a Workflow or individual Step. + +See [Writing Jobs docs](/documentation/next/build/steps/job-expressions) for +more on writing custom logic, and see +[this article](/documentation/next/build/steps/state) for more on the concept of +"state" when writing Jobs and building OpenFn Workflows. diff --git a/docs/build/steps/steps.md b/docs/build/steps/steps.md index 81ca33052f6c..3fbae426f9de 100644 --- a/docs/build/steps/steps.md +++ b/docs/build/steps/steps.md @@ -15,7 +15,7 @@ click on existing Step to view or configure its key components. When configuring a Step, you must understand its basic anatomy. -![Step Anatomy](/img/step-anatomy.png) +![Step Anatomy](/img/anatomy_of_step.png) A Step includes these key components: @@ -27,10 +27,33 @@ A Step includes these key components: learn more. - `Credentials` - The Credential used to authorize connections to the target app related to this Step. -- `Expression` - The step "script" or `Job expression` that defines the business - logic and/or sequence of operations to be executed +- `Job` - The custom code that defines the business logic and/or sequence of + operations to be executed in the connected app. -## Choose an Adaptor +:::tip Writing Jobs + +Writing Jobs to add custom logic for business or data transformation rules +typically requires basic knowledge of JavaScript. See the +[Job-writing docs](/documentation/next/build/job-expressions) for a detailed +overview and the [Library Examples](/adaptors/library) for sample code. + +::: + +## 1. Name your Step + +First, give your Step a `Name` that describes its purpose (e.g., +`create patient`, `map form data`); + +## 2. Choose an Adaptor + +Next, select an `Adaptor` to define which app your Step will connect with. + +:::tip + +Each Step can only have 1 Adaptor. If you want to connect with 2 different apps, +you should create 2 different Steps. + +::: We've got a whole section on creating new [Adaptors](/adaptors), but the critical thing to be aware of when writing a step is that you've got to choose @@ -45,7 +68,12 @@ For example, `create` means one thing in the `salesforce` Adaptor and another thing entirely in `dhis2`. For this reason, before you can begin writing a step you have to decide which [Adaptor](/adaptors/) to work with. -### Choose an Adaptor Version +### 3.Choose an Adaptor Version + +Pick the Adaptor Version you want to use. We recommend selecting the latest +version available, unless you want to use an older version that is compatible +with an older version of the API you're connecting with. See the +[Adaptor docs](/adaptors) for details on each Adaptor. Adaptors change over time. They're open source, and we encourage as much contribution as possible—releasing new versions for use on OpenFn.org as soon as @@ -76,7 +104,7 @@ Versions for run f470a3da-8b90-480e-a94f-6dd982c91afe: ...more logs here... ``` -### Managing Adaptor Versions +#### Managing Adaptor Versions While it may be beneficial to upgrade as part of your routine maintenance, these upgrades should be carefully tested. Most often, customers upgrade to a new @@ -101,29 +129,10 @@ want this and to avoid the risk of accidental upgrades on live Workflows. ::: -## Run & Test Steps - -When running Steps to test the configuration, every Run will have an initial -state (which may contain an `Input`) and results in a final state that will -include `Logs` and an `Output`. - -- `Input` - Data (JSON) that is used as the starting Input for a Step to utilise - in its run. An Input can exist for a Work Order and individual Steps within a - Run, though it is possible for either to exist without an Input. -- `Output` - Data (JSON) that is created as the Output of a Step's execution. An - Output can exist for a Work Order and individual jobs within a run, and - typically contains the data sent to the target app. -- `Logs` - A record generated by the workflow execution engine that details the - activities performed when running a Workflow or individual Step. - -See [Edit Steps](/docs/build/steps/step-editor.md) for more on making changes -and testing, and see [this article](/docs/build/steps/state.md) for more on the -concept of "state" when writing Jobs and building OpenFn Workflows. - -## Add business logic or data transformation rules +### 4. Write a Job for custom business logic or data transformation rules Click the code button `` displayed on the configuration panel to write or edit a Job expression to define the "rules" or the specific tasks to be completed by your Step. See the pages on [the Inspector](/docs/build/steps/step-editor.md) and -[writing Job expressions](/docs/build/steps/job-expressions.md) to learn more. +[writing Jobs](/docs/build/steps/job-expressions.md) to learn more. diff --git a/docs/build/steps/test-steps.md b/docs/build/steps/test-steps.md new file mode 100644 index 000000000000..71783ad443fe --- /dev/null +++ b/docs/build/steps/test-steps.md @@ -0,0 +1,22 @@ +--- +title: Configure Steps +--- + +## Run & Test Steps + +When running Steps to test the configuration, every Run will have an initial +state (which may contain an `Input`) and results in a final state that will +include `Logs` and an `Output`. + +- `Input` - Data (JSON) that is used as the starting Input for a Step to utilise + in its run. An Input can exist for a Work Order and individual Steps within a + Run, though it is possible for either to exist without an Input. +- `Output` - Data (JSON) that is created as the Output of a Step's execution. An + Output can exist for a Work Order and individual jobs within a run, and + typically contains the data sent to the target app. +- `Logs` - A record generated by the workflow execution engine that details the + activities performed when running a Workflow or individual Step. + +See [Edit Steps](/docs/build/steps/step-editor.md) for more on making changes +and testing, and see [this article](/docs/build/steps/state.md) for more on the +concept of "state" when writing Jobs and building OpenFn Workflows. \ No newline at end of file diff --git a/docs/build/triggers-cron.md b/docs/build/triggers-cron.md deleted file mode 100644 index abec1ba55b1a..000000000000 --- a/docs/build/triggers-cron.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Cron Triggers ---- -# Cron Triggers -Incl. cron expressions diff --git a/docs/build/triggers-webhook.md b/docs/build/triggers-webhook.md deleted file mode 100644 index 3f8e11556ba6..000000000000 --- a/docs/build/triggers-webhook.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Webhook Triggers ---- -# Webhook Triggers -Incl. authentication methods diff --git a/docs/build/triggers.md b/docs/build/triggers.md index 74844d9a3ac4..4301a60f9cf1 100644 --- a/docs/build/triggers.md +++ b/docs/build/triggers.md @@ -3,113 +3,49 @@ title: Triggers --- Triggers are responsible for starting workflows automatically. They come in 2 -types: "cron" triggers and "webhook event" triggers. +types: "cron" triggers and "webhook event" triggers. ## Trigger types -### Message Filter Triggers +### Webhook Event Triggers -Message Filter triggers watch for inbound messages and check to see if the data -in those messages meet their **inclusion criteria** and _don't_ meet their -**exclusion criteria**. If they pass these tests and if there are active jobs -configured to use that trigger, a run will be started for each message/job -combination. +`Webhook Event Triggers` watch for inbound http requests or messages, and enable +real-time event-based automation. -You, the user, specify the inclusion and exclusion criteria which determines -which inbound messages should trigger job runs. Broadly speaking, if part of a -message body **matches** the JSON you provide as the inclusion filter, and -_doesn't_ match the JSON you provided as the exclusion filter, a job will run -(assuming you created one with `autoprocess` turned on). +- These Triggers are fired by “pushing” data to OpenFn (i.e., by sending a HTTP + “POST” request to your trigger’s designated URL). +- This triggering HTTP request might be sent via a webhook in an external app, + another OpenFn workflow, or manually (i.e., via cURL request). -The filter criteria takes the form of a string of valid JSON like this: -`{"Name":"Aleksa Iwobi"}`. In an SQL query, this string will be used in the -WHERE clause and make use of special `jsonb` operators like this: - -```sql -SELECT * FROM messages - WHERE body::jsonb @> '{"Name":"Nicholas Pépé"}'::jsonb; -``` - -If you provide a exclusion criteria like `{"type": "fake-data"}` the resulting -query will look something like this: - -```sql -SELECT * FROM messages - WHERE body::jsonb @> '{"Name":"Nicholas Pépé"}'::jsonb - AND NOT (body::jsonb @> '{"type":"fake-data"}'::jsonb); -``` - -There is a more detailed explanation of filter matching -[below](#filter-matching-in-detail). +![Webhook Trigger](/img/webhook_trigger.png) ### Cron Triggers (formerly timers) -Cron triggers run jobs based on a cron schedule. They can run as frequently as +`Cron Triggers` run Workflows based on a cron schedule, and are good for repetitive tasks that are time-based (e.g., every day at 8am, sync financial data). +- These Triggers enable users to “pull” data from connected systems. +- You can pick a standard schedule (e.g., every day, or every month), or define a custom schedule using cron expressions. + +These Triggers enable Workflows to be run as frequently as once every minutes, or as infrequently as you desire and can be scheuled on very -specific dates or times. Each time a timed job succeeds, its `final_state` will +specific dates or times. + +Each time a timed job succeeds, its `final_state` will be saved and used as the `initial_state` for its next run. See "Managing state" and "Keeping a cursor" below for implementation help. +![Cron Trigger](/img/cron_trigger.png) + +:::tip Help with cron expressions + The best way to learn about `cron`, if you're not already familiar, is through the OpenFn interface or crontab.guru. -### Flow Triggers - -Flow triggers will execute a job _after_ another specified job finishes -successfully. E.g., a flow trigger which specifies the succesful run of Job A -can be used by Job B. Each time Job A succeeds, Job B will start to run with the -`final_state` of Job A as its `initial_state`. - -### Fail Triggers - -Fail, or "catch", triggers work just like flow triggers, except that they watch -for the failure, rather than the success, of a specified job. (E.g., Job A pays -a CHW via MPESA. If Job A _fails_ we should initiate Job B, which sends an SMS -to the district manager instructing them to manually pay the CHW.) - -## Processing cron jobs - -**On-demand processing for cron jobs.** If you’re leveraging cron triggers to -run jobs at specific times, you can also run that cron triggered job on demand. -This way you don’t have to wait for the timer to expire before testing! Simply -click the process/ “play” button now available via the Job, Run, and Activity -History pages. - -![Runs list run time trigger button](/img/timetriggerunslist.png) - -![Run history time trigger button](/img/runtimetrigger1.png) +::: -#### Keeping a cursor in `state` for timer Jobs +#### Managing the size of `state` for Cron Workflows -Because many timer jobs require keeping some sort of record of their previous -run to modify their later actions, `state` is passed between the runs. One -example might be keeping a "cursor" to select only new records from a database. -We'd expect the following logic: - -1. `job-1` fetches patients from the database -2. `job-1` does something important with those patient records -3. `job-1` saves the `id` of the last successfully processed patient to - `final_state` -4. when `job-1` runs again, it fetches patients whose `id` is greater than the - `id` of the last successfully processed patient. - -To achieve this you might write: - -```js -fetchPatient({ type: 'referral', offset: state.lastId }, state => { - // Assuming the system returned an array of patients in the "data" key. - state.lastId = state.data.patients.sort((a, b) => b.id - a.id)[0]; - return state; -}); -``` - -The initial offset will be `null`, but the subsequent runs will automatically -only fetch "new" patients. - -### Managing the size of `state` for Timer Jobs - -Since state is passed between each run of a timer job, if your job adds +Since state is passed between each run of a cron Workflow, if your Workflow Step adds something new to state each time it runs, it may quickly become too large to be practically handled. Imagine if a server response were adding, via `array.push(...)`, to `state.references` each time the job ran. OpenFn supports @@ -122,7 +58,7 @@ succeed but its `final_state` will not be saved and the next time that job runs it will inherit the previous, un-updated final state. (I.e., the last state that was < 50,000 bytes.) -### A quick fix for final state bloat +#### A quick fix for final state bloat Most often, final state bloat is due to improper handling of `state.references` or `state.data`. This can be fixed by adding the following lines _either_ to the @@ -137,121 +73,3 @@ fn(state => { return state; }); ``` - -## Filter Matching in Detail - -To illustrate filter matching, refer to the filters and message samples below. - -- Message "a" will match filter 1, but message "b" will not. -- Message "c" will match filter 2, but message "d" will not. - -### Filter 1, simple inclusion - -The inclusion criteria is `{ "formID": "patient_registration_v7" }` and the -exclusion criteria is left blank. - -#### Message "a" will match - -```json -{ - "submissionDate": "2016-01-15", - "formID": "patient_registration_v7", - "name": "Jack Wilshere", - "dob": "1986-05-16", - "medications": ["anaphlene", "zaradood", "morphofast"] -} -``` - -#### Message "b" will NOT match - -```json -{ - "submissionDate": "2016-01-16", - "formID": "patient_registration_v8", - "name": "Larry Bird", - "dob": "1982-03-21", - "medications": ["anaphlene", "zaradood", "morphofast"] -} -``` - -Message 'b' does not include `"formID":"patient_registration_v7"` and will not -match filter '1'. - -### Filter 2, inclusion _and_ exclusion - -The inclusion criteria is `{ "name": "john doe" }` and the exclusion criteria is -`{"allowedToShare": false}`. - -#### Message "c" will match - -```json -{ - "submissionDate": "2016-01-15", - "name": "john doe", - "dob": "1986-05-16" -} -``` - -#### Message "d" will NOT match - -```json -{ - "submissionDate": "2016-01-15", - "name": "john doe", - "dob": "1986-05-16", - "allowedToShare": false -} -``` - -## More filter samples - -### Match messages `WHERE` the `formId` is `"Robot_Photo_21.04.2015"` - -| inclusion | exclusion | -| ---------------------------------------- | --------- | -| `{ "formId": "Robot_Photo_21.04.2015" }` | | - -### Match a message with two fragments inside an array called `data` - -(This is useful when gathering data via ODK) - -| inclusion | exclusion | -| --------------------------------------------------------------------- | --------- | -| `{ "data": [{ "outlet_call": "TRUE", "new_existing": "Existing" }] }` | | - -### Match a message `WHERE` this `AND` that are both included - -| inclusion | exclusion | -| ------------------------------------------------------------ | --------- | -| `{ "formId": "Robot_Photo_21.04.2015", "secret_number": 8 }` | | - -### Match a message using exclusion - -| inclusion | exclusion | -| ---------------------------------------- | ---------------------------- | -| `{ "formId": "Robot_Photo_21.04.2015" }` | `{ "safeToProcess": false }` | - -### Match a message with a fragment inside another object called `form` - -| inclusion | exclusion | -| ------------------------------------------------------------------------------------- | --------- | -| `{"form": {"@xmlns": "http://openrosa.org/formdesigner/F732194-3278-nota-ReAL-one"}}` | | - -## An exclusion demo - -Imagine that we had a filter which included messages with `form == 'bns_survey'` -but we then want to start _excluding_ those that have -`body.survey_type == 'practice'`. Our filter trigger would look need to like -this: - -| inclusion | exclusion | -| -------------------------- | --------------------------------------- | -| `{ "form": "bns_survey" }` | `{"body": {"survey_type": "practice"}}` | - -We'd set it up from the trigger form like this: - -![img](/img/exclusion.gif) - -And verify the result on the inbox: - -![img](/img/demo-exclusion.gif) diff --git a/docs/build/workflows.md b/docs/build/workflows.md index 659e3443d2c0..d70f5707e295 100644 --- a/docs/build/workflows.md +++ b/docs/build/workflows.md @@ -1,802 +1,34 @@ --- -title: What are "workflows"? +title: About4. Workflows +sidebar_label: Workflows --- -A job defines the specific series of "operations" (think: tasks or database -actions) to be performed when a triggering message is received (even-based), -another run finishes (flow- or catch-based) or a pre-scheduled (and recurring) -time is reached. +`Workflows` are automated processes or sets of instructions that accomplish a +task. In OpenFn configuration, a Worklfow consists of a Trigger, Steps, and +Paths that define automation logic. Read on to learn how to configure Workflows. -## The properties of a job +## Create a new Workflow -- `Name` - a human-readable name describing the series of operations -- `Project` - the project the job belongs to -- `Trigger` - the trigger that is used to automatically initiate a run of the - job -- `Adaptor` - the adaptor that is used to provide tool-specific functionality - for this job (e.g., `language-dhis2` or `language-commcare`) -- `Auto-process?` - a true/false switch which controls whether the trigger - should be used to automatically run this job when its criteria are met -- `Expression` - the job "script" itself; a sequence of operations +To create a new Workflow in your Project: +1. Go to the `Workflows` page. +2. Click `Create new workflow` button. +3. Give your Workflow a descriptive `Name` (e.g., `Register patients`, `Refer cases`, `Monthly payroll`). +4. Choose your [Trigger](/documentation/next/build/triggers) +5. Edit your first [Step](/documentation/next/build/step/steps) +6. Modify the [Path Condition](), if needed, to define _when_ the Workflow should proceed to the next Step. +7. Configure more Steps as needed -## Adaptors +Check out the video overview below to learn how to create a Workflow. + -We've got a whole section on creating new -[Adaptors](/adaptors), but the critical thing to be aware of -when writing a job is that you've got to choose an **adaptor**, and an **adaptor -version**. +## Run Workflows +To run a Workflow, you can either activate the Trigger (e.g., send a request to the Webhook Event Trigger's URL, or wait for the cron timer to be activated), or run your workflow manually. -All of the discussion below of helper functions like `create` or `findPatient` -requires some understanding of adaptors. When you run a job, you're borrowing a -layer of functionality that's been built to connect with some specific API, type -of API, or database. + -For example, `create` means one thing in `language-salesforce` and another thing -entirely in `language-dhis2`. For this reason, before you can begin writing a -job you have to decide which `adaptor` to work with. - -### Adaptor Versions - -Adaptors change over time. They're open source, and we encourage as much -contribution as possible—releasing new versions for use on OpenFn.org as soon as -they pass our security reviews. New features may be added and bugs may be fixed, -but in order to make sure that an existing integration is not broken, we -recommend that you select a specific version (rather than using the -"auto-upgrade" feature) when you choose an adaptor. The highest released version -is the default choice here. - -:::tip - -The _first 4 lines_ in the log of any run on OpenFn will tell you what adaptor -you're running. (As well as the version of core and NodeJs) This is incredibly -important, particularly if you're trying to troubleshoot jobs in various -environments (like your own shell, OpenFn.org, OpenFn/microservice, etc.). - -::: - -Pay careful attention to which `version` you're using to write a job. Consider -the following run logs: - -```sh -╭───────────────────────────────────────────────╮ -│ ◲ ◱ @openfn/core#v1.3.12 (Node.js v12.20.1) │ -│ ◳ ◰ @openfn/language-http#v2.4.15 │ -╰───────────────────────────────────────────────╯ -...more logs here... - -Finished. -``` - -Note that here, OpenFn/core version `1.3.12` is running on Node.js `12.20.1` and -using `@openfn/language-http#v2.4.15` which might have very different helper -functions from `@openfn/language-http#v3.1.5` - -:::info - -See [the npm section](/adaptors#install-on-platform-via-npm) -on the adaptors docs page to learn how to install an adaptor from `npm` while -using `platform`. - -::: - -### Upgrading to newer adaptor versions - -While it may be beneficial to upgrade as part of your routine maintenance, these -upgrades should be carefully tested. Most often, customers upgrade to a new -adaptor version for an existing job when they are making business-drives changes -to that job. Some business-driven changes may actually _require_ upgrading the -version in order to use a new feature from the adaptor. Even if those changes -don't require and upgrade, if the technical team must spend time testing -job-specific changes anyway, it may be an ideal opportunity to test also test an -upgrade. - -Adaptors follow [SEMVER](https://semver.org/) so you can be reasonably assured -that upgrading from `x.1.z` to `x.2.z` will not lead to existing job code -failing, but an upgrade from `3.y.z` to `4.y.z` may—in SEMVER _major_ upgrades -(those that change the first number in the `x.y.z` version number) have -"breaking" or "non-backwards compatible" changes. - -## Composing job expressions - -In most cases, a job expression is a series of `create` or `upsert` actions that -are run after a message arrives, using data from that message. It could look -like this: - -### A basic expression - -```js -create( - 'Patient__c', - fields( - field('Name', dataValue('form.surname')), - field('Other Names', dataValue('form.firstName')), - field('Age__c', dataValue('form.ageInYears')), - field('Is_Enrolled__c', true), - field('Enrollment_Status__c', 3) - ) -); -``` - -That would create a new `Patient__c` in some other system. The patient's `Name` -will be determined by the triggering message (the value inside `form.surname`, -specifically) and the patient's `Is_Enrolled__c` will _always_ be `true`. See -how we hard coded it? - -What you see above is OpenFn's own syntax, and you've got access to dozens of -common "helper functions" like `dataValue(path)` and destination specific -functions like `create(object,attributes)`. While most cases are covered -out-of-the-box, jobs are **evaluated as Javascript**. This means that you can -write your own custom, anonymous functions to do whatever your heart desires: - -### dataValue - -The most commonly used "helper function" is `dataValue(...)`. This function -takes a single argument—the _path_ to some data that you're trying to access -inside the message that has triggered a particular run. In the above example, -you'll notice that `Is_Enrolled__c` is _always_ set to `true`, but `Name` will -change for each message that triggers the running of this job. It's set to -`dataValue('form.surname')` which means it will set `Name` to whatever value is -present at `state.data.form.surname` for the triggering message. It might be Bob -for one message, and Alice for another. - -:::note - -Note that for message-triggered jobs, `state` will always have it's `data` key -(i.e., `state.data`) set to the body of the triggering message (aka HTTP -request). - -I.e., `dataValue('some.path') === state.data.some.path`, as evaluated at the -time that the operation (`create` in the above expression) is executed. - -::: - -### An expression with custom Javascript - -```js -create( - 'Patient__c', - fields( - field('Name', state => { - console.log('Manipulate state to get your desired output.'); - return Array.apply(null, state.data.form.names).join(', '); - }), - field('Age__c', 7) - ) -); -``` - -Here, the patient's name will be a comma separated concatenation of all the -values in the `patient_names` array from our source message. - -## Available Javascript Globals - -For security reasons, users start with access to the following standard -Javascript globals, and can request more by opening an issue on Github: - -- [`Array`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array) -- [`console`](https://nodejs.org/api/console.html) -- [`JSON`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON) -- [`Number`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number) -- [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) -- [`String`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String) - -## Examples of adaptor-specific functions - -**N.B.: This is just a sample.** There are lots more available in the -language-packs. - -### language-common - -- `field('destination_field_name__c', 'value')` Returns a key, value pair in an - array. - [(source)](https://github.com/OpenFn/language-common/blob/master/src/index.js#L248) -- `fields(list_of_fields)` zips key value pairs into an object. - [(source)](https://github.com/OpenFn/language-common/blob/master/src/index.js#L258) -- `dataValue('JSON_path')` Picks out a single value from source data. - [(source)](https://github.com/OpenFn/language-common/blob/master/src/index.js#L71) -- `each(JSON_path, operation(...))` Scopes an array of data based on a JSONPath - [(source)](https://github.com/OpenFn/language-common/blob/master/src/index.js#L194). - See beta.each when using multiple each()'s in an expression. -- `each(merge(dataPath("CHILD_ARRAY[*]"),fields(field("metaId", dataValue("*meta-instance-id*")),field("parentId", lastReferenceValue("id")))), create(...))` - merges data into an array then creates for each item in the array - [(source)](https://github.com/OpenFn/language-common/blob/master/src/index.js#L272) -- `lastReferenceValue('id')` gets the sfID of the last item created - [(source)](https://github.com/OpenFn/language-common/blob/master/src/index.js#L96-L100) -- `function(state){return state.references[state.references.length-N].id})` gets - the sfID of the nth item created - -#### each() - -Read more about each here: [The each(...) operation](/documentation/jobs/each) - -```js -each( - dataPath('csvData[*]'), - upsertTEI( - 'aX5hD4qUpRW', //piirs uid - { - trackedEntityType: 'bsDL4dvl2ni', - orgUnit: dataValue('OrgUnit'), - attributes: [ - { - attribute: 'aX5hD4qUpRW', - value: dataValue('aX5hD4qUpRW'), - }, - { - attribute: 'MxQPuS9G7hh', - value: dataValue('MxQPuS9G7hh'), - }, - ], - }, - { strict: false } - ) -); -``` - -#### beta.each - -```js -beta.each(JSON_path, operation(...)) -``` - -Scopes an array of data based on a JSONPath but then returns to the state it was -given upon completion -[(source)](https://github.com/OpenFn/language-common/blob/master/src/beta.js#L44). -This is necessary if you string multiple `each(...)` functions together in-line -in the same expression. (E.g., Given data which has multiple separate 'repeat -groups' in a form which are rendered as arrays, you want to create new records -for each item inside the first repeat group, then _RETURN TO THE TOP LEVEL_ of -the data, and then create new records for each item in the second repeat group. -Using `beta.each(...)` lets you enter the first array, create your records, then -return to the top level and be able to enter the second array. - -### Salesforce - -- `create("DEST_OBJECT_NAME__C", fields(...))` Create a new object. Takes 2 - parameters: An object and attributes. - [(source)](https://github.com/OpenFn/language-salesforce/blob/master/src/Adaptor.js#L42-L63) -- `upsert("DEST_OBJECT_NAME__C", "DEST_OBJECT_EXTERNAL_ID__C", fields(...))` - Creates or updates an object. Takes 3 paraneters: An object, an ID field and - attributes. - [(source)](https://github.com/OpenFn/language-salesforce/blob/master/src/Adaptor.js#L65-L80) -- `relationship("DEST_RELATIONSHIP_NAME__r", "EXTERNAL_ID_ON_RELATED_OBJECT__C", "SOURCE_DATA_OR_VALUE")` - Adds a lookup or 'dome insert' to a record. - [(source)](https://github.com/OpenFn/language-salesforce/blob/master/src/sourceHelpers.js#L21-L40) - -### dhis2 - -- `event(...)` Creates an event. - [(source)](https://github.com/OpenFn/language-dhis2/blob/master/src/Adaptor.js#L31-L60) -- `dataValueSet(...)` Send data values using the dataValueSets resource - [(source)](https://github.com/OpenFn/language-dhis2/blob/master/src/Adaptor.js#L62-L82) - -### OpenMRS - -- `person(...)` Takes a payload of data to create a person - [(source)](https://github.com/OpenFn/language-openmrs/blob/master/src/Adaptor.js#L31-L60) -- `patient(...)` Takes a payload of data to create a patient - [(source)](https://github.com/OpenFn/language-openmrs/blob/master/src/Adaptor.js#L62-L90) - -## Snippets and samples - -Below you can find some examples of block code for different functions and data -handling contexts. - -### Job expression (for CommCare to SF) - -The following job expression will take a matching receipt and use data from that -receipt to upsert a `Patient__c` record in Salesforce and create multiple new -`Patient_Visit__c` (child to Patient) records. - -```js -upsert( - 'Patient__c', - 'Patient_Id__c', - fields( - field('Patient_Id__c', dataValue('form.patient_ID')), - relationship('Nurse__r', 'Nurse_ID_code__c', dataValue('form.staff_id')), - field('Phone_Number__c', dataValue('form.mobile_phone')) - ) -), - each( - join('$.data.form.visits[*]', '$.references[0].id', 'Id'), - create( - 'Visit__c', - fields( - field('Patient__c', dataValue('Id')), - field('Date__c', dataValue('date')), - field('Reason__c', dataValue('why_did_they_see_doctor')) - ) - ) - ); -``` - -### Accessing the "data array" in Open Data Kit submissions - -Notice how we use "each" to get data from each item inside the "data array" in -ODK. - -```js -each( - '$.data.data[*]', - create( - 'ODK_Submission__c', - fields( - field('Site_School_ID_Number__c', dataValue('school')), - field('Date_Completed__c', dataValue('date')), - field('comments__c', dataValue('comments')), - field('ODK_Key__c', dataValue('*meta-instance-id*')) - ) - ) -); -``` - -### ODK to Salesforce: create parent record with many children from parent data - -Here, the user brings `time_end` and `parentId` onto the line items from the -parent object. - -```js -each( - dataPath('data[*]'), - combine( - create( - 'transaction__c', - fields( - field('Transaction_Date__c', dataValue('today')), - relationship( - 'Person_Responsible__r', - 'Staff_ID_Code__c', - dataValue('person_code') - ), - field('metainstanceid__c', dataValue('*meta-instance-id*')) - ) - ), - each( - merge( - dataPath('line_items[*]'), - fields( - field('end', dataValue('time_end')), - field('parentId', lastReferenceValue('id')) - ) - ), - create( - 'line_item__c', - fields( - field('transaction__c', dataValue('parentId')), - field('Barcode__c', dataValue('product_barcode')), - field('ODK_Form_Completed__c', dataValue('end')) - ) - ) - ) - ) -); -``` - -> **NB - there was a known bug with the `combine` function which has been -> resolved. `combine` can be used to combine two operations into one and is -> commonly used to run multiple `create`'s inside an `each(path, operation)`. -> The source code for combine can be found here: -> [language-common: combine](https://github.com/OpenFn/language-common/blob/master/src/index.js#L204-L222)** - -### Create many child records WITHOUT a repeat group in ODK - -```js -beta.each( - '$.data.data[*]', - upsert( - 'Outlet__c', - 'Outlet_Code__c', - fields( - field('Outlet_Code__c', dataValue('outlet_code')), - field('Location__Latitude__s', dataValue('gps:Latitude')), - field('Location__Longitude__s', dataValue('gps:Longitude')) - ) - ) -), - beta.each( - '$.data.data[*]', - upsert( - 'Outlet_Call__c', - 'Invoice_Number__c', - fields( - field('Invoice_Number__c', dataValue('invoice_number')), - relationship('Outlet__r', 'Outlet_Code__c', dataValue('outlet_code')), - relationship('RecordType', 'name', 'No Call Card'), - field('Trip__c', 'a0FN0000008jPue'), - relationship( - 'Sales_Person__r', - 'Sales_Rep_Code__c', - dataValue('sales_rep_code') - ), - field('Date__c', dataValue('date')), - field('Comments__c', dataValue('comments')) - ) - ) - ); -``` - -### Salesforce: perform an update - -```js -update("Patient__c", fields( - field("Id", dataValue("pathToSalesforceId")), - field("Name__c", dataValue("patient.first_name")), - field(...) -)); -``` - -### Salesforce: Set record type using 'relationship(...)' - -```js -create( - 'custom_obj__c', - fields( - relationship( - 'RecordType', - 'name', - dataValue('submission_type'), - field('name', dataValue('Name')) - ) - ) -); -``` - -### Salesforce: Set record type using record Type ID - -```js -each( - '$.data.data[*]', - create( - 'fancy_object__c', - fields( - field('RecordTypeId', '012110000008s19'), - field('site_size', dataValue('size')) - ) - ) -); -``` - -### Telerivet: Send SMS based on Salesforce workflow alert - -```js -send( - fields( - field( - 'to_number', - dataValue( - 'Envelope.Body.notifications.Notification.sObject.phone_number__c' - ) - ), - field('message_type', 'sms'), - field('route_id', ''), - field('content', function (state) { - return 'Hey there. Your name is '.concat( - dataValue('Envelope.Body.notifications.Notification.sObject.name__c')( - state - ), - '.' - ); - }) - ) -); -``` - -### HTTP: fetch but don't fail! - -```js -// ============= -// We use "fetchWithErrors(...)" so that when the -// SMS gateway returns an error the run does not "fail". -// It "succeeds" and then delivers that error message -// back to Salesforce with the "Update SMS Status" job. -// ============= -fetchWithErrors({ - getEndpoint: 'send_to_contact', - query: function (state) { - return { - msisdn: - state.data.Envelope.Body.notifications.Notification.sObject - .SMS__Phone_Number__c, - message: - state.data.Envelope.Body.notifications.Notification.sObject - .SMS__Message__c, - api_key: 'some-secret-key', - }; - }, - externalId: state.data.Envelope.Body.notifications.Notification.sObject.Id, - postUrl: 'https://www.openfn.org/inbox/another-secret-key', -}); -``` - -### Sample DHIS2 events API job: - -```js -event( - fields( - field('program', 'eBAyeGv0exc'), - field('orgUnit', 'DiszpKrYNg8'), - field('eventDate', dataValue('properties.date')), - field('status', 'COMPLETED'), - field('storedBy', 'admin'), - field('coordinate', { - latitude: '59.8', - longitude: '10.9', - }), - field('dataValues', function (state) { - return [ - { - dataElement: 'qrur9Dvnyt5', - value: dataValue('properties.prop_a')(state), - }, - { - dataElement: 'oZg33kd9taw', - value: dataValue('properties.prop_b')(state), - }, - { - dataElement: 'msodh3rEMJa', - value: dataValue('properties.prop_c')(state), - }, - ]; - }) - ) -); -``` - -### Sample DHIS2 data value sets API job: - -```js -dataValueSet( - fields( - field('dataSet', 'pBOMPrpg1QX'), - field('orgUnit', 'DiszpKrYNg8'), - field('period', '201401'), - field('completeData', dataValue('date')), - field('dataValues', function (state) { - return [ - { dataElement: 'f7n9E0hX8qk', value: dataValue('prop_a')(state) }, - { dataElement: 'Ix2HsbDMLea', value: dataValue('prop_b')(state) }, - { dataElement: 'eY5ehpbEsB7', value: dataValue('prop_c')(state) }, - ]; - }) - ) -); -``` - -### sample openMRS expression, creates a person and then a patient - -```js -person( - fields( - field('gender', 'F'), - field('names', function (state) { - return [ - { - givenName: dataValue('form.first_name')(state), - familyName: dataValue('form.last_name')(state), - }, - ]; - }) - ) -), - patient( - fields( - field('person', lastReferenceValue('uuid')), - field('identifiers', function (state) { - return [ - { - identifier: '1234', - identifierType: '8d79403a-c2cc-11de-8d13-0010c6dffd0f', - location: '8d6c993e-c2cc-11de-8d13-0010c6dffd0f', - preferred: true, - }, - ]; - }) - ) - ); -``` - -### merge many values into a child path - -```js -each( - merge( - dataPath("CHILD_ARRAY[*]"), - fields( - field("metaId", dataValue("*meta-instance-id*")), - field("parentId", lastReferenceValue("id")) - ) - ), - create(...) -) -``` - -### arrayToString - -```js -arrayToString(arr, separator_string); -``` - -### access an image URL from an ODK submission - -```js -// In ODK the image URL is inside an image object... -field("Photo_URL_text__c", dataValue("image.url")), -``` - -### alterState (alter state) to make sure data is in an array - -```js -// Here, we make sure CommCare gives us an array to use in each(merge(...), ...) -fn(state => { - const idCards = state.data.form.ID_cards_given_to_vendor; - if (!Array.isArray(idCards)) { - state.data.form.ID_cards_given_to_vendor = [idCards]; - } - return state; -}); - -// Now state has been changed, and we carry on... -each( - merge( - dataPath('form.ID_cards_given_to_vendor[*]'), - fields( - field('Vendor_Id', dataValue('form.ID_vendor')), - field('form_finished_time', dataValue('form.meta.timeEnd')) - ) - ), - upsert( - 'Small_Packet__c', - 'sp_id__c', - fields( - field('sp_id__c', dataValue('ID_cards_given_to_vendor')), - relationship('Vendor__r', 'Badge_Code__c', dataValue('Vendor_Id')), - field( - 'Small_Packet_Distribution_Date__c', - dataValue('form_finished_time') - ) - ) - ) -); -``` - -### Login in to a server with a custom SSL Certificate - -This snippet describes how you would connect to a secure server ignoring SSL -certificate verification. Set `strictSSL: false` in the options argument of the -`post` function in `language-http`. - -```js -post( - `${state.configuration.url}/${path}`, - { - headers: { 'content-type': 'application/json' }, - body: { - email: 'Luka', - password: 'somethingSecret', - }, - strictSSL: false, - }, - callback -); -``` - -## Anonymous Functions - -Different to [Named Functions](#examples-of-adaptor-specific-functions), -Anonymous functions are generic pieces of javascript which you can write to suit -your needs. Here are some examples of these custom functions: - -### Custom replacer - -```js -field('destination__c', state => { - console.log(something); - return dataValue('path_to_data')(state).toString().replace('cats', 'dogs'); -}); -``` - -This will replace all "cats" with "dogs" in the string that lives at -`path_to_data`. - -> **NOTE:** The JavaScript `replace()` function only replaces the first instance -> of whatever argument you specify. If you're looking for a way to replace all -> instances, we suggest you use a regex like we did in the -> [example](#custom-concatenation-of-null-values) below. - -### Custom arrayToString - -```js -field("target_specie_list__c", function(state) { - return Array.apply( - null, sourceValue("$.data.target_specie_list")(state) - ).join(', ') -}), -``` - -It will take an array, and concatenate each item into a string with a ", " -separator. - -### Custom concatenation - -```js -field('ODK_Key__c', function (state) { - return dataValue('metaId')(state).concat('(', dataValue('index')(state), ')'); -}); -``` - -This will concatenate two values. - -### Concatenation of null values - -This will concatenate many values, even if one or more are null, writing them to -a field called Main_Office_City_c. - -```js -... - field("Main_Office_City__c", function(state) { - return arrayToString([ - dataValue("Main_Office_City_a")(state) === null ? "" : dataValue("Main_Office_City_a")(state).toString().replace(/-/g, " "), - dataValue("Main_Office_City_b")(state) === null ? "" : dataValue("Main_Office_City_b")(state).toString().replace(/-/g, " "), - dataValue("Main_Office_City_c")(state) === null ? "" : dataValue("Main_Office_City_c")(state).toString().replace(/-/g, " "), - dataValue("Main_Office_City_d")(state) === null ? "" : dataValue("Main_Office_City_d")(state).toString().replace(/-/g, " "), - ].filter(Boolean), ',') - }) -``` - -> Notice how this custom function makes use of the **regex** `/-/g` to ensure -> that all instances are accounted for (g = global search). - -### Custom Nth reference ID - -If you ever want to retrieve the FIRST object you created, or the SECOND, or the -Nth, for that matter, a function like this will do the trick. - -```js -field('parent__c', function (state) { - return state.references[state.references.length - 1].id; -}); -``` - -See how instead of taking the id of the "last" thing that was created in -Salesforce, you're taking the id of the 1st thing, or 2nd thing if you replace -"length-1" with "length-2". - -### Convert date string to standard ISO date for Salesforce - -```js -field('Payment_Date__c', function (state) { - return new Date(dataValue('payment_date')(state)).toISOString(); -}); -``` - -> **NOTE**: The output of this function will always be formatted according to -> GMT time-zone. - -### Use external ID fields for relationships during a bulk load in Salesforce - -```js -array.map(item => { - return { - Patient_Name__c: item.fullName, - 'Account.Account_External_ID__c': item.account - 'Clinic__r.Unique_Clinic_Identifier__c': item.clinicId, - 'RecordType.Name': item.type, - }; -}); -``` - -### Bulk upsert with an external ID in salesforce - -```js -bulk( - 'Visit_new__c', - 'upsert', - { - extIdField: 'commcare_case_id__c', - failOnError: true, - allowNoOp: true, - }, - dataValue('patients') -); -``` +## Turn off Workflows +To "turn off" or disable a Workflow: +1. Open the Workflow +2. Click on the Trigger +3. Select the `Disable this trigger` checkbox +4. Select `Save` to save your changes \ No newline at end of file diff --git a/docs/monitor-history/inspect-runs.md b/docs/monitor-history/inspect-runs.md index 71029f193b42..10e600cd3afd 100644 --- a/docs/monitor-history/inspect-runs.md +++ b/docs/monitor-history/inspect-runs.md @@ -26,4 +26,4 @@ To learn how to search and filter Work Order and Run history via the History page, check out the below video tutorial ([or see link](https://youtu.be/XIUykmLCxwQ?si=pCzefw4zyLxG1voE)). - + diff --git a/docs/build/tutorial.md b/docs/tutorials/tutorial.md similarity index 100% rename from docs/build/tutorial.md rename to docs/tutorials/tutorial.md diff --git a/sidebars-main.js b/sidebars-main.js index b9327456da0f..eb9a59f5a2f3 100644 --- a/sidebars-main.js +++ b/sidebars-main.js @@ -28,32 +28,33 @@ module.exports = { //'design/discovery' ], }, - // { - // type: 'category', - // label: 'Tutorials', - // items: ['tutorials/kobo-to-dhis2'], - // }, + { + type: 'category', + label: 'Tutorials', + items: ['tutorials/tutorial', 'tutorials/kobo-to-dhis2'], + }, { type: 'category', label: 'Build & manage Workflows', items: [ - 'build/tutorial', + 'build/workflows', + 'build/triggers', + 'build/steps/steps', + 'build/steps/step-editor', + 'build/steps/step-design-intro', + 'build/steps/paths', 'build/limits', - // 'build/triggers' - //====== STEPS SUBCATEGORY =============// - //TODO: @Mtuchi pls clean up the below pages and decide which to keep/refresh/delete for V2 docs// + //'build/troubleshooting', { type: 'category', - label: 'Steps', + label: 'Jobs', items: [ - 'build/steps/steps', - 'build/steps/step-design-intro', 'build/steps/job-expressions', + 'build/steps/job-examples', 'build/steps/operations', 'build/steps/multiple-operations', 'build/steps/state', 'build/steps/each', - 'build/steps/step-editor', 'build/steps/editing-locally', 'build/steps/working-with-branches', ], @@ -106,9 +107,7 @@ module.exports = { { type: 'category', label: 'Manage Users & Credentials', - items: ['manage-users/user-profile', - 'manage-users/user-credentials' - ], + items: ['manage-users/user-profile', 'manage-users/user-credentials'], }, { type: 'category', diff --git a/static/img/anatomy_of_step.png b/static/img/anatomy_of_step.png new file mode 100644 index 000000000000..5d5876d8b59c Binary files /dev/null and b/static/img/anatomy_of_step.png differ diff --git a/static/img/cron_trigger.png b/static/img/cron_trigger.png new file mode 100644 index 000000000000..4f4ef93e0e05 Binary files /dev/null and b/static/img/cron_trigger.png differ diff --git a/static/img/edit_steps.png b/static/img/edit_steps.png new file mode 100644 index 000000000000..6aaac825bafb Binary files /dev/null and b/static/img/edit_steps.png differ diff --git a/static/img/path_conditions.png b/static/img/path_conditions.png new file mode 100644 index 000000000000..3648c209d7a1 Binary files /dev/null and b/static/img/path_conditions.png differ diff --git a/static/img/path_js_expression.png b/static/img/path_js_expression.png new file mode 100644 index 000000000000..53c16ba7aa10 Binary files /dev/null and b/static/img/path_js_expression.png differ diff --git a/static/img/settings_credentials.png b/static/img/settings_credentials.png new file mode 100644 index 000000000000..089a0f295351 Binary files /dev/null and b/static/img/settings_credentials.png differ diff --git a/static/img/webhook_trigger.png b/static/img/webhook_trigger.png new file mode 100644 index 000000000000..b1a070a07c58 Binary files /dev/null and b/static/img/webhook_trigger.png differ