Merge pull request #17 from OpenFn/16-spec-template

Re-organize workflow files and update spec template

.github/.github/pull_request_template.md

@@ -8,7 +8,8 @@ Reference any related issues here.
 
 ## AI Usage
 
-Please disclose how you've used AI in this work (it's cool, we just want to know!):
+Please disclose how you've used AI in this work (it's cool, we just want to
+know!):
 
 - [ ] Code generation (copilot but not intellisense)
 - [ ] Learning or fact checking
@@ -18,12 +19,15 @@ Please disclose how you've used AI in this work (it's cool, we just want to know
 - [ ] Other
 - [ ] I have not used AI
 
-You can read more details in our [Responsible AI Policy](https://www.openfn.org/ai#pull-request-templates)
+You can read more details in our
+[Responsible AI Policy](https://www.openfn.org/ai#pull-request-templates)
 
 ## Review Checklist
 
 Before merging, the reviewer should check the following items:
 
-- [ ] Does the PR do what it claims to do (Mention jobs changes)? 
-- [ ] Does the PR have adaptor version?
+- [ ] Does the PR do what it claims to do (Mention jobs changes)?
+- [ ] Does the PR have adaptor version?, Adaptor version should be locked to a
+      specific version
 - [ ] Have you ticked a box under AI Usage?
+- [ ] Have you removed any sensitive logs after testing?

.github/ISSUE_TEMPLATE/bug_report.md

@@ -4,38 +4,62 @@ about: Create a report to help us improve
 title: ''
 labels: bug
 assignees: ''
-
 ---
 
-## Describe the bug and expected behavior 
-A clear and concise description of what the bug is. Include any error messages from the run logs and the expected behavior. 
+## Describe the bug and expected behavior
+
+A clear and concise description of what the bug is. Include any error messages
+from the run logs and the expected behavior.
 
 ## To Reproduce
-1. Here is a link to a failed run on OpenFn.org which is indicative of the bug: __________
 
-### expression.js
-Link to the job itself in Github: _____________  
-Adaptor:
+Here is a [link to a failed run] on OpenFn.org which is indicative of the bug:
+
+1. Using a initial input `{data: {"name": "John Doe"}}` or
+   `{"lastSync": "2020-01-01T00:00:00.000Z"}`
+2. Run [Name of step] or `step.js`
+3. See failed logs
+
+### Step(s) to be updated
+
+- Provide a link to the job itself in GitHub.
+- Mention the adaptor being used.
+- Provide the state directly or link to a file.
+
+  > ```json
+  > {
+  >   "configuration": ["SEE LAST PASS: 'client cred'"],
+  >   "data": {LINK TO STATE},
+  >   "cursor": "2020-01-19 00:00:00"
+  > }
+  > ```
+
+- Redact any sensitive information and provide instructions for where it can be
+  found.
 
+## Testing Guidance
 
-### state.json
+Link to test suite and/or provide examples of scenarios with sample input/output
+data to help the dev validate the implementation.
 
-Either provide state directly, or link to a file. If sensitive information
-should be in state, redact it and provide instructions for where it can be
-found.
+## Toggl
 
-```json
-{
-  "configuration": ["SEE LAST PASS: 'client cred'"],
-  "data": {LINK TO STATE},
-  "cursor": "2020-01-19 00:00:00"
-}
-```
+`Name of Toggl project`
 
-## To test/resolve
-1. After the desired output is working locally (from the CLI), please [push commits to master || open a pull request].
-2. [Please test the change on OpenFn.org by re-running this run (link) and confirming success.]
+## Pre-Development Checklist
 
-## Toggl 
-Name of the Toggle project to log work
+Before handling this issue to a developer, ensure the following items are
+checked:
 
+- [ ] Credentials: Ensure all necessary credentials are available and
+      documented.
+- [ ] Sample Input Data: Ensure sample input data is provided and linked.
+- [ ] PII: Verify if any Personally Identifiable Information (PII) is involved
+      and ensure proper handling.
+- [ ] Collections: Confirm if collections are needed and pre-configure with
+      sample data if required.
+- [ ] Mapping Spec: Ensure mapping specifications are complete and linked.
+- [ ] API Docs: Ensure all relevant API documentation is linked.
+- [ ] Workflow Diagrams: Ensure workflow diagrams are complete and linked.
+- [ ] VPN Access: Ensure VPN Access is provided if required to run the workflow
+- [ ] Toggl: Ensure the Toggl project name is provided.

.github/ISSUE_TEMPLATE/feature-request.md

@@ -4,74 +4,90 @@ about: For new workflows & change requests
 title: ''
 labels: feature request
 assignees: ''
-
 ---
+
 ## Background, context, and business value
 
-A clear and concise description of what the client wants and WHY. 
+A clear and concise description of what the client wants and WHY.
 
 For example: [Insert use case here]
 
 ## The specific request, in as few words as possible
-**Request Type:** New Workflow? Change Request?
 
 A clear and concise description of what you want to happen.  
 Things to include as needed:
-- The number of workflows needed to be created or updated 
-- The function of each workflow including specific resources and operations
-- Unique identifiers 
-- Links to mapping specifications, data flow diagrams, sample input/output data, and any API documentation
-- Links to the data model of target systems, if available
 
-```md
-Create a workflow in which OpenFn will: 
-1. Get new rows from the PostgreSQL database every 1 hour
-2. Clean & transform the data according to the specified mapping rules, and then 
-3. Upsert cases in the Primero case management system via externalId `case_id`
-(Note: 1 DB row will = 1 case record.)
-
-See [links] below for the workflow diagrams, mapping specs, & Primero data model.
-```
+- Workflow Diagram: [Link to workflow diagram]
+- Mapping Specs: [Link to field-level mapping specifications]
+- API Docs: [Link to relevant API & system documentation]
 
 ## Data Volumes & Limits
-How many records do we think these jobs will need to process in each run? For example: 
+
+How many records do we think these jobs will need to process in each run? For
+example:
+
 ```md
-When you GET data from the DB, this may return up to 1000 records. 
-There are no known Primero API limits for # of records, but there is API paging to consider.
+When you GET data from the DB, this may return up to 1000 records. There are no
+known Primero API limits for # of records, but there is API paging to consider.
 ```
 
-## Workflow Spec
-
-For each new Workflow, describe the business process to be automated and its objectives. 
+## [Workflow Name] Workflow Steps
 
-- **Workflow Diagram**: {add link to diagram describing each step & logic}
-- **Mapping Specs**: {add link to field-level mapping specifications - in most cases you will have 1 spec per WF Step)
-- **API Docs:** {add links to relevant API & system documentation}
+Create a workflow in which OpenFn will:
 
-### Trigger
-What is the trigger type: cron, webhook, or kafka? Be sure to provide a sample input. 
+### Trigger: Cron Schedule `Every 1 hour`
 
-### Adaptors
-Which adaptor(s) do you expect to use? Any specific adaptor version considerations? 
+> What is the trigger type: cron, webhook, or kafka? Be sure to provide a sample
+> input.
 
-### Collections
-Do you plan to use `collections` feature in this workflow? If yes, please (1) specify how the collection should be used, and (2) make sure to either pre-configure the collection with sample data or spec how the collection data should be structured. 
+### Step 1: Get new rows from the PostgreSQL database every 1 hour
 
+- **Adaptor:** [PostgreSQL]
+- **Input**: [Link to sample input data]
+- **Collections (optional):** [Collection details if required]
+- **Credential (optional):** [Also specify If VPN Access is required]
+- **Desired Output:** [Description of the desired output]
 
-## Input
+### Step 2: Clean & transform the data according to the specified mapping rules
 
-### Credentials
-Which credentials can be used to access the target system(s)? Do these target systems have test data? 
-(Do NOT share credential secrets on this issue -> rather point to where it can be found).
+- **Adaptor:** [Common]
+- **Edge Condition**: [Eg: on success]
+- **Mapping Spec**: [Link to mapping spec]
+- **Credential (optional):** [Credential details if required]
+- **Desired Output:** [Description of the desired output]
 
-### Sample Input Data
-Describe how the "input" for this workflow will be generated (e.g., webhook request, timer-based query to be sent). Either provide an example directly, link to a file, or describe how a query can be executed to extract data. 
+### Step 3: Upsert cases in the Primero case management system via externalId `case_id`
 
-Be sure to redact any sensitive data and to not paste here. 
+> Note: 1 DB row will = 1 case record.
 
+- **Adaptor:** [Primero]
+- **Edge Condition**: [Eg: `!state.errors && state.patients.length > 0`]
+- **Credential (optional):** [Credential details if required]
+- **Desired Output:** [Description of the desired output]
 
 ## Testing Guidance
-Link to test suite and/or provide examples of scenarios with sample input/output data to help the dev validate the implementation. 
+
+Link to test suite and/or provide examples of scenarios with sample input/output
+data to help the dev validate the implementation.
 
 ## Toggl
-Name of Toggl project
+
+`Name of Toggl project`
+
+## Pre-Development Checklist
+
+Before handling this issue to a developer, ensure the following items are
+checked:
+
+- [ ] Credentials: Ensure all necessary credentials are available and
+      documented.
+- [ ] Sample Input Data: Ensure sample input data is provided and linked.
+- [ ] PII: Verify if any Personally Identifiable Information (PII) is involved
+      and ensure proper handling.
+- [ ] Collections: Confirm if collections are needed and pre-configure with
+      sample data if required.
+- [ ] Mapping Spec: Ensure mapping specifications are complete and linked.
+- [ ] API Docs: Ensure all relevant API documentation is linked.
+- [ ] Workflow Diagrams: Ensure workflow diagrams are complete and linked.
+- [ ] VPN Access: Ensure VPN Access is provided if required to run the workflow
+- [ ] Toggl: Ensure the Toggl project name is provided.

.gitignore

@@ -1,6 +1,6 @@
 # temp
 tmp
-
+.DS_Store
 # Logs
 logs
 *.log

workflows/sample-workflow/get-data.js

@@ -0,0 +1,15 @@
+cursor($.lastSync || '2024-12-31T00:00:00.000Z');
+cursor('now', { key: 'lastSync' });
+// Fetch data from database
+sql({ query: `select(*) from patients where created_at > ${$.lastSync}` });
+
+fn(state => {
+  state.patients = state.data.map(patient => {
+    return {
+      id: patient.uuid,
+      name: patient.first_name + ' ' + patient.last_name,
+      bod: patient.birth_date,
+    };
+  });
+  return state;
+});

workflows/sample-workflow/post-data.js

@@ -0,0 +1 @@
+post('patients', { body: $.patients });

workflows/lightning-workflow.json → workflows/sample-workflow/workflow.json

@@ -7,23 +7,23 @@
     "steps": [
       {
         "id": "get-data",
-        "adaptor": "[email protected]",
-        "state": {},
+        "adaptor": "[email protected]",
+        "state": {
+          "lastSync": "2025-01-01T00:00:00.000Z"
+        },
         "configuration": "./tmp/db-credential.json",
-        "expression": "./jobs/1-getPatientData.js",
+        "expression": "get-data.js",
         "next": {
-          "post-data": {
-            "condition": "true"
-          }
+          "post-data": "!state.errors && state.patients.length > 0"
         }
       },
       {
         "id": "post-data",
         "adaptor": "http@latest",
         "configuration": "./tmp/api-credential.json",
         "state": {},
-        "expression": "./jobs/2-postData.js"
+        "expression": "post-data.js"
       }
     ]
   }
-}
+}