Unify AI flex flow example (#3616)

# Description

Adds demonstration of promptflow code generation/evaluation flex flow
using Unify AI (instead of OpenAI or Azure OpenAI APIs). Code is added
under `promptflow/flex-flows/examples/unify-ai`.

# All Promptflow Contribution checklist:
- [ ] **The pull request does not introduce [breaking changes].**
- [ ] **CHANGELOG is updated for new features, bug fixes or other
significant changes.**
- [ ] **I have read the [contribution
guidelines](https://github.com/microsoft/promptflow/blob/main/CONTRIBUTING.md).**
- [ ] **I confirm that all new dependencies are compatible with the MIT
license.**
- [ ] **Create an issue and link to the pull request to get dedicated
review from promptflow team. Learn more: [suggested
workflow](../CONTRIBUTING.md#suggested-workflow).**

## General Guidelines and Best Practices
- [ ] Title of the pull request is clear and informative.
- [ ] There are a small number of commits, each of which have an
informative message. This means that previously merged commits do not
appear in the history of the PR. For more information on cleaning up the
commits in your PR, [see this
page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md).

### Testing Guidelines
- [ ] Pull request includes test coverage for the included changes.

---------

Co-authored-by: Honglin <[email protected]>

examples/flows/integrations/unify-ai/.env.example

@@ -0,0 +1,2 @@
+UNIFY_AI_API_KEY=<your_Unify_AI_api_key>
+UNIFY_AI_BASE_URL=https://api.unify.ai/v0/ #Please refer https://unify.ai/docs/concepts/unify_api.html

examples/flows/integrations/unify-ai/README.md

@@ -0,0 +1,107 @@
+# Basic standard flow with Unify AI
+A basic standard flow define using function entry that calls Unify AI. 
+
+Unify AI helps you use a LLM from a wide variety of models and providers using a single Unify API key. You can make an optimal choice by comparing trade-offs between quality, cost and latency. 
+
+Refer [Unify AI documentation](https://unify.ai/docs).
+
+## Prerequisites
+
+Install promptflow sdk and other dependencies:
+```bash
+pip install -r requirements.txt
+```
+
+## Run flow
+
+- Prepare your Unify AI account follow this [instruction](https://unify.ai/docs/index.html#getting-started) and get your `api_key` if you don't have one.
+
+- Setup environment variables
+
+Ensure you have put your Unify key in [.env](./.env) file. You can create one refer to this [example file](./.env.example).
+
+```bash
+cat ./.env
+```
+
+- Run/Debug as normal Python file
+```bash
+python programmer.py
+```
+
+- Test with flow entry
+```bash
+pf flow test --flow programmer:write_simple_program --inputs text="Java Hello World!"
+```
+
+- Test with flow yaml
+```bash
+# test with sample input value in flow.flex.yaml
+pf flow test --flow .
+```
+
+```shell
+# test with UI
+pf flow test --flow . --ui
+```
+
+- Create run with multiple lines data
+```bash
+# using environment from .env file (loaded in user code: hello.py)
+pf run create --flow . --data ./data.jsonl --column-mapping text='${data.text}' --stream
+```
+
+You can also skip providing `column-mapping` if provided data has same column name as the flow.
+Reference [here](https://aka.ms/pf/column-mapping) for default behavior when `column-mapping` not provided in CLI.
+
+- List and show run meta
+```bash
+# list created run
+pf run list
+
+# get a sample run name
+
+name=$(pf run list -r 10 | jq '.[] | select(.name | contains("basic_")) | .name'| head -n 1 | tr -d '"')
+# show specific run detail
+pf run show --name $name
+
+# show output
+pf run show-details --name $name
+
+# visualize run in browser
+pf run visualize --name $name
+```
+
+## Run flow in cloud with connection
+
+```bash
+# set default workspace
+az account set -s <your_subscription_id>
+az configure --defaults group=<your_resource_group_name> workspace=<your_workspace_name>
+```
+
+- Create run
+```bash
+# run with environment variable reference connection in azureml workspace
+pfazure run create --flow . --data ./data.jsonl --column-mapping text='${data.text}' --environment-variables UNIFY_AI_API_KEY='<unify_api_key>' UNIFY_AI_BASE_URL='https://api.unify.ai/v0/' --stream
+# run using yaml file
+pfazure run create --file run.yml --stream
+```
+
+- List and show run meta
+```bash
+# list created run
+pfazure run list -r 3
+
+# get a sample run name
+name=$(pfazure run list -r 100 | jq '.[] | select(.name | contains("basic_")) | .name'| head -n 1 | tr -d '"')
+
+# show specific run detail
+pfazure run show --name $name
+
+# show output
+pfazure run show-details --name $name
+
+# visualize run in browser
+pfazure run visualize --name $name
+```

examples/flows/integrations/unify-ai/data.jsonl

@@ -0,0 +1,3 @@
+{"text": "Python Hello World!"}
+{"text": "C Hello World!"}
+{"text": "C# Hello World!"}

examples/flows/integrations/unify-ai/eval_code_flow/code-eval-flow.flex.yaml

@@ -0,0 +1,6 @@
+$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json
+# flow is defined as python function
+entry: code_quality_unify_ai:CodeEvaluator
+environment:
+  # image: mcr.microsoft.com/azureml/promptflow/promptflow-python
+  python_requirements_txt: requirements.txt

examples/flows/integrations/unify-ai/eval_code_flow/code_quality_unify_ai.py

@@ -0,0 +1,61 @@
+import json
+from pathlib import Path
+from typing import TypedDict
+
+from jinja2 import Template
+
+from promptflow.core import OpenAIModelConfiguration
+from promptflow.core._flow import Prompty
+from promptflow.tracing import trace
+
+BASE_DIR = Path(__file__).absolute().parent
+
+# Derived from https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/eval-code-quality/
+
+
+@trace
+def load_prompt(jinja2_template: str, code: str, examples: list) -> str:
+    """Load prompt function."""
+    with open(BASE_DIR / jinja2_template, "r", encoding="utf-8") as f:
+        tmpl = Template(f.read(), trim_blocks=True, keep_trailing_newline=True)
+        prompt = tmpl.render(code=code, examples=examples)
+        return prompt
+
+
+class Result(TypedDict):
+    correctness: float
+    readability: float
+    explanation: str
+
+
+class CodeEvaluator:
+    """ Uses Unify AI's LLM to evaluate a code block.
+    Note:
+    OpenAI client is being repurposed to call Unify AI API, Since Unify AI API is competable with OpenAI API.
+    This enables reusing Promptflow's OpenAI integration/support with Unify AI.
+
+    """
+    def __init__(self, model_config: OpenAIModelConfiguration):
+        self.model_config = model_config
+
+    def __call__(self, code: str) -> Result:
+        """Evaluate the code based on correctness, readability."""
+        prompty = Prompty.load(
+            source=BASE_DIR / "eval_code_quality.prompty",
+            model={"configuration": self.model_config},
+        )
+        output = prompty(code=code)
+        output = json.loads(output)
+        output = Result(**output)
+        return output
+
+    def __aggregate__(self, line_results: list) -> dict:
+        """Aggregate the results."""
+        total = len(line_results)
+        avg_correctness = sum(int(r["correctness"]) for r in line_results) / total
+        avg_readability = sum(int(r["readability"]) for r in line_results) / total
+        return {
+            "average_correctness": avg_correctness,
+            "average_readability": avg_readability,
+            "total": total,
+        }

examples/flows/integrations/unify-ai/eval_code_flow/eval_code_quality.prompty

@@ -0,0 +1,39 @@
+---
+name: Evaluate code quality
+description: Evaluate the quality of code snippet.
+model:
+  api: chat
+  configuration:
+    type: unify
+    model_name: llama-3.1-8b-chat
+    provider_name: together-ai
+  parameters:
+    temperature: 0.2
+inputs: 
+  code:
+    type: string
+sample: ${file:sample.json}
+---
+# system:
+You are an AI assistant. 
+You task is to evaluate the code based on correctness, readability.
+Only accepts valid JSON format response without extra prefix or postfix.
+
+# user:
+This correctness value should always be an integer between 1 and 5. So the correctness produced should be 1 or 2 or 3 or 4 or 5.
+This readability value should always be an integer between 1 and 5. So the readability produced should be 1 or 2 or 3 or 4 or 5.
+
+Here are a few examples:
+
+**Example 1**
+Code: print(\"Hello, world!\")
+OUTPUT:
+{
+    "correctness": 5,
+    "readability": 5,
+    "explanation": "The code is correct as it is a simple question and answer format. The readability is also good as the code is short and easy to understand."
+}
+
+For a given code, valuate the code based on correctness, readability:
+Code: {{code}}
+OUTPUT:

examples/flows/integrations/unify-ai/eval_code_flow/sample.json

@@ -0,0 +1,3 @@
+{
+    "code": "print(\"Hello, world!\")"
+}