Creating bots for Check API

Please note that this page is a work in progress

---

Developing a bot that connects to Check API makes easier to automate tasks that should be executed when some action occurs and perform changes to your workspace data. Bots can interact with Check API by subscribing to events and modifying the data using GraphQL queries.

Table of Contents generated with DocToc

• Bot Workflow
  • Listen to events
  • Execute something
  • Respond to Check API
• Developing a bot
  • Setup your bot
  • Write the code
  • Generate ZIP file
• Deploying a bot to AWS Lambda
• Examples
  • YouTube Data Bot
  • Alegre
  • EXIF Bot
  • Health Desk Bot

Bot Workflow

This is the basic steps a bot will take when interacting with Check API.

Listen to events

The bot should define the events it will listen to. When an action occurs on your workspace and your bot is listening to the related event, Check API will make an HTTP POST request to the the webhook URL with the event data information. The webhook URL is configured on your bot.

Your bot can subscribe to creation and updates of project_media, source and annotations. The exact list of events can be accessed through Check API Rails console by calling BotUser::EVENTS.

Execute something

The bot can help giving more information about an item, integrating with other services, notify people when some specific action happens and many more things.

Some actions executed by bots already used on Check:

• Get thumbnails, dates and geographic information about a Youtube video
• Get title, description and all the available informations of an URL
• Extract metadata from uploaded images
• Find similar images on Google
• Let users interact with items directly on Slack
• Connect a social media tipline
• Identify languages

Respond to Check API

The bot can call a GraphQL mutation and change its workspace. The bot can edit lists, items, create comments, answer tasks, add tags, change status and do all the things the mutations and its role allows to.

Check will identify the requests with the token and all the content added by your bot will have the bot name as the author.

To know how to get an API key token for your bot see how to get an authentication token. To learn how to query, please read Querying the database through Check API and GraphQL query examples.

Developing a bot

In order to receive messages when an event happens on Check API and to send mutations to Check API your bot must have at least a function to handle the received data, a token to authenticate and a request to Check API to send the mutation.

Your bot can be written in any programming language. In this example we'll create a bot written in Node.js using a terminal.

To follow the steps below you need npm, which is automatically installed when you download Node.js. It's a simple bot that is notified when a new item is created and add a comment to it.

Setup your bot

Get an authentication token

To interact to Check API the bot needs an API key token. Read Authentication and authorization on Check API to learn how to obtain a workspace-specific token and how to authenticate.

Workspace-specific API tokens allow Check to identify the bot and give the permissions it needs. The bot will have the same permissions a regular user with the same role in this workspace has. By default, the bots are created with the role editor.

When you ask for a token please send:

• Bot name, description and image
• List of the events your bot will listen to
• GraphQL fields needed
• URL that should be notified by Check

If you want a different permission to your bot, please inform that too.

If you want to try it locally, please read Create Bots on Check

Create a folder for your project

$ mkdir hello-check
$ cd hello-check

Initialize the project

When running this command it will ask to fill in some fields. The texts in brackets are the default values and it will be used if you leave it blank and hit enter. You'll see the content generated and can review it. If it's correct, confirm and it will generate the file package.json.

$ npm init
package name: (hello-check) 
version: (1.0.0) 
description: Sample bot for Check
entry point: (index.js) 
test command: mocha
git repository: 
keywords: 
author: Meedan
license: (ISC) 

Install the dependencies

As it's a simple bot, these two dependencies are a GraphQL client for JavaScript. It will be used when sending the mutation to Check API.

$ npm install lokka --save
$ npm install lokka-transport-http --save

Create a config file

It's not required to have a configuration file, the information can be added directly on file. To keep it more organized we will leave it in a separate file.

Create a file named config.js and add the Check API domain and the workspace-specific token you received to access it.

const config = {
  checkApiUrl: 'https://check-api.checkmedia.org',
  checkApiAccessToken: <token>
};
module.exports = config;

Define the build script

This bot will be hosted on AWS Lambda and it will be uploaded as a compacted file. Edit your package.json file and add on scripts the command that should run when building.

The command used here will delete if there's already a zip file, generate a new zip file with all the files inside the folder and then show a message asking to upload to AWS Lambda.

  "scripts": {
    "test": "mocha",
    "build": "rm -f hello-check.zip && zip -9 -r hello-check.zip * && echo 'Now upload hello-check.zip to AWS Lambda'"
  }

Write the code

The code blocks below is part of a single file.

Include the required modules

const config = require('./config.js'),
      util = require('util'),
      Lokka = require('lokka').Lokka,
      Transport = require('lokka-transport-http').Transport;

Define the handler

The handler is the method in the Lambda function that will process the event when Check API calls the bot webhook.

When the bot is notified it will verify if the event sent is the one we are subscribed to and call the function replyToCheck.

The bot is configured with this GraphQL fragment: dbid, media { url quote }. The dbid is required to send the mutation to Check API and the url and quote will be used on comment content.

exports.handler = (event, context, callback) => {
  const data = JSON.parse(event.body);
  if (data.event === 'create_project_media') {
    const content = data.data.media.url || data.data.media.quote;
    const pmid = data.data.dbid.toString();
    replyToCheck(pmid, data.team.slug, 'Hello from bot! You added ' + content + ' to ' + data.team.slug, callback);
  }
  else {
    callback(null);
  }
};

Send the mutation to Check

The replyToCheck function uses the config file and the arguments to create the mutation request. If the mutation succeeds or fails the details will be written in the log.

const replyToCheck = (pmid, team_slug, text, callback) => {
  const vars = {
    text,
    pmid,
    clientMutationId: 'hello-check' + parseInt(new Date().getTime()),
  };

  const mutationQuery = `($text: String!, $pmid: String!, $clientMutationId: String!) {
    createComment: createComment(input: { clientMutationId: $clientMutationId, text: $text, annotated_id: $pmid, annotated_type: "ProjectMedia"}) {
      comment {
        text
      }
    }
  }`;

  const headers = {
    'X-Check-Token': config.checkApiAccessToken
  };

  const transport = new Transport(config.checkApiUrl + '/api/graphql?team=' + team_slug, { headers, credentials: false, timeout: 120000 });
  const client = new Lokka({ transport });

  client.mutate(mutationQuery, vars)
  .then(function(resp, err) {
    console.log('Response: ' + util.inspect(resp));
    callback(null);
  })
  .catch(function(e) {
    console.log('Error when executing mutation: ' + util.inspect(e));
    callback(null);
  });
};

Bots that run lengthy tasks

If a bot needs to run a lengthy task, it may be necessary to split the job into two different lambdas, so the request initiated by Check-API doesn't stay hanging or time out. A common solution is having the bot split across two lambdas, where the first one just needs to invoke the second one and return. Then the second lambda can take its time processing and writing back to Check when it's done. See the Health Desk Bot for a full example.

const aws = require('aws-sdk');

aws.config.loadFromPath('./aws.json');
      const lambda = new aws.Lambda({ region: config.awsRegion });
      const payload = JSON.stringify({ data: data });
      console.log('payload', payload);

      const lambdaRequest = lambda.invoke({ FunctionName: config.functionName, InvocationType: 'Event', Payload: payload });
      lambdaRequest.send();

Where payload is the data relayed to the second lambda and config.functionName contains the name of the lambda function being invoked.

Important note: Make sure to invoke the lambda without passing any callbacks to lambda.invoke, as it would cause the execution to be synchronous and it defeats the purpose of having a second lambda.

Generate ZIP file

To generate the ZIP file to upload to AWS Lambda:

npm run-script build

Deploying a bot to AWS Lambda

AWS Lambda is a serverless compute service. You can run your code on it in response to events without having to manage servers.

• Create the function
  • Open the AWS Lambda console
  • Choose Create a function
  • For Function name, enter hello-check
  • For language, choose Node.js 12.x
  • Click on Create function
• Get the function URL
  • Add a trigger "API Gateway" to the Lambda function
  • Click on API Gateway and get the URL displayed on API endpoint:. This URL is the webhook URL Check will use to notify your bot
• Upload you code. On "Function Code" click on "Actions", select Upload a .zip file, choose the .zip file you generated and click on Save
• On "Basic settings" confirm the "Handler" field is index.handler.

Examples

There are some bots that can be connected to Check API:

YouTube Data Bot

Source code: https://github.com/meedan/check-bots/blob/develop/youtube.js

Listening to event: create_project_media

What it does:

For YouTube URLs, it does the following:

• Extracts thumbnails and runs reverse image search over each of them
• Extracts upload date and creation data
• Extracts geographic information
• Send these information to Check as annotations.

Alegre

Source code: https://github.com/meedan/alegre

Listening to event: create_project_media

What it does:

Alegre identifies the language of texts. The claims included in your workspace will be analyzed and the language will be added to the item as an annotation.

EXIF Bot

Source code: https://github.com/meedan/check-bots/blob/develop/exif.js

Listening to events: create_project_media, create_annotation_task_geolocation

What it does:

This bot listen to two different events and it does different actions for each event:

• For create_project_media event: if it's an uploaded image, it extracts EXIF data and posts it to Check as a comment, with a link for the full report.
• For create_annotation_task_geolocation and update_annotation_task_geolocation: if it's an uploaded image, it extracts GPS data, geocodes it and posts the result as a task response to any unanswered geolocation task.

Health Desk Bot

Source code: https://github.com/meedan/check-bots/tree/develop/health-desk-bot

Listening to events: create_project_media

What it does:

This bot listens for new item creation events and it looks up similar content in Health Desk articles indexed in Alegre. The functionality is split across two different lambda functions. The first one, health-desk-bot-lambda, receives the events from Check-API and starts the lookup jobs by the second lambda health-desk-bot-lambda-bg which will respond with a note to the Check item if the content matches a Health Desk article.