part2-api-workshop-curl.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b95deda8-5dc6-4422-80ca-992980be8cc4",
   "metadata": {},
   "source": [
    "# eLabFTW - API workshop - Part 2 - Using curl\n",
    "\n",
    "This notebook will guide you towards using the `curl` command line tool to make requests. But first, we need a key!"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3ea849d2-11a3-46d1-a2f3-efd5add15ad0",
   "metadata": {},
   "source": [
    "## **Chapter 1: Create an API Key for authentication**\n",
    "\n",
    "Each request needs to carry a key to authenticate ourselves with the server. So we will need to create that key. As an eLabFTW user, you already have an account and can navigate to your settings to generate the **API key** for authentication.\n",
    "\n",
    "For this part, we will need to login via the web interface to manage the api keys.\n",
    "\n",
    "Here is the link to your local instance:\n",
    "\n",
    "⇒ [API Workshop Instance](https://api-workshop.elab.one) ⇐\n",
    "\n",
    "#### 1. **Log in and go to your `settings`**\n",
    "\n",
    "![login settings](https://github.com/elabftw/api-workshop/blob/master/assets/1-login-settings.png?raw=true)\n",
    "\n",
    "*(Click on the settings icon from the dropdown menu.)*\n",
    "\n",
    "#### 2. **Navigate to the `API KEYS` tab**\n",
    "\n",
    "![api keys navigation tab](https://github.com/elabftw/api-workshop/blob/master/assets/2-api-keys-tab.png?raw=true)\n",
    "\n",
    "\n",
    "*(Find the API Key section where you can manage your keys.)*\n",
    "\n",
    "#### 3. **Create a read/write key**\n",
    "- Enter a name for the key (*e.g.* `api-for-test`).  \n",
    "- Select the appropriate permission level. In this case, **read/write** would be appropriate for the course.\n",
    "- Click **Generate API Key**.\n",
    "\n",
    "\n",
    "![api key full tab](https://github.com/elabftw/api-workshop/blob/master/assets/3-api-key-full-tab.png?raw=true)\n",
    "\n",
    "Congratulations! The API Key has been generated. **Make sure to save it somewhere safe** as you will use it often. We will need it now to send requests to the API.\n",
    "\n",
    "![api key generated](https://github.com/elabftw/api-workshop/blob/master/assets/4-api-key-generated.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e713331b-4a39-4421-bfce-1b5d15571c25",
   "metadata": {},
   "source": [
    "## **Chapter 2: Make your first curl request with an API key**\n",
    "\n",
    "### 1. **Setup your environment**\n",
    "\n",
    "Before interacting with the API, the first step is to configure the necessary environment variables. These variables enable seamless communication with the API without requiring repeated manual input.\n",
    "\n",
    "You will need to define the following:\n",
    "- `API_URL`: The base URL of your eLabFTW instance.\n",
    "- `API_KEY`:  The API key you recently generated.\n",
    "\n",
    "In **Jupyter Lab**, you can use the `%env` magic command to set these environment variables, making them reusable by assigning them a key (name) instead of manually entering the full values each time.\n",
    "\n",
    "#### **Execute the commands**\n",
    "\n",
    "Use your **generated API Key** and your **API URL** to adapt the code below. For example:\n",
    "\n",
    "%env API_URL=**yourdomain.elab/api/v2/**\n",
    "\n",
    "%env API_KEY=**ADD YOUR KEY HERE**\n",
    "\n",
    "Then, as explained in the [first section](./0-api-workshop-intro.ipynb#How-to-navigate-through-this-Notebook), **select the cell below** and click the play button!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4094eda-0522-4de0-87da-3d138ddeaafe",
   "metadata": {},
   "outputs": [],
   "source": [
    "%env API_URL=https://api-workshop.elab.one/api/v2/\n",
    "%env API_KEY=<ADD YOUR KEY HERE>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58629361-957c-4383-91b1-af38c49ae8fd",
   "metadata": {},
   "source": [
    "Now, verify the configuration by executing the following cell.\n",
    "\n",
    "On success, this will display all environment variables containing **\"API_\"**, ensuring they are correctly stored and accessible for further API interactions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bbbd2bf-2fa9-493a-9781-2b47525fcb09",
   "metadata": {},
   "outputs": [],
   "source": [
    "!env | grep API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49890716-936a-4aaf-82f6-17025eccc195",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\"><b>You should see:</b>\n",
    "    <p>API_URL=https://api-workshop.elab.one/api/v2/</p>\n",
    "    <p>API_KEY=9-349864..............................................649</p></div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3287ac1-c5db-4dc4-a4bd-72fc85cae28e",
   "metadata": {},
   "source": [
    "### 2. **Make your first request to the API**\n",
    "\n",
    "We will now use `curl` to have our first interaction with the API.\n",
    "<b>curl</b> is a command-line tool used to make HTTP requests.\n",
    "It will be completed by an `Authorization` header : when making API requests, you often need to send an authorization token in the headers to <b>authenticate</b>. We will provide it with the `-H` flag, like this:\n",
    "\n",
    "```sh\n",
    "curl $API_URL -H \"Authorization: $API_KEY\"\" \n",
    "```\n",
    "\n",
    "- `-H \"Authorization: $API_KEY\"`: This is the way of adding an HTTP header for authentication, using your API key.\n",
    "\n",
    "<div class=\"alert alert-block alert-warning\"><b>💡 Important:</b> Always use double quotes (\") around <b>\"Authorization: ...\"</b> to ensure the terminal correctly interprets the $API_KEY variable. Using single quotes (') would pass the string literally instead of expanding the variable.</div>\n",
    "\n",
    "\n",
    "#### **Send it using curl**\n",
    "\n",
    "We will now complete the previous command with an [API Endpoint](./part1-api-workshop-intro.ipynb#2-what-are-api-endpoints?).\n",
    "\n",
    "Let us run the cell below to send a `curl` request using the `GET` method to the `/info` endpoint. This will retrieve basic information about your eLabFTW instance.\n",
    "\n",
    "📖 [See here](https://doc.elabftw.net/api/v2/#/Info/get-info) for the documentation related to this endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aff8b221-c88b-449e-8402-4cf58d362537",
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl \"$API_URL\"/info -H \"Authorization: $API_KEY\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e70a0f2-94b3-4c29-874e-8912bd8afaa7",
   "metadata": {},
   "source": [
    "You should get this answer back from the server:\n",
    "\n",
    "(if not, please refer to the [troubleshooting section](#Troubleshooting))\n",
    "\n",
    "*Formatted JSON:*\n",
    "~~~json\n",
    "{\n",
    "   \"elabftw_version\":\"5.2.0-alpha\",\n",
    "   \"elabftw_version_int\":50200,\n",
    "   \"ts_balance\":0,\n",
    "   \"ts_limit\":0,\n",
    "   \"uploads_filesize_sum\":116507,\n",
    "   \"uploads_filesize_sum_formatted\":\"113.78 KiB\",\n",
    "   \"all_users_count\":2,\n",
    "   \"active_users_count\":2,\n",
    "   \"items_count\":0,\n",
    "   \"teams_count\":2,\n",
    "   \"experiments_count\":1,\n",
    "   \"experiments_timestamped_count\":0,\n",
    "   \"entities_timestamped_count_last_30_days\":0\n",
    "}\n",
    "~~~\n",
    "\n",
    "**Congratulations! You just did your first request to the eLabFTW API using curl.**\n",
    "\n",
    "You can find a full list of the eLabFTW API's endpoints (*e.g.* `apikeys`, `experiments`, `config` *etc.*), with their different verbs (*e.g.* `GET`, `PATCH`) on this page:\n",
    "\n",
    "https://doc.elabftw.net/api/v2/\n",
    "\n",
    "![api swagger display full](https://github.com/elabftw/api-workshop/blob/master/assets/6-api-swagger-display.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cce320bb-4a74-496d-bbeb-60d5b310f664",
   "metadata": {},
   "source": [
    "## **Chapter 3: Perform a full operation, from create to delete, with eLabFTW's API**\n",
    "\n",
    "Now that you have all the tools at your disposal, let us perform some actions from A to Z with the API. We are going to **C**reate, **R**ead, **U**pdate and **D**elete an experiment (**CRUD**)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5a978bb1-4ff7-470a-a0e5-769bb7b47f79",
   "metadata": {},
   "source": [
    "### 1. **Create an experiment with the POST verb**\n",
    "\n",
    "![api experiments endpoint](https://github.com/elabftw/api-workshop/blob/master/assets/7-api-experiments-endpoint.png?raw=true)\n",
    "\n",
    "We will start by creating an experiment with the command below (click to execute):\n",
    "\n",
    "<details><summary>The command is in one line. <b>Click here</b> to see what the extended code looks like</summary>\n",
    "    \n",
    "~~~bash\n",
    "curl -X 'POST' \\\n",
    "  'https://elab.local:3148/api/v2/experiments' \\\n",
    "  -H 'accept: */*' \\\n",
    "  -H \"Authorization: $K\" \\\n",
    "  -H 'Content-Type: application/json' \\\n",
    "  -d '{\n",
    "  \"body\": \"<h1>Section title</h1><p>Main text of resource</p>\",\n",
    "  \"canread\": \"{\\\"base\\\": 20, \\\"teams\\\": [], \\\"users\\\": [1], \\\"teamgroups\\\": [1]}\",\n",
    "  \"canwrite\": \"{\\\"base\\\": 20, \\\"teams\\\": [], \\\"users\\\": [1], \\\"teamgroups\\\": [1]}\",\n",
    "  \"category\": 3,\n",
    "  \"content_type\": 1,\n",
    "  \"metadata\": {\n",
    "    \"extra_fields\": {\n",
    "      \"Information\": {\n",
    "        \"type\": \"text\",\n",
    "        \"value\": \"Lorem ipsum.\",\n",
    "        \"required\": true,\n",
    "        \"description\": \"This is a field of type: text\"\n",
    "      }\n",
    "    }\n",
    "  },\n",
    "  \"rating\": 0,\n",
    "  \"status\": 1,\n",
    "  \"tags\": [\n",
    "    \"test\",\n",
    "    \"confidential\",\n",
    "    \"replication\"\n",
    "  ],\n",
    "  \"title\": \"Test replicating the results of Naomi\",\n",
    "  \"template\": -1\n",
    "}'\n",
    "~~~\n",
    "</details>\n",
    "<br>\n",
    "<div class=\"alert alert-block alert-info\"><b>Note:</b> The lines starting with a number sign <b>(#)</b> are comments, which will be ignored during the execution of the command.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f328bdf9-4ade-4ffa-8326-fac72b23c7c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# we add \"-w\" option to display the response status code.\n",
    "!curl  -w \"%{http_code}\" -X POST \"$API_URL\"/experiments -H \"accept: */*\" -H \"Authorization: $API_KEY\" -H \"Content-Type: application/json\" -d '{\"body\":\"<h1>Section title</h1><p>Main text of resource</p>\",\"canread\":\"{\\\"base\\\": 20, \\\"teams\\\": [], \\\"users\\\": [1], \\\"teamgroups\\\": [1]}\",\"canwrite\":\"{\\\"base\\\": 20, \\\"teams\\\": [], \\\"users\\\": [1], \\\"teamgroups\\\": [1]}\",\"category\":3,\"content_type\":1,\"metadata\":{\"extra_fields\":{\"Information\":{\"type\":\"text\",\"value\":\"Lorem ipsum.\",\"required\":true,\"description\":\"This is a field of type: text\"}}},\"rating\":0,\"status\":1,\"tags\":[\"test\",\"confidential\",\"replication\"],\"title\":\"Test replicating the results of Naomi - using curl\",\"template\":-1}'\n",
    "# By default, there would be no output."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82c87403-73c5-45bc-b708-888cf995359a",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\"><b>Success:</b> 201 Created: The resource was successfully created.</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29bf0d67-f965-4e50-ab86-5b4ef4ba3561",
   "metadata": {},
   "source": [
    "### 2. **Read an experiment with the GET verb**\n",
    "\n",
    "In this section, we use the tool `jq`, a tool for processing JSON data, and we will use it to format API responses. \n",
    "\n",
    "Install it with your package manager, or for **Google Colab** run the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3461ebf9-00e7-4b85-9cae-45484a211a83",
   "metadata": {},
   "outputs": [],
   "source": [
    "# only for google colab\n",
    "!apt install jq"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6673593c-4121-4b5c-917f-3f974c43d42a",
   "metadata": {},
   "source": [
    "Let us read the content of an experiment. The command will retrieve a list of experiments from the eLabFTW API. We will use jq to extract only the `id` and `title` fields from the response.\n",
    "\n",
    "- The `curl` command sends a `GET` request to the API at `https://api-workshop.elab.one/api/v2/experiments`.\n",
    "- The `-s` option makes the request silent, meaning it won’t show progress or error messages unless necessary.\n",
    "- and the `-H \"Authorization: $K\"` header provides the required authentication using the API key stored in the `$K` environment variable.\n",
    "\n",
    "The output of the `curl` command is piped (`|`) to `jq`. The filter `.[] | {id, title}` instructs `jq` to process each item in the returned JSON list and extract only the `id` and `title` fields from each experiment. The result is a list of objects containing just these two fields, making it easier to read and work with the relevant information without extra details from the full API response.\n",
    "\n",
    "Click the cell below and execute to see the result :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d03be706-94b3-40a5-bc0e-2569ac2cbd60",
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl -s -X GET \"$API_URL\"/experiments -H \"Authorization: $API_KEY\" | jq '.[] | {id, title}'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a98b48f-82dd-4b2c-9ead-244a4a555620",
   "metadata": {},
   "source": [
    "This should be your response from the server:\n",
    "\n",
    "~~~json\n",
    "{\n",
    "  \"id\": 1,\n",
    "  \"title\": \"Test replicating the results of Naomi - using curl\"\n",
    "}\n",
    "~~~\n",
    "\n",
    "If you need not to filter the result for the id and title, you can of course remove the part with `| jq '....'`\n",
    "\n",
    "### **Going further**\n",
    "\n",
    "Try to extract the `created_at` and `modified_at` fields by modifying the previous command.\n",
    "\n",
    "### 3. **Read one specific experiment with the GET verb**\n",
    "\n",
    "We can now use the latest id, which is the experiment we created, to try and get **one specific experiment** instead of all of them.\n",
    "\n",
    "The command below will retrieve one experiment and its information.\n",
    "\n",
    "Here, replace the **\"X\"** in \"/experiments/<span class=\"alert-danger\">X</span>\" with an <b>id</b> from the previous <b>GET</b> request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45ef39a0-17b1-4cd1-9655-e6415397e55a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl -s -X GET \"$API_URL\"/experiments/X -H \"Authorization: $API_KEY\" | jq '.'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e103b00-3533-43a1-93d1-47cfdd707189",
   "metadata": {},
   "source": [
    "### 4. **Update an experiment with the PATCH verb**\n",
    "\n",
    "This command patches the experiment with the ID you provide.\n",
    "Download the\n",
    "Here, replace the **\"X\"** in \"/experiments/<span class=\"alert-danger\">X</span>\" with an <b>id</b> from the <b>GET</b> request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "785b43c9-6689-41b1-a718-ad670e953fd2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!curl -s -X PATCH \"$API_URL\"/experiments/X -H \"Authorization: $API_KEY\" -H 'Content-Type: application/json' -d '{\"title\": \"Updated the title through curl\"}' | jq '.'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "582a0fe7-cf1a-400a-a2f3-e41478eca878",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\"><b>Success:</b> 200 Success: The resource is returned with updated content.</div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aadbf3f9-64c4-49a7-a20c-8316bb3a6d8d",
   "metadata": {},
   "source": [
    "Should you see any other answer, please refer to the 👉 [troubleshooting section](#Troubleshooting).\n",
    "\n",
    "### 5. **Delete an experiment with the DELETE verb**\n",
    "\n",
    "Deleting an experiment is a simple task. You need to provide the ID as the parameter, and the response will be empty on success, just like the POST verb.\n",
    "\n",
    "Here, replace the **\"X\"** in \"/experiments/<span class=\"alert-danger\">X</span>\" with an <b>id</b> from the previous <b>GET</b> request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac4e6359-273a-438a-b3e9-576ef48057df",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!curl -s -X DELETE \"$API_URL\"/experiments/X -H 'Content-Type: application/json' -H \"Authorization: $API_KEY\" | jq '.'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1db4a6d1-482c-47e2-8086-1704b96bf8f6",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-success\"><b>Success:</b> 204 No Content: The resource was successfully deleted, and no further content is returned.</div>\n",
    "\n",
    "Congratulations on finishing the second part of this course ! If you want to improve even further your lab's workflow, you can proceed to the next part of this course and learn about **python** programming, to automate your actions.\n",
    "\n",
    "➡️ See [last part](./part3-api-workshop-python➡️.ipynb) of this course.\n",
    "\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d62f7209-7092-422a-b94b-a8ed806b2928",
   "metadata": {},
   "source": [
    "# Troubleshooting\n",
    "\n",
    "Here are some issues you could have encountered during this course.\n",
    "\n",
    "### Error 400: Bad Request\n",
    "\n",
    "~~~json\n",
    "{\n",
    "   \"code\":400,\n",
    "   \"message\":\"Bad Request\",\n",
    "   \"description\":\"Invalid endpoint: available endpoints: apikeys, batch, config, idps, idps_sources, import, info, experiments, exports, items, experiments_templates, items_types, event, events, extra_fields_keys, favtags, team_tags, teams, todolist, unfinished_steps, users\"\n",
    "}\n",
    "~~~\n",
    "\n",
    "**Explanation:**\n",
    "The error message indicates that the request was made to an invalid or non-existent endpoint. The API responded with a 400 Bad Request error, meaning the request is not correctly formatted or does not match an available resource.\n",
    "\n",
    "The description provides a list of valid endpoints, such as `apikeys`, `config`, `experiments`, `users`, etc. This suggests that the issue is likely due to a typo in the endpoint name or an incorrect URL structure. To resolve this, on the next step we will use one of the listed endpoints in the request.\n",
    "\n",
    "You can also receive a **400** error such as :\n",
    "~~~json\n",
    "{\n",
    "  \"code\": 400,\n",
    "  \"message\": \"Bad Request\",\n",
    "  \"description\": \"Cannot check permissions without an id!\"\n",
    "}\n",
    "~~~\n",
    "\n",
    "### Error 401: Authentication required\n",
    "\n",
    "~~~json\n",
    "{\n",
    "    \"code\":401,\n",
    "    \"message\":\"Unauthorized\",\n",
    "    \"description\":\"Authentication required\"\n",
    "}\n",
    "~~~ \n",
    "\n",
    "**Explanation:**\n",
    "This error occurs when the API key is incorrect or missing. The server rejects the request because authentication is required but not properly provided. Ensure that:\n",
    "\n",
    "- Your API key is correctly set in the environment\n",
    "- You are using double quotes around \"Authorization: $API_KEY\" in your curl request.\n",
    "\n",
    "### Error 401: No corresponding API Key\n",
    "\n",
    "~~~json\n",
    "{\n",
    "    \"code\":401,\n",
    "    \"description\":\"No corresponding API key found!\"\n",
    "}\n",
    "~~~ \n",
    "\n",
    "**Explanation:**\n",
    "This message indicates that the provided API key exists and is correctly formatted, but it does not match any valid key on this eLabFTW instance. Possible reasons include:\n",
    "\n",
    "- The API key was created in a different eLabFTW instance.\n",
    "- The key has been revoked or deleted.\n",
    "- The key was not copied correctly when set in the environment.\n",
    "\n",
    "🔹 Solution: Double-check that you're using an active API key from the correct eLabFTW instance."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}