API

Candid has an extensive API (Application Programming Interface) that allows programmatic access to retrieve and update data in the system.

API Documentation

Candid has an API Help page that lists all the available API methods, their structure and examples (in JSON) of what they return.

This documentation is only available to users that have API access enabled.

To access this documentation click the API link under the Help main menu item. If you do not see the API link under Help, please contact your Candid Administrator about enabling API access for your account.

Candid’s API documentation contains all the calls you can make to create, read, update and delete data programmatically.

API call are grouped by topic in bold, black text. For exmple, Analysis, Dataset, Output.

Click a topic name to expand the list of methods available for that topic. The list of methods will appear as colored buttons. Blue for GET, Green for POST, Orange for PUT, and Red for DELETE.

Click an API method button to expand the documenation for that method. When expanded, you will see the list of parameters for that method and the response description.

How to Use the Documentation

The API documentation is interactive. You can use the GUI to learn how to use the interface as well as try it out.

1. Click the Try it out button to activate parameter inputs to execute a query against your database.
2. Fill in the required inputs with values (For example: enter the number '1' for Organization Id).
3. Click the Execute button. An API call will be made to your database and the results will be returned in the Responses section.

Example of the Responses section when executing a method from the API documentation:

How to Retrieve Data

For performance and flexibility the strategy for retrieving data is done in two parts.

For example if you want to retrieve metadata for all the TFLs on a given analysis:

1. First call the output GET method to retrieve an array of TFL IDs for a given analysis.
2. Then call the output's POST load method, passing in the list of IDs from above in the request body to get the details for each ID.

API Security

There are 3 security modes for Candid. On premise Windows authentication, forms authentication, and single sign-on.

If your installation is using Windows authentication then calls you make will be authenticated against your Active Directory account managed by your IT department. What API methods your account has access to is governed by your Candid user account role.

If your installation is using forms authentication, single sign-on or you want to automate calls via a proxy account, then you will need to generate and use a Candid API token in your calls.

How to use a Candid generated API token to authenticate your API calls:

1. Click your Name in the main menu of the page. Your user Profile and customizations page will open.
2. Click the API Tokens link. The list of tokens you have created will be shown. You can have only 1 enabled token at a time.
3. Click the Generate Candid New Token button to generate a new token for use in your API calls.
4. Your new token will be generated. Immediately copy and paste the token into a safe, secure place before navigating away from this page. For security reasons it will not be displayed again.
5. Use this token in your method calls to authenticate your requests.

Contact support@zeroarc.com for more help using the Candid API.

Candid API integration using R

Introduction

* big thank you to Alisa Solomatina for preparing this content (during her Summer 2024 internship)

Welcome to this comprehensive tutorial on Candid API integration using R! This guide is designed to equip you with the essential skills to interact with Candid’s API, unlocking the potential to view, extract and manipulate data.

This tutorial will walk you through the fundamentals of API interaction, covering topics such as:

• Understanding the structure and role of APIs
• Authentication and authorization
• Making HTTP requests (GET, POST, PUT, DELETE)
• Handling API responses

There are also real R code examples to help visualize the process of making an API call.

What is an API?

An API (Application Programming Interface) is a set of rules and protocols that allows different software applications to communicate with each other. It defines how requests and responses should be formatted and enables data exchange between different systems.

API acts as the interface between R and the Candid system’s data repository. It provides a structured way to formulate server requests (specifying which data is needed and in what format) and receive the corresponding data in a usable format. This ensures that R can effectively “communicate” with the Candid system and seamlessly access the required data

How is an API Used with R?

With APIs, R can be used to extract, modify and add or delete data in the Candid WebApp. Here how it operates:

Sending HTTP Requests:

R can send HTTP requests (like GET, POST, PUT, DELETE) to an API endpoint (URL for data access) using functions from libraries such as httr or curl. For example, you might use a GET request to retrieve data such as TFL IDs, and then use a POST request to submit these IDs to the server and get back the TFL metadata. PUT request is usually used for updating records, while delete request deletes them (e.g issues, comments or test results).

Handling API Responses:

Once a request is made, the API responds with data, usually in formats like JSON or XML. R can parse this data using libraries like jsonlite (for JSON) or xml2 (for XML) to convert it into R data structures like lists or data frames which are easier to read and manipulate.

Authentication:

Candid API requires API token authentication to ensure secure access. There are two types of tokens in Candid - Read-only (GET and POST requests only) and Full-access (PUT and DELETE requests). They can be generated in the user profile (they need to be stored securely as they will be displayed only once). These tokens are necessary for running any API requests and are included in the request headers in every R script.

Data Manipulation:

After receiving and parsing the data from the API, R can be used for data manipulation, utilizing packages like dplyr or tidyr to clean, analyze, and visualize the data.

Get an API token. API tokens are essential for authenticating requests with the server. They act like a digital key, ensuring that the server recognizes you and grants you access to its resources. Without a valid API token, this code won't be able to connect to the server and retrieve the data. API token can be generated in your user profile in Candid > API Tokens tab. Make sure the token is copied somewhere secure before leaving the page, since for security reasons it will not be displayed again.

Constructing an API Request

To construct an API request in R, you will need:

1. R packages: jsonlite, httr, and stringr
2. API Token
3. API URL Endpoint
4. Filter Parameters
5. Headers
6. Request Body (for Post and PUT requests)

R packages: jsonlite, httr, and stringr

httr is used for making HTTP requests (GET, POST, PUT and DELETE) and handling the responses.

httr functions used in the code:

• GET(): Sends an HTTP GET request to a specified URL.
• POST(): Sends an HTTP POST request, used to submit data to a server and get back a response.
• PUT() : Sends an HTTP PUT request to update a record.
• DELETE(): Sends an HTTP DELETE request to delete a record.
• content(): Extracts the content from an HTTP response, which could be in text, JSON, or other formats.
• add_headers(): Adds headers to your request, useful for including API keys, input and output data formats.

jsonlite is used for converting between JSON and R objects, making it easy to work with JSON data in R. It is an essential package since Candid API requests operate (accept and return data) through JSON.

jsonlite functions used in the code:

• fromJSON(): Converts JSON strings into R objects (e.g., data frames, lists).
• toJSON(): Converts R objects into JSON strings.

The stringr package provides a set of consistent, simple, and convenient functions for working with strings. In our code, the str_replace_all() function is used when constructing API url endpoint to prevent blank spaces between the values.

Packages can be installed in R studio via the “packages” tab in the Files pane, or through running the following code: install.packages("httr","jsonlite", "stringr")

After the packages are installed, they will need to be imported into the code:

library(httr)

library(jsonlite)

library(stringr)

API Token

Candid API requires an API token for all server requests. After it is generated in the user profile and securely stored, the token can be inserted in the R-script, for example:

api_token <- "INSERT TOKEN HERE"

After running this line, your token will be stored as a value in the Environment pane. It will be stored across multiple files in the same session, so only needs to be assigned once in the beginning of the script.

API URL endpoint

An API URL endpoint is a specific URL that represents a resource or an action in a web-based API. It is the address where the API listens for requests and provides responses. An endpoint is a part of the API’s URL that directs your request to a specific resource or action on the server.

Every Candid API call consists of a unique combination of a request type (GET, POST etc) and an endpoint URL. The details for each call can be found in the Candid API help page. Here are a few examples:

URL values in the { } brackets are essential base parameters and API request will not work if they are not set. Organization ID is one of these parameters, and it is always “1”. This also needs to be set in the beginning of the script:

organisationId <- 1

Same as the API token, this will be stored as a value.

Other essential parameters can include various IDs, depending on the request: analysis ID, issue ID etc. If they are present in the URL, they must be defined before running the code.

Taking the GET request from the screenshot above as an example, the URL for it looks like this:

url <- str_replace_all(paste("https://dev.candid.zeroarc.com/api/v4/", organisationId, "/output"), " ", "")

Note, that this is also being stored as a value.

Filter Parameters

Filter parameters are included in API requests to filter, sort, or search through data returned by the server. Parameters in the code match the filtering options in Candid API help page. You can also refer to this page for parameter data types and whether any of them are mandatory for the request.

After setting up these parameters in R, they will be stored as a list object.

Headers

Headers define the nature of the request and the expected response. They provide metadata about the request or response, such as the type of data being sent or received and authentication details.

There are three main types of headers used in Candid API calls:

1. 'Accept' header: tells the server the format in which you expect the response. This ensures that the response is returned in a format that the user can understand and process (like JSON, plain text, etc.).

2. 'Content - type' header: specifies the format of the data being sent to the server (in our case it is always in JSON format), which is particularly important in POST requests where data is included in the body of the request. This tells the server how to interpret the data being sent.

3. 'Authorization' header: Provides credentials or tokens that authenticate the client with the server. Our code uses a Bearer token (a type of OAuth token) to authenticate the request. This token confirms that the user has permission to access the API and perform the operations being requested.

Request Body (for POST or PUT requests only)

The request body is an essential component of POST and PUT requests. These types of requests are used to send data to a server, whether it’s to create a new record or update an existing one.

The request body contains the data that you want to send to the server. In Candid API, this data is in the JSON format. The server processes this data and performs the corresponding action, such as returning data, creating a new entry in a database or updating an existing record.

In R, we use toJSON function to convert a request body into JSON format that server can accept.

If unsure how to construct the request body, refer to the Candid API help page - it has example of both values and responses for each API call.

Error Handling

Error handling is a crucial aspect of making API requests, as it allows R to respond appropriately when something goes wrong, such as when a request fails or the server returns an unexpected response.

If problem is with R code itself, the error can be seen in the console. However, to view any errors from the request itself, additional code is required.

After making an API request, the server returns a status code as part of the response. This code indicates whether the request was successful or if an error occurred:

• Success: Typically, a status code of 200 indicates a successful request. For POST requests it could also be 201, and 204 for PUT requests.

• Client Errors (4xx): These errors occur when the request is not valid:

  • 400 Bad Request- The server cannot understand or process the request due to malformed syntax or invalid data. Common causes include:

    • Incorrectly formatted JSON or XML in the request body
    • Missing required parameters or fields in the request
    • Invalid parameter values, such as incorrect data types

  • 401 Unauthorized: The request lacks valid authentication credentials for the target resource. Common cause: Missing/incorrect/expired API token

  • 404 Not Found - The server cannot find the requested resource. The requested URL does not correspond to any existing resource. Common cause: trying to update/delete non-existing or wrongly specified record, i.e issue, test result, etc.

• Server Errors (5xx): These errors occur when something goes wrong on the server side (e.g., 500 Internal Server Error - The server encountered an unexpected condition that prevented it from fulfilling the request)

Code example: GET Request

In this code example, GET request is used to extract output IDs

The printed output:

(Notice the format - an array of integers)

Code example: POST request

In this example, POST request is used to create a new test result for a TFL.

In this particular request, there is no output being displayed. However, in some POST requests where data is being retrieved, the response can be viewed under the current code chunk as a dataframe. As an example, here is a table of TFL metadata, after an array of TFL IDs had been sent to a sever through a POST request: