Web API

From Flametree Technologies
(Difference between revisions)
Jump to: navigation, search
(API endpoints)
(API endpoints)
Line 333: Line 333:
 
! width="40%"|Sample call
 
! width="40%"|Sample call
 
|-
 
|-
|'''www.flametreeapi.com/user'''
+
|'''/user'''
 
|POST
 
|POST
 
|username, password, admin
 
|username, password, admin

Revision as of 10:02, 14 August 2019

Contents

Why a Web API?

Flametree’s Web API allows you to run FIA remotely from any location, using an internet connection.

Providing FIA’s capabilities through a Web API provides the following advantages:

To run the examples shown here, you will need

The API's web address is

www.flametreeapi.com

The Python API wrapper

The most straightforward way to use the Web API is to use our Python wrapper routines to interact with the program, which provide short-cuts to commonly used functions. For instance, to see a list of your files in the filestore, you can either run

ListFiles(username, password)

or

url = base_url + '/file'   
try:          
    response = requests.get(url, auth=HTTPBasicAuth(username, password) )    
    print(response.status_code, response.text)
except:
    print('Error found when listing files')
    sys.exit(1)

Both code fragments return the same information, but the first shields you from the details of calling the API directly.

We assume that the Python wrapper routines are used throughout this document. If you prefer, you can call the API directly using the Python requests (or similar) library from any programming language. Full documentation is provided here.

Architecture and workflow

The filestore

In principle, FIA can be run by uploading all required data, together with values of any configuration settings required, and then running the program. For most users, this approach has several disadvantages:

To avoid these issues, the API is designed so that file operations are distinct from program execution. FIA uses a server-based filestore, to which raw data files are written, and from which results are read. The program is run using the following sequence of commands:

  1. Files required by the program are uploaded to the filestore.
  2. Once all files are in place, the user sends an ‘execute’ command, together with parameter values to describe the type of attribution analysis required.
  3. The API submits this request to a batch queue on the remote server.
  4. The server runs the attribution analysis as soon as resources become available.
  5. When the analysis is complete, results and reports are written to the filestore as a compressed file.
  6. The results file is downloaded.

The batch queue

By using a batch queue for processing, Flametree Attribution allows ‘fire and forget’ processing for large numbers of portfolios.

Each time an attribution analysis is requested, a new batch job is created. This is submitted to the batch queue and is processed when previous jobs are complete. When a batch job is created, it is assigned a unique job ID that can be used to interrogate the batch queue and determine its current status.

The batch queue uses multiple worker processes so that several jobs can be executed at the same time.

A batch processing job has one of the following status values:

Security

Each call to the API requires that the user provide credentials identifying themselves as a registered user of the system. This must be either:

No user can access any other user’s data or usage records. Passwords are stored internally in hashed form so that nobody has access to the plaintext versions.

Flametree Attribution automatically encrypts all traffic to and from the site using HTTPS, so that your identity and results are protected.

Calling the API

These examples assume:

The examples shown use our suite of wrapper functions, which encapsulate the complexities of calling the API directly. If you prefer to call the API directly, please refer to Appendix X.

Import reference to the fia_wapper functions

from fia_wrapper import *	

Assign values for username and password. These are used each time an API function is called:

username = 'username'
password = 'password'

Assign values to be passed to the API to define the attribution. Some fields (PortfolioFile, SecurityFile, YieldCurveFile, ZipFile) must have values assigned; others are optional:

data = {
   "PortfolioFile" : "fia_p.csv",
   "SecurityFile" : "fia_s.csv",
   "YieldCurveFile" : "fia_y.csv",
   "IndexFile"	: "fia_i.csv",
   "batch_id" : "1001",
   "SovereignCurveDecomposition" : "KRD",
   "DateFormat" : "%Y-%b-%d",
   "SummaryAttributionReport" : "true",
   "InteractiveAttributionReport" : "true",
   "SecurityAttributionReport" : "true",
   "CurveReport" : "false",
   "ExPostRiskReport" : "false",
   "CSVreport" : "true",
   "XLSreport" : "true",
   "ZipFile" : "results.zip"
}	

Upload required files to the filestore. If a file is already present in the filestore, it need not be uploaded again:

UploadFile (data['PortfolioFile'],    username, password)
UploadFile (data['SecurityFile'],     username, password)
UploadFile (data['YieldCurveFile'],   username, password)
UploadFile (data['IndexFile'],        username, password)

(Optional) Show the names of all this user’s files present in the filestore:

ListFiles(username, password)	

Submit this job to the batch queue. The job is assigned a unique job ID. In this instance, we are only submitting one job, but you can submit as many as you like at the same time:

job_id = RunJson (data, username, password)

Then, repeatedly interrogate the batch queue to find this job’s status. When it has run, its status is ‘COMPLETE’:

while True:   
   if IsRunning (job_id, username, password) == 'COMPLETE': 
       break 

Now that the job has run, download the results in data[‘ZipFile’]:

DownloadFile(data['ZipFile'], username, password)

Uploading files

To upload a file, call the following statements in Python, which issue a ‘post’ command to the API and some diagnostic data:

UploadFile(filename, username, password)

If all has gone well, this will display the message

{200, "message": "File uploaded OK"}

If there is already a file with this name in the filestore, Python will display

{403, "message": "Duplicate file name"}

If you want to replace a file with another that has the same name, delete the older copy from the filestore by using the commands

DeleteFile(filename, username, password)

If the file was deleted successfully, you will see the message

{200, "message": "File ‘fia_s.csv’ deleted"}

You can now upload a new copy of the file.

To see a list of all files in the filestore, issue the commands

ListFiles (username, password)

The names of all the files present in this user's account will be returned as a list in the ‘response.text’ field.

The same file can be called from multiple runs. For instance, a common security master, or benchmark, can be referenced during multiple runs. In this case there is no need to upload the file repeatedly.

Uploaded files are subject to the following restrictions:

Running the program

Once you have uploaded all the files required by FIA, the next step is to set the program’s parameters and run the program. This is done by setting up a JSON structure and initializing its values. If you have not used JSON before, please refer to the examples below.

FIA has a long list of parameters that can be set to tailor your analysis. A few must be set, but most are optional and have sensible defaults. For a full list of the parameters and their defaults, refer to Appendix A.

Required fields

Values for PortfolioFile, SecurityFile, YieldCurveFile and ZipFile must be set each time the API is called. A 404 error will be generated if any of these quantities has not been set.

Parameter lists

FIA is run using the execute command. This function requires a JSON structure to be passed as an argument.

In this example, we set up a JSON structure directly. However, many languages allow conversion from other data structures to JSON. For instance, the Python ‘json.dumps’ function converts a Python dictionary to JSON.

The parameters shown here are represented in JSON as

data = {
"PortfolioFile"	: "fia_p.csv",
"SecurityFile" 	: "fia_s.csv",
"YieldCurveFile": "fia_y.csv",
"SovereignCurveDecomposition": "STB",
"xls_reports": True,
"batch_id": 100,
"ZipFile" : "DemoZipFile.zip"
}

The program is run by issuing the command

job_id = execute(data, username, password)

When the user issues one or more of the above run commands, two things happen:

Interrogating the batch queue

Once you have a job’s ID code, the Status command allows you to interrogate the batch queue and return the current status of that job.

For instance, to run a single job you can use the following commands:

id = RunJson (data, username, password)
while True:
    if Status (id, username, password) == 'COMPLETE': 
        break

This code fragment submits a job to the queue, records the job ID, and repeatedly polls the task queue until the job is complete. At this point the results are available and can be downloaded from the filestore.

Wrapper functions

General user commands

These commands can be run by any user. Username and password must be added to parameters (if any). If a login token is used, this can be substituted for the username, and the password field can then be left blank.

Command Description Arguments Returns
LatencyTest Ping the Flametree server. This command can be used to check that you have an active Internet connection, and to monitor traffic speeds. (none) Time taken to get a response from the Flametree server
ListFiles Generate list of all files in user’s account (username, password) Python list of file names
UploadFile Upload single file to user’s account. If the file already exists, an error message is returned. (filename, username, password)

'filename' can include path on host system.

Boolean, indicating whether file was uploaded successfully
DeleteFile Delete single file from user’s account (filename, username, password) Boolean, indicating whether file was deleted successfully
DeleteAllFiles Delete all file from user’s account (username, password) Boolean, indicating whether all files were deleted successfully
DownloadFile Download single file from user’s account (filename, username, password) Boolean, indicating whether file was downloaded successfully
Execute Submits job with parameters supplied in JSON structure ‘settings’ to batch queue (settings, username, password) If successful, returns UUID string job id. This can be used to interrogate the batch queue via the Status command.
GetQueue Shows list of all jobs submitted that have status PENDING (not yet started). Note that this does not include jobs that are currently being processed, so a queue with zero length does not necessarily mean that your job has completed. (username, password) List of job IDS
Status Interrogate queue to find status of job with given ID. (job_id, username, password) PENDING, RUNNING, COMPLETE, FAILED, NOT_FOUND
GetToken Returns security token that can be used in place of user name and password (username, password) String token that can be used in place of a username and password.

Administrator commands

These commands can only be run by the system’s administrator. The admin password must be added to parameters, if any.

Command Description Arguments Returns
CreateUser Create new user account, using supplied name and password. Requires administrator privileges. (new_user_name, new_password, admin_password) Boolean, indicating whether user account was created successfully
DeleteUser Delete user account and associated filestore. Requires administrator privileges. (user_name, admin_password) Boolean, indicating whether user account was deleted successfully
ListUsers Generate list of all users. Requires administrator privileges. (admin_password) Python list of all user accounts
UsageReport Generates raw system usage report for all users (user ID, date and time of call, CPU usage, IP address of caller). Requires administrator privileges. (admin_password) JSON structure, containing all usage data

API endpoints

These commands can only be run by the system’s administrator. The admin password must be added to parameters, if any.

All URLs are prefixed with www.flametreeapi.com.

URL Method Parameters Data parameters Success response Error response Sample call
/user POST username, password, admin 200 AUTHORIZED 401 UNAUTHORIZED
data = {}
data['username'] = username
data['password'] = password
data['master_password'] = master_password
json_data = json.dumps(data)
response = requests.post(url, data=json_data, headers=headers)   

API status codes

Flametree Attribution uses REST (Representational State Transfer), an API interface protocol that is rapidly becoming the standard for communication between software components. Commands are issued using a simple set of verbs (GET, POST) and a data dictionary of parameters that can be set by the user.

All interaction with the API happens through the use of simple verbs (get, post, delete) which are appended to URLs – the paths that tell you which web page to load.

The API returns information about the status of each request in the ‘response’ object. A response is returned every time the API is called, and this contains both a status code and some text with information about the all.

If everything is working correctly, all your status codes will be of the form 2XX. You may wish to check this from within the program that calls FIA.

Personal tools
Namespaces
Variants
Actions
Manual
File formats
API
Architecture
Additional information
Company
Downloads
Toolbox