Web API

From Flametree Technologies
(Difference between revisions)
Jump to: navigation, search
(API endpoints)
(API endpoints)
Line 320: Line 320:
 
|JSON structure, containing all usage data
 
|JSON structure, containing all usage data
 
|}
 
|}
 +
 +
<!-- This is a comment
  
 
==API endpoints==
 
==API endpoints==
Line 379: Line 381:
  
 
If everything is working correctly, all your status codes will be of the form 2XX. You may wish to check this from within the program that calls FIA.
 
If everything is working correctly, all your status codes will be of the form 2XX. You may wish to check this from within the program that calls FIA.
 +
-->

Revision as of 11:53, 15 August 2019

Contents

Why a Web API?

Flametree’s Web API allows you to run FIA remotely from any location, using an internet connection.

Providing FIA’s capabilities through a Web API provides the following advantages:

To run the examples shown here, you will need a username and a password, available from our Customer Support desk, and a Python development environment.

The API's web address is

www.flametreeapi.com

The Python API wrapper

The most straightforward way to use the Web API is to use our Python wrapper routines to interact with the program, which provide short-cuts to commonly used functions. For instance, to see a list of your files in the filestore, you can either run

ListFiles(username, password)

or

url = base_url + '/file'   
try:          
    response = requests.get(url, auth=HTTPBasicAuth(username, password) )    
    print(response.status_code, response.text)
except:
    print('Error found when listing files')
    sys.exit(1)

Both code fragments return the same information, but the first shields you from the details of calling the API directly.

We assume that the Python wrapper routines are used throughout this document. If you prefer, you can call the API directly using the Python requests (or similar) library from any programming language. Full documentation is provided here.

Architecture and workflow

The filestore

In principle, FIA can be run by uploading all required data, together with values of any configuration settings required, and then running the program. For most users, this approach has several disadvantages:

To avoid these issues, the API is designed so that file operations are distinct from program execution. FIA uses a server-based filestore, to which raw data files are written, and from which results are read. The program is run using the following sequence of commands:

  1. Files required by the program are uploaded to the filestore.
  2. Once all files are in place, the user sends an ‘execute’ command, together with parameter values to describe the type of attribution analysis required.
  3. The API submits this request to a batch queue on the remote server.
  4. The server runs the attribution analysis as soon as resources become available.
  5. When the analysis is complete, results and reports are written to the filestore as a compressed file.
  6. The results file is downloaded.

The batch queue

By using a batch queue for processing, Flametree Attribution allows ‘fire and forget’ processing for large numbers of portfolios.

Each time an attribution analysis is requested, a new batch job is created. This is submitted to the batch queue and is processed when previous jobs are complete. When a batch job is created, it is assigned a unique job ID that can be used to interrogate the batch queue and determine its current status.

The batch queue uses multiple worker processes so that several jobs can be executed at the same time.

A batch processing job has one of the following status values:

Security

Each call to the API requires that the user provide credentials identifying themselves as a registered user of the system. This must be either:

No user can access any other user’s data or usage records. Passwords are stored internally in hashed form so that nobody has access to the plaintext versions.

Flametree Attribution automatically encrypts all traffic to and from the site using HTTPS, so that your identity and results are protected.

Calling the API

This example shows the basics of uploading files, calling FIA, and downloading results. We assume that

The code shown uses our suite of wrapper functions, which encapsulate the complexities of calling the API directly. If you prefer to call the API directly, please refer to Appendix X.

Import reference to the fia_wapper functions

from fia_wrapper import *	

Assign values for username and password. These are used each time an API function is called:

username = 'username'
password = 'password'

Assign values to be passed to the API to define the attribution. Some fields (PortfolioFile, SecurityFile, YieldCurveFile, ZipFile) must have values assigned; others are optional:

data = {
   "PortfolioFile" : "fia_p.csv",
   "SecurityFile" : "fia_s.csv",
   "YieldCurveFile" : "fia_y.csv",
   "IndexFile"	: "fia_i.csv",
   "batch_id" : "1001",
   "SovereignCurveDecomposition" : "KRD",
   "DateFormat" : "%Y-%b-%d",
   "SummaryAttributionReport" : "true",
   "InteractiveAttributionReport" : "true",
   "SecurityAttributionReport" : "true",
   "CurveReport" : "false",
   "ExPostRiskReport" : "false",
   "CSVreport" : "true",
   "XLSreport" : "true",
   "ZipFile" : "results.zip"
}	

Upload required files to the filestore. If a file is already present in the filestore, it need not be uploaded again:

UploadFile (data['PortfolioFile'],    username, password)
UploadFile (data['SecurityFile'],     username, password)
UploadFile (data['YieldCurveFile'],   username, password)
UploadFile (data['IndexFile'],        username, password)

(Optional) Show the names of all files present in the filestore:

ListFiles(username, password)	

Submit this job to the batch queue. The job is assigned a unique job ID. In this instance, we are only submitting one job, but you can submit as many as you like at the same time:

job_id = RunJson (data, username, password)

Then, repeatedly interrogate the batch queue to find this job’s status. When it has run, its status is ‘COMPLETE’:

while True:   
   if IsRunning (job_id, username, password) == 'COMPLETE': 
       break 

Now that the job has run, download the results in data[‘ZipFile’]:

DownloadFile(data['ZipFile'], username, password)

Uploading files

To upload a file, call the following statements in Python, which issue a ‘post’ command to the API and some diagnostic data:

UploadFile(filename, username, password)

If all has gone well, this will display the message

{200, "message": "File uploaded OK"}

If there is already a file with this name in the filestore, Python will display

{403, "message": "Duplicate file name"}

If you want to replace a file with another that has the same name, delete the older copy from the filestore by using the commands

DeleteFile(filename, username, password)

If the file was deleted successfully, you will see the message

{200, "message": "File ‘fia_s.csv’ deleted"}

You can now upload a new copy of the file.

To see a list of all files in the filestore, issue the commands

ListFiles (username, password)

The names of all the files present in this user's account will be returned as a list in the ‘response.text’ field.

The same file can be called from multiple runs. For instance, a common security master, or benchmark, can be referenced during multiple runs. In this case there is no need to upload the file repeatedly.

Uploaded files are subject to the following restrictions:

Running the program

Once you have uploaded all the files required by FIA, the next step is to set the program’s parameters and run the program. This is done by setting up a JSON structure and initializing its values. If you have not used JSON before, please refer to the examples below.

FIA has a long list of parameters that can be set to tailor your analysis. A few must be set, but most are optional and have sensible defaults. For a full list of the parameters and their defaults, refer to Appendix A.

Required fields

Values for PortfolioFile, SecurityFile, YieldCurveFile and ZipFile must be set each time the API is called. A 404 error will be generated if any of these quantities has not been set.

Parameter lists

FIA is run using the execute command. This function requires a JSON structure to be passed as an argument.

In this example, we set up a JSON structure directly. However, many languages allow conversion from other data structures to JSON. For instance, the Python json.dumps function converts a Python dictionary to JSON.

The parameters shown here are represented in JSON as

data = {
"PortfolioFile"	: "fia_p.csv",
"SecurityFile" 	: "fia_s.csv",
"YieldCurveFile": "fia_y.csv",
"SovereignCurveDecomposition": "STB",
"xls_reports": True,
"batch_id": 100,
"ZipFile" : "DemoZipFile.zip"
}

The program is run by issuing the command

job_id = execute(data, username, password)

When the user issues this command, two things happen:

Note that you can submit multiple jobs to the queue at the same time; there is no need to wait for one job to complete before you submit another. For instance, if you have a large number of portfolios to process, you can submit them all to the batch queue at the same time and download the results when they are ready.

Interrogating the batch queue

Once you have a job’s ID code, the Status command allows you to interrogate the batch queue and return the current status of that job.

For instance, to run a single job you can use the following commands:

id = RunJson (data, username, password)
while True:
    if Status (id, username, password) == 'COMPLETE': 
        break

This code fragment submits a job to the queue, records the job ID, and repeatedly polls the task queue until the job is complete. At this point the results are available and can be downloaded from the filestore.

You can also see if the batch queue is empty, or whether your report file has been written to the filestore. In either case, your job has run.

Wrapper functions

General user commands

These commands can be run by any user. If a login token is used, this may be substituted for the username, and the password field left blank.

Command Description Arguments Returns
LatencyTest Ping the Flametree server. This command can be used to check that you have an active Internet connection, and to monitor traffic speeds. (none) Time taken to get a response from the Flametree server
ListFiles Generate list of all files in user’s account (username, password) Python list of file names
UploadFile Upload single file to user’s account. If the file already exists, an error message is returned. (filename, username, password)

'filename' can include path on host system.

Boolean, indicating whether file was uploaded successfully
DeleteFile Delete single file from user’s account (filename, username, password) Boolean, indicating whether file was deleted successfully
DeleteAllFiles Delete all file from user’s account (username, password) Boolean, indicating whether all files were deleted successfully
DownloadFile Download single file from user’s account (filename, username, password) Boolean, indicating whether file was downloaded successfully
Execute Submits job with parameters supplied in JSON structure ‘settings’ to batch queue (settings, username, password) If successful, returns UUID string job id. This can be used to interrogate the batch queue via the Status command.
GetQueue Shows list of all jobs submitted that have status PENDING (not yet started). Note that this does not include jobs that are currently being processed, so a queue with zero length does not necessarily mean that your job has completed. (username, password) List of job IDS
Status Interrogate queue to find status of job with given ID. (job_id, username, password) PENDING, RUNNING, COMPLETE, FAILED, NOT_FOUND
GetToken Returns security token that can be used in place of user name and password (username, password) String token that can be used in place of a username and password.

Administrator commands

These commands can only be run by the system’s administrator.

Command Description Arguments Returns
CreateUser Create new user account, using supplied name and password. (new_user_name, new_password, admin_password) Boolean, indicating whether user account was created successfully
DeleteUser Delete user account and associated filestore. (user_name, admin_password) Boolean, indicating whether user account was deleted successfully
ListUsers Generate list of all users. (admin_password) Python list of all user accounts
UsageReport Generates raw system usage report for all users (user ID, date and time of call, CPU usage, IP address of caller). (admin_password) JSON structure, containing all usage data


Personal tools
Namespaces
Variants
Actions
Manual
File formats
Local API
API
Architecture
Additional information
Company
Downloads
Toolbox