Running FIA

From Flametree Technologies
Jump to navigation Jump to search

General remarks

FIA's core library can be called in many different ways.

Running FIA from the command line

To run FIA from the command line, enter

fia64 -c config.cnf

and press enter. Click here for information about how to set up a configuration file.

Running from the GUI

Running via the local API

Running via the Web API

Flametree’s Web API allows you to run FIA remotely from any location, using an internet connection.

Providing FIA’s capabilities through a Web API provides the following advantages:

  • No third-party software is installed on your computer network. All you need to run FIA is the ability to put together raw data files and to call the Web API’s commands from your own software.
  • No specialized or high-powered hardware is required. The Web API allows Flametree to be run from any computer with an Internet connection, even a tablet or a low-powered netbook.
  • You always run the most up-to-date version of our software.

To run the examples shown here, you will need a username and a password, available from our Customer Support desk, and a Python development environment.

The API's web address is

The Python API wrapper

The most straightforward way to use the Web API is to use our Python wrapper routines to interact with the program, which provide short-cuts to commonly used functions. For instance, to see a list of your files in the filestore, you can either run

url = base_url + '/file'   
    response = requests.get(url, auth=HTTPBasicAuth(username, password) )    
    print(response.status_code, response.text)
    print('Error found when listing files')


ListFiles(username, password)

Both code fragments return the same information, but the second shields you from the details of calling the API directly.

We assume that the Python wrapper routines are used throughout this document. If you prefer, you can call the API directly using the Python requests (or similar) library from Python, or a similar library from any programming language. Full documentation is provided here.

Architecture and workflow

The filestore

In principle, FIA can be run by uploading all required data, together with values of any configuration settings required, and then running the program. For most users, this approach has several disadvantages:

  • It requires that the user wait while files are uploaded, the calculation runs, and results downloaded. This ties up resources.
  • The same data may have to be uploaded many times – for instance, where the same security master file or benchmark is used for all portfolios.

To avoid these issues, the API is designed so that file operations are distinct from program execution. FIA uses a server-based filestore, to which raw data files are written, and from which results are read. The program is run using the following sequence of commands:

  1. Files required by the program are uploaded to the filestore.
  2. Once all files are in place, the user sends an ‘execute’ command, together with parameter values to describe the type of attribution analysis required.
  3. The API submits this request to a batch queue on the remote server.
  4. The server runs the attribution analysis as soon as resources become available.
  5. When the analysis is complete, results and reports are written to the filestore as a compressed file.
  6. The results file is downloaded.

The batch queue

By using a batch queue for processing, Flametree Attribution allows ‘fire and forget’ processing for large numbers of portfolios.

Each time an attribution analysis is requested, a new batch job is created. This is submitted to the batch queue and is processed when previous jobs are complete. When a batch job is created, it is assigned a unique job ID that can be used to interrogate the batch queue and determine its current status.

The batch queue uses multiple worker processes so that several jobs can be executed at the same time.

A batch processing job has one of the following status values:



Each call to the API requires that the user provide credentials identifying themselves as a registered user of the system. This must be either:

  • A valid user ID and associated password, or
  • A valid security token, generated previously using the GetToken command. The security token contains the user ID, so once the token is obtained the user can run the program anonymously.

No user can access any other user’s data or usage records. Passwords are stored internally in hashed form so that nobody has access to the plaintext versions.

Flametree Attribution automatically encrypts all traffic to and from the site using HTTPS, so that your identity and results are protected.

Calling the API

This example shows the basics of uploading files, calling FIA, and downloading results. We assume that

  • you have been issued a username and password to access the system;
  • you have data files for security master, portfolio weights and returns, and yield curves, called fia_s.csv, fia_p.csv, fia_y.csv respectively;
  • you are using Python, and have a copy of Flametree’s file.

The code shown uses our suite of wrapper functions, which encapsulate the complexities of calling the API directly. If you prefer to call the API directly, please refer to Appendix X.

Import reference to the fia_wapper functions

from fia_wrapper import *	

Assign values for username and password. These are used each time an API function is called:

username = 'username'
password = 'password'

Assign values to be passed to the API to define the attribution. Some fields (PortfolioFile, SecurityFile, YieldCurveFile, ZipFile) must have values assigned; others are optional:

data = {
   "PortfolioFile" : "fia_p.csv",
   "SecurityFile" : "fia_s.csv",
   "YieldCurveFile" : "fia_y.csv",
   "IndexFile"	: "fia_i.csv",
   "batch_id" : "1001",
   "SovereignCurveDecomposition" : "KRD",
   "DateFormat" : "%Y-%b-%d",
   "SummaryAttributionReport" : "true",
   "InteractiveAttributionReport" : "true",
   "SecurityAttributionReport" : "true",
   "CurveReport" : "false",
   "ExPostRiskReport" : "false",
   "CSVreport" : "true",
   "XLSreport" : "true",
   "ZipFile" : ""

Upload required files to the filestore. If a file is already present in the filestore, it need not be uploaded again:

UploadFile (data['PortfolioFile'],    username, password)
UploadFile (data['SecurityFile'],     username, password)
UploadFile (data['YieldCurveFile'],   username, password)
UploadFile (data['IndexFile'],        username, password)

(Optional) Show the names of all files present in the filestore:

ListFiles(username, password)	

Submit this job to the batch queue. The job is assigned a unique job ID. In this instance, we are only submitting one job, but you can submit as many as you like at the same time:

job_id = RunJson (data, username, password)

Then, repeatedly interrogate the batch queue to find this job’s status. When it has run, its status is ‘COMPLETE’:

while True:   
   if IsRunning (job_id, username, password) == 'COMPLETE': 

Now that the job has run, download the results in data[‘ZipFile’]:

DownloadFile(data['ZipFile'], username, password)

Uploading files

To upload a file, call the following statements in Python, which issue a ‘post’ command to the API and some diagnostic data:

UploadFile(filename, username, password)

If all has gone well, this will display the message

{200, "message": "File uploaded OK"}

If there is already a file with this name in the filestore, Python will display

{403, "message": "Duplicate file name"}

If you want to replace a file with another that has the same name, delete the older copy from the filestore by using the commands

DeleteFile(filename, username, password)

If the file was deleted successfully, you will see the message

{200, "message": "File ‘fia_s.csv’ deleted"}

You can now upload a new copy of the file.

To see a list of all files in the filestore, issue the commands

ListFiles (username, password)

The names of all the files present in this user's account will be returned as a list in the ‘response.text’ field.

The same file can be called from multiple runs. For instance, a common security master, or benchmark, can be referenced during multiple runs. In this case there is no need to upload the file repeatedly.

Uploaded files are subject to the following restrictions:

  • File name must be valid: at least N characters, ‘.’ in name, must have csv format
  • The API is the only way to access the filestore. The filestore is not accessible by FTP.
  • File sizes are restricted to a maximum of 100 MB
  • Limit of 1 GB storage per user. Please contact us if you need more remote storage.

Running the program

Once you have uploaded all the files required by FIA, the next step is to set the program’s parameters and run the program. This is done by setting up a JSON structure and initializing its values. If you have not used JSON before, please refer to the examples below.

FIA has a long list of parameters that can be set to tailor your analysis. A few must be set, but most are optional and have sensible defaults. For a full list of the parameters and their defaults, refer to Appendix A.

Required fields

Values for PortfolioFile, SecurityFile, YieldCurveFile and ZipFile must be set each time the API is called. A 404 error will be generated if any of these quantities has not been set.

Parameter lists

FIA is run using the execute command. This function requires a JSON structure to be passed as an argument.

In this example, we set up a JSON structure directly. However, many languages allow conversion from other data structures to JSON. For instance, the Python json.dumps function converts a Python dictionary to JSON.

The parameters shown here are represented in JSON as

data = {
"PortfolioFile"	: "fia_p.csv",
"SecurityFile" 	: "fia_s.csv",
"YieldCurveFile": "fia_y.csv",
"SovereignCurveDecomposition": "STB",
"xls_reports": True,
"batch_id": 100,
"ZipFile" : ""

The program is run by issuing the command

job_id = execute(data, username, password)

When the user issues this command, two things happen:

  • the job is placed into the API’s batch queue for processing. The queue contains the IDs of all jobs that have not yet started execution. If there are no other pending jobs, a job will immediately be executed, in which case it will not appear in the queue at all.
  • a unique ID code is returned that uniquely labels each job.

Note that you can submit multiple jobs to the queue at the same time; there is no need to wait for one job to complete before you submit another. For instance, if you have a large number of portfolios to process, you can submit them all to the batch queue at the same time and download the results when they are ready.

Interrogating the batch queue

Once you have a job’s ID code, the Status command allows you to interrogate the batch queue and return the current status of that job.

For instance, to run a single job you can use the following commands:

id = RunJson (data, username, password)
while True:
    if Status (id, username, password) == 'COMPLETE': 

This code fragment submits a job to the queue, records the job ID, and repeatedly polls the task queue until the job is complete. At this point the results are available and can be downloaded from the filestore.

You can also see if the batch queue is empty, or whether your report file has been written to the filestore. In either case, your job has run.

Wrapper functions

General user commands

These commands can be run by any user. If a login token is used, this may be substituted for the username, and the password field left blank.

Command Description Arguments Returns
LatencyTest Ping the Flametree server. This command can be used to check that you have an active Internet connection, and to monitor traffic speeds. (none) Time taken to get a response from the Flametree server
ListFiles Generate list of all files in user’s account (username, password) Python list of file names
UploadFile Upload single file to user’s account. If the file already exists, an error message is returned. (filename, username, password)

'filename' can include path on host system.

Boolean, indicating whether file was uploaded successfully
DeleteFile Delete single file from user’s account (filename, username, password) Boolean, indicating whether file was deleted successfully
DeleteAllFiles Delete all file from user’s account (username, password) Boolean, indicating whether all files were deleted successfully
DownloadFile Download single file from user’s account (filename, username, password) Boolean, indicating whether file was downloaded successfully
Execute Submits job with parameters supplied in JSON structure ‘settings’ to batch queue (settings, username, password) If successful, returns UUID string job id. This can be used to interrogate the batch queue via the Status command.
GetQueue Shows list of all jobs submitted that have status PENDING (not yet started). Note that this does not include jobs that are currently being processed, so a queue with zero length does not necessarily mean that your job has completed. (username, password) List of job IDS
Status Interrogate queue to find status of job with given ID. (job_id, username, password) PENDING, RUNNING, COMPLETE, FAILED, NOT_FOUND
GetToken Returns security token that can be used in place of user name and password (username, password) String token that can be used in place of a username and password.

Administrator commands

These commands can only be run by the system’s administrator.

Command Description Arguments Returns
CreateUser Create new user account, using supplied name and password. (new_user_name, new_password, admin_password) Boolean, indicating whether user account was created successfully
DeleteUser Delete user account and associated filestore. (user_name, admin_password) Boolean, indicating whether user account was deleted successfully
ListUsers Generate list of all users. (admin_password) Python list of all user accounts
UsageReport Generates raw system usage report for all users (user ID, date and time of call, CPU usage, IP address of caller). (admin_password) JSON structure, containing all usage data

API endpoints

All URLs are prefixed with

URL Method Parameters Data parameters Success response Error response Usage and sample call
/user POST username, password, admin_password JSON structure containing username, password, admin_password 201 AUTHORIZED 403 UNAUTHORIZED PASSWORD Create a new user with supplied username and password
url = ''
data = {}
data['username'] = username
data['password'] = password
data['master_password'] = admin_password
json_data = json.dumps(data)
response =, data=json_data, headers=headers) 
/user DELETE username, admin JSON structure containing username, admin_password 200 AUTHORIZED 403 UNAUTHORIZED PASSWORD


Delete existing user with given username
url = ''
data = {}
data['username'] = username
data['master_password'] = admin_password
json_data = json.dumps(data)
response = requests.delete(url, data=json_data, headers=headers)   

API status codes

Flametree Attribution uses REST (Representational State Transfer), an API interface protocol that is rapidly becoming the standard for communication between software components. Commands are issued using a simple set of verbs (GET, POST, DELETE) appended to the site's URL, and a data dictionary of parameters that can be set by the user.

The API returns information about the status of each request in the ‘response’ object. A response is returned every time the API is called, and this contains both a status code and some text with information about the all.

  • A status code of 200 means everything worked OK.
  • A status code of 201 means something was created successfully.
  • A status code of 402 means a request was unauthorized – for instance, a wrong password was used.
  • A value of 403 means a request was forbidden. For instance, the API does not let you overwrite a file with another that has the same name. In this case a 403 error is appropriate.
  • A value of 404 means something was not found. This error can be issued if one of the files FIA needs to run is not present.
  • A value of the form 5XX means that the server is not running.

If everything is working correctly, all your status codes will be of the form 2XX. You may wish to check this from within the program that calls FIA.

As far as possible, the API follows accepted industry standards for the use of response codes. For more details, see


The type of each field is indicated in its name.

Each label refers to an internal integer label.

User-supplied and program generated data

Field Description Value
FX_MATRIX_PORTFOLIO Name of user-supplied portfolio weights and returns file, including file path if required 0
FX_MATRIX_BENCHMARK Name of user-supplied benchmark weights and returns file, including file path if required 1
FX_MATRIX_SECURITY Name of user-supplied security definitions file, including file path if required 2
FX_MATRIX_YIELDCURVE Name of user-supplied yield curve file, including file path if required 3
FX_MATRIX_STRESS Name of user-supplied stress test scenario file, including file path if required 4
FX_MATRIX_MAP Name of user-supplied security ID map file, including file path if required 5
FX_MATRIX_FX Name of user-supplied FX rate file, including file path if required 6
FX_MATRIX_INDEX Name of user-supplied index data, including file path if required 7
FX_MATRIX_RESULTS Name of attribution results data 8
FX_MATRIX_RISKS Name of risk results data 9
FX_MATRIX_CURVES Name of curve results data 10
FX_MATRIX_DIAGNOSTIC Name of diagnostic data 11


All data is supplied as two-dimensional matrices of type STRING_MATRIX. Since individual entries have type string, numerical values should be converted to doubles before being used.

File format data

Field Description Value
FT_BOOL_FILE_HEADERS Indicates whether user-supplied data has column header data 100
FT_STRING_DATE_FORMAT Default format for user-supplied dates. This format is applied to all dates unless a different format is supplied using one of the other settings in this section 101
FT_STRING_SECURITY_DATE_FORMAT Format for dates in security master data 102
FT_STRING_PORTFOLIO_DATE_FORMAT Format for dates in portfolio data 103
FT_STRING_BENCHMARK_DATE_FORMAT Format for dates in benchmark data 104
FT_STRING_YIELDCURVE_DATE_FORMAT Format for dates in benchmark data 105
FT_STRING_FX_DATE_FORMAT File format for dates in benchmark data. 106
FT_STRING_INDEX_DATE_FORMAT File format for dates in benchmark data. 107


Data structures

Data parameters used by the API are of the following standard C++ types:

  • integer
  • double
  • long
  • boolean
  • string
  • vector<string>

where STRING_MATRIX is defined by

typedef vector <vector<string> > STRING_MATRIX;