Home > Technical Reference > Workflow Automation > Daily Process

Daily Process

Introduction

The end-to-end process for a Vector Risk workflow involves the following steps:

Loading data

Use the ashx web service (called DailyProcess.ashx) to load data. The load service can be used to populate “Load” type tasks in the workflow (eg trade, rate, counterparty data), or static configuration data (such as the businessunits configuration which contains books, desks and hierarchies).

The ashx call has four input parameters: q, e, d, and datafile. When calling the service with curl, each of these inputs will be preceeded by a -F flag.


Input Parameter Details
q=load
Setting the “q” flag (short for query) to “load” tells DailyProcess.ashx that we are loading data.
e=org.env.level The “e” flag specifies the exact workflow environment. An environment has three components. For example “ABCBank.xVA.Production” means:
Organisation=ABCBank
Environment=xVA
Level=Production
d=date  (yyyy-MM-dd) This is the calculation date for the workflow. An environment contains workflows for multiple dates and this flag selects the correct one.

Note: This flag is optional for Vector Risk load files because the system will extract the date from the second row in the load file if it is not supplied.
datafile=@name This parameter is a string containing the load data. The @ in front of it tells the system that it is actually a file being sent. The system will save the file into a temporary location on the server.

Typically the csv format is the easiest to map to. The system will convert the file from csv into Vector Risk objects on loading into the system. Once loaded, the data can be viewed in the GUI in an xml tree format, and can be exported by the user in either xml or csv format.

Note:
When loading csv data the command includes the parameter:  type=text/csv.
If loading xml data, the command would instead use:  type=text/xml.


Monitoring progress

Use the asmx web service “GetStatus” in DailyProcess.asmx to monitor progress of tasks in a workflow. The primary goal is to be alerted when reports have been published so we can retrieve them, but the service also allows the external caller to monitor progress and capture any errors. This visibility allows for prompt error resolution.

If a process needs to be re-run the external caller only needs to re-submit the data. The system will reset all of the calculations and publish tasks to their original state “Pending”, allowing the external caller (or script) to monitor the tasks as before.

For the GetStatus call the specifics of the environment are included in the body of the soap envelope. The result object contains a list of the current status (and other details) of all of the tasks in the workflow. The user only need cycle through the list and either display or act on the status of particular tasks.


Input Parameter Details
Soap Header The body of the soap envelope contains a GetStatus object with three input parameters: environment, date and statusToGet.

Eg: In the example below:
environment should be configured as org.env.level
date should have the format yyyy-MM-dd.
statusToGet = all

<soapenv:Envelope
xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/' xmlns:web='http://boundaryrider.com.au/webservices/'>
   <soapenv:Header/>
   <soapenv:Body>
      <web:GetStatus>
         <web:environmentName>$environment</web:environmentName>
         <web:environmentDate>$date</web:environmentDate>
         <web:statusToGet>all</web:statusToGet>
      </web:GetStatus>
   </soapenv:Body>
</soapenv:Envelope>

Soap call The soap call specifies:
soap_url
userpass (username and password separated by “:”)
Content-Type = text/xml
charset = utf-8
soapaction (GetStatus) 
soapheader (above)

Curl example:
curl.exe -s `
"https://workflow.ch.vectorrisk.com/DailyProcess.asmx" `
 -u $userpass `
 -H 'Content-Type: text/xml; charset=utf-8' 
 -H 'SOAPAction: http://boundaryrider.com.au/webservices/GetStatus' `
 -d $soapheader 

Response object The response is a list of objects (one per task) containing details of each task in the workflow. We are generally most interested in the “Status” field.

The Status may be one of the following:
Pending
InProgress
Success
Failure
Locked

A typical use case would be to poll the web service in a loop (sleeping for some seconds between each poll) until a particular task (most often a publish task) is successful. This loop can also check the status of other tasks and create an alert if any of the tasks have a status of “Failure” as this means something has gone wrong with the run and intervention (or re-run) is required.

Note: When a re-run is initiated by reloading data to the workflow, all of the calculation and publish tasks statuses are reset to “Pending” so the GetStatus loop can start polling all over again.


Retrieving Reports

The system will generate reports that reflect the type of calculations that have been performed. In the workflow, each task for which reports are required is included in the dependency list for the “Publish” task.
Reports are available to view in the GUI and export in various formats (csv, pdf etc). Reports are also saved in csv format to a secure location in Azure blob storage.

Access these reports in two steps:
  1. Use the asmx web service “GetZipReportUrl” to get access to the reports. This web service returns a temporary url for retrieving the reports (this URL times out after 5 minutes and is obfuscated by a GUID in the address).
  2. Pull down the file behind this URL. The file is a zip file containing all of the csv reports created for the environment in question. Unpack the zip file in a local directory to access the reports.

Input Parameter Details
Soap Header
The body of the soap envelope contains a GetZipReportUrl object with three input parameters: environment, date and taskName.

Eg: In the example below:
environment should be configured as org.env.level
date should have the format yyyy-MM-dd.
taskName = Publish


<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/' xmlns:web='http://boundaryrider.com.au/webservices/'>
   <soapenv:Header/>
   <soapenv:Body>
      <web:GetZipReportUrl>
         <web:environmentName>$environment</web:environmentName>
         <web:environmentDate>$date</web:environmentDate>
         <web:taskName>Publish</web:taskName>
      </web:GetZipReportUrl>
   </soapenv:Body>
</soapenv:Envelope> 

Soap call The soap call specifies:
soap_url
userpass (username and password separated by “:”)
Content-Type = text/xml
charset = utf-8
soapaction (GetZipReportUrl) 
soapheader (above)

Curl example:
curl.exe -s `
"https://workflow.ch.vectorrisk.com/DailyProcess.asmx" `
 -u $userpass `
 -H 'Content-Type: text/xml; charset=utf-8' 
 -H 'SOAPAction: 
     http://boundaryrider.com.au/webservices/GetZipReportUrl ' `
 -d $soapheader

Response object The response is a string containing the URL giving temporary access to the zip file on Azure blob storage containing the reports.


An example powershell command for getting a file from a URL and placing it in the current directory is:
Invoke-WebRequest -Uri $reportUrl -OutFile .\report.zip 

Note: The caller is able to choose the name of the file (in this example they chose the name "report.zip").


Controlling Tasks

The workflow is usually configured so that tasks will run when dependencies are met. This process can be controlled from outside the GUI via “Post” calls to the asmx web service. 

The post call specifies:
soap_url
userpass (username and password separated by “:”)
Content-Type = application/x-www-form-urlencoded
endpoint =  soap_url/Command
a string containing the list of parameter=value, separated by &

Return code should be 200 (successful).

The most useful commands and curl script examples are shown below:

Function HTTP Post Command
Create a daily folder
This command is normally not necessary, as simply calling a load task against a new date will trigger the system to generate an environment for that date.

curl.exe -s -w "%{http_code}" -u $userpass 
 -d "environmentDate=$date&environmentName=$environment"
 -H "Content-Type: application/x-www-form-urlencoded"
 -X POST $soap_url/CheckAndCreateDailyFolder 

Set task status
This command would be used to set a task to “Locked” so it will not run, or “Pending” which makes the task respond to a command to run it. It can also be used to set load tasks to pending so a user can make a fresh start on an environment.

curl.exe -s -w "%{http_code}" -u $userpass 
 -d "environmentDate=$date&environmentName=$environment
    &taskName=LoadTrades&taskStatus=Pending&optionalLogMessage=" 
 -H "Content-Type: application/x-www-form-urlencoded" 
 -X POST $soap_url/SetTaskStatus 

Execute task
This command can be used to trigger a re-run of a task that is in a “Pending”, “Success”, “Failure”, or “InProgress” state. Note: this command will not trigger a re-run of a task in a “Locked” state.

curl.exe -s -w "%{http_code}" -u $userpass 
 -d "environmentDate=$date&environmentName=$environment
     &name=ReconcileTrades" 
 -H "Content-Type: application/x-www-form-urlencoded" 
 -X POST $soap_url/ExecuteTask 






See also
PreDeal Check