Overview

This article will cover how to configure the Data Sets skill setting when writing your skills in the IDE. 

This setting is not available for all clients. If you'd like to use this setting or learn more about it, please contact us at support@andi.com.

 

In this Article

 

What is the Data Sets skill setting?

Data Sets is a type of skill setting that allows skill managers the ability to upload large data files to a temporary data storage area, which will allow for faster querying when running a semi-static skill. This is achieved by using the dataSets property.

 

When to use Data Sets

Data sets should be used when you need a structured, indexed, queryable set of data to help deliver insights to your team.

Data sets are currently limited to 2 MB of data per row and a total data set size of 100 MB.

For example, you may want to notify lenders who are pricing an opportunity for a product associated with a particular industry of the total value of similar loans you currently have in your portfolio based on data compiled by your data analyst team. Because this data is stored in an Excel file on your Intranet, you’ll need a way to pull in and query this information to provide this insight to your lenders. Utilizing Data Sets when writing your skill would allow you the ability to upload and query this large data file to return a field tag message to your lenders.

 

How to define Data Sets

You will define Data Set properties in the skillmanifest.json file and write the query in the run.ts file in the IDE. It is very important that not only the data fields be defined correctly in the IDE, but that they also match the format of the file contents itself. If the file’s column and/or data types differ from how they’re ordered and entered in the code, then the file upload in the Skills Manager will be unsuccessful.

In the skillmanifest.json

For any Data Set, you will need to define the following:

  • Name of the Data Set
  • A description of the Data Set – not required but helpful if the filename isn’t clear
  • The data fields
    • Column Name
    • Type of data contained in the column (string, number, or Boolean)
    • A description of the data in the column

Once the data set has been defined and the skill deployed, any changes to the schema definition made in the skillmanifest will need a different data set schema name. For instance, in the example below for IndustryDataV3, to change the Region to a dataType of number, you would also need to change the IndustryDataV3 name to IndustryDataV3.1 for the change to take affect.

This will look like the following:

"description": "This is a description of your data set skill",
    "skillConfigKeys": [],
    "dataSets": [
        {
            "name": "DataSetNameHere",
            "description": "This file contains data on this topic",
            "fields": [
                {
                    "name": "Column A",
                    "dataType": "string",
                    "description": "This is what is in column A"
                },
                {
                    "name": "Column B",
                    "dataType": "number",
                    "description": "This is what is in Column B"
                }
            ]
        }
    ]    

Using our earlier example, we want to write a skill that delivers insights to lenders pricing construction loans. We need to define our Industry Data file for the 5 columns of data it contains. The skillManifest will look like this:

"description": "This skill will create a field tag letting lenders know the total value of loans in our portfolio at or above the commitment amount of the loan they are pricing if it is a loan in the construction industry.”
    "skillConfigKeys": [],
    "dataSets": [
        {
            "name": "IndustryDataV3",
            "description": "",
            "fields": [
                {
                    "name": "Industry",
                    "dataType": "string",
                    "description": "Industry name"
                },
                {
                    "name": "Region",
                    "dataType": "string",
                    "description": "Region for the industry"
                },
                {
                    "name": "Loan Amount",
                    "dataType": "number",
                    "description": "Loan amount"
                },
                {
                    "name": "Product",
                    "dataType": "string",
                    "description": "Product name"
                },
                {
                    "name": "Amount",
                    "dataType": "number",
                    "description": "Amount of product"
                }
            ]
        }
    ]    
}

In the shouldIRun.ts file

You will then define the parameters for which your skill should run in the shouldIRun file. For our example, we want the skill to run only if there is a construction loan in the opportunity.

export function shouldIRun(skillContext: andiSkills.ISkillContext): Promise | boolean {
    
    let isLoanAccountChangeEvent = skillContext.powers.precisionLender.opportunity.isLoanAccountChangeEvent();
    if(!isLoanAccountChangeEvent) return  skillContext.powers.andi.shouldIRun.shouldIRunFalse();

    let loanAccounts = skillContext.powers.precisionLender.opportunity.getLoanAccounts();
    let hasLoans = loanAccounts.length > 0;
    
    // if the opportunity doesn't have loans, don't run
    if(!hasLoans)
        return skillContext.powers.andi.shouldIRun.shouldIRunFalse(); 
    
    let eventData = skillContext.powers.precisionLender.opportunity.getOpportunityChangeEventData() as andiExternal.PrecisionLenderOpportunityChangeEvent.OpportunityChangeEvent;

    let currentContext = skillContext.event.currentContext;

               
    let currentLoan = eventData.engineModel.commercialLoanAccounts.find((loan) => {
            return loan.id == currentContext;
    });
    
        // are there any loans?
    if (!currentLoan) {
        return skillContext.powers.andi.shouldIRun.shouldIRunFalse();
    }

    let loanProduct: any = currentLoan.product;
    // check if the product is the product of interested
    let product: string = loanProduct.name.trim();
    
    let productOfInterest: string = "construction";
    
    if (product.toLowerCase().indexOf(productOfInterest) < 0) {
         return skillContext.powers.andi.shouldIRun.shouldIRunFalse();
    }
    
    return skillContext.powers.andi.shouldIRun.shouldIRunTrue();
}

In the run.ts file

In the run.ts file, you will need to define an interface for the dataset that will define the properties of the dataset used when writing your query. If a property name has a space, for example more than one word, it will need to be entered with quotes. The properties defined in the interface need to match the name and dataType entered in the skillmanifest.

interface ObjectName {
    Name: string,
    Name2: number,
    "Two WordName": number,
}

Returning to our example, the interface will be defined as the following:

interface IndustryData {
    Industry: string,
    Region: string,
    "Loan Amount": number,
    Product: string,
    Amount: number,
}

You will also need to call the Data Set client using the name you defined in the skillmanifest before writing your query. This will look like the following:

const dataSetClient = skillContext.powers.andi.dataSets.getDataSet("DataSetNameHere");

After you call the Data Set client, you will then use the dataSetClient to write your query. Your query must always contain a ‘where’ clause, but ‘parameters’ are optional. Any parameters entered in your ‘where’ clause will need to be prefixed with ‘@’:

dataSetClient.query({
        where: 'row.fromInterface = @parameter and row.fromInterface = @parameter',
        parameters: [
            { name: "fromInterface", value: "maybeItIsAString" },
            { name: " fromInterface ", value: orVariable }
        ]

Continuing with our example, our run.ts file will be written as follows to generate a field tag that displays a message ("There are X loans in the Construction industry totaling $XX,XXX.") to the lender when pricing a loan in the construction industry:

export function run(skillContext: andiSkills.ISkillContext): Promise | andiSkills.ISkillActivity {

    let eventData = skillContext.powers.precisionLender.opportunity.getOpportunityChangeEventData() as andiExternal.PrecisionLenderOpportunityChangeEvent.OpportunityChangeEvent;

    let currentContext = skillContext.event.currentContext;
    
    let currentLoan = eventData.engineModel.commercialLoanAccounts.find((loan) => {
        return loan.id === currentContext;
    });

    const loanAmount = currentLoan.amount;

    // Get the DataSet client by supplying a DataSet name defined in the skillmanifest.json
    const dataSetClient = skillContext.powers.andi.dataSets.getDataSet("IndustryDataV3");

    // Use the dataSetClient to query the DataSet, supplying the 'where' and 'parameters' (parameters are optional)
    // Parameters within the 'where clause' should be prefixed with '@'
    return dataSetClient.query({
        where: 'row.Industry = @industry and row["Loan Amount"] > @loanAmount',
        parameters: [
            { name: "industry", value: "Construction" },
            { name: "loanAmount", value: loanAmount }
        ]
    }).then((results: IndustryData[]) => {
        var message = "No results found";
        if (results.length > 0) {
            const totalValue = results.map(r => r["Loan Amount"]).reduce(getSum);
            message = `There are ${results.length} loans in the Construction industry totaling $${totalValue}`;
        }

        return createFieldTag(message, skillContext);
    });
}

function getSum(acc, num) {
    return acc + num;
}

function createFieldTag(message: string, skillContext: andiSkills.ISkillContext) {
    // send field tag using powers
    // consult powers documentation for more details
    return skillContext.powers.andi.fieldTag.sendFieldTag(
        andiSkills.FieldTagTypes.Info,
        // this is just a key to uniquely identify your fieldtag
        "opportunity-change-loan-dataset-skill",
        1,
        message,
        [],
        null,
        function (api, model, comparehash, custom) { 
            let shouldIShow = custom.message !== "No results found";           
            return shouldIShow;
        },
        null,
        null,
        { message: message}
    );
}


Once the skill has been deployed, your skill manger(s) will be able to upload the appropriate data file(s).

Was this article helpful?
0 out of 0 found this helpful