Overview
The Analyze section of the Andi Skills Manager provides ways to evaluate the performance and usage of your skills. Understanding how your skills are utilized by your users is key to meeting desired outcomes.
In this Article
Skill Usage
The Skill Usage section shows near real-time skill performance and usage through three metrics:
- Impressions (views)
- Actions (clicks)
- Action Rate (clicks per view)
These metrics paint a picture of how often your skills are seen and used. Actions can be clicks on a link, downloading a file, sending an email, or any other support application action.
Using the top Date Range selector, you can adjust data for the whole page to any dates within the last 90 days.
Skill Highlights
The Skill Highlights section shows the highest and lowest skills regarding impressions and action rate. For the action rate attribution, skills with only at least one action are evaluated as some skills do not provide users with any action. This dashboard provides a quick view of skill performance within the pages’ filtered date range.
Performance
The Performance section renders the three metrics, impressions, actions, and action rate over time by day. The chart can be filtered by skill, audience, and the metrics you want to show. Hover your mouse over the chart dots to view specific numbers.
In the top right, the Export button allows you to do a raw export of the data you are looking at with the currently applied filters. This may be helpful in reporting or using the data in your own analytics’ views.
All Skills
The All Skills section allows for the review of data through table sorting and aggregation. This is helpful to get an idea of how skills roll up by these metrics. You may notice several skills that you do not recognize. These skills do not have an Andi Skills Manager page and are embedded into the application. You will notice that these particular skills do not have a clickable title and the Source will show 'Application'.
Skill Performance
The skill performance dashboard provides context on skill timing and failures. These measures help isolate failure points and understand any hold ups in skill execution.
What's Measured
- Skill Executions – The number of times your skill runs and produces a result or error. This excludes any should I run false calls.
- Skill Failures – The number of times your skill ran and produced an error result.
- Dependency Failures – The number of external calls that your skill makes and fails.
- Skill Response Time – The average time it takes for your skill to respond to an event.
- Should I Run Time – The average time it takes for your “should I run” code to execute.
- Run Time – The average time it takes for your “run” code to execute.
- Dependency Response Time – The average time it takes for your dependencies to complete.
Understanding Timings
The timeline view below demonstrates a measure for a single event going to a single skill. The user interacts in the browser and generates an event. This event takes some time to reach Andi based on any network (internet) latency.
From there, there is a slight delay before the event is passed to a skill which marks the start of the “Skill Response Time.” Measures of Should I Run (SIR) and Run times are strictly bounded around the code calls. Measure of dependencies begin immediately when a power is invoked. The “dep” bars below signify four separate dependency calls. Sometimes these calls happen at close to the same time—represented by a stacking.
Dashboard Sections
Summary
The summary section provides a brief overview of key measures for your skills. Hovering over each measure provides more context about the performance. Green highlights indicate measures that are in a safe range while shades of orange and red may indicate a high degree of failure or performance issues.
Performance
The performance section charts skill performance measures by day. The line color indicates which axis a measure is plotted against. The two supported y-axes are count (frequency) and duration (average time taken in milliseconds). By default, three measures are plotted. More measures are available in the “Show” dropdown. Viewing performance over time helps understand any performance trends and if any fixes helped a particular skill.
All Skills
This section on the skill performance dashboard presents the information in a tabular format for easy review. Selecting the skill failure numbers or dependency error numbers will open a breakdown in a new tab.
Managing Bottlenecks and Errors
With information of how your skills perform in hand, you can begin to take steps to ensure a more efficient and resilient experience for your users.
High Failure Rate or Skill Errors
Begin by reviewing your skills that experience errors the most often. Check the errors on the related skill page or by clicking the skill failure counts in the skills summary table. Common errors that point to something in your code being incorrect are type, syntax, or reference errors. See JavaScript error reference.
Another common error is a skill timeout. Skills have a limited execution window of 15 seconds to complete and return a result. This is done to keep skill results near real-time for users and avoid stale skill results. For tips on avoiding skill timeouts, see tips below on reducing skill response and dependency time. To troubleshoot skill timeouts, view this article here.
High SIR or Run Time and High Dependency Time
Your skill has one or more dependency calls that take a long time. If it’s in your control to fix the performance of the remote endpoint, this is beneficial. If not, ensure you are not calling several dependencies sequentially and try to load things at the same time. To make calls concurrently, see the use of Promise.all().
If all else fails and if it is possible, consider uploading the data you need and loading from skill storage.
High SIR or Run Time and Low Dependency Time
Your skill does a lot of processing or looping while not waiting for any external calls. If there is a need to process a lot of data real-time, it helps to pre-aggregate data to be more readily consumed during skill execution.
High Skill Response Time and Low SIR and Run Time
Your skill is extremely busy by processing many events. Ensure that your skill is setup to respond to only the events that it needs to.
High Dependency Failure Rate
If your remote calls outside of the skill fail often, try to resolve any issues with the remote endpoint. If the remote endpoint has a high failure rate or has specific rate limits, configure retry options in calls made with HttpPowers using SkillHttpRequestOptions.
Chat
The Chat Tab will be available to users who have the 'Access Chat Feed Telemetry' permission.