- What are topics?
- When do topics populate?
- Highlighted topics
- Topic extract
- Topic themes
- Multiple language topics
- Viewing a particular topic
- Removing topics
- The Peakon topic algorithm
1. What are topics?
Generally, the amount of written comments can vary through different levels of an organisational hierarchy, and it may not be feasible to capture all feedback one by one. Therefore, senior leaders may be looking to discover the general themes and issues that people care about. Topics are a way of automatically summarising a large number of comments based on a common theme.
You can review topics by driver, open-ended questions and values.
2. When do topics populate?
Once the survey round closes, at 1:00 AM UTC on the following day, your dashboard will calculate its comment topics. Topics are typically calculated if 200 or more comments are associated with each driver. The topic algorithm only includes the latest comments over the last three months.
It’s therefore unlikely that managers further down the hierarchy will have topics populate, when you consider the survey frequency and the manager’s team size, in comparison to managers further up the hierarchy, who have access to more survey results and comments.
3. Highlighted topics
Peakon will provide highlighted topics to help you understand where to focus your attention in relation to the qualitative data on your dashboard. There are five categories of highlighted topics:
- High Scoring - The topic with the highest average score. This informs managers of the aspects of work employees are most satisfied with.
- Low Scoring - The topic with the lowest average score. This informs managers of the aspects of work employees are least satisfied with.
- Most Comments - The topic made up of the most amount of comments. This informs managers of the themes that are on employees’ minds or of the most used words/language within employee comments.
- Consistent Comments - The topic made up of comments repeated in different survey rounds over time. This informs managers of the consistent themes coming through employee comments that are consistent over time.
- Comment Spike - The topic made up of a large number of comments within a short period of time. This informs managers of emerging themes from a one-time event or question.
4. Topic extract
Each topic contains an extract. The extract is a paragraph of the sentences that are most indicative of all the comments that make up a topic – giving you insight into a broad conversation in a few seconds.
5. Topic themes
Topics can also be grouped into themes when the same words appear in topics. This allows you to quickly spot similarities as well as key differences in the topics identified.
There is a maximum of 3 themes per driver of engagement. If more than 3 themes are generated in a single driver, these will be displayed as single topics.
In the example below, we can see three topics grouped together under a “Bonus” theme.
6. Multiple language topics
The language filter on the topics overview page allows you to see topics generated from comments in a specific language.
Where enabled, the translate button will allow you to translate these topics into your dashboard language. When viewing a specific topic, the translate option will also be available on the individual comments that make up the topic.
7. Looking at a particular topic
Each topic will display a score, which is the average of the scores associated with the comments that make up each topic. This score enables you to quickly judge the sentiment around each topic, for example the 5.9 on the "Meeting Rooms" topic in the screenshot below shows that there is room for improvement.
The graph at the top of the page shows the number of comments generated into the topic over time (as long as you have had that topic on your dashboard), along with the NPS breakdown. This insight is key in understanding the number of comments feeding into a topic as well as its degree of satisfaction over time.
8. Removing topics
In some instances, topics might not be helpful in understanding the themes from employee comments, or perhaps redundant topics have been generated. Removing specific topics is possible in two ways:
1. From within the driver topics list by clicking on the “x” icon.
2. From the topic dashboard that shows its comments by clicking on the delete icon in the top right corner of your topic extract.
Removing a topic from the dashboard will blacklist the word from being generated as a topic in the future. You’ll also be asked to provide feedback on the reason why a certain topic is being removed, to help train our algorithm in learning to generate more meaningful topics in future.
Considerations when removing topics:
- When removing a topic as an admin, it is removed from all contexts and manager views within Peakon.
- When removing a topic as a manager, it is removed only from the manager’s view. This is to allow individual users the flexibility to focus on topics that help them interpret their employees’ comments.
- Removing topics works across languages. As an example, if the word “feedback” is blacklisted in English but also appears as a topic in French, both will be blacklisted. This is only the case for exact words across languages.
Managing blacklisted topics:
- Blacklisted topics are managed from the topics overview page (scroll to the bottom of the page)
- The blacklist contains all the topics that have been removed
- Removing a blacklisted topic from this list allows the topic to be generated again from future survey rounds’ comments
In the example below you can see that blacklisted topics are accessed and managed from the blacklisted section at the bottom of the page.
9. The Peakon topic algorithm
For a manager to begin seeing topics, her team will need to have provided more than 200 comments per driver. Peakon's proprietary machine learning algorithm can then begin to discover topics.
The challenge with finding topics in short text answers is that the same words are used in many contexts. Take two examples of comments in which the word manager is used:
- In response to one of Peakon's growth questions ("I feel that I'm growing professionally") an employee could comment:"I'm definitely growing in a direction that's more interesting to me, thanks to the development plan that my manager created."
- In response to an open-ended question ("If you had a magic wand what’s the one thing you would change about Kinetar?") an employee could comment:"I really wish we would provide more line manager training, this would be really helpful."
In example 1, the word "manager" is used in relation to an employee’s interaction with the manager, whereas in example 2 it is used by a manager asking for more training.
The traditional (simple) approach to aggregating text is to count the number of times noun words are used. Often the counts of words can be represented in a word-cloud that, while visually compelling, gives you little information to act on. The reason for this is that it provides no context. Using example 1 and 2 we would find that ‘manager’ was important as it is used twice. But being told that ‘manager’ is mentioned often gives no context to act on. We get no interesting actionable information about why ‘manager’ is important.
The algorithm does five important things to make topics more actionable and interesting:
- Attach topics to a driver: the most innovative aspect of our algorithm is that it will attach the topics to a driver. This provides a context to the topic. Using the answer from example 1 the algorithm might find "Development plan" as a topic if it is a word used often in relation to growth.
- Find multi-word topics: in example 1 the answer included the phrasing "development plan". If this is commonly used in many answers, Peakon will highlight the topic as "Development plan" rather than just "Development". From example 2 the topic could be "Line manager training", "Manager training" or "Training" depending on how often the words are used across answers.
- Remove words from the question phrasing: people have a tendency to repeat the words of the question in the answer. In example 1 the answer includes the word ‘growing’ which was also in the question. It is not very useful to find this word as a topic since we already know that we are asking about growth. Therefore Peakon automatically removes the words from the question phrasing.
- Down-prioritize commonly occurring words: peakon down-prioritizes words that are used throughout all the survey answers with no specific meaning. This means that "Manager" is only found as a topic if it has a specific meaning in a driver context. This provides topics that are specific to the driver, for example, "Office" or "Room" for the environment driver or "Pay-rise" for the driver reward. If we did not down-prioritize commonly occurring words we may find "Manager" as a topic for all drivers because it is a frequently used word.
- Deliver a topic extract: peakon generates a paragraph of the sentences that are most indicative of all the comments that make up a topic – giving you an insight to a broad conversation in a few seconds.
- Topics only analyse the last 3 months of comments: this ensures that your topics will change over time. The system will also use any older comments that are also relevant to the topic, however the topic will depend on comments made in the last 3 months of time.