- Data Aggregation
- Time for all data
- Time for each driver of engagement
- Time for former employees
- The process for aggregating data
- Engagement and driver score aggregation
- Segment score aggregation
- Data Anonymity
- Minimum segment size
- Data visibility threshold
- Difference anonymity level
- Comment anonymity level
- Score anonymity process
- Topic anonymity process
- Topic aggregation
Traditionally employee engagement surveys are performed once a year, where all employees in the company are asked the full set of questions. This means that the company and its managers only have up-to-date engagement data once a year, and that the employees have to go through a very long survey.
Peakon instead provides a more frequent survey, that will ask a subset of questions for each survey round. In addition, Peakon also supports different survey frequencies and questions for different parts of the company. This provides up-to-date engagement data for the company and its managers, and a better survey experience for the employees. Specifically, it saves employees from survey fatigue.
Even when multiple question sets and frequencies are used, Peakon is able to give an overview of the full set of engagement drivers across the company and its segments. This article describes how Peakon aggregates data over time and over different questions as well as how anonymity is ensured.
The following options are configurable through the product and used in the aggregation and anonymity protection – these are shown in the screenshot below. This area of the dashboard is available to administrators and accessed via the settings option in the left menu.
3. Data Aggregation
On Peakon, the “Time for all data” and “Time for each driver of engagement” aggregation settings allow for an employee’s most recent survey results to be included in scores. This is especially relevant since Peakon uses question sampling to reduce the amount of questions being asked in surveys running on an automatic frequency mode.
To illustrate how these two settings work, a simplified example will be used below:
Imagine we have a survey that runs monthly, for just two employees with just two sets of driver questions being asked: the Autonomy driver question and its two sub-driver questions, and the workload question. The survey started running in January 2018, and it is currently February 2019.
Peakon doesn’t ask every question in every round, instead Peakon uses question sampling to reduce the survey length. This table shows the answers given to each question at each round. An ‘x’ means that the employee answered that question - the score doesn’t actually matter for these examples.
In reality, employees normally answer more frequently within each driver, which reduces the impact of these filters. These are exaggerated examples designed only to show how they work.
4. Time for all data
This setting sets a global window for scores to be considered. Peakon calculates the end-point of the window automatically. This is not measured from today, instead this is set by the last score to any question by any employee. In this case our end date is January 2019, since nobody has answered in February yet.
For now, ignore the impact of the Time for each driver of engagement setting, this will cover that next.
Option: Setting "Time for all data" to "1 year"
We go back 12 months from January 2019, and throw out all data from outside this window.
Peakon then takes the latest score for each question.
Option: Setting "Time for all data" to "3 months"
We go back 3 months from January 2019, and throw out all data from outside this window.
Then we take the latest score for each question, but in this example that has no impact as there is only one score per question within the time window.
Compare this to the previous result:
- The scores given by Harriet to Autonomy - Flexibility and Autonomy - Remote Work will not be included
- Matilda will not contribute any data at all
5. Time for each driver of engagement
This setting is designed to ensure that each driver average consists of up-to-date scores, even when the overall time window is long. In this example we assume that Time for all data is set to 1 Year, and Time for each driver of engagement is set to 3 months.
Starting with the 12 month window of data:
We then draw an additional 3 month window (in orange) around each driver for each employee, starting with the most recent answer per employee and driver.
Peakon then takes the latest score per question, but again this has no further impact.
Compare this to the result seen with a 1 Year Time for all data but without the Time for each driver of engagement set to 3 months. The scores marked in red are no longer included, because although they are the latest score for that question, and they fall within the overall time window, they are now too old compared to the latest score for that driver.
6. Time for former employees
This setting controls how long scores will be retained and aggregated for employees that have left the company. Once an employee leaves, either by setting the separation date or deleting them completely, their most recent scores will still be included for either one, three, or six months from the leaving day or day of deletion.
7. The process for aggregating data
8. Engagement and driver score aggregation
The base date for the following calculations is always the end of a survey round. The current score for a company or segment is calculated from the end of the last round.
Data aggregation happens via the following process:
- Any answers from survey rounds that fall outside of the “Time for all data” window are excluded.
- Any answers from former employees that were deleted from Peakon are excluded if they fall outside of the “Time for former employees” window.
- Any answers for a given employee and driver question, that fall outside of the “Time for driver of engagement” window, are excluded.
- Only the most recent answer for each question for each employee is retained.
- The average score for each employee and driver/sub-driver is calculated.
- The average score for each driver/sub-driver is calculated.
This process provides the following benefits:
- Enables sampling of questions from survey to survey, while still retaining an accurate picture of all drivers across the company.
- Allows inclusion of results from different parts of the company being surveyed at different frequencies.
- Ensures that only recent answers for each employee are utilised to provide the most accurate picture of the company engagement.
- Responses from former employees are only kept around for a short amount of time.
9. Segment score aggregation
When scores are calculated for an individual segment, the same process as above is followed. However between step 4 and 5, only scores from employees within the given segment are considered.
10. Data Anonymity
The anonymity levels control the minimum number of individual employees that must be, either part of a specific survey round, or have answered a survey over a given time period in order to have aggregate data shown. This is used to ensure the anonymity of employees. The following sections, detail how this is applied.
11. Minimum Segment Size
Controls the minimum number of responses required to break out a segment for comparison. Lower numbers mean less relative anonymity, but may be required for small teams. For example, if set to three, a manager with more than three reports will have a dashboard populated with scores only if three or more of those reports have completed a survey.
12. Data Visibility Threshold
The minimum number of answers for a given driver or sub-driver that must be present, for the aggregated score to be calculated. This ensures that there is a minimum number of data points for the aggregated score to be visible.
13. Difference anonymity level
The difference level controls the minimum difference in number of employees with answers between two different groups of employees, where there is a large overlap in employees between the two groups. This is used to ensure answers from small groups cannot be inferred through the difference between the scores for the groups.
This setting solves problems of anonymity that arise when a set of employees can be partitioned into two or more groups, with one much larger than the other. As an example, imagine a case where a company of 10 people is made up of 7 female employees and 3 male.
A minimum segment size of 5 would hide the male segment, but leave the female segment visible. The problem is that if you know the company score, and you know the female segment score, you can work out what the male segment score must have been.
The difference anonymity level setting prevents segments that are too close to the context size (10, in the example above) from being visible.
For this reason, you can view this setting as a counterpart to the minimum segment size setting. While the minimum segment size hides segments that are too small to view, this setting hides segments that are too large instead.
In the diagram above, the context size is set to 30. The red area eliminates segments that are too small using the minimum segment size setting of 5. The purple area eliminates segments that are too big using a difference anonymity setting of 5, where the limit works back from the context size: 30 - 5 = 25.
In this example, only segments sized between 5 and 25 employees would be visible.
The difference level checks are performed in three ways:
- When filtering data by a subsegment within a context (the company or a segment): In this case there must be at least the configured number of employees that are both within the context and outside the subsegment.
- When viewing direct reports segment: In this case there must be at least the configured number of employees that are both within the all reports segment and outside the direct reports segment.
- When viewing any segment with rights to view other segments: In this case, for any other segment the manager has access to, there must be at least the configured number of employees that are both within the other segment and outside this segment. This check was added on the 8th of August 2018.
To illustrate the above rules, a few examples can be helpful:
- A manager is viewing results for their segment. Overall 10 employees have answered of which 7 are female and 3 are male. The anonymity level is set to 5. Without a difference level, the manager can view the results for the whole segment as well as the results for female. Results for male are not available as it is below the anonymity level. Given the scores for the female segment as well as the scores for the segment overall, the manager can infer the aggregate results for the male segment. With a difference level of 4, the results for female will no longer be available, as the difference between the overall segment (10 employees) and the subsegment female (7 employees) is below the difference level (4 employees).
- A manager has been given access to both their own segment (10 employees) as well as their department segment (12 employees). The only difference in employees between the two segments is the manager themselves as well as another employee reporting to a manager in a different department. Without a difference level the manager can see the results for both their own segment and the department segment and in turn infer the aggregate results for the group outside their own segment. With a difference level of 3 the results for the managers own segment, will no longer be available to the manager. If access to the department segment is removed for the manager, the results for their own segment will again be available.
Note: The second case is not performed for users with access to view engagement data across the whole company. This is to ensure HR and C-level can get data for all segments and support managers in evaluating and improving their engagement score.
14. Comment anonymity
Controls the minimum amount of responses required to display comments. Comments are often the most actionable part of the feedback from employees. It is therefore important to surface as many comments as possible while preserving anonymity for each individual employee. This is particularly important on weekly or biweekly frequencies, where it is generally expected that a smaller part of a given segment completes the survey every round.
15. Comment anonymity process
The following process is followed when listing comments:
- Only answers that match a given segment are retained.
- If the number of employees that answered over the last year is less than the anonymity level, no comments are shown.
- Only comments from survey rounds where the total number of employees asked are above the anonymity level, are shown.
This process provides the following benefits:
- It ensures no comments are shown if the number of employees that responded is below the anonymity level.
- It allows access to comments in the case of high frequency surveying without requiring a high participation rate on every round.
- It protects anonymity by ensuring that only comments from rounds where enough employees were asked, are included.
16. Score anonymity process
To ensure that individual answers remain anonymous, the following process is applied after the score aggregation:
- If the total number of employees that are included across the full aggregated results is less than the anonymity level, the full set of scores is not shown.
- For each driver and sub-driver, if the total number of answers is less than the significance level, the scores for the driver/sub-driver are not shown.
This protects anonymity overall, and ensures that there is a significant number of responses to each driver/sub-driver before any further analysis.
17. Topic aggregation
Comments to include in topic analysis follows a process similar to the way engagement and driver scores are aggregated.
- Any answers from survey rounds, that are older than a year, are excluded
- Any answers from former employees, that left more than 12 weeks ago, are excluded
- Any answers for a given employee and driver/open-ended question, that are older than 12 weeks from the last answer of the employee and driver/open-ended question, are excluded.