Echo 360 - Engagement and Analytics Metrics Defined
Instructions for defining engagement and analytics metrics in Echo 360.
Engagement and analytic information uses six different metrics that combine to determine the overall "Engagement" score given for students and classes. The Engagement Settings page lets you assign relative weight to each metric, depending on what is important to you, or what aspects you want to keep an eye on.
However, in order to properly interpret the information being presented, you need to know how we define each of them, and what information is actually being tracked/logged.
Analytics for LIVE classes - There are some student participation statistics that cannot be applied or are tracked differently for live-streamed classes, because Echo360 has no concept of a "time stamp" of a live-stream. For example, "video views" apply to a processed video, not to a live-stream. If the student views the capture of the class again later, they will get credit for a video view. Notes word counts are tracked, as well as questions and responses. Activity participation and Presentation views are counted for any presentation also published to the class.
Definition: Whether or not the student attends a class DURING the time it is scheduled. If the student clicked on the class and viewed the classroom during the date/time the class is scheduled, they get credit for attendance.
This is a binary statistic, meaning "yes they entered the classroom during that time" or "no they didn't". This metric does not track how long they were in the classroom or whether they viewed any of the materials. Students can enter the classroom anytime between 10 minutes before the scheduled class starts, and 5 minutes before it ends.
It's an easy metric to understand but can be hard to quantify, especially if you have classes that simply hold media. If the only classes you create for your section are for the actual classes being taught, then attendance has very specific meaning. Otherwise just know that if a student doesn't watch the media during the time-frame of the class, their attendance count will be skewed. You should weigh this metric in your engagement settings appropriately.
Technical Details: Attendance percentage for a class is calculated as: total number of students who attended the class/total number of students in the section.
Attendance percentage for a student is calculated as: total number of classes attended/total number of class that have already occurred. Future classes (those with a future date/time) are not taken into consideration when calculating attendance.
Definition: The number of times students have viewed 5% or more of the class video. You can see this for each class or for individual students. Every student view is counted, whether by different students or multiple times by a single student. Repeated views by a student are counted if there is a 15 minute gap between views.
Depending on where you are on the Analytics page, the video views information can be more or less granular. The examples below provide more details.
Analytics for LIVE classes - There are some student participation statistics that cannot be applied or are tracked differently for live-streamed classes, because Echo360 has no concept of a "time stamp" of a live-stream. For example, "video views" apply to a processed video, not to a live-stream. If the student views the capture of the class again later, they will get credit for a video view. The graph (shown below) identifies time or location stamps for student activity in the class, but a live-stream doesn't have a trackable "time" until the capture is processed and published as a video. Notes word counts are tracked, as well as questions and responses. Activity participation and Presentation views are counted for any presentation also published to the class.
Video views for all classes in the section
In the Classes section of the Analytics tab, if you are viewing All Classes and click Video views, the number indicated provides the total number of views of the video, by class date. However, if 32 views are shown for the date, it could be one student viewing it 32 times, or 32 students viewing it once. As stated above, each view is counted as soon as a student watches 5% or more of the video, and repeated views by a student are counted if there is a 15 minute gap between views.
Video views for a specific class
In the Classes section of the Analytics tab, select a specific class from the drop-down list. The bar graph changes to show information for only that class. In addition, a second drop-down list appears, allowing you to see Video view or Presentation view information. Video is shown by default.
NOTICE that the X axis (horizontal) measurement now reflects the timestamp/location in the video for the view, and the Y axis (vertical) indicates how many views that location in the video has received. Once again, the count is cumulative; 32 views by one student of a location is counted the same as 1 view of the location by 32 students.
Hover your mouse over a location in the graph to see the timestamp for that particular location.
Video views by students
In the Students section of the Analytics tab, select All Classes from the class drop-down list. All video views are shown for each student. Again this figure is cumulative; a student with 20 video views could mean 20 views of a single video, or one view of 20 different videos. As long as 5% or more of the video was viewed, and a gap of at least 15 minutes occurred between repeated views by a single student.
Video views by student for a class
In the Students section of the Analytic tab, select a particular class from the class drop-down list, then select Video.
The Video views for that class are shown for each student. This lets you know which students have viewed that video at all, and which have viewed it multiple times. It also shows what percentage of the video the student viewed. Remember that a video view count is logged if the student watches 5% or more of a video. This page lets you know how much of the video the student actually watched.
Definition: The number of times students have viewed 5% or more of the class presentation (with at least 5 seconds on each slide). You can see this for each class or for individual students. Every student view is counted, whether by different students or multiple times by a single student. Repeated views by a student are counted if there is a 15 minute gap between views.
Depending on where you are on the Analytics page, the presentation views information can be more or less granular.
The Video view metric information provided in the sections above are the same for Presentations, except that a student must view each slide for at least 5 seconds. Like video views, view counts are cumulative up until the point you choose to view presentation views for a particular class in the Students section. And as indicated for Video views, the method for counting views and the granularity available from the different selections is the same.
The only difference in the information is on the Classes section of the Analytics page; if you select a particular class, then select Presentation from the drop-down list, the Y axis shows slide number instead of video timestamp/location. These are shown in the below figure.
Definition: The number of questions and responses posted by students, either in total for the section or viewable by class. It does not take into account Instructor-posted questions or responses.
This metric is reasonably straightforward, but there is one odd-looking detail you may encounter. In the Classes section of the Analytics page, select a particular class, then click the Questions metric to highlight that line. The graphic below shows this for the Presentation media in the class.
NOTICE that the Question tally at the top reads "4". This counts all student questions/responses posted for this class. The graph shows only one question posted (for slide 6). Selecting Video shows a graph with zero questions posted. All this means is that some questions posted do not reference specific content. The Q&A tab will show 4 questions and responses for that class, but only one of them references a particular location in the slide deck.
Remember also that the total question count at the top includes only STUDENT posted questions/responses; instructor posts are not counted.
Definition: The number of words entered by students into the Notes panel and/or Study Guide for the section or class.
This is another reasonably straightforward metric, simply letting you know which students are taking notes in the Echo360 interface, and how extensive their note-taking is. You cannot see student notes. All you can do is identify who is taking notes.
The word count is determined by counting the number of distinct groups of letters students are entering. A "word" is qualified as a set of letters between spaces. (This is as opposed to typing test word-per-minute counts, where a "word" is qualified as a group of 5 keystrokes.) All this really means is that a student who uses very big words in their notes will have a smaller "notes word count" than a student who uses lots of small words.
Definition: Whether or not students are answering the interactive questions/slides posted in class presentations. There are two parts to this metric: Activity participation, and Activity correct.
Activities Download provides raw numbers: The analytics interface shows the activity participation/correct as a percentage. However the analytics download (export to CSV) allows you to select activity participation/correct as either percentages OR raw numbers. If you select the raw number download for either, you see the number of activities responded to and the number of correct answers given, by each student for each class.
Activity participation for each student is the percentage of the total activities the student responded to (number of responses/total number of activities). This is shown on the Students analytics page, and can be displayed for All Classes or for a specific class.
Activity correct is only shown for individual students, and is a percentage calculated as: number of correct answers given/number of activities that have a correct answer identified. In this case, short answer questions, or other activities where you have not identified a "correct" answer are not counted.
Activity participation for the entire section is shown in the bar graph on the Classes analytics page, and only when All Classes is selected. Hover over a particular class date to see the total student participation percentage in that class' activities, as shown in the below figure.
Technical Details: The Activity Participation percentage for a class is calculated as: total number of student responses/(total number of activities * total number of students in the section). It does not take into account the number of students who viewed the presentation containing the activities.
For example: The section has 20 students, and 15 of them view the presentation and answer ALL of the activities. The Activity Participation percentage for that class will be 75%. You would get the same percentage if all 20 students viewed the presentation but only 15 of them answered the activities.
© 2017 Echo360 | www.echo360.com