By Guest Author | October 15th, 2017 | No Comments

*Interested in discussing these findings with other teachers? Follow the **Edmodo Topic** for this series!*

Understanding how students think is an art and a science. It is also an invaluable skill in a teacher’s repertoire, as it helps the teacher anticipate student errors and, in a sense, explain concepts from a student’s perspective. Knowing how students think about a topic and how that changes as they gain mastery in it, allows a teacher to use explanations that exactly address the doubts students face as their understanding develops. Some of this develops with time and experience and ongoing student interaction.

Yet, up until now, there have been few systematic ways in which teachers can study and develop this skill. Repositories of large-scale student responses to questions are not easily available to teachers. Sure, teachers have access to their own students, but it turns out that data from only one classroom is often influenced by the teacher’s own methods. On the other hand, large-scale data provides unique insights some of which apply to the way all students learn.

Edmodo is a massive collaboration platform for students and teachers that leverages technology and has over 80 million registered users. It not only forms one of the largest learning networks on the planet, it is also one of the largest repositories of data on student learning. Edmodo’s mission is to connect all learners with the people and resources they need to reach their full potential — to improve learning across all schools, in other words. In this age of big data, could this massive student data repository contain insights that can be used to advance learning for all? You bet!

Recently, Edmodo hosted tests from the American Mathematics Competitions (AMC). Almost *3 million students* answered these questions over a period of a few months! It is next to impossible to collect such volumes of data through a physical test. Would this data contain insights on student learning — insights that can help teachers in actionable ways in class? We embarked on an analysis to answer this question over the past many months, and the result is a 5 article series that we shall share with you over the coming few weeks.

In this new series, we shall explore some of these ideas with specific topics in Mathematics and provide insights on how students seem to be thinking about and understanding key concepts. We will also provide practical insights and actionable advice that teachers could use in the class to help improve student learning. Not only will teachers gain specific ways of approaching student learning issues in the topics discussed, there are some larger takeaways about how students learn and how large-scale data can be used by teachers to easily gain these insights.

Each article will relate to a topic or concept in Mathematics. While this article will introduce the series and a look at some question examples that indicate student errors and how students think, the following articles will focus on these themes:

- Students understanding and misconceptions about ratios and proportions
- Understanding area and perimeter — common errors students make
- Solving word problems — using data of student errors to improve teaching effectiveness
- How well have students mastered concepts from prior (lower) grades

An important note on data tracking and student privacy — Edmodo has always been committed to safeguarding the privacy of all its users and maintaining the highest standards of confidentiality with respect to student data. All the analysis presented here was done with anonymized data, meaning that even the researchers doing the analysis are not aware of the identities of the students who answered the questions. What is being tracked is patterns of large groups — students as a whole or high-scoring students; with no attempt to analyze performance or scores of individual students or classes or schools. Also, all these articles and analyses are based on student responses to particular tests (the AMC Grade 8, 10 and 12) only and no other tests or questions.

Methodology of the Analysis: A total of over 1250 questions asked in the AMC to students of classes 8, 10 and 12 were analyzed. On Edmodo, over half a million students answered at least 15 of these AMC questions (with many enthusiastic students answering hundreds of them). On the Edmodo platform, students could answer as few or as many questions as they wanted — it was not a competition and they did not have to answer a complete set of questions. Also, students could answer questions for any grade, making the data even more interesting and rich as it allows us to compare learning levels of students of different grade levels on the same topics.

For some of the analysis, we focused on a subset of questions that were all answered by a group of students. Specifically, we selected 50 questions each in grades 8 and 10 (and 24 questions in grade 12). For example, 47,000 students answered all the 50 selected grade 8 questions. This allowed us to analyze the relative performance of high performing and lower performing students (as we will show in the graph below)

Let us examine a few questions and see the kind of insights we get into how children think.

The first AMC 8 was given in 1985 and it has been given annually since that time. Samantha turned 12 years old the year that she took the seventh AMC 8. In what year was Samantha born?A. 1979 (36%)

B. 1980 (23%)

C. 1981 (15%)

D. 1982 (15%)

E. 1983 (12%)

Attempted: 430763 | Skipped: 480

As we can see above, over 430,000 students have answered this question, resulting in very robust data which can be relied upon. We can see that about 36% of grade 8 students answered this question correctly. Technically, this question relates to a grade 4 standard “4.NBT.4: Fluently add and subtract multi-digit whole numbers using the standard algorithm.” But, needless to say, the challenge in this question is that of interpreting the question correctly and determining the solution strategy.

Students need to reason that if Samantha turned 12 when she took the 7th AMC, she was (12–6 =) 6 years old in the year of the 1st AMC. Thus she was 6 years old in 1985, and hence she was born in 1979. A common mistake is to calculate her age at the time of the 1st AMC, not as 12–6 but 12–7 = 5. This group thus concludes that she was 5 years old in 1985 and gets the answer as 1980 (which we can see is the most common wrong answer).

But can we tell what stronger students answered and which options confused the weaker students? For this, we can draw the graph shown below on the left.

Option vs. Average Score in full ‘paper’

A 58%

B 29%

C 24%

D 23%

E 23%

*Students who chose A in this question scored twice as much in the paper as those who chose B

On the right, it shows the correct answer in yellow highlight and the percentage of students who chose each option. The horizontal axis of the graph represents the total score in the full question set’. Thus points on the left represent students who scored lower in the paper and the right extreme consists of students who answered almost all (50) questions correctly. The vertical axis represents the percentage of students. Thus we can see that at a score of about 10, all options are equally popular among students. Up to a score of about 18 options A and B are equally popular! This shows us exactly what weaker students are thinking and the errors prevalent at different levels of proficiency. We can also see that the wrong options (with the possible exception of B) become extremely rare beyond a score of about 20/50.

The table on the right reinforces that students choosing option A are the higher proficiency students. They score significantly better than students selecting any other option.

So what can a teacher do? Note that it is possible in a classroom to group the students based on their choices and this exercise should be done for a few important questions in any test. The ‘error of 1’ in ‘calculating Samantha’s age during the 1st AMC if she was 12 during the 7th AMC’ is widely observed in between 20–30% of students, and hence should be illustrated with examples and more such questions. It often helps to have the different groups write 1–2 line explanations for their answers and then discuss some of the common errors in thinking.

Do share your comments on the question and the student performance data in the comments below. And think about the following too.

Sometimes, the data throws up surprises. Look at the following question:

Rachelle uses 3 pounds of meat to make 8 hamburgers for her family. How many pounds of meat does she need to make 24 hamburgers for a neighborhood picnic?

A. 6

B. 6 2/3

C. 7 ½ Most common wrong answer

D. 8E. 9 Correct and most common answer

Why do you think students choose 7 ½? It is the most common wrong answer, selected by far more students than 8 or 6 2/3!

*Edmodo would like to thank and acknowledge the inputs of Educational Initiatives, Inc. which provided the question performance analysis used in the article.*

*Interested in discussing these findings with other teachers? Follow the **Edmodo Topic** for this series!*