Interested in discussing these findings with other teachers? Follow the Edmodo Topic for this series!
Understanding how students think is an art and a science. It is also an invaluable skill in a teacher’s repertoire, as it helps the teacher anticipate student errors and, in a sense, explain concepts from a student’s perspective. Knowing how students think about a topic and how that changes as they gain mastery in it, allows a teacher to use explanations that exactly address the doubts students face as their understanding develops. Some of this develops with time and experience and ongoing student interaction.
Yet, up until now, there have been few systematic ways in which teachers can study and develop this skill. Repositories of largescale student responses to questions are not easily available to teachers. Sure, teachers have access to their own students, but it turns out that data from only one classroom is often influenced by the teacher’s own methods. On the other hand, largescale data provides unique insights some of which apply to the way all students learn.
Edmodo is a massive collaboration platform for students and teachers that leverages technology and has over 80 million registered users. It not only forms one of the largest learning networks on the planet, it is also one of the largest repositories of data on student learning. Edmodo’s mission is to connect all learners with the people and resources they need to reach their full potential – to improve learning across all schools, in other words. In this age of big data, could this massive student data repository contain insights that can be used to advance learning for all? You bet!
Recently, Edmodo hosted tests from the American Mathematics Competitions (AMC). Almost 3 million students answered these questions over a period of a few months! It is next to impossible to collect such volumes of data through a physical test. Would this data contain insights on student learning – insights that can help teachers in actionable ways in class? We embarked on an analysis to answer this question over the past many months, and the result is a 5 article series that we shall share with you over the coming few weeks.
In this new series, we shall explore some of these ideas with specific topics in Mathematics and provide insights on how students seem to be thinking about and understanding key concepts. We will also provide practical insights and actionable advice that teachers could use in the class to help improve student learning. Not only will teachers gain specific ways of approaching student learning issues in the topics discussed, there are some larger takeaways about how students learn and how largescale data can be used by teachers to easily gain these insights.
Each article will relate to a topic or concept in Mathematics. While this article will introduce the series and a look at some question examples that indicate student errors and how students think, the following articles will focus on these themes:
 Students understanding and misconceptions about ratios and proportions
 Understanding area and perimeter – common errors students make
 Solving word problems – using data of student errors to improve teaching effectiveness
 How well have students mastered concepts from prior (lower) grades
An important note on data tracking and student privacy – Edmodo has always been committed to safeguarding the privacy of all its users and maintaining the highest standards of confidentiality with respect to student data. All the analysis presented here was done with anonymized data, meaning that even the researchers doing the analysis are not aware of the identities of the students who answered the questions. What is being tracked is patterns of large groups – students as a whole or highscoring students; with no attempt to analyze performance or scores of individual students or classes or schools. Also, all these articles and analyses are based on student responses to particular tests (the AMC Grade 8, 10 and 12) only and no other tests or questions.
Methodology of the Analysis: A total of over 1250 questions asked in the AMC to students of classes 8, 10 and 12 were analyzed. On Edmodo, over half a million students answered at least 15 of these AMC questions (with many enthusiastic students answering hundreds of them). On the Edmodo platform, students could answer as few or as many questions as they wanted – it was not a competition and they did not have to answer a complete set of questions. Also, students could answer questions for any grade, making the data even more interesting and rich as it allows us to compare learning levels of students of different grade levels on the same topics.
For some of the analysis, we focused on a subset of questions that were all answered by a group of students. Specifically, we selected 50 questions each in grades 8 and 10 (and 24 questions in grade 12). For example, 47,000 students answered all the 50 selected grade 8 questions. This allowed us to analyze the relative performance of high performing and lower performing students (as we will show in the graph below)
Let us examine a few questions and see the kind of insights we get into how children think.

As we can see above, over 430,000 students have answered this question, resulting in very robust data which can be relied upon. We can see that about 36% of grade 8 students answered this question correctly. Technically, this question relates to a grade 4 standard “4.NBT.4: Fluently add and subtract multidigit whole numbers using the standard algorithm.” But, needless to say, the challenge in this question is that of interpreting the question correctly and determining the solution strategy.
Students need to reason that if Samantha turned 12 when she took the 7^{th} AMC, she was (12 – 6 =) 6 years old in the year of the 1^{st} AMC. Thus she was 6 years old in 1985, and hence she was born in 1979. A common mistake is to calculate her age at the time of the 1^{st} AMC, not as 12 – 6 but 12 – 7 = 5. This group thus concludes that she was 5 years old in 1985 and gets the answer as 1980 (which we can see is the most common wrong answer).
But can we tell what stronger students answered and which options confused the weaker students? For this, we can draw the graph shown below on the left.
Option vs. Average Score in full ‘paper’ A 58% B 29% C 24% D 23% E 23% *Students who chose A in this question scored twice as much in the paper as those who chose B 
On the right, it shows the correct answer in yellow highlight and the percentage of students who chose each option. The horizontal axis of the graph represents the total score in the full question set’. Thus points on the left represent students who scored lower in the paper and the right extreme consists of students who answered almost all (50) questions correctly. The vertical axis represents the percentage of students. Thus we can see that at a score of about 10, all options are equally popular among students. Up to a score of about 18 options A and B are equally popular! This shows us exactly what weaker students are thinking and the errors prevalent at different levels of proficiency. We can also see that the wrong options (with the possible exception of B) become extremely rare beyond a score of about 20/50.
The table on the right reinforces that students choosing option A are the higher proficiency students. They score significantly better than students selecting any other option.
So what can a teacher do? Note that it is possible in a classroom to group the students based on their choices and this exercise should be done for a few important questions in any test. The ‘error of 1’ in ‘calculating Samantha’s age during the 1^{st} AMC if she was 12 during the 7^{th} AMC’ is widely observed in between 2030% of students, and hence should be illustrated with examples and more such questions. It often helps to have the different groups write 12 line explanations for their answers and then discuss some of the common errors in thinking.
Do share your comments on the question and the student performance data in the comments below. And think about the following too.
Sometimes, the data throws up surprises. Look at the following question:

Why do you think students choose 7 ½? It is the most common wrong answer, selected by far more students than 8 or 6 2/3!
Edmodo would like to thank and acknowledge the inputs of Educational Initiatives, Inc. which provided the question performance analysis used in the article.
Interested in discussing these findings with other teachers? Follow the Edmodo Topic for this series!
22 comments. Leave new
This is an incredibly insightful article – I can’t wait to read the whole series. Although I don’t teach math specifically, I try to teach the ideas mentioned in the article by integrating them into my class content. I look forward to this series making me a better teacher over all.
As a math teacher I find this really interesting and am looking forward to the whole series of articles.
Seems cool, thanks for sharing that!
Really interesting! It is very important to reflect students’ thinking to understand their learning!
As a retired educator I appreciate the practicality of the article. Tutoring students in person and via Skype getting professional insights is very helpful.
Indeed insightful and useful article. Looking forward to the whole series.
Interesting article on the research data. Would be great to read about further results and insights. Thank you.
This article is absolutely of great help to teachers as well as to us, future teachers. I am hoping to read more of it.
The finding is so interesting!
I agree by asking students to write steps on solving multiple choice questions is a very useful stragegy for discussing some of the common errors in their thinking.
I am a teacher of year 4 and as a kid growing up was never taught problem solving strategies. Have learnt a lot through teaching maths to kids over years. I used a different strategy to you – drew a time Line image Sam 12 in. 1991 on 7th amc. 1991 – 12 is 1979.
Great to consider your strategy and why children made errors.
Thank you.
Think my maths skills have certainly improved over years simply by teaching a variety of practical methods to approach problems – e.g. draw a pic, work backwards, make a table etc.
Too many kids get stuck in head stuff – get it out in front of you as an image or a table and ‘see’ the patterns – I have always been a concrete practical learner – so are most of the kids I teach – we often expect everyone in High School to operate in the abstract realm. Kids feel inadequate if they draw an image – but we should get over this negative mind set as teachers and encourage all strategies – to suit all types of minds.
Thank you. Look forward to the series.
If you would like to know what a student is thinking and feeling during a test, why not take a survey? It would be intriguing to view the results of this especially if it is taken before and after a test.
Nice
I took the Mathematical Mindset course from youcubed.org this summer which changed me as a teacher. I am interested in how students think and am working on my Learning and Technology Master’s degree.
Regarding the question: “Why do you think students choose 7 ½? It is the most common wrong answer, selected by far more students than 8 or 6 2/3?”
Hmmm…. things to think about.
This made my day now. A lot of realizations and ideas earned in this brilliant article. I love It!
Thanks indeed, very useful info. Looking forward for the next part.
I found it very interesting reading and I’ll be checking for the whole series. I hope to keep reading some more of the articles.
Una forma técnicopedagógica de verificar el grado de desarrollo lógicomatemático del alumnado, estaré pendiente de los artículos siguientes. muy formativo.
Good. i hope my english can usefull
I really like to read the entire article. As a teacher, it would definitely give me an insight as to how students think during exams.
It was very informative
Thanks any way, it’s very useful activity and knowledge for me and my students
I love this and anticipate more of this. Great job