A Look Into The Student Mind: Understanding Performance Across Grades (Part 5)

By Guest Author | March 30th, 2018 | No Comments

In our final entry, we take a look at how learning changes as students grow in their education.

This article is Part 5 in a five-part series. Find the previous articles here: Part 1. Part 2. Part 3. Part 4.

Interested in discussing these findings with other teachers? Follow the Edmodo Topic for this series!

How strong is student mastery in topics that they have studied years earlier? How proficient is the average 7th grader in concepts from the 4th grade? We’ll look at some very interesting data from Edmodo Play that helps us answer this question.

We’re using the anonymized data from large scale student responses to gain insights into how students learn and where they struggle. All these questions are from the American Mathematics Competition (AMC) which were hosted and answered by students on the Edmodo platform. Almost 3 million students answered these questions over a period of a few months. In earlier articles in this series, we examined student errors in topics like ratios and proportions, area and perimeter, and word problems.

When looking for information on student proficiency in lower-grade content, we find that the answers are not easy to get in regular school data or even educational research. It’s normally assumed that students have learned earlier grade content as they progress to new grades. Student testing does not systematically test on lower grade content either. In this context, the data we are able to draw upon from Edmodo is even more unique and remarkable — because it finally lets us answer some of these questions.

When the AMC questions were made available on Edmodo, any student of any grade level could opt to take the test. Some students may be mapped to multiple grades, like grade 6–8. For this analysis, we only considered students who are tagged to only 1 grade between 4th and 12th grades.

We’re starting with the hypothesis that the learning gradient (the change in learning levels across grades) will be different for questions of different difficulty. We’ll therefore examine 3 questions ranging from easy to medium to difficult. We’ll then study how performance on these questions varied from grade 4 to grade 12.

(Did enough of these students answer these questions? Yes, the minimum number of students who answered any of these questions at any grade level was 7,426 (and the maximum was 62,460!). This should remove any doubts about the robustness of the data.)

Here is the first question:

Margie’s car can go 32 miles on a gallon of gas, and gas currently costs $4 per gallon. How many miles can Margie drive on $20 worth of gas?
A. 64 (8%) 
B. 128 (13%) 
C. 160 (58%)
D. 320 (11%) 
E. 640 (10%) 
Attempted: 472214 | Skipped: 514

This looks like a simple and straight-forward question. It was one of the best performing questions in the entire set of AMC questions. The graph below also shows that right from those who scored about 8/50 in the paper, students go with option C. Over 470,000 students answered this question.

A relatively easy question on the distance a car would go for a given value of gas. The table shows that students who chose C scored much better in the overall paper than others.

Option Selected vs. Average score in full ‘paper’
A. 22%
B. 23%
C. 48%
D. 21%
E. 23%

The second question is also straightforward. It requires students to navigate 50% and ‘half’ carefully and note that the question asked for the regular price for a full pound of fish.

A sign at the fish market says, “50% OFF, TODAY ONLY: HALF-POUND PACKAGES FOR JUST $3 PER PACKAGE.” What is the regular price for a full pound of fish, in dollars?
 A. 6 (24%) 
B. 9 (13%) 
C. 10 (14%) 
D. 12 (36%)
E. 15 (12%) 
Attempted: 250798 | Skipped: 159

Over 250,000 students answered this question. We can see that option A is a source of confusion for students (unlike in the previous question where no other options competed for the students’ attention.)

In this question on the price of fish, students who answered D scored much higher in the overall paper.

Option Selected vs. Average score in full ‘paper’
A. 26%
B. 24%
C. 24%
D. 58%
E. 23%

The final question was a difficult one. Only 23% of the over 213,000 students who attempted it got it correct. More students — 36% — chose the wrong option C.

Four children were born at City Hospital yesterday. Assume each child is equally likely to be a boy or a girl. Which of the following outcomes is most likely?
A. All 4 are boys (7%) 
B. All 4 are girls (8%) 
C. 2 are girls and 2 are boys (36%) 
D. 3 are of one gender and 1 is of the other gender (23%) 
E. All of these outcomes are equally likely (26%) 
Attempted: 213299 | Skipped: 89

The graph below depicts the difficulty the students faced. Wrong options, especially C but even E are more popular at almost all score levels than the correct answer. While we expected the best performing students to have answered correctly, we can see from the table below that many of the better performers did select E and C.

This tough question has stumped a number of students. We can see that E and C were the favored wrong answers even among the better performers.

Option Selected vs. Average score in full ‘paper’
A. 27%
B. 27%
C. 32%
D. 55%
E. 34%

Overall this was the performance of the 3 questions:

Now when we map the performance on these questions via grade, we can see that they show very different patterns.

While the performance on the easy question on Maggie’s car mileage increases from below 48% in grade 4 to 75% in grade 12, the same for the question on the price of fish is from 27% to 44%. On the question on city hospital births, there is no increase at all — it stays at 19% in grade 4 as well as grade 12!

What can we learn from this?

Firstly, we see that even for the easy car mileage question, the performance in grade 12 is only at 75%. Thus we see that learning levels are much lower that what they should be as per standards like the Common Core.

Secondly, we see that the slope of learning is higher for easier concepts and questions. This means clearly that learning happens more slowly than teachers expect.

Finally, we find that for difficult concepts, the learning slope is virtually flat.

We should constantly question the assumption that simply because a student has passed a certain grade, they will have complete knowledge of that grade’s content and standards. Grade, and age by extension, is not a reliable indicator of a student’s skill with specific concepts.

Overall, it helps teachers to know what students are finding easy and difficult based — not on our expectations — but on actual data. This allows us to focus our attention on what students need help on, not what we may think they need help on.

Edmodo acknowledges the input of Educational Initiatives, Inc. which provided the question performance analysis used in the article.

About the Author: Guest Author