New research shows that some of the tools used in the US to test children’s thinking skills are based on richer, mostly White, populations. This may have led to inaccurate results for poor, urban students.

New research indicates that several of the assessment tools used to measure children’s cognitive skills in the United States may have generated inaccurate data concerning low-income, urban kids as they are based on wealthier, predominantly White communities.

A recently published study involving nearly 500 children from high-poverty urban communities in the United States found that a popular assessment, which measures the growth of cognitive abilities known as “executive functions,” did not provide a complete and accurate picture of the students’ academic development. The paper attributes this to a likely cultural bias in the assessment process and warns that this may occur with other similar measures.

Any such design defect may have contributed to a growing body of evidence indicating that children from disadvantaged families enter school with poorly developed executive functions. Executive functions refers to a group of critical reasoning abilities and learning that are necessary to complete daily tasks. Working memory, self-control, the capacity to ignore distractions, and the ability to swiftly switch between projects are a few of them. Children with strong executive functions typically perform better on tests, have better mental health, and have more possibilities for career.

Asking teachers to fill out questionnaires about the kids’ actions is a common way to see if they are developing these skills in a healthy way. The findings may be used to identify individual kids or groups who could benefit from additional care. They are also a significant source of data for studies on the development of executive functions.

In the latest study, researchers discovered that one of these widely used teacher rating systems in the United States was of limited usefulness when analyzing urban, low-income kids. Specifically, they discovered that the executive function screener of the BASC-2, a version of the Behaviour Assessment System for Children (BASC), “is not a good representation of everyday executive function behaviors by children from schools in high-poverty communities.”

This scale and others like it were developed using an unrepresentative sample of children, according to the team from the University of Cambridge in the UK and Virginia Commonweath University in the US.

Researchers have pointed out in the past that these tests are usually based on children from comfortable socioeconomic backgrounds. By mapping observable behaviors to executive functions, one may erroneously conclude that these behaviors are ‘normal’ for children of the same age. In fact, the varied origins and lived experiences of children may mean that different groups of children display their executive functions in different ways.

“There is a big question around how we measure executive functions: are we actually using the right tools?,” says first author Annie Zonneveld, adding, “If they are based on white, middle-class students, we cannot be sure that they would actually work for the whole population. We may be seeing evidence of that here.”

“Teachers can provide us with really valuable data about children’s executive functions,” adds Prof Michelle Ellefson, “because they can monitor development in ways we could not possibly replicate in a lab, but they need effective measures to do this.” 

“Teachers can provide us with really valuable data about children’s executive functions,” adds Prof Michelle Ellefson, “because they can monitor development in ways we could not possibly replicate in a lab, but they need effective measures to do this.” 

This means that data about kids from various backgrounds must be used in the assessments.

The Children’s Defense Fund estimates that 14% of American children are living in poverty. While roughly 50% of all children are from ethnic minority families, 71% of the poor are from these backgrounds. However, the majority of psychometric research on executive functioning focuses on white, middle-class, or wealthy households. The extent to which its results can be generalized is unclear.

The current study assessed the executive function components of the BASC-2 and BASC-3 instruments. These ask instructors to watch how kids behave in ordinary situations and score how much they agree with statements like “acts without thinking,” “is easily distracted,” “cannot wait to take turn,” “is a self-starter,” and “argues when denied own way” on a scale of “never” to “always.” Based on the answers, they then figure out more about the child’s executive functions.

The researchers analyzed two sample groups of children aged nine or ten from public schools in urban areas of the United States with significant rates of poverty. A total of 472 kids participated. The BASC-2 and BASC-3 were used to evaluate the first and second samples, respectively.

Both groups also finished six computer-based tests that psychologists and neuroscientists use to measure particular executive functions in lab settings. The researchers examined the correlation between the results from the teacher-administered questionnaires and the scores from these computerized tasks, which are accurate but challenging to deliver to large groups.

The results showed that while the BASC-2 gives students a reasonable overview of their overall executive functioning, it lacks precise information about particular abilities like working memory and self-control. The BASC-3 was significantly more efficient, most likely as a result of the use of a unique and more targeted set of questions.

“The BASC-2 has been used extensively in archived datasets and contributes to academic research about how executive functions develop,” Ellefson adds. “It is really important to recognise that without modification, it is not an appropriate basis for making judgements about certain groups of children.”

The test is just one of many that are done in different countries to find out how smart children are getting. It’s critical to understand how these tools are forming their foundational concept of “typical” growth, according to Zonneveld. 

“If they are based on mostly white populations from affluent suburbs, they won’t necessarily be as representative as we might hope.”

The findings of the study were published in the journal Developmental Science.

Source: 10.1111/desc.13319

Image Credit: Getty

You were reading: There’s A Big Problem With Tools Used In The US To Test Kids’ Thinking Skills – Says Study