| Croll and Lee report |
[Mar. 17th, 2008|10:50 pm] |
In deciding to scrap Early French Immersion in New Brunswick schools, Education Minister Kelly Lamrock made use of a document examining French Second Language instruction put together by Patricia Lee and Jim Croll. This report is statistically flawed, and would not stand up to scrutiny by students in any introductory statistics course. The fact that a report of this nature is being used to dictate government policy is outrageous. Citizens of New Brunswick cannot tolerate this.
Here are some of the more grievous errors in the Croll and Lee report.
*Drop-out rates are calculated erroneously. For example, Croll and Lee take one year, and compare the number of students in grades 1 through 5 EFI. For example, in 1997 there were 1,822 Grade 1 students in EFI, and 1,201 students in Grade 5 EFI. They then say that the net change is 621 students, and imply that the drop-out rate for EFI that year is 34.1%. The problem is that they are comparing two different classes here: obviously students who are in Grade 1 in 1997 are not in Grade 5 in the same year! Unless exactly the same number of students register in EFI every single year (which obviously doesn't happen), comparing classes like this gives no real information. Dropout rates for a class can only be calculated by following that same class through the years. When this is done correctly, the dropout rate is substantially smaller.
*A student is determined to have "dropped out" of the EFI program if he/she did not complete an oral proficiency exam in Grade 12 and receive a bilingualism certificate. This exam is not mandatory for students, and so there is a good chance that many students who could have scored well chose not to take the exam. Also, students who may have completed French Immersion in a province other than New Brunswick are dropouts for the purposes of this study. In addition, students attending smaller schools have fewer options when they reach the upper grades, and are not offered math, science, etc. in French. In order to prepare for university, these students may be forced out of the program in their final years. They may be proficient in French, but because they could not take Grade 12 Language Arts, they were not given the option of an oral proficiency exam. Therefore, though they may be bilingual, they are failures as far as this report is concerned.
*Croll and Lee take Grade 2 Provincial Reading Assessment results from 2006-7 and imply that because Core students had a mean score of 69.29 and EFI students a mean score of 68.23, that this indicates that students in the Core program are performing better. However, nowhere do they do a statistical test that proves the difference in scores is significant. In fact, it is not. In addition, the proportion of EFI students who attain the standards set by the province is actually higher than the proportion of Core students that do.
Croll and Lee then compare Grade 4 results for the same year (2006-7), and the difference is still slight: EFI: 66.61 versus Core: 68.63. This difference is enough to be statistically significant due to the large sample size they used. However, they imply that this means that the differences in performance increase with age. In this, they make the same mistake they made earlier: the kids writing the Grade 2 test in 2006-7 did not write the Grade 4 test that same year, so you can't compare the results as if they had.
These are just a few of the major problems with the report. Many more can be found at http://hamlit2008.googlepages.com/
Many thanks to Dr. Diana J. Hamilton at Mount Allison and Dr. Matthew K. Litvak at UNBSJ who have taken the time to rip this report apart. Most of what I have written here I've shamelessly taken from them.
The future of the children of New Brunswick should not be decided by such a flawed report. |
|
|
![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small}
macbethann's journal
