Analysis |

Ultra-Orthodox Test Scores Are Israel’s Most Secretive Statistic

Ultra-Orthodox students don’t take Education Ministry standardized tests. It says something about state oversight of their schools: There isn’t any

Meirav Arlosoroff
Send in e-mailSend in e-mail
An ultra-Orthodox Jewish man, works in his shop in the Israeli settlement of Beitar Illit in the occupied West Bank, June 19, 2019.
An ultra-Orthodox Jewish man, works in his shop in the Israeli settlement of Beitar Illit in the occupied West Bank, June 19, 2019. Credit: NIR ELIAS/ REUTERS
Meirav Arlosoroff

The expose that the Israel Defense Forces erred on – or faked – its draft statistics regarding the ultra-Orthodox raised a storm. To its credit, the IDF is taking responsibility. But this isn’t something you can say about the government ministries, which are no smaller than the IDF.

Take the Education Ministry, with its 58 billion shekel ($16.7 billion) annual budget – second only to the Defense Ministry. Its statistics regarding the ultra-Orthodox are even more deeply classified than military secrets, and as opposed to the IDF, the Education Ministry has been carrying on this way for years, and there’s no sign that anyone is taking responsibility for this.

Israel has several school systems – general Jewish public, national religious Jewish, ultra-Orthodox Jewish, and Arabic – and the performance of the ultra-Orthodox education system is apparently Israel’s most classified information. Try to find in the 134-page report summarizing Israel’s last Meitzav standardized test, from December 2018, the words “ultra-Orthodox.” Even if you try, you won’t find them. The statistics on ultra-Orthodox schools, few of which give their students the Meitzav test, are well classified, so that no one, horror, might see their actual scores.

Fortunately, two decades ago Israel made the mistake of subjecting its students to international standardized testing, the PISA exam, conducted by the OECD (the Organization of Economic Cooperation and Development, which counts 36 developed countries as members). The OECD has no secrets, and it doesn’t hide data. The report on the 2018 PISA scores, published a week ago, drew horrified responses in Israel due to the drop in students’ performance. It included data on ultra-Orthodox students who take the PISA exam – or more accurately, who don’t.

The last test, administered to 15-year-olds, included 22 ultra-Orthodox schools, including 15 girls’ high schools, six boys’ schools and one mixed school, which raises questions as to whether the latter is actually ultra-Orthodox. The girls were tested in keeping with standards, and did reasonably on the test.

The boys, not so much.

Falling behind.

First off, the boys who were tested constituted much less than a representative sample – as opposed to their 9% of the population, they represented only 2% of all test takers. Furthermore, the six schools that agreed to be tested were registered only after another 21 schools declined to participate. This suggests that these six schools are relatively liberal and open, and not representative of the high-school yeshivas where ultra-Orthodox boys typically learn.

The Education Ministry’s testing authority fully admits this, stating that these schools may be exceptions within the ultra-Orthodox education system, and may even teach subjects that are not typically taught, such as math and English. “Due to the exceptionality of these schools that took part in the test and their small numb er, these students cannot be considered a representative sample of ultra-Orthodox boys,” it states in its summary of the 2018 PISA results.

It declined to publish their scores. There’s reason to suspect that if ultra-Orthodox boys were properly represented in keeping with their proportion of the population, Israel would not have ranked 40th in the world on the PISA test – more like 50th or even 60th.

This suspicion is substantiated by the hidden Meitzav statistics. There, without OECD oversight keeping things transparent, the Education Ministry does everything it can to hide the numbers. In order to find the average for ultra-Orthodox students, you need to go over the list of schools, find every ultra-Orthodox institution in every town, and calculate the average scores manually – the ministry doesn’t make it readily available. Here, too, though, the figure is skewed and not representative.

The Meitzav test is conducted among fifth- and eighth-graders, in reading, math, science and English. Israeli schools administer the Meitzav test systematically, except for the ultra-Orthodox schools. These latter, by and large, do what they want. Up until 2015, no ultra-Orthodox students took the Meitzav. Then, in 2015, the Finance Ministry accountant general audited the Mayaan Torah Education Network of schools (now renamed Bnei Yosef), run by Israel’s ultra-Orthodox Shas party, and found improprieties. In order to get itself out of trouble, the school network agreed to accept more significant ministry oversight, including Meitzav testing for its students.

Since then, some 100 ultra-Orthodox schools have participated in Meitzav testing, most of them from the Bnei Yosef network. But only 5th-graders participate, and they take only the Hebrew and math portions of the test, and they also take a version of the test adapted to ultra-Orthodox culture. In quite a few classes, a very small number of students are tested – which raises suspicions that the students who participated in the test are being specially selected. In any case, the large majority of ultra-Orthodox schools – including United Torah Judaism’s independent education network, and the so-called independent network – give themselves exemptions and don’t take the test. The ultra-Orthodox Meitzav score is not representative in the slightest.

According to the Education Ministry, since only some 100 ultra-Orthodox schools take the Meitzav test every year, and only from within one education network, there’s no reason to publish ultra-Orthodox scores – the figure is not representative and not based on a proper sample. So thus, it hides it.

So in order to figure out what’s going on inside ultra-Orthodox schools, we need to rely on outside research. Dr. Neri Horowitz of the Haredi Institute for Public Affairs examined the Meitzav scores of the 100 schools that participated. He found shocking results: Compared against nationwide scores, the girls scored within the 20th or 30th percentile, while the boys scored in the bottom 10th percentile. A similar review conducted by TheMarker’s Lior Dattel found that some 72% of ultra-Orthodox students failed the math portion of the exam.

This gives us a hint about what would happen were ultra-Orthodox boys to participate in the PISA exams - just how low Israel would rank in the international comparison. but Horowitz says that even though the community’s independent education network exempts itself from the Meitzav and the PISA exams, it doesn’t mean the education is necessarily at a lower level. It may well be that the Ashkenazi ultra-Orthodox schools have a higher level of instruction than Shas’ schools, and they simply don’t want the state interfering in their schools. But due to the lack of data, we can only guess how low – or not – their level is.

Ultimately, the Education Ministry and the state of Israel have no idea what quality education ultra-Orthodox boys are receiving in elementary, middle or high school. Regarding the girls, we have no idea about the elementary education – the Meitzav group is too small – but we have an encouraging picture of high school studies, as the PISA results are more or less on par with Israel’s average.

The fact that ultra-Orthodox students don’t take Education Ministry standardized tests says something about state oversight into ultra-Orthodox schools: There isn’t any. Israel has about 230,000 ultra-Orthodox pupils, and they attend schools that receive some 2.7 billion shekels in state funding every year. What is this money being used for, and to what extent is it helping students acquire skills for the modern world - through studies in core subjects? No one knows. Certainly not the Education Ministry, which takes care not to ask hard questions.

Let’s note that the two largest ultra-Orthodox education networks – Shas’ and the independent network - receive generous funding, in that they are fully funded. In terms of their right to public funding, ultra-Orthodox students are equal to all other Israeli students. But in terms of meeting commitments – oversight, standardized testing and core subjects – they’re exempt. That’s what happens when the political system is busy trying to satisfy the ultra-Orthodox parties, and it lets them sacrifice 230,000 students to narrow rabbinic interests.

Comments