Ei-iE

This blog post presents two major critiques of the SABER-Teachers domain: The World Bank’s inconsistent use of empirical evidence in order to promote policies that reflect its ideological biases related to teachers and the teaching profession, and the way the SABER-Teachers domain framework paper serves to de-professionalize teachers.

A key focus of the SABER-Teachers domain is determining the role of teacher policies in promoting teacher effectiveness to improve student learning, emphasizing that effective teacher policies are those that “are most closely aligned with improved student performance” ( Vegas et al. 2013, p. 7).

The above-quoted framework developed for analyzing teacher policies in the SABER-Teachers domain consists of two core components, “policy mapping” and “policy guidance.” The purpose of the policy mapping component is to provide a descriptive answer to the question “What do education systems do in terms of teacher policies?” ( Vegas et al. 2013, p. 7). This was done by identifying “a number of teacher policy dimensions that are central to producing a comprehensive descriptive account of the policies education systems put in place to manage their teacher force” and then developing a set of categories, or “policy dimensions,” from these ( p. 7). The results of the policy mapping process were used to inform the second component, policy guidance. This policy guidance component consists of the policy recommendations that are actually included in the rubric for the SABER-Teachers domain; these are the Teacher Policy Goals.

Through this process, the SABER-Teachers domain mapped 10 key policy dimensions and developed 8 Teacher Policy Goals. The 10 key dimensions of teacher policies include ( Vegas et al. 2013, p. 12):

1. requirements for entering and remaining in the teaching profession;
2. initial teacher preparation;
3. recruitment and employment;
4. teacher workloads and autonomy;
5. professional development;
6. compensation: salary and non-salary benefits;
7. retirement rules and benefits;
8. monitoring and evaluation of teacher quality;
9. teacher representation and voice; and
10. school leadership.

The eight policy goals for SABER-Teachers (Figure 1) that emerged from this policy mapping process are ( p. 24):

1. setting clear expectations for teachers;
2. attracting the best into teaching;
3. preparing teachers with useful training and experience;
4. matching teachers’ skills with students’ needs;
5. leading teachers with strong principals;
6. monitoring teaching and learning;
7. supporting teachers to improve instruction; and
8. motivating teachers to perform.

Figure 1. SABER-Teachers 8 Teacher Policy Goals ( Vegas et al. 2013, p. 24)

Though there is much that could be criticized about the SABER domain of Teachers, this blog post focuses on two primary critiques. First, drawing upon Steiner-Khamsi’s façades of precision and rationality( as outlined in the first blog post in this series), the decisions about which policy indicators to include as Teacher Policy Goals and as policy recommendations seem to be based just as much on the World Bank’s ideological biases as they actually are on the available research findings or the policies and practices of “best-performing” countries. Secondly, the content of the framework, the rhetoric used within the framework, and the portrayal of teachers in the framework promote the de-professionalization of teachers, and this is counterproductive to what the World Bank claims to be the ultimate goal of SABER—improving learning outcomes for all ( 2013).

Turning to the first criticism, the World Bank asserts throughout the SABER documents that all of their policy recommendations are supported by the available research; they emphasize that policies are excluded if “there is no empirical basis” for them, either because “evidence on a particular policy intervention is unclear or because top-performing education systems take different approaches to reach a particular objective” ( Vegas et al. 2013, p. 23). Yet, if you examine the policies more closely, it is easy to see how World Bank ideological biases trump empirical evidence in influencing the policy recommendations of SABER-Teachers. This is demonstrated by the inconsistent use of “unclear” or “contested” research findings to determine what policies to recommend, or not.

In other words, while some policies that are in opposition to the World Bank’s ideological biases are excluded on the grounds of unclear or contested research, there are other policies that are in line with World Bank ideology that are included in spite of unclear or contested research. Two examples within the SABER-Teachers domain demonstrate this clearly.

The first example involves Policy Dimension #9: Teacher Representation and Voice. As mentioned above, in the SABER-Teachers framework paper, ten policy dimensions were mapped in the process of developing the eight Teacher Policy Goals. The SABER-Teachers framework paper explains that, “developing a comprehensive description of the policies an education system puts in place to manage its teaching force is a necessary first step to assess the strength of these policies and their potential to improve education quality in a given system” ( Vegas et al. 2013, p. 12). Although SABER-Teachers recognizes that Teacher Representation and Voice is an important dimension of teacher policies by including it in the policy mapping process, this dimension was ultimately excluded from the eight Teacher Policy Goals. This exclusion is justified as follows:

The 8 Teacher Policy Goals exclude objectives and policies that countries might want to pursue to improve teacher effectiveness, but on which there is no empirical basis to make specific policy recommendations […] For example, there is no clear trend on whether (and if so, how) governments should engage with teacher organizations. [...] Evidence on the relationship between teacher organizations and student achievement is contested, and top-performing countries differ widely in how much they engage, to what extent they regulate, and how they organize teacher unions. Therefore, regulations related to teacher organizations were not included in the 8 Teacher Policy goals ( p. 23-24).

Because evidence is contested and top-performing countries take different approaches to teacher unions, this dimension of teacher policies is completely left out of the Policy Goals. Yet, a second example—Policy Goal #8: Motivating Teachers to Perform—demonstrates how this logic and reasoning is ignored in order to include a Policy Goal that is strongly in line with World Bank ideology—despite contested research and differing approaches by top-performing countries.

Policy Lever C under Teacher Policy Goal #8: Motivating Teachers to Perform asks “Is teacher compensation linked to performance?” ( Vegas et al. 2013, p. 37). In order for an education system in a particular country to be assessed as “Established” or “Advanced” for this policy goal, the country must be able to answer “yes” by having policies in place that link teacher compensation to performance.

Based on the Bank’s assertion that all policy recommendations given in the SABER framework papers must be those that are “most closely aligned with improved student performance […] based on the available research evidence to date,” if performance-based pay is included as a policy recommendation, then it must be supported by the available research evidence to date.

However, based on the available research evidence to date, this issue is actually quite contested. While there is some research supporting Pay-for-Performance policies, there is also a great deal of research contesting them ( Gratz, 2009; Yuan et al. 2013). In fact, this is such a contentious issue that even the Bank itself has questioned PFP policies (Breeding, Beteille, & Evans, 2019)!

For example, the abstract for a presentation given by World Bank representatives at the 2019 Annual Conference of the Comparative and International Education Society entitled “Teacher Pay-for-Performance (PFP) Systems: What works? Where? And how?” states:

The impact of evaluated PFP programs on student learning outcomes varies substantially. Across programs [58 reported outcomes of impacts on student test scores in 11 evaluations], effect sizes range in magnitude with a minimum effect of -0.08 SD and a maximum effect of a 0.32 SD increase in student test scores. The median reported effect size is a 0.056 SD increase. Most programs produce a mix of outcomes across different measures of test scores and subjects, most of which are substantively and statistically insignificant” (Breeding, Beteille, & Evans, 2019).

Here the Bank’s own research shows a lack of clear evidence on the effectiveness of PFP policies, yet these are still required within the SABER-Teachers framework for a country to be assessed highly.

Turning now to the second major critique of the SABER-Teachers domain, the framework paper and related documents contribute to the de-professionalization of teachers and the teaching profession.

The ongoing phenomenon of the de-professionalization of teachers and teaching has been well-documented (Filson, 1988; Milner, 2013). This is especially problematic because de-professionalization is associated with teacher shortages and lower teaching quality, and the converse of this is also true ( García & Weiss, 2019; Harris-Van Keuren, 2011).

In what ways does SABER de-professionalize teachers?

First, key indicators of a profession include decision-making authority, a strong voice in the shaping of relevant public policy, a large degree of control over the exercise of professional responsibilities, a high degree of autonomy, and relatively high compensation, among others (Hoyle, 1982; Ingersoll & Merrill, 2011).

As discussed above, by excluding the policy dimension of “Teacher Representation and Voice” from the Teacher Policy Goals and policy recommendations that the World Bank is promoting worldwide through SABER, the SABER-Teachers domain refrains from promoting “a strong voice in the shaping of relevant public policy,” one hallmark of a profession (Hoyle, 1982).

Furthermore, it is evident from the language used to present the 8 Teacher Policy Goals that, within the SABER-Teachers framework, teachers are not seen as a group of professionals deserving of decision-making authority, significant control over their professional responsibilities, and autonomy. For example, in Policy Goal #1, expectations should be set for teachers, rather than with or by teachers. Policy Goals #5, #6, and #7 present teachers as those who must be monitored, led, and motivated by others. The rhetoric of these goals implies that teachers are not to be trusted (as they must be monitored by others) and that they are not intrinsically motivated (as, again, they must be motivated by others).

To elaborate further, within the SABER-Teachers framework paper, the authors use 15 pages of text to introduce and explain the 8 Policy Goals. Within these 15 pages, the word “autonomy” is used only once, and when it is used, it is to promote policies that give autonomy to school principals, not teachers. Similarly, the word “authority” appears 21 times in these same 15 pages. Interestingly, these 21 mentions within the SABER Teachers framework recommend authority to be granted to the central government, the state government, the local government, schools, “a national educational authority,” education policy makers, school principals, state professional standards boards, management agents, and oversight agents. Policies recommending any type of authority to teachers are not mentioned even one time.

This is even more concerning given the fact that “Teacher Workloads and Autonomy” is included as one of the 10 teacher policy dimensions in the policy mapping component of the SABER-Teachers framework:

Awarding teachers a certain level of autonomy to carry out the tasks they are assigned is desirable for several reasons. Autonomy allows teachers to use their creativity, to innovate, to feel a sense of ownership for their work and thus be more motivated, and it enables teachers to adapt their teaching methods in order to better address the particular needs of each individual student ( Vegas et al. 2013, p. 16).

Just like Policy Dimension #9: Teacher Representation and Voice, the World Bank acknowledges that these are important policies, but refrains from including them in the 8 Teacher Policy Goals that they use to assess education systems worldwide. Thus, instead of empowering and promoting a strong and professional teacher workforce, the SABER-Teachers framework recommends policies that limit teachers’ agency and grants power to make decisions about teachers and teaching to others, further contributing to the ongoing de-professionalization of teaching.

To summarize, throughout the SABER-Teachers framework, World Bank ideology informs what the Bank recommends as “best practices” just as much as, if not more than, available research actually does; additionally, the SABER-Teachers Framework promotes the de-professionalization of teachers, further reflecting Bank ideology.

If the World Bank really wants to understand the policies that promote teacher effectiveness and student learning, perhaps they should begin by listening to teachers themselves.

--

The World Bank’s Systems Approach for Better Education Results (SABER) assesses education systems against “global best practices”, judging countries as latent, emerging, established or advanced in their policy maturity. Though SABER has been around since 2011, it has not yet received much critical scrutiny. This 4-part blog series analyses SABER, pointing out the fundamental flaws in both its conception and application.

References

Breeding, M., Beteille, T. & Evans, D. (2019, April). Teacher pay-for-performance (PFP)

systems: What works? Where? And how? Paper presented at the 63rd annual meeting of the Comparative and International Education Society, San Francisco, CA.

Filson, G. (1988). Ontario Teachers' Deprofessionalization and Proletarianization.

Comparative Education Review, 32(3), 298-317.

Harris-Van Keuren, C. (2011). Influencing the Status of Teaching in Central Asia. In

Silova, I. (Ed.) Globalization on the Margins, (p. 173-201). Information Age Publishing; Greenwich CT.

Hoyle, E. (1982). The Professionalization of Teachers: A Paradox, British Journal of

Educational Studies, 30(2), 161-171.

Yuan, K., Le, V.-N., McCaffrey, D. F., Marsh, J. A., Hamilton, L. S., Stecher, B. M., &

Springer, M. G. 2013. “Incentive pay programs do not affect teacher motivation or reported practices: Results from three randomized studies,” Educational Evaluation and Policy Analysis, 35(1), 3–22.