Reliability and Utility of Measures of Academic Progress (MAP) Test and Its Impact on Students’ Learning: A Study in an American Curriculum School in Dubai موثوقيّة وفائدة امتحانات التقّدم األكاديمي (MAP) ، وتأثيرها على تعلّم الطّّلب: تّم البحث في مدرسة تابعة للمنهاج األميركي في دبّي. by RANA MOHAMAD EL DOR Dissertation submitted in fulfilment of the requirements for the degree of MASTER OF EDUCATION at The British University in Dubai October 2019 DECLARATION I warrant that the content of this research in the direct result of my own work and that any use made in it of published or unpublished copyright material falls within the limits permitted by international copyright conventions. I understand that a copy of my research will be deposited in the University Library for permanent retention. I hereby agree that the material mentioned above for which I am author and copyright holder may be copied and distributed by the British University in Dubai for the purpose of research, private study or education and that the British University in Dubai may recover from the purchasers the costs incurred in such copying and distribution, where appropriate. I understand that the British University in Dubai may make a digital copy available in the institutional repository. I understand that I may apply to the University to retain the right to withhold or to restrict access to my thesis for a period which shall not normally exceed four calendar years from the congregation at which the degree is conferred, the length of the period to be specified in the application, together with the precise reasons for making that application. Rana El Dor Student Signature COPYRIGHT AND INFORMATION TO USERS The author whose copyright is declared on the title page of the work has granted to the British University in Dubai the right to lend his/her research work to users of its library and to make partial or single copies for educational and research use. The author has also granted permission to the University to keep or make a digital copy for similar use and for the purpose of preservation of the work digitally. Multiple copying of this work for scholarly purposes may be granted by either the author, the Registrar, or the Dean only. Copying for financial gain shall only be allowed with the author’s express permission. Any use of this work in whole or in part shall respect the moral rights of the author to be acknowledged and to reflect in good faith and without detriment the meaning of the content, and the original authorship. ABSTRACT Education has been given a major focus by educators and educational authorities in the United Arab Emirates (UAE) since the launch of the National Agenda (NA) by His Highness Sheikh Mohammad Bin Rashid Al Maktoum in 2010. Accordingly, the education reform started to pave its way towards internationalism and national emulousness. Educational authorities were established to contribute to this reform. All Dubai private schools are supervised and followed up closely and regularly by the Knowledge and Human Development Authority (KHDA) to ensure they are on the right track towards the change in education and towards meeting the targets of the National Agenda. This study aimed at investigating the reliability and utility of Measures of Academic Progress (MAP) testing and its impact on students’ learning. MAP testing was mandated by the KHDA for all American curriculum schools based on the National Agenda Parameter (NAP). MAP is used to benchmark students’ performance in schools following the American curriculum as it is aligned with the American curriculum standards. The researcher used actual MAP data from students’ scores, conducted surveys to students, and led interviews with both teachers and leaders in order to investigate how the school is using and benefiting from MAP testing and to test the impact this standardized test has on students’ learning. The study concluded that MAP testing allows for students’ academic progress. However, due to not utilizing MAP effectively and authentically in terms of modifying instructions to meet the needs of students, the test was not reliable and was found to have no impact on students’ learning. Also, findings emphasized the need for professional development for teachers on MAP testing and its resources in order to equip them with the needed strategies to benefit from this well designed and beneficial standardized assessment. Keywords: MAP Testing, Reliability, Utility, Impact, Curriculum Modification, Instructional Adaptation, Use of Data ملّخص البحث منذ إطالق األجندة الوطنيّة من قِبل صاحب السمّو الشيخ محّمد بن راشد آل مكتوم، حظَي قطاع التربية والتعليم باهتماٍم ودعٍم كبيرين من قبل الجهات التربوية وجميع المعنيين بالشأن األكاديمي، وبدأت عمليّة إعادة هيكلة هذا القطاع لكي يصبح في مصاف العالميّة، حيث تّم إنشاء العديد من الهيئات التربوية، من ضمنها هيئة المعرفة والتنمية )KHDA( والتي ساهمت وال تزال في نهضة التربية والتعليم في معظم مدارس دبي الخاصة، والتي تُعنى إلى حٍد بعيد بمراقبة العمليّة التعليميّة، وتوجيهها نحو المسار الصحيح، وذلك من أجل تحقيق أهداف األجندة الوطنيّة. يهدف هذا البحث إلى دراسة مدى صّحة وفائدة امتحانات ال MAP )قياس التقدم األكاديمي لدى الطالب( التي تُعتمد كمقياٍس عالميٍ لتحصيل وتقدّم الطالب، والتي تّم فرض اعتمادها من قبل هيئة المعرفة والتنمية على جميع المدارس ذات المنهاج األميركي، تطبيقاً لبنود األجندة الوطنية. وقد استخدمت الباحثة في هذه الدراسة النتائج الفعلية للطلبة، كما اعتمدت على العديد من استبيانات الرأي الخاصة بهم، وأجرت مقابالٍت مع عدٍد من المعلمين والمسؤولين التربويين في المدرسة، وذلك لمعرفة مدى فائدة هذه االمتحانات وأثرها في عمليّة التعلّم لديهم. ،MAP وقد توّصلت هذه الدراسة إلى استنتاجٍ مفادُه أنّه وعلى الرغم من فعالية امتحانات ال ومساهمتها في التطّور األكاديمي للطلبة، إال أّن تطبيقها الخاطئ في المدرسة المعنية، وعدم ِدّ إلى أي تطوير استخدام نتائجها في تعديل المناهج بما يتوافق مع احتياجات الطالب لديها لم يُؤ أو أّي تأثير إيجابي على مستوى التعلّم. كما أظهرت نتائج البحث احتياج المعلّمين إلى دورات تدريبيّة في االستراتيجيات الخاصة بتطبيق هذه االمتحانات، وذلك لالستفادة على أفضل وجه من فوائد هذا التقييم المقياسي. DEDICATION To my amazing husband for his never-ending support at all times To my precious mom for her care and guidance To my dear dad for his love and motivation To my son and my daughter who have been my strength in the challenging days To my most valuable sisters and brothers for their encouragement To my dearest friend and honest mirror, Shadi, for his ongoing support and inspiration ACKNOWLEDGMENTS I am extending my deepest sense of gratitude to Allah Almighty for granting me the strength and courage to go through this journey. Thanks to my teacher and leader, Mr. Nafez El Hayek, for inspiring and motivating me to take the right direction and achieve my goals. I am sincerely grateful to Dr. Christopher Hill, my role model. His support, constructive and helpful feedback, and valuable suggestions have been of great help to me to finish my dissertation and overcome all challenges. He has also been an excellent support and mentor throughout my two-year Masters Program. I have gained a broad and an excellent academic and personal experience due to his unique approaches and instructions. Many thanks and much appreciation to the British University in Dubai for providing me quality education and for giving me the opportunity to learn and gain a professional experience in a safe and motivational environment. TABLE OF CONTENTS I. CHAPTER 1: INTRODUCTION 1 1.1. Chapter Overview 1 1.2. Introduction 1 1.3. Background of the Study 2 1.4. Statement of the Problem 3 1.5. Purpose of the Study and Research Questions 4 1.6. Structure of the Dissertation 5 II. CHAPTER 2: LITERATURE REVIEW 7 2.1. Chapter Overview 7 2.2. History of Educational Reform in the UAE 7 2.3. What Is MAP Testing in Specific, and What Are Benchmark Tests in General? 8 2.4. Use of MAP Testing to Modify Curriculum and Inform Instruction 9 2.5. MAP Testing and the KHDA 10 2.6. Positive and Negative Aspects of MAP Testing 11 2.7. Importance of Benchmark and MAP Tests 12 2.8. Purpose of MAP Testing 14 2.9. Impact of MAP Testing on Students’ Learning in School and Beyond (University and College) 15 2.10. Utility, Validity, and Reliability of MAP Testing 18 2.11. Psychological Impact of Testing on Students and Teachers 20 2.12. Summary 21 III. Chapter 3: Methodology 22 3.1. Chapter Overview 22 3.2. Research Design 22 I 3.3. Population and Sampling 24 3.4. Data Collection and Instrumentation 26 3.4.1. Interviews 26 3.4.2. Classroom Observation 27 3.4.3. Students’ Survey 28 3.5. Data Analysis 29 3.5.1. Quantitative Data Analysis 29 3.5.2. Qualitative Data Analysis 30 3.6. Site and Accessibility 30 3.7. Key Research Considerations 31 3.7.1. Ethical Consideration 31 3.7.2. Triangulation 32 3.7.3. Validity 33 3.7.4. Reliability 33 3.7.5. Role of the Researcher 34 3.8. Research Limitations and Scope for Further Research 34 3.9. Summary 35 IV. Chapter 4: Results, Data Analysis, and Findings 36 4.1. Chapter Overview 36 4.2. Analysis of Quantitative Data 36 4.2.1. Students’ Surveys 36 4.2.2. Analysis of MAP Results 38 4.2.3. Class Observation Reports 41 4.3. Analysis of Qualitative Data 44 4.3.1. The Analysis of the Open-ended Questions in the Students’ Survey 44 4.3.2. Analysis of Teachers’ Interviews 48 4.3.2. Analysis of Leaders’ Interviews 53 V. CHAPTER 5: DISCUSSION 57 5.1. Chapter Overview 57 5.2. Discussion of the Findings 57 5.2.1. Building a Culture of In-depth Understanding of the Test and Its Purposes 58 5.2.2. Effective Use of MAP Data to Modify Curriculum and Adapt Instruction 59 5.2.3. Use of MAP Testing to Impact Students’ Learning 60 5.2.4. Professional Development for Teachers 61 5.2.5. Availability of Resources and Testing Atmosphere 62 II 5.3. Summary of the Chapter 63 VI. CHAPTER 6: CONCLUSION 64 6.1. Chapter Overview 64 6.2 Implications of the Study 64 6.3. Key Findings and Conclusion 65 6.4. Limitations 66 6.5. Recommendations 67 REFERENCES 70 APPENDICES 75 Appendix A: KHDA Standards and Indicators to Measure MAP and Other External Assessments 75 Appendix B: Classroom Observation Rubrics 76 Appendix C: Students’ Survey Questions 77 Appendix D: Teachers’ Interview Questions 78 Appendix E: Leaders’ Interview Questions 79 Appendix F: Samples of Transcribed Interview 80 Sample 1 80 Sample 2 81 Sample 3 82 III LIST OF ACRONYMS UAE United Arab Emirates NA National Agenda KHDA Knowledge and Human Development Authority MAP Measures of Academic Progress NAP National Agenda Parameter NWEA Northwest Evaluation Association USA United States of America NEASC New England Association for Schools and Colleges SLT Senior Leadership Team MLT Middle Leadership Team IV LIST OF TABLES Table 1. Numbers of Students’ Survey Participants ..................................................................... 36 Table 2. Students’ Suggested Resolutions to the Challenges ....................................................... 48 V LIST OF FIGURES Figure 1. Percentages of Students Participating in the Survey ..................................................... 37 Figure 2 Students' Responses ........................................................................................................ 37 Figure 3. MAP Attainment Analysis ............................................................................................ 40 Figure 4. MAP Progress Analysis ................................................................................................. 41 Figure 5. Math Teachers Class Observation Results .................................................................... 42 Figure 6.Science Teachers Class Observation Results ................................................................. 42 Figure 7 English Teachers Class Observation Results . ............................................................... 43 Figure 8. Testing Atmosphere Analysis........................................................................................ 63 VI VII I. Chapter 1: Introduction 1.1. Chapter Overview This chapter demonstrates an overview of the research study and the purpose behind conducting the investigation. It includes information about the background of the study and states the problem. The research questions are indicated, and the methodology adopted for this study is also summarized. The chapter also highlights the structure of the dissertation. 1.2. Introduction Private schools in Dubai were put on track towards improving the education sector in the United Arab Emirates (UAE) as targeted in the National Agenda (NA). The Knowledge and Human Development Authority (KHDA) constructed and published all needed documents that guided the schools towards meeting expectations in academic provisions from teaching and learning to curriculum and to assessment. Assessment was among the targeted standards that align with the NA targets. The inspection framework includes all the standards and indicators for evaluation. The last framework was published in 2015-2016. Many schools showed improvement in meeting expectations and improved their rating from weak to acceptable and from acceptable to good or from good to very good and so on so forth. The study will be beneficial to the school where the researcher investigated the reliability, utility, and impact of Measures of Academic Progress (MAP) testing. MAP is a benchmark test that measures students’ progress and provides students with their results immediately after the test (Talmage, 2017). The School Management will be able to understand what goes well in MAP and what needs improvement; accordingly, plans will be put in place to bridge the gaps and accelerate 1 the progress in this area to improve results. The study will also support other schools that face similar challenges and aim at showing progress in MAP and in the overall rating of the school. 1.3. Background of the Study External assessments and benchmark tests play a vital role in the school’s evaluation based on the KHDA Inspection Framework. The impact of those assessments on students’ learning is evaluated in standards 1, 3, 4, and 6 in the Framework. Progress and meeting targets in those assessments affect the overall evaluation of the school positively or negatively. Accordingly, improving students’ results in those assessments should be granted full attention. Additionally, there should be accurate planning and professional implementation in order to improve students’ scores and consequently improve the school’s evaluation. The need to meet the target in all external assessments in all schools in the UAE and in MAP in American curriculum schools in Dubai has led schools to set their plans and develop effective processes. American curriculum schools put a major focus on MAP testing as it is linked with different standards that affect the school’s rating as mentioned earlier in the introduction. Although support and guidance were provided to schools by the KHDA in this regard, schools still needed to put extra efforts to find the strategies and resources that allow for effective implementation and positive results. Implementing MAP testing also requires schools to make some investments that help them meet the desired targets. Investments should target resources, curriculum, and teaching and learning. 2 1.4. Statement of the Problem Assessments have been given a major focus in the UAE since the launch of the National Agenda in 2010. The Education Sector was among the sectors in the National Agenda; it was given a great attention by the public and private sectors to ensure all of its targets get met. Benchmark tests and external assessments are considered the core targets of the schools in the UAE to work in response to the targets set by his Highness Mohammad Bin Rashid in the NA. Since the KHDA is the higher authority for all private schools in Dubai, they provided schools with the framework that includes standards which reinforce and highlight the targets in the Education Sector in the NA. Students’ attainment and progress in benchmark tests and external assessments are measured and evaluated against specific criteria set by the KHDA in the National Agenda parameter (NAP) that was constructed in response to the National Agenda targets, especially those that address assessments. The researcher does not tend to investigate the reasons behind mandating MAP testing in American curriculum schools; however, she aims at highlighting the key findings that were the result of the key problems that brought the purpose of the study. Problem is stated below: 1. Meeting expectations in MAP has been challenging for most American curriculum schools in Dubai. Despite the progress schools are showing in MAP results, this progress remains slow and inconsistent, which suspects the reliability of the test. 2. Schools are required by the KHDA to achieve in benchmark testing based on the curricula they adopt. All American curriculum schools are requested to present data about their students’ performance in MAP Tests. Benchmark test results are linked to different standards and indicators. Thus, failing to meet the expected results in attainment and progress will lead to low performance of the school overall. 3 The NAP was created by the KHDA for schools to organize and undergo assessments for their students. Moreover, The Schools’ Supplement is formed by the KHDA to guide schools towards evaluating different aspects of their schools focusing on the ones that are targeted in the NA. Nevertheless, schools needed one on one support in their schools based on their weaknesses or challenges. They needed best fit strategies as what works for one school does not work for others. Despite the fact that many schools showed improvement in their MAP results and in the overall performance, the picture in Dubai schools is still not positive and the results are not meeting the expectations. This study investigates the specific aspects and areas that might be hindering the improvement in MAP results based on the practices in the school where the study took place. The selected school has been undergone in-depth study by the researcher to examine best practices in MAP testing and to depict what is still hindering the expected growth. 1.5. Purpose of the Study and Research Questions This dissertation aims at investigating the reliability and utility of MAP testing and its impact on students’ learning in an American curriculum school in Dubai in order to uncover what needs to be done to move forward in the right track. The study highlights the practices of teachers and leaders and the views of students. It focuses on use of data to inform curriculum and instruction. The study aims at answering the following questions: a. How reliable are MAP Tests in reflecting students’ achievement, and how effective are they in curriculum modification and instruction adaptation? b. How does effective utilization of MAP testing impact students’ learning? c. What is the impact of MAP tests on students’ learning? 4 Relevant literature has proven that effective utilization of MAP testing to inform curriculum and instruction has a positive impact on students’ learning and their academic performance. That was also proven in the study at hand; due to the ineffective use of MAP data in the studied school, students’ learning as well as their growth remained inconsistent. Moreover, investigating the effectiveness of this benchmark test would serve all educational authorities and providers as assessment plays a vital role in the life of the school due to its relevance to measuring different aspects and areas in the school or any other educational institutions. Assessments, especially those that are linked with the learning outcomes, are used as effective tools to measure the usefulness of the curriculum and the efficacy of teaching and learning (Jabbarifar, 2009). 1.6. Structure of the Dissertation The dissertation at hand consists of six chapters.  The first chapter introduces the topic and gives an overview of the study and its background. It also highlights the significance and purpose of the study and indicates the research questions.  The second chapter is devoted to the Literature Review that incorporates conceptual and empirical frameworks relevant to the study. It also sheds light on the background and the understanding of benchmark testing in general and MAP testing in particular. The chapter also displays definitions, theories, and analytical studies about reliability, utility, and impact of MAP testing; importance of MAP testing in regards to National Agenda targets and to other academic standards is also emphasized. 5  Additionally, chapter three constitutes the methodology used by the researcher to undergo the study. It demonstrates the research design, sampling and population, instrumentation, different tools of data analysis, and site and accessibility. The chapter also discusses ethical considerations, the role of the researcher, and the research limitations.  Chapter four presents the data analysis and results. It also provides critical analysis, discussion, and interpretation of data and highlights the findings.  The discussion of findings is explicated in chapter five.  Finally, chapter six concludes the dissertation and showcases the implication and limitations of the research study. It also provides recommendations and scope for further research in the future. 6 II. Chapter 2: Literature Review 2.1. Chapter Overview The Literature Review chapter includes reference to conceptual as well as empirical frameworks. The conceptual framework incorporates the definitions of the key words and terminologies used in the study at hand. MAP testing and Benchmark tests are defined, and their characteristics are demonstrated. The empirical framework comprises literature about studies conducted to investigate the use of MAP testing to modify instruction and inform planning. Also, studies investigating the positive and negative aspects of MAP are also highlighted. Moreover, this chapter incorporates facts about the history of education in the UAE in terms of assessment targets in relation to KHDA expectations. Additionally, relevant literature about the importance, utility, validity, and reliability of MAP is showcased in this chapter. 2.2. History of Educational Reform in the UAE Improving the standards of education was a chief emphasis in the UAE (Tabari, 2014). The UAE government has invested in human capacity through devoting more than a third of its budgeting to education reform which has been processed in response to the demands brought by globalization (Zahran, Pettaway, Waller, and Waller, 2016). In the education reform progress, each Emirate created its plans towards meeting the targets and achieving the UAE 2021 vision. The unified Inspection Framework was established to move education towards approaching the targets of the National Agenda and the 2021 vision in a unified outline and through unified strategies (KHDA, 2015). Assessments were among the targets in the education reform in the UAE. Educational authorities mandated assessments that would play an important role in improving education. Those 7 assessments aimed at comparing students’ results, continuously improving learning and instruction, and evaluating teaching and instruction (Popham, 2016). One of the targets of the UAE National Agenda 2021 is to provide first rate education system. Additionally, the NA aims at providing students the opportunity to score among the highest ranks in the world in science, mathematics, and reading assessments.1 The Knowledge and Human Development Authority (KHDA) was established in response to the 2006 decree as directed by his Highness Sheikh Mouhammad Bin Rashid Al Maktoum, aiming at improving the education sector in Dubai2. The private education in Dubai falls under the accountability of the KHDA in terms of promoting education growth and quality3. The KHDA has led different initiatives that focused on promoting positive education in Dubai through sharing best practices among private schools in Dubai4. 2.3. What Is MAP Testing in Specific, and What Are Benchmark Tests in General? Measures of Academic Progress (MAP) is one of the most broadly used programs; it includes benchmark assessments and training and online resources for administrators and teachers that 1 Uaecabinet.ae. (2019). National Agenda. [online] Available at: https://uaecabinet.ae/en/national-agenda [Accessed 2 Oct. 2019]. 2 En.wikipedia.org. (2019). Knowledge and Human Development Authority. [online] Available at: https://en.wikipedia.org/wiki/Knowledge_and_Human_Development_Authority [Accessed 2 Oct. 2019]. 3 Khda.gov.ae. (2019). About Us. [online] Available at: https://www.khda.gov.ae/en//about-us [Accessed 2 Oct. 2019]. 4 edarabia.com. (2019). Education Policy Reform in the UAE: Building Teacher Capacity. [online] Available at: https://www.edarabia.com/education-policy-reform-uae-teacher-capacity/ [Accessed 2 Oct. 2019]. 8 allow them to use assessment data to differentiate instruction (Cordray, Pion, Brandt, Molefe, and Toby, 2012). MAP is a benchmark/interim assessment that is scheduled to take place three or four times during an academic year. It is a computer-based adaptive test that assesses students in Reading, Language Usage, and Math (Brown, 2007). MAP test is a Common Core – aligned assessment, which does not require the use of extra time for practicing for the test as it is aligned with the curriculum standards (Lazarin, 2014). Like other benchmark assessments, it is utilized to measure students’ progress on a group of standards that are to be mastered by students towards reaching end-of-year learning outcomes (Cordray, Pion, Brandt, Molefe, and Toby, 2012). In addition to impartially and broadly measuring students’ performance, benchmark tests also hold teachers accountable. Using benchmark tests appropriately leads educators and policy makers to make suitable decisions about students’ learning and school programs (Walberg, 2012). Students who participate in benchmark tests in their school become more prepared for future goals, and teachers who are provided with professional development and training about the test and how it is used become encouraged and confident to respect the test and believe in it (Walberg, 2012). Benchmark tests assess students’ knowledge and skills against clear long-term learning expectations at certain times throughout the academic year and during a curriculum sequence. Moreover, benchmark assessments provide data that helps in informing policy, instructional development, and decision making at different levels (Herman, Osmundson, and Dietel, 2010). 2.4. Use of MAP Testing to Modify Curriculum and Inform Instruction Assessments are considered in education means for educational improvement. Assessments play an important role in guiding educators towards using test data to make decisions (Popham, 2016). Nevertheless, not only are assessments linked with learning, but they also focus on results and 9 liability (Vey, 2005). As assessments should aim at enhancing students’ learning (Ven, 2005), an adaptive test is a test that responds to students and adjusts to the learning level of each student. The level of difficulty changes up or down depending on the student’s answers to questions (NWEA, 2017). Northwest Evaluation Association (NWEA) MAP assessment was investigated as a predictive tool for the State Standardized test and for college readiness. This benchmark test is used to identify students’ academic needs for intervention (Brown, 2014). Two of the main purposes of MAP as mentioned by NWEA (2017) in their guide to parents are the use of assessment data by teachers to teach proper content for individual students based on their needs and to monitor the academic growth of each student over time. Teachers use reports released by NWEA to understand students’ academic level and their performance. The Learning Continuum report is one of the reports teachers receive immediately after the students’ completion of the MAP test. It provides instructors details about the student’s academic standing and what he/she is ready to study. Accordingly, the teacher modifies curriculum and instruction to cater to students’ needs and respond to their abilities. 2.5. MAP Testing and the KHDA Adopting benchmark assessments has been taken into account by states across the Midwest region to provide schools and teachers with predictive measures for enhancing instruction as well as state test results (Cordray, Pion, Brandt, Molefe, and Toby, 2012). However, in this school where the research took place, MAP assessment is used to measure students’ academic attainment and progress in line with the set of standards that align with the school’s adopted curriculum. As per the KHDA Framework and the National Agenda Parameter, MAP test is one of the mandated assessments in all American Curriculum Schools in Dubai. It is a benchmark test used to measure students’ attainment and progress according to criteria set by the KHDA. 10 In the KHDA Inspection Framework 2015-2016, the KHDA express commitment towards contributing to the UAE National Agenda target of achieving world-class education through maintaining a high-quality evaluation system. The Framework comprises a set of six standards, and each standard is detailed into indicators, elements, and descriptors (KHDA, 2015). Among the six standards and their indicators and elements, there are ones (Appendix A) that target and evaluate the external assessments including the use of MAP testing. 2.6. Positive and Negative Aspects of MAP Testing MAP testing is conducted three to four times a year. Doing a test more than once during an academic year has a positive impact on students’ learning as sitting for the test at multiple times will grant them more opportunities to show their actual level avoiding bad circumstances that might affect their results. Also, students’ attainment and progress will be measured consistently across the year. Students get acquainted with the type of test and feel more comfortable with the test when getting familiar with it (Benjamin and Pashler, 2015). MAP is an adaptive test which is designed to challenge students throughout (Benjamin and Pashler, 2015) and allow them to work to their full potential during the test. Moreover, students become motivated around their learning when they are involved with their teachers in setting their learning goals (NWEA, 2013) based on their MAP scores and targeted growth. Additionally, students’ motivation increases when teachers make it clear to students about how they will use their test scores to set learning goals together with them (NEWA, 2013). On the other hand, mandated tests put students and parents under pressure (Zimmer, 2018; Suval, 2018). Despite the fact that standardized and benchmark tests are of great importance to enhance teaching and learning, there are some concerns that have negative impacts on students’ well-being (Simpson, 2016) and should be taken into account. In addition to students’ concerns of the 11 additional stress they face from testing, teachers also worry about the instructional time they waste in favor to tests administration (Simpson, 2016). MAP testing is one target in the NA, and students’ scores have a great consideration by the higher authorities in evaluating and rating schools. Consequently, schools, including all stakeholders, feel overly burdened with the fact that students’ scores reflect the school’s rating which accordingly affect students’ engagement and interest in school and their views of their own capabilities (Simpson, 2016). The increasing demands for benchmark and standardized tests are leaving teachers in frustration and students in stress (Cox, 2015). MAP testing has a great impact on teaching and learning but has a negative impact on students’ psychology and emotions. So, it is mandated that educators put efforts to minimize the negative effects in policy as well as in practice (Simpson, 2016). According to Harris, Smith, and Harris (2012), standardized testing cannot measure students’ critical thinking and creativity as well as other qualities that students need to acquire. They believe achievement is not a score on a standardized test only; it also incorporates class participation, forms of students’ coursework, and forms of teachers’ professional development (Harris, Smith, and Harris, 2012). On top of that, one of the weaknesses of MAP is that it is intended to measure students’ academic level and monitor their progress with no connection to their age nor their grade level (Shaw, 2013). 2.7. Importance of Benchmark and MAP Tests Assessment is the bridge that allows teachers to know that instructional activities have resulted in the students’ projected learning (Wiliam, 2013). Schools’ commitment to data driven decision making drives schools to collect data from a variety of assessments (Wiliam, 2013). Benchmark assessments play a role in providing teachers and other stakeholders like parents, students, and 12 administrators with appropriate and needed information that allows for planning opportunities that support learning (Bergan, Bergan, and Burnham, 2009). Bergan, Bergan, and Burnham (2009) define the benchmark test as a locally adapted, district-wide assessment intended to measure the accomplishment of standards. The main and important purpose of benchmark assessment is providing information that is utilized to guide instruction and accordingly provide students with appropriate learning opportunities and then track students’ mastery of taught standards. Additionally, benchmark assessments identify the skills that students need to acquire to master standards and support the interventions that focus on allowing students to learn those skills (Bergan, Bergan, and Burnham, 2009). Furthermore, benchmark tests can be used as a predictive tool to measure to what extent students are ready to meet or not meet state standards (Bergan, Bergan, and Burnham, 2009). MAP is a computer-based adaptive benchmark test that provides data about students’ learning (Bjorklund-Young and Borkoski, 2016) immediately after test administration regardless of the grade level at which a student is performing (NWEA, 2013). It allows teachers to monitor students’ learning goals to meet summative goals measured against state summative tests; additionally, it provides information about students’ learning so that teachers get the opportunity to modify teaching accordingly (Bjorklund-Young and Borkoski, 2016). The advantage of NWEA MAP is that it gives students a test that is accurate and better suited to them (Cronin, Kingsbury, McCall, and Bowe, 2005) According to NWEA (2013), MAP is considered a trusted resource by hundreds of thousands of teachers, allowing them to measure individual student attainment and progress, predict student proficiency on high-risk/summative assessments, and compare students’ progress to those across the country. MAP also has accurate standards of measurement that allow for determining and 13 comparing progress of different students; accordingly, MAP has been utilized by different universities in the United States of America (USA). All teachers and administrators should ensure effective use of assessment data to inform instructions by setting a process for data analysis, identification of potential factors intruding students’ performance levels, and finally discussion of suggestions for curriculum planning and teaching and learning practices (Nisbet, 2005). 2.8. Purpose of MAP Testing The main purposes of benchmark tests are to help teachers select the best instructional strategies that support students’ learning based on their needs, to inform students of their strengths and weaknesses, and to provide parents with information about their kids’ competencies to get them to support their kids effectively (Nisbet, 2005). Additionally, another purpose of benchmark assessments is communicating learning expectations, planning curriculum and instruction, monitoring and gauging instructional and program efficiency, and envisaging future performance (Herman, Osmundson, and Dietel, 2010). MAP assessment is a benchmark test that shares the same above-mentioned purpose in addition to restructuring instructional planning and differentiating instruction (NWEA, 2014). Additionally, the purpose of MAP is to improve instruction. Research has not demonstrated a positive and statistically noteworthy impact of benchmark tests on improving students’ performance (Nelson, 2013). In a MAP test presentation on YouTube, the presenter stressed on the importance of MAP test to measure progress. This test is designed to figure out where a student is at. The questions adjust based on the actual level of the 14 student. According to the presenter, the purpose of MAP test is to tell if the student is below, at, or above grade level so he/she would receive support accordingly 5. NWEA MAP provides top ten reports that allow students to improve and show progress (Set, 2016). Those reports engage teachers, students, and parents in understanding the students’ data and demonstrating students’ performance and their readiness for learning (Set, 2016). One of the most important reports is the Learning Continuum Report which provides information that allows for differentiating instruction for all groups of students and accordingly involve students in their learning (NWEA, 2014). Moreover, teachers use the Learning Continuum to identify students’ readiness for learning and improvement and to pinpoint their needs as well. Teachers may also use the learning statements provided in the same report to drive instruction from a starting point that describes the skills and concepts that students are ready to learn (NWEA, 2014). 2.9. Impact of MAP Testing on Students’ Learning in School and Beyond (University and College) Use of valid assessment data allows for differentiating instruction. Studies examining the impact of MAP on students’ learning are rare 6. To demonstrate impact of MAP, NWEA shares stories of progress of schools in the USA and in other schools in different countries. One example is the Elmira City School District in New York. According to NWEA (2019), the school’s families and other stakeholders are excited about using MAP data. Accordingly, they all speak the same 5 YouTube. (2019). The MAP Test: How it works. [online] Available at: https://www.youtube.com/watch?v=jTaY3AdiB8M [Accessed 2 Oct. 2019]. 6 Ies.ed.gov. (2019). The Impact of the Measures of Academic Progress on Differentiated Instruction and Student Achievement. [online] Available at: https://ies.ed.gov/ncee/edlabs/projects/rct_245.asp?section=ALL [Accessed 2 Oct. 2019]. 15 language and understand what data means. Using MAP reports effectively, students showed academic progress and overcame the challenges in their education. Another example is the story of progress and growth of students in New Academy School in Dubai. Due to the effective implementation of MAP Growth, students were observed showing academic growth (NWEA, 2017). MAP Growth was used as a starting point assessment to measure students’ progress, and it was also used to measure the impact of instructional modifications and interventions on students’ learning. MAP Growth is also used by the school to improve the procedures of self-evaluation and performance review through using quantitative data from assessments to show students’ progress over time. On the other hand, the case study conducted by Cordray, Pion, Brandt, Molefe, and Toby (2012) concluded that implementing MAP did not have a significant impact on students’ reading achievement for fourth and fifth graders in Illinois. Additionally, the study concluded that teachers who implemented MAP did not likely differentiate instruction based on the information they received from MAP reports about individual students. Accordingly, there was no evidence provided by the study that use of MAP raises student learning (Bjorklund-Young and Borkoski, 2016). In his study about NWEA MAP Interim assessments, Finnerty (2018) found out that despite the evidence of growth students displayed, there was not clear evidence of continual or consistent growth within year and across year. The scores produced by MAP tests help in monitoring students’ growth from year to year in relation to scales of progressive curriculum (NWEA, 2014). However, according to Finnerty (2018), even if MAP has impact on students’ progress, the same assessments use up a substantial amount of instructional time. Additionally, Nelson (2013) believed that testing time has replaced students’ learning time and teachers’ professional development and learning and collaboration 16 time. Another negative impact of MAP on students is that standardized tests put students under pressure and stress (Nicholson, 2016). It is also found that this generation is considered to be the most tested cohort in history as students spend most of their time sitting for tests including the benchmark assessments (Radosh, 2018). Actually, schools with perfect MAP scores demand excellence. This was concluded by a school in the USA which had the perfect scores that ensured students’ readiness for their university study. As watched in the YouTube video, teachers explained that those scores were due to the doubled- up efforts in teaching ELA, math, and science7. In a YouTube Video, Free Speech TV show discussed the issue of boycotting MAP testing in Garfield High School. According to the teacher leading the speech, the test has negative impact on students’ learning as it blocks creativity and critical thinking; students focus on the test questions and are prevented from thinking out of the box. Additionally, the teacher believes that the test is unfair to the Special Education Needs students since their IEPS do not support interventions in this specific test. Besides, the teacher explained that the test does not fully align with the curriculum standards as students are asked questions that are not based on the tested grade level standards. Based on the aforementioned reasons, all teachers in Garfield High School refused the test and asked for boycotting it8. 7 YouTube. (2019). Schools with perfect MAP scores demand excellence. [online] Available at: https://www.youtube.com/watch?v=dVG4NOH1QYU [Accessed 2 Oct. 2019]. 8 YouTube. (2019). Seattle Teacher Uprising: Boycotting Standardized MAP Tests. [online] Available at: https://www.youtube.com/watch?v=N5ODEoqZZHs [Accessed 2 Oct. 2019]. 17 2.10. Utility, Validity, and Reliability of MAP Testing Tests are important tools to measure students’ learning; however, it is more important to greatly understand the utility and benefits of any tests especially benchmark ones as this will lead to resolving any concern over standardized testing (Benjamin and Pashler, 2015) and will also serve common goals of education. Utility or usefulness of a benchmark test is an important criterion that should be considered by schools and policy makers; it should be clear to all concerned how this selected test will help them achieve the targeted purposes (Herman, Osmundson, and Dietel, 2010). A test is considered reliable when a student sits for the test more than once or when he/she receives a different version but still receives congruent scores. The test gains validity when it is used for prediction purposes. Standardized tests are qualified for both validity and reliability. (Benjamin and Pashler, 2015). Reliability of benchmark assessments is measured against the consistency of information given by this assessment (Bergan, Bergan, and Burnham, 2009). Also, the validity of benchmark assessments is measured against the correlation between the benchmark tests in a certain state and the statewide test for the same state (Bergan, Bergan, and Burnham, 2009). Dedicated to its mission of “Partnering to Help All Kids Learn” (NWEA, 2013), Northwest Evaluation Association (NWEA) published MAP which allows teachers to tailor instruction to cater to the specific needs of students in their classrooms (NWEA, 2013). NWEA had their internal researchers conduct research to study its validity (Bjorklund-Young and Borkoski, 2016). Consequently, MAP validity remains problematic and questioned. Validity studies have three approaches: content validity, criterion-related validity, and construct validity (Bjorklund-Young and Borkoski, 2016). 18 1. Comparing MAP assessments to state standards establishes content validity. 2. Comparing MAP assessments to summative pre-test scores or practice materials establishes criterion-related validity. 3. Comparing student aligned summative test scores establishes construct validity. Moreover, MAP can be used as a formative assessment to predict the students’ performance on summative assessments which are state standardized or Common Core tests (Bjorklund-Young and Borkoski, 2016). Accordingly, in order for MAP tests to be considered valid, they should be able to assess state standards or be able to predict students’ scores on states standardized tests (Bjorklund-Young and Borkoski, 2016). However, MAP tests cannot be assessed if they are actually purposeful assessments to increase students’ performance as recent research on MAP could not offer information on its content validity or its construct validity (Bjorklund-Young and Borkoski, 2016). Reliability of a test is defined as the consistency of an assessment in measuring its projected target and the degree of accuracy of test scores (Herman, Osmundson, and Dietel, 2010). Measuring reliability of a test is done by experts in different ways; one way can be measuring if test results are consistent regardless of who scores the test or when or at what times of the day or year (Herman, Osmundson, and Dietel, 2010; Gawthrop, 2014). Benchmark tests are designed to assess students’ knowledge and skills objectively and in an unbiased manner; accordingly, the student’s score reflects an accurate measurement of ability and progress (Gawthrop, 2014). Test validity is indicated through accuracy of test results and through precisely measuring what the test is intending to measure (Gawthrop, 2014). 19 Many debates were made about the validity of MAP testing. Although NWEA assures that MAP tests are fully aligned with the curriculum standards, some critics agree that it only partially aligns (Shaw, 2013). Additionally, some teachers agree that MAP test tells teachers exactly if a student still needs any remedial classes based on their MAP progress test, while others confirm that MAP cannot tell teachers what specific questions students are not good at (Shaw, 2013). On the other hand, a research conducted by Brown (2007) concluded that there was no clear evidence of MAP as a predictive assessment tool. 2.11. Psychological Impact of Testing on Students and Teachers Teachers face a lot of stress due to the augmented use of testing in relation to making important decisions in education; this stress negatively affects teachers’ job satisfaction (Embse, Sandilos, Pendergast, and Mankin, 2016). According to a study conducted by Fulton (2016), students’ standardized test scores decreased as students’ anxiety increased; the study concluded that there is a negative relationship between test anxiety and students’ performance. Accordingly, since anxious students do not perform well, teachers must be provided with professional development to become acquainted with standardized tests, strategies of test taking, and relaxation procedures (Fulton, 2016). Consequently, students gain self-esteem, become motivated, and have increase in their test scores. Additionally, parental involvement helps decrease test anxiety; parents should also be given information about how to encourage their children and help them improve (Fulton, 2016). One of the reasons why students become anxious and stressed out is the pressure their teachers and leaders put on them to do well (Fulton, 2016) due to the fact that leaders and teachers themselves are put under stress since their school’s evaluation is linked with students’ performance 20 in benchmark and external tests. Hence, results should not be used by higher authorities to punish schools (Nelson, 2013) or to judge or evaluate schools. 2.12. Summary The Literature Review chapter defined the key words and terminologies in this study. MAP test and Benchmark/Standardized assessments were defined, and their characteristics were showcased. Empirical literature was demonstrated in this chapter to provide accurate and reliable information about the topic discussed in this research. Utility, reliability, and impact of MAP testing were focus areas in the investigated research studies. Findings and conclusions from other case studies or research were used as evidence for the accuracy of the information. Due to the shortage of research and literature about the topic of this dissertation, the researcher resorted to watching YouTube videos in which the topic was discussed on different occasions. This issue will be highlighted in the Limitation Section. All YouTube videos were transcribed by the researcher, and they were also referenced and cited. 21 III. Chapter 3: Methodology 3.1. Chapter Overview This chapter discusses the methodology of the research. Research methodology is a means to solve a research problem in an orderly manner; it is the discipline of studying how research is done logically (Kothari, 2004, p8). The chapter also entails the research design which is defined as the conceptual structure that is followed to conduct the research study. It includes the plan for the collection, measurement, and analysis of data (Kothari, 2004, p31). Methods of data collection and data analysis are discussed. The chapter at hand also includes the site and population of the study and the instrument of data collection. Additionally, ethical consideration, validity, reliability, and the establishment of trustworthiness as well as role of the researcher have been discoursed in this chapter of the research. Considering that research makes it possible to show progress in a field and overcome or solve problems (Rajasekar, Philominathan, and Chinnathambi, 2013), the researcher conducted the study at hand aiming at investigating the reliability, utility, and impact of MAP testing in a school in Dubai and at suggesting solutions to bridge the gaps and improve students’ scores. The researcher also aimed at reaching viable findings through in-depth studies of literature reviews and most importantly through close involvement of stakeholders working directly with MAP testing. Emphasizing the teachers’ voices and those of students’ and leaders’, surveys and interviews were conducted to ensure valid data are collected. 3.2. Research Design Research is looking logically and thoroughly for new and beneficial information (Rajasekar, Philominathan, and Chinnathambi, 2013). It aims at finding resolutions to scientific and social problems. Hidden truths are discovered through research (Rajasekar, Philominathan, and 22 Chinnathambi, 2013; Kothari, 2004, p2) in which knowledge is searched for (Kothari, 2004, p1) and becomes a need for digging into solutions and conclusions. Based on the aforementioned definition of research, the study at hand aimed at unveiling the truth about MAP testing in a school in Dubai, and it investigated the impact of MAP testing on students’ learning. The mixed methods approach was selected by the researcher due to the strength it gains from quantitative and qualitative data together and because it reduces the limitations of both approaches (Creswell, 2014). Additionally, the mixed methods approach allows for a comprehensive understanding of the research problems through comparing various perspectives collected from both qualitative and quantitative data (Creswell, 2014). The mixed methods research aims at obtaining the strengths and reducing the weaknesses of each method used separately in research (Johnson and Onwuegbuzie, 2004). Consequently, the researcher used a mixed approach of qualitative and quantitative data to ensure findings are based on reliable data and statistical analysis of data. According to Creswell (2014), mixed methods is combining or mixing qualitative and quantitative research and data in a research study. More specifically, the researcher used the convergent parallel mixed approach through which quantitative and qualitative data are merged so that the research problem is comprehensively analyzed (Creswell, 2014). The researcher collected both forms of data simultaneously and then integrated the information in the explanation of the results. Contradictions or incongruent findings are explained or further probed in this design. The researcher aimed at investigating how students and teachers see MAP testing and what impact it actually has on students’ learning based on analysis of its reliability, validity, and utility. Accordingly, the qualitative and quantitative methods were used to serve the purpose of the study. 23 Quantitative data is numerical and provides evidence that is assessed, and tables and graphs are used to present the results (Rajasekar, Philominathan, and Chinnathambi, 2013). However, the qualitative data is descriptive and uses words, aiming at getting the meanings and feelings and describing the situations (Rajasekar, Philominathan, and Chinnathambi, 2013). The quantitative data were retrieved from students’ scores in spring 2018 and were compared to fall and winter 2018 and then fall and winter 2019 to investigate the validity of MAP testing. Also, students’ views of MAP testing based on their own experience were translated into percentages of their agreement and disagreement with the indicators in the student survey. In this study, the researcher collected the qualitative data from teachers’ and leaders’ perspectives of the utility and impact of MAP testing and then compared the qualitative data with the quantitative results of the students’ surveys on their experience with MAP testing and its impact on their learning. Quantitative data from students’ MAP test results were also used in the comparison. The findings were accordingly compared. The researcher used the data collectively to show results and looked at both databases simultaneously. In the discussion section, the quantitative data were reported initially followed by the report on the qualitative data. The researcher gave more emphasis to the quantitative results, especially the students’ scores as they came from statistical reports that came from NWEA themselves. 3.3. Population and Sampling Various methods of sampling were used by the researcher in this study. Deliberate or purposive sampling was used; the population was deliberately selected (Kothari, 2004, p15) in a school in Dubai to represent the whole population involved in MAP testing. Classes that participated in the survey were selected based on recommendation from their teachers. Additionally, convenience sampling through which the researcher has easy access to the data as well as the population 24 (Kothari, 2004, p15) was evident in the research study. considering that the selected group can be studied intensively (Kothari, 2004, p15), the researcher applied the judgement sampling method by excluding grades three, four, and five from the completion of the survey. The main purpose was to investigate whether MAP testing is valid, reliable, and utilized effectively. It also aimed at examining its impact on students’ learning. Consequently, it was demanded that the researcher would explore the views of students, teachers and leaders about all aspects of MAP, focusing on the efforts made by the school to meet the expectations. The study took place during the last week of the term when students were preparing for their end of term exams. Accordingly, only two classes, one male section and one female section, from each grade level were given the survey. The sections were selected by the researcher based on their teachers’ recommendations. The recommended groups were the better performing students than others in the internal and external examinations, which made their views to be the most reliable. Students in grades three through nine sat for MAP in the academic year 2019/2020. All students in grades six to nine were surveyed about MAP testing; grades three, four, and five students were excluded from the survey because they were found very hesitant in expressing their views and talk about their experience, and that appeared in the results of the repeated survey. The Head of Section, their teachers, and the assessment coordinator concluded that their input might not be reliable. A pilot survey was conducted before the actual survey, and it was noticed that third and fourth graders tended to be hesitant about expressing their views. Accordingly, it was decided that they were to be excluded from the survey. Besides, senior students were also surveyed as they were the first group that sat for MAP testing in the school when they were in grade nine in the academic year 2015-2016. The purpose of the interview was to investigate impact of MAP testing on students’ learning and on their college readiness. Ten math teachers, nine science teachers, and eight English 25 teachers were interviewed during school hours. A permission was taken from their Heads of Departments and from the principal of the school. They were met for the interview according to a schedule that was set in an earlier agreement with them. The leaders of English, math, and science departments and the assessment coordinator were also interviewed, each one at a time. 3.4. Data Collection and Instrumentation Real life problems require the collection of appropriate data (Kothari, 2004, p17). Data may be collected though experiment or survey. Conduction of experiment data requires quantitative measurements, and survey data can be collected through observations, personal interviews, questionnaires, or through schedules (Kothari, 2004, p17). In this research, the researcher collected the data from students’ results and through the conduction of class observations, personal interviews with teachers, and students’ surveys. The researcher then combined all the methods in this study to ensure triangulation of data, allowing for accurate conclusions through depending on data from various methods (Rossman and Wilson, 1985). 3.4.1. Interviews For the qualitative data, the researcher conducted semi-structured interviews with a focus group to explore the attitudes, opinions, and perceptions of the interviewees (Kumar, 2011) about MAP testing through an open discussion between the researcher and the members of the focus group, the teachers, the leaders of English, Math, and Science, and the assessment coordinator. The interview was selected to ensure consistent information that allows for comparability of data (Kumar, 2011). All participants, teachers and leaders, were encouraged to talk about all issues relevant to MAP testing through answering open-ended questions in one-to-one interviews (Tong, Sainsbury, and Craig, 2007). 26 The researcher aimed at gathering as much information as possible from teachers and leaders in an interview session of 40 to 45 minutes. Accordingly, the research questions were designed to target utility and understanding of MAP by teachers and leaders. Leaders and teachers were interviewed in order to make a live discussion with each one of them and get into the details of needed information. The questions were open-ended designed by the researcher with the help of the assessment coordinator in the school to serve the study in terms of use of MAP assessment data to inform planning and instruction. All teachers and leaders were asked about their experience in MAP testing and their use of data to benefit from this standardized test. Besides, the questions targeted the challenges that students, teachers, and leaders face in the test and how the school responds to those challenges. Other questions investigated the positive and negative aspects of MAP testing as well the impact it has on students’ learning. A consent of agreement for the interview was signed by the interviewee. The interviews took place in the office of the Curriculum Coordinator in the school’s administration, and the interviewee arrived at the venue based on a schedule that was shared with them earlier. 3.4.2. Classroom Observation The researchers conducted class observations to collect data about use of MAP data to inform planning and vary instruction and instructional material. The observations aimed at spending enough time with students and teachers as participants in the study in order to gain much deeper, more valuable, and more precise information (Kumar, 2011). The researcher has recorded in the analysis section the observations in a narrative descriptive form through which the interactions were described in the researcher’s own words (Kumar, 2011). The researcher has followed the class observation rubrics (Appendix B) used in the school to measure teaching, learning, 27 attainment, and progress. The rubrics also align with the KHDA indicators of teaching and learning and students’ achievement. Four English teachers, four math teachers, and four science teachers were visited in the classes of the students who sit for MAP testing. Each teacher was visited twice in different classrooms before making the final evaluation. The purpose of the visit and the rubrics for evaluation were shared with the visited teachers, and their approval was granted. After each visit and before the final evaluation, teachers were met, and the report was shared with them for their approval. 3.4.3. Students’ Survey Quantitative data incorporate established structures and methods that promote validity and reliability. However, this is not the case for qualitative data due to the flexibility, freedom, and impulsiveness that the researcher is given in qualitative data collection (Kumar, 2011). The survey items and the interviews questions covered the issue of MAP testing utility and its impact on students and were able to ensure content validity through establishing a logical link with the main goal of the research (Kumar, 2011). All questions were primarily discussed with an assessment expert and a data analyst before being finalized and used as research methods. A research tool is considered reliable if it is consistent and steady and then predictable and credible (Kumar, 2011). The students’ surveys were conducted and analyzed twice to ensure reliability and credibility. The results remained the same after the repetition of the survey for grades six through nine and the senior students. However, results changed after repeating the survey for grades three through five, which led to a conclusion that students in this age might not be mature enough to control their emotions when responding to personal questions and indicators. Accordingly, those 28 groups of students were not included in the survey in order to maintain reliability of the survey as a research method. 3.5. Data Analysis In mixed methods research, data analysis is relevant to the type of research strategy selected by the researcher for the procedures (Creswell, 2009). The researcher collected qualitative and quantitative data concurrently and analyzed both data bases by changing the qualitative data into counts and comparing them with the descriptive quantitative data, creating an integration between the two databases (Creswell, 2009). Quantitative data received from students’ results in MAP testing during the academic years 2017-2018 and 2018-2019 was used to measure students’ attainment and progress overtime. Other data from surveys, interviews, and observations were collected to gain strong findings and ensure triangulation. 3.5.1. Quantitative Data Analysis Quantitative data was analyzed using descriptive statistics. Descriptive statistics summarized the academic performance of students in MAP. Attainment in MAP was demonstrated in percentages of students scoring in the 41st percentile or the 61st percentile, which are the criteria set by the KHDA for rating students’ attainment in benchmark tests. Students’ progress was measured against students’ growth from spring 2018 to spring 2019. Besides, percentages of students’ agreement and disagreement with the survey indicators about the topic of the study were summarized in tables and graphs. Additionally, the researcher showed the percentage and number of classrooms undergoing modification of instruction based on MAP data. 29 3.5.2. Qualitative Data Analysis Open-ended questions in students’ surveys and teachers’ interviews were analyzed using the thematic analysis method which aims at recognizing, evaluating, and reporting themes within data. It describes data in detail and interprets different aspects of the research topic (Clarke and Braun, 2006). The researcher tended to analyze the responses of the students in the surveys and those of the interviewee through interpretatively developing themes (Clarke and Braun, 2006). Analyzing the qualitative data in the research in hand, the researcher followed the steps demonstrated by Clarke and Braun (2006). Data from surveys and interviews were read and reread to acquire in-depth understanding of the collected data. Recorded data was then transcribed and written down. After that, all data was coded, and themes were mapped out. Later, themes were reviewed for more accuracy. Finally, the report was published. In addition to the surveys and interviews, the researcher wrote a descriptive report of the class visits that took place to investigate the use of MAP data to inform planning through observing the instructional strategies used by teachers in classrooms and the impact of modifying those strategies on students’ attainment and progress in lessons. 3.6. Site and Accessibility The study took place in a school in Dubai. The school was selected by the researcher because she works in the same school in the first place which makes it possible for her to meet as many teachers as needed for the study and to be able to conduct the survey for as many students as possible. The researcher was able to get full support from the management team, the staff members, and the students to conduct the survey and the interviews and to observe classrooms of different grade 30 levels. Additionally, the researcher has been working in the school for 16 years and is fully aware of its history and challenges. The school was founded in 1985. It provides education to 2123 students, enrolled in grade levels from the KG through grade 12. It follows the American curriculum and seeking the accreditation of the New England Association for Schools and Colleges (NEASC). The school has been rated acceptable by the KHDA since 2010; the school management is putting efforts and making investments in resources to improve and meet the KHDA expectations. Consequently, the research study in hand will help the school pinpoint the areas for improvement and find ways to construct strategic planning and update the existing plans and policies to bridge the gaps and address the KHDA recommendations and improve in MAP. 3.7. Key Research Considerations 3.7.1. Ethical Consideration Application of suitable ethical principles to protect human themes is important in any research study (Arifin, 2018). The researcher explained to the school’s management the purpose of the study and made sure all participants were aware of the need to conduct the research due to the importance of the topic and its impact on students’ education. Anonymity of participants was maintained; all participants who agreed to participate in the study were informed that their responses would remain anonymous and their names would never be mentioned. Ethical considerations in qualitative research, in specific, is quite needed as face to face interviews might expose participants to become hesitant to express their views due to feeling stressed (Arifin, 2018). Leaders and teachers who showed willingness to be interviewed were promised by the researcher that their responses would be recorded for research purposes only and their names 31 would not appear in any documents. As soon as the data was analyzed, all recorded interviews were deleted. The researcher included in the dissertation samples of the transcriptions of responses from the interviews (Appendix F) after getting the permission and validation from the interviewees. As for assessment data and MAP scores, all reports were without students’ names, and the data was derived from reports that were already published on the school’s website. Classroom observation reports were written without mentioning the names of the teachers and the sections where they were observed. Teachers were informed of the purpose of the visit and the rubrics were shared with the visited teachers ahead of time. The class visits focused mainly on modifying instruction based on MAP data, and the rubrics were planned to serve the purpose. 3.7.2. Triangulation Triangulation plays an effective role in compensating the weaknesses of one method with the strengths of another (Jick, 1979). Triangulation is dominantly used to integrate the survey method with the research. The researcher used more than one strategy of collecting data so that the study would not be deemed to one method of data collection. Using various instruments engendered a rich and comprehensive picture of MAP testing, its reliability, utility, and impact. Surveys, interviews, and class observation reports mirrored a range of insights, qualitatively and quantitatively analyzed. Quantitative data from test results supplemented the other data. Accordingly, the researcher was provided with the opportunity to feel more confident of the results due to the use of triangulation (Jick, 1979). 32 3.7.3. Validity Using the convergent approach might threaten validity as sample sizes may be unequally pictured in favor of quantitative data over qualitative data. Additionally, the use of different variables on both quantitative and qualitative data sides makes it difficult to compare and combine findings (Creswell, 2014). The researcher was allowed to bring data together to check if there is convergence in the results, a function described as corroboration (Rossman and Wilson, 1985). Qualitative data were quantified to become measurable and combine with the quantitative findings; likewise, quantitative findings were analyzed and elaborated through qualitative methods to give richness to the findings, creating side by side integration to ensure well-validated and substantiated results (Creswell, 2009). 3.7.4. Reliability Reliability of data is ensured through the consistency of results after repeating the use of the research instrument at different times (Golafshani, 2003). To ensure reliability of data, the researcher endeavored to design research methods that accurately measure the intended aspects in the topic. The researcher used different aspects extracted from literature review to design the research instruments. Besides, the researcher made students repeat the survey to test the stability in their responses (Heale and Twycross, 2015) which consequently measures the reliability of the research instrument used by the researcher. On the other hand, although results were found to be reliable, results cannot be generalized to a broader population since this study was conducted in a single school. 33 3.7.5. Role of the Researcher Considering ethical issues in research serves the purpose of the research which incorporates the spreading of knowledge, reporting the truth, and the need to respond to errors (Akaranga and Makau, 2016). Accordingly, the researcher becomes obliged to use an appropriate methodology and relevant methods of data collection and data analysis and then to present the information in a logical order (Akaranga and Makau, 2016). The researcher made sure that ethical issues be considered and that the purpose of the study should be to serve the school and help all stakeholders use MAP testing more purposefully and effectively to positively impact students’ learning and improve their attainment and progress. Consequently, the research findings were cautiously revealed to maintain the positive relations with the school and work in harmony with its policies (Akaranga and Makau, 2016). The researcher holds a huge responsibility when collecting the qualitative data; her role becomes very crucial. Surveys and interviews include important and rich data which is collected and analyzed by the researcher. The researcher has a professional experience in curriculum and assessment and has a direct role in monitoring teaching and instruction. 3.8. Research Limitations and Scope for Further Research Although the researcher was able to access the needed data for the analysis, there remained some challenges that hindered the full access towards valid and reliable conclusions. Leaders, teachers, and students lacked full knowledge of MAP testing and its goals and utility. Consequently, their views and answers to open-ended questions were limited to their restricted experiences, which led to imperfect scope of findings. Additionally, the researcher could find limited research about the topic; only few researchers wrote about impact of MAP testing on student learning. The studies on this topic were conducted in the USA where MAP testing is used for purposes that are not similar 34 to those in Dubai. No studies about MAP testing were found in the UAE or Dubai context. The topic of this research is important as it is a target for all American Curriculum schools in Dubai. However, no studies could be found that would serve the study in hand. 3.9. Summary The chapter at hand served in discussing the methodology adopted by the researcher in this study. The chapter also highlighted the different components of research design and methods as well as instrumentation and data analysis procedures. Additionally, it provided details about how those components were utilized. 35 IV. Chapter 4: Results, Data Analysis, and Findings 4.1. Chapter Overview Chapter four provides an overall picture of the research findings. The chapter comprises analysis of results and interpretation of data. Qualitative and quantitative data collected through different methods are analyzed. Tables and graphs that display data are also demonstrated in this chapter. The researcher divided this chapter into two main sections, analysis of quantitative data and analysis of qualitative data. Each section is inclusive of subdivisions that represent the methods and instruments used to collect targeted data. Data collected from each tool were analyzed and interpreted, and findings were then made. 4.2. Analysis of Quantitative Data 4.2.1. Students’ Surveys Students in grades six through nine and those in grade 12 participated in the survey (Appendix C). Numbers of participants were as follows: Grade Number of Boys Number of Girls Total Number of Students 6 15 19 34 7 19 25 44 8 21 21 42 9 16 25 41 12 15 17 32 Table 1. Numbers of Students’ Survey Participants The overall analysis of the students’ survey concluded a positive feedback on their experience with MAP testing when questions addressed general knowledge about MAP testing. However, the questions that would support the study and the purposes of MAP testing concluded high percentages of gaps! The survey results confirmed a high percentage of students’ strong agreement 36 with the survey indicators. Although the study did not aim to compare students’ views per gender, it was found worth mentioning that girls were more serious and accurate in their responses in the survey than the boys. Answering the open-ended questions in the survey, the girls seemed clearer in expressing their perspectives. The graph below demonstrates students’ responses. Figure 1. Percentages of Students Participating in the Survey Figure 2 Students' Responses 37 50% of students strongly agreed and 33% agreed that they understand what MAP testing is. 37% of students strongly agreed and 45% agreed that they are aware of the importance of MAP testing. Evaluating the impact of MAP testing on students’ learning, a good percentage of students showed disagreement. 26% of students strongly agreed and 38% agreed that MAP testing has an impact on their learning; however, a total of 36% disagreed with the aforementioned indicator. Moreover, although a total of 59% of students agreed that their academic status has improved since they started doing MAP, a high percentage of students (41%) still disagreed and strongly disagreed with this indicator that is considered an important aspect of MAP testing as mentioned earlier in the literature review. Additionally, 15% strongly agreed and 47% agreed that they feel MAP tests are linked to their curriculum, and a total of 37% disagreed with this indicator. 27% of students strongly agreed and 47% agreed that MAP sets realistic targets for improvement. Furthermore, 24% of students strongly agreed and 45% agreed that the targeted scores are appropriate for their grade level. Though, a total of 31% disagreed about the appropriateness of the scores set by NWEA in relation to their grade levels. 19% strongly agreed and 42% agreed that the level of the test questions meets their expectations; still a total of 39% showed disagreement with this statement. Besides, a total of 66% agreed that MAP test results reflect their actual level. Also, 66% of students agreed that the testing environment at their school is appropriate, but 32% of students expressed disagreement. Finally, 80% of students agreed that the allotted time for the test is appropriate; only 20% of students disagreed with this statement. 4.2.2. Analysis of MAP Results Since the beginning of MAP Administration, students have showed progress in their academic journey. Students’ attitude towards MAP testing has changed and students have started to express positive feedback towards this benchmark test. Despite the challenges students have expressed 38 they are going through in MAP testing, they still have been very much convinced that it has been beneficial for their academic progress and they enjoy the experience. However, this progress remains very slow. The National Agenda Parameter (NAP) was initiated by the KHDA to ensure all schools are working in line with the National Agenda targets (KHDA, 2018). In their letter to school principals, the KHDA assessment team (2018), announced the mandated grade levels, grades three through nine, who would be sitting for MAP in the academic year 2018-2019. The National Agenda Parameter was launched by the KHDA in 2015 (KHDA, 2017). In the KHDA Supplement for the academic year 2017-2018, the criteria for evaluating students’ attainment in MAP is explained as the following (KHDA,2017):  If at least 75% of students score at or above international standard of 41st percentile, the students’ attainment is judged “Acceptable”.  If at least 50% of students score at or above the international standard of 61st percentile, the students’ attainment is judged “Good”.  If at least 61% of students score at or above the international standard of 61st percentile, the students’ attainment is judged “Very Good”.  If at least 75% of students score at or above the international standard of 61st percentile, the students’ attainment is judged “Outstanding”. The MAP results of students in the studied school were analyzed based on the aforementioned criteria. Students in grades three through nine sat for MAP three times during the academic year 2018-2019. The first time was in October 2018 (Fall), the second time was in February 2019 (Winter), and the third time was in April 2019 (Spring). 39  In Math, all students in grades three to nine had a weak attainment in the three MAP tests. In Fall 35%, in Winter 28%, and in Spring 31% of students scored in the proportion of 41st percentile.  In Reading, all students in grades three to nine had a weak attainment in the three MAP tests. In Fall 37%, in Winter 30%, and in Spring 34% of students scored in the proportion of 41st percentile.  In Language Usage, all students in grades three to nine had a weak attainment in the three MAP tests. In Fall 53%, in Winter 41%, and in Spring 41% of students scored in the proportion of 41st percentile.  In science, all students in grades three to nine had a weak attainment in the three MAP tests. In Fall 67%, in Winter 48%, and in Spring 65% of students scored in the proportion of 41st percentile. Figure 3. MAP Attainment Analysis 40 Figure 4. MAP Progress Analysis The above graph demonstrates students’ progress from Spring 2018 to Spring 2019. It shows an overall picture of weak progress in all grade levels and all subjects. 4.2.3. Class Observation Reports The researcher conducted 24 class visits to grades three through nine; she visited four English teachers, four science teachers, and four math teachers. Each teacher was visited twice in two different classrooms. The researcher used class visits rubrics (Appendix B) that are used in the same school by leaders to evaluate teaching and learning, attainment, and progress. The same rubrics were designed by the school leadership based on KHDA descriptors and indicators in the KHDA framework. The class visits aimed at investigating the use of MAP data to inform instructions and improve students’ attainment and progress through using teaching and learning strategies that cater to different needs of students based on data from MAP tests. The following graphs demonstrate the evaluation of the 24 visited classrooms. 41 Figure 5. Math Teachers Class Observation Results 42 Figure 6.Science Teachers Class Observation Results F igure 7 English Teachers Class Observation Results . Only two teachers were found to be using MAP data effectively in their visited classrooms. Both of them modified instructions in their lesson plans based on students’ MAP scores and based on the grouping of students in the Learning Continuum Report from NWEA. The two teachers, one English and one science, used differentiated strategies that catered to the needs of students; students were grouped into five groups according to their reports that specify their readiness for learning and the skills they are ready to master. Accordingly, this resulted in good attainment and progress in their four classrooms as measured against the rubrics. The other ten teachers implemented whole group instructions and addressed all students similarly. Although their plans were well designed and constructed, the implementation was not appropriate and lacked use of effective strategies to support students and cater for their learning needs. All ten teachers did not display any evidence of using MAP data to inform instruction and differentiation was not obvious in their classrooms. Consequently, teaching and learning was measured weak 43 based on KHDA descriptors, and students displayed weak attainment and progress based on the KHDA criteria (Appendix A). 4.3. Analysis of Qualitative Data Qualitative data from students’ responses in the open-ended questions in the survey (Appendix C) and that from the teachers’ and leaders’ interviews was analyzed in this section. Answers from respondents and interviewees are also quoted to support analysis and findings. 4.3.1. The Analysis of the Open-ended Questions in the Students’ Survey The students’ survey included three open-ended questions. Students’ answers were all gathered into one spreadsheet to compare their responses and analyze their feedback. 1. Question 1: Do you think students should sit for MAP Tests? Explain why or why not? Most participants (78%) believed that students should sit for MAP testing. Most of them related importance of MAP to measuring students’ actual academic level. Student X: “Students should sit for MAP to understand what their actual level is and to reveal the weaknesses in specific skills and work on improving them.” Only one student thought “it compares students’ levels to other students all over the world”. Another student believed that “MAP testing allows students to challenge themselves against international standards”. Twelve students related MAP importance to measuring students’ progress. 44 Student Y: “MAP actually tracks and recognizes students’ abilities. It also draws an exact path for improvement of students’ academic achievement.” Only 28 students thought students should not sit for MAP testing. Almost all of students confirmed that “it is waste of time”. All senior students believed that “MAP testing did not have any impact on their learning, and it would not help with their college studies.” Fourteen students said they did not know why they would sit for the test. 2. Question 2: What challenges do you face during the test? Students’ answers to this question showed inconsistencies in their views towards MAP testing. Students’ seemed not aware about the real and genuine challenges they face in MAP testing. A number of eight students showed concerns about time as a stress factor, as they thought they always needed extra time to finish the test. 42 students (21%) said they did not face any challenges in their MAP tests. A number of 105 (54%) expressed worry about difficulty level of questions and how much this will be affecting their scores overall. Student X “The questions are not relevant to the standards for the current grade level; we have chemistry questions and we don’t study chemistry.” Five students related their challenges to difficulty of Math MAP questions. Only one student thought it is tiring and a break is needed during the test. Another student confirmed that the targeted score is challenging to achieve. Additionally, one student believed that stress factor is a challenge. Female students were more specific than males; they were more accurate in communicating their challenges. Thirteen girls talked about the testing environment and expressed concerns about the noise and discomfort in the venue. Student Y “It is freezing in the room.” Student Z “I mostly struggle with the environment where the test takes place as it is always noisy with 45 unnecessary announcement.” Moreover, students, mostly females, said that they face technical issues that hinder their effective completion of the test. Student B “Laptop problem is the only challenge I face during a MAP test; it wastes almost an hour of my time to open NWEA server.” 3. Question 3: What kind of support would you request from your teachers to improve your MAP scores? Students made suggestions to improve their MAP scores; those suggestions appeared to be based on students’ personal experiences in MAP Testing. The researcher compared students’ answers in question two about their challenges with their answers here; it was found out by the researcher that almost all students requested support that is based on the challenges they faced in MAP. Additionally, the researcher demonstrated the students’ responses and suggestions in the table below, believing that this would help the targeted school and other schools understand their students’ challenges and how they think they would improve in order to provide them with the needed support that can lead to the overall improvement of their results in MAP. Number/ Percentage of Students’ Suggested Support/ Quotes of Some Students Students 34 Students (17.6%) No support was suggested. 63 Students (32.6%) Students requested more MAP practice in lessons and at home. …… “ a 20 to 30 minute practice period everyday” …… “Create after-school clubs that seek to help students strengthen their education and knowledge in a subject that they are weak at, and give students sufficient sample and practice tests.” ….. “ Give us a website so we can practice MAP questions and solve better.” 46 …… “Do more screening tests.” 66 Students (34.1%) Students requested help from teachers during testing time. Some needed help in reading and clarifying the questions; others wanted more explanation of what’s needed; few students wanted support in understanding the meanings of words or concepts. …… “Help me through tests” …… “Answer our questions during the test.” ….. “Help students who suffer with the questions.” ….. “Help us with unfamiliar words.” Eight Students Students wanted better alignment of teaching and learning and curriculum with MAP tests. …… “Expand the curriculum to cover more topics.” ….. “Introduce topics that we never studied and are required for MAP test.” ….. “Make sure we learn everything required for grade 8.” …… “Cover most of the topics that are mostly included in MAP tests.” One Student ….. “Provide more critical thinking opportunities to students.” Three Students Students suggested to be given tips on how to study for and/or do the test. ….. “Provide overall test-taking skills.” One Student …. “Influential and economic support” Four Students Students wanted to have the option to do the test or not to do it. …. “Leave it to students to decide if they take the test or no” One Student ….. “Provide more time for the completion of the test.” Two Students Students requested breaks and beverages. 47 ….. “Provide food and water to students.” One Student Students requested a better testing environment. ….. “They should remove the extra chairs from the testing hall.” …… “Organize the system and the room in general” Two Students ….. “Provide constructive feedback and encouraging words.” One Student ….. “Apply MAP testing for all grades.” One Student ….. “Take the test more seriously by students.” Two Students Students requested more resources for practice. ….. “Provide me with resources that I can use to prepare for the test.” One Student ….. “Align lessons with MAP.” One Student ….. “I don’t know, really. The test is pretty random and I don’t think teachers can change that.” One Student ….. “Decrease the score and be realistic in the target.” Table 2. Students’ Suggested Resolutions to the Challenges 4.3.2. Analysis of Teachers’ Interviews Teachers were interviewed and their responses were recorded on the answer sheet. The interviews (Appendix D) were conducted with eight English teachers, ten math teachers, and nine science teachers. Question 1. How long have you been using MAP at your schools? Teachers’ responses ranged between 1 and 5 years with two teachers saying they were “not sure” about the number of years. 48 Question 2. Why have you decided to use MAP? Most teachers said that they started using MAP testing because it was mandated by their schools. Only two science and five English teachers responded that they decided to use MAP to measure students’ progress and to modify instruction. Besides, one math teacher had a clear picture of why she is using MAP and related her answer to specific features of MAP testing. She said: “The RIT (Ready for Instruction Today) score tells me at what level my students are ready to learn. The score is tied to learning statements which allow me to provide additional support/resources that are tailored for each student.” Question 3. What are you using MAP data for? Almost all teachers explained that they would use MAP data to monitor students’ attainment and progress and to modify curriculum and instruction accordingly. Teacher X said: “As MAP test measures students’ progress, I use its data to make decisions concerning students’ academic level and best ways to improve their different skills.” Another teacher related the use of data to meeting KHDA criteria for the school’s rating: “To my understanding, one of the requirements for a school to be rated “good” by the KHDA is for at least 50% of students to be performing at the 61st percentile in relation to MAP progress and attainment data. Because of this, I use my MAP quadrant data to see where students fall in relation to the 61st percentile and track their progress throughout the year to try to get them “over the line”.” Two teachers said that they use the data to understand the strengths and weaknesses of students, and only one teacher said that data is not used appropriately in the school. 49 Question 4. How are you relating MAP data to curriculum, teaching and learning, and assessment? As per differentiating instructions in teaching and learning, teachers had different views in this regard; some thought MAP data is helping them select instructional strategies for individual students; others thought it helps them modify curriculum standards in terms of learning outcomes rather than teaching strategies. Most teachers gave general answers to this question; their answers were that they made this relation between MAP and curriculum and teaching and learning. However, they failed to explain how. Only one teacher was able to fully explain how MAP data is specifically related to the aforementioned standards: “ I relate MAP data to teaching and learning through use of the Learning Continuum, which provides me information on where students need help and where they are ready to advance…..” Question 5. How are your students feeling about MAP testing? Almost all teachers agreed that students do not take MAP tests seriously. Most teachers though thought students feel stressed out and are challenged during the test. Only two teachers agreed that students find it interesting and enjoy doing it. Question 6. Does MAP testing have impact on students’ learning? Seven science teachers agreed that MAP testing has impact on students’ learning; one teacher disagreed without giving any explanation; another teacher also disagree, claiming that due to the lack of students’ and parents’ seriousness, students are not benefiting from this test. Math teachers were mostly hesitant about answering this question and showed inability to measure its impact on students’ learning. Three out of ten teachers said it has impact without explaining 50 how. Only one teacher said it does not have impact. Six teachers said “sometimes” it has impact without being able to give a clear explanation. All eight English teachers agreed that MAP testing does have impact on students’ learning. Two teachers explained that this impact is due to curriculum modifications. The rest did not explain. Question 7. How is MAP helping you as a teacher? Almost all teachers explained that MAP test helps them understand the level of their students as measured against curriculum standards and allows them to modify their plans to cater to students’ needs and bridge curriculum gaps accordingly. Only two teachers said that it does not help them as teachers. Question 8. What are the challenges that your students face in MAP testing? Teachers mentioned technical issues and time as the major challenges, with more focus on the technical issues in terms of weak internet connection and bad quality of laptops that would lead to interruptions in the test. Some teachers showed concerns about the lack of full alignment between the test and the adopted curriculum standards. Question 9. What are the challenges that you face as a teacher whose students sit for MAP testing? Most teachers complained about the technical issues which lead to disruptions to students during the test and accordingly requiring students to sit longer for the test. The majority of teachers expressed their concerns about the time “wasted” in testing, as, according to them, students sit for tests more than they sit for learning. Some teachers referred to the test alignment with the curriculum standards. Few teachers were not satisfied with the testing environment and said that 51 students ask for support when they have technical issues and consequently disturb other students in the hall. Few teachers explained that encouraging students to take the test seriously is an area of concern. Question 10. What are the positives and negatives of MAP Testing? They gave positive feedback about the effective use of MAP testing as a way for measuring students’ progress and a means for modifying the curriculum to meet the needs of different learners. They thought that the results helped in deciding students’ readiness in studying adopted standards as this test (according to some teachers) is widely aligned with California Common Core ELA and Mathematics Standards and Next Generation Science Standards. Although all teachers expressed a positive attitude towards MAP testing, all of them had concerns about the loss of instructional time. They stated that they are spending at least three weeks every term for MAP testing which is putting them under stress to finish the curriculum standards needed for each grade level. According to teachers, this problem can be easily resolved by availing all needed tools and resources for testing like scheduling and infrastructure. Two teachers had different views of the negatives that were expressed by their colleagues; they claimed that MAP testing has a negative impact on students’ psychology as the results might frustrate them. They also expressed concerns about the validity of data as “there are irregularities with test administration”. Question 11. Do you recommend MAP testing in schools? Why or Why not? 8% of teachers showed disagreement about using MAP testing in schools as, according to them, it does not actually reflect students’ mastery of standards. It is also considered by some of them as an extra burden on teachers who get overloaded to analyze data that is not used effectively due to lack of clarity and professional training. 52 92% of teachers agreed that MAP test should be used in schools as, according to them, it helps teachers understand students’ levels and pinpoint their weaknesses and then modify/adapt plans to cater to students’ needs. One teacher linked her agreement with a condition which is that “it should not be the main focus for judging the students, the educators, or the schools”. 4.3.2. Analysis of Leaders’ Interviews Four leaders were interviewed by the researcher. One was the Assessment Coordinator in the school, and the other three were the leaders of English, math, and science. Leaders were met in the administration, each one at a time. They all answered the following questions (Appendix E). Question 1. How long have you been using MAP at your school? Three leaders have been using MAP for three years in this current school, and one leader has five years of experience in using MAP at this school and other schools. Question 2. Why have you decided to use MAP? Three leaders answered that it is a requirement by the KHDA for all the US curriculum schools to use MAP. One leader said that it allows for aligning the curriculum with benchmark tests. Question 3. What are you using MAP data for? All four leaders agreed that MAP data is used to measure the performance of students in grades 3 to 9. Students are informed about their current level and what they are ready to learn next, and teachers use data to modify instruction and review curriculum based on students’ needs. The Assessment Coordinator: “MAP growth data helps us pinpoint the instructional areas of our students and measure their growth over time. Our teachers use the data to differentiate instruction and identify individual student need.” 53 Question 4. How are you relating MAP data to curriculum, teaching and learning, and assessment? All leaders discussed the following:  Curriculum modifications, using Learning Continuum, take place based on the identified areas of strengths and weaknesses.  Teaching and learning is differentiated as per individual student’s needs.  Alignment of internal assessment with the MAP assessment takes place after the outcomes of the MAP data analysis. Question 5. How are your students feeling about MAP testing? All leaders agreed that students did not find it comfortable taking the MAP assessment. However, it has considerably changed this year and students have started taking interest in it. Leaders also agreed that most students strive to reach their individual goals in the test. Question 6. Does MAP testing have impact on students’ learning? According to the four leaders, MAP testing streamlines students’ learning. Teachers and students have a clear understanding of what students already know and what they need to learn next. “……. Students Goal setting worksheet and learning continuum play a vital part in it.” Question7. How is MAP helping in improving students’ attainment and progress? All leaders confirmed that MAP helps in improving students’ attainment and progress through effective use of data to inform planning. 54 The Assessment Coordinator: “MAP provides a deeper understanding of what students know and need to learn. It helps us to more effectively address instructional needs and gaps in skills. MAP data is used to support goal setting, lesson planning, interventions, and more.” Question 8. What are the challenges that your students face in MAP testing? The leaders’ answers matched those of teachers. They mention that concentration, lack of seriousness, types of questions, and time management are some of the challenges that students are dealing with. Question 9. What are the challenges that your teachers face in MAP testing? According to the Assessment Coordinator and the leaders of math and science, teachers are facing challenges with the time management and proctoring as well as, from the science leader’s perspective, convincing students to take the test seriously. However, the Leader of English Language Art explained that “teachers are not clear on the purpose of MAP or how to use the data.” Question 10. What challenges do you face as leaders in ensuring that MAP testing is done professionally and that MAP data is used consistently? The four leaders had different perspectives in this regard based on their role and their experience with the test. According to the Assessment Coordinator, the main challenges are: “Test administration as per international standards, the use of data by the teachers, and interpretation of RIT score by the students and their parents.” 55 The leader of English Language Art believes that “the problem is with the test security and accurate reporting as there is no assessment policy for MAP testing in the school”. The math leader explained that: “The biggest challenge is achieving progress from session to session and from year to year. The use of data in its core is a challenge since many involved parties need continuous training.” The science leader expressed concerns about covering the standards relevant to each grade level before each MAP test. He also considered test time a challenge. Question 11. What are the positives and negatives of MAP Testing? All leaders talked about the same positive and negative aspects of MAP testing. Positives: It provides clear lines of actions at all levels for senior leaders, middle leaders, teachers, students, and parents. Negatives: Teaching time is taken by the MAP three times a year. It also puts all stakeholders under stress. Question 12. Do you recommend MAP testing in schools? Why or Why not? All four leaders recommended MAP testing in school as it supports teaching and learning by providing clearly identified goals. They also agreed that it helps bridge curriculum gaps and address students’ needs. 56 V. Chapter 5: Discussion 5.1. Chapter Overview In this study, the researcher aimed at investigating the utility, reliability, and impact of MAP testing in a school in Dubai. The researcher is aware of the importance of external assessments in schools in Dubai in general and MAP testing in American Curriculum schools in particular; this importance is built based on the relevance of those assessments to UAE National Agenda and its Vision. The researcher intended to fully understand the strengths and weaknesses of MAP testing through studying its implementation in a school in Dubai, aiming at pinpointing the gaps and finding some solutions that can be helpful to the school where the study took place and to other schools using this benchmark test. Additionally, the researcher is ambitious that this study will be considered a starting point for other deeper and wider future studies that will help make effective use of MAP testing and reach the desired results. 5.2. Discussion of the Findings The researcher concluded the findings based on students’ results in their MAP tests during the academic year 2018-2019. She also analyzed the surveys and interviews that she conducted, and then drew the conclusions consequently. Besides, the researcher conducted class visits and analyzed the reports to pinpoint the gaps and relate them to students’ results. Moreover, the researcher referred to literature which highlighted best practices in the utilization of MAP testing in order to be able to suggest recommended strategies for improvement. 57 The findings were made in relevance to the following research questions: a. How reliable are MAP Tests in reflecting students’ achievement, and how effective are they in curriculum modification and instruction adaptation? b. How does effective use of MAP testing impact students’ learning? c. What is the impact of MAP tests on students’ learning? All results derived from the different methods used in this study concluded gaps and weaknesses in the school’s practices which negatively affected students’ results and had a negative impact on their learning. Furthermore, it was found that this lack of effective implementation of the test led to negatively affecting its reliability, utility, and efficacy. In order to bring change to the situation in this school and other schools, there should be strict measures in place, and schools should take drastic and immediate actions in response to the requirements of accurate operation of the test. 5.2.1. Building a Culture of In-depth Understanding of the Test and Its Purposes As mentioned earlier in this chapter, the reliability of MAP testing is concluded to be not existent in the study at hand due to the inconsistency of students’ results in their attainment and progress as measured against KHDA criteria. Overall, students’ attainment and progress remained weak despite the slight improvements that were evident in some grade levels. Additionally, there were ups and downs in students’ scores from fall to winter and then to spring. Almost all students’ scores appeared to be very low in fall, then dropped again in winter, and later improved in spring to drop again in next fall. The researcher believes that this inconsistency is due to the lack of clear and deep understanding of MAP testing and its purposes by the students and teachers which was quite evident in their responses in the surveys and interviews. “… teachers are not clear on the purpose 58 of MAP or how to use the data.” Although some students were positive about why they were sitting for the test, many of them were not sure about its importance and its impact on their learning. However, even the teachers who deeply understood MAP testing and its purposes, they were not using it effectively in their classrooms because it was evident that they needed support on how to use the data and how to make use of the purposes of the test. Additionally, some students understood the purposes of the test; however, they needed more awareness of the importance of the test, and they also needed the resources to be availed for them. 5.2.2. Effective Use of MAP Data to Modify Curriculum and Adapt Instruction Effective use of data is of great importance as it leads to reliable self-evaluation and effective teaching and learning (Demie, 2913). The studied school failed to use MAP data to modify curriculum and instruction, which led to ineffective teaching and learning and accordingly inconsistent growth is students’ achievement. The students’ results, the class visit reports, and the students’ and teachers’ responses reflected inconsistencies in the overall school’s response to MAP requirements. Results, data analysis, and gathered information provided evidence that the school’s curriculum doesn’t undergo regular reviews based on MAP data. Although subject leaders and teachers stressed the need to modify curriculum and adapt instruction based on MAP data analysis, implementation of modified plans was not evident in most visited classrooms. In most cases, there was no evidence of the use of data to modify the curriculum and make adjustments on the curriculum maps. Not all curriculum modifications were visible in classes through instructional modifications that were supposedly made by SLT (Senior Leadership Team), MLT (Middle Leadership Team), and teachers to provide learning opportunities that match students’ abilities. 59 In one English class which showed good attainment and progress as evaluated during a class visit by the researcher using KHDA rubrics, students’ learning was happening in learning centers and station work with focused activities targeting specific learning outcomes which are mapped into the curriculum showing both differentiation and independent learning. Additionally, there is a good story in one science class as well. The effective use of assessment information was evident, demonstrating that the curriculum is more carefully aligned with the required standards. This enabled the visited teacher to plan work in lessons that was better matched to students’ learning needs and preferences. Generally speaking, most teachers showed evidence of knowledge of what is expected of them in terms of use of MAP data and what this test is used for; however, this was not visible in their classrooms. This might be reflecting the reason why students’ scores are inconsistent, and why the test is not proved to have utility and effectiveness. 5.2.3. Use of MAP Testing to Impact Students’ Learning “Students are given the Goal Setting worksheets and are encouraged to set their own targets based on their projected growth”, said one of the interviewed science teachers. However, examining few samples, it was noticed by the researcher that students’ goals remained general and not specific to their gaps in learning. Students receive their MAP test results in the form of a RIT score which shows their knowledge, skills, and abilities (NWEA, 2017). The student’s score shows his/her progress at different times of the year. Growth is what is always expected from MAP results, and that is how this target is explained to parents by NWEA (2017) in their guide to parents. However, the picture is different at the school in this study. Students show growth in spring MAP results compared to winter and 60 then in fall they show drop and more drop occurs drastically again in winter and then the picture becomes better and shows some progress in next spring. So, the comparison made by NWEA does not fit all students and all schools. All in all, this study concludes no strong impact of MAP on students’ learning. In the cases when MAP was used effectively and its data was used purposefully and accurately, students showed impact of good learning which resulted in good attainment and progress in class. However, this progress was not precisely reflected in MAP test results from one session to another. 5.2.4. Professional Development for Teachers The analysis of teachers’ and leaders’ interviews demonstrated inconsistencies in their responses from one side and contradiction between their knowledge and their classroom practices from another side. Almost all teachers knew the purpose of MAP and how they can benefit from the test; however, they were not making use of its data in their classrooms. Analyzed data was with all teachers, and modified curriculum plans were also available with them. However, there was not implementation of the plans. This led to the conclusion that teachers needed professional development sessions about the use of data to inform planning and instruction. To avoid any discrepancies between what assessment data is actually used for and how schools are using data, schools should be following an approach that puts them in the right direction in order to improve students’ learning and their performance. Professional development programs should be based on deep analysis and understanding of the teachers’ work, how they get motivated, and how they learn (Kennedy, 2016). Consequently, professional development sessions should target teachers’ needs and their weaknesses. 61 5.2.5. Availability of Resources and Testing Atmosphere The paper at hand investigates the reliability and utility of MAP testing in a school in Dubai. The school has been using MAP for three years since it was mandated by the KHDA for all American Curriculum Schools. MAP windows are opened three times a year, in fall, winter, and then spring. It is conducted three times a year, once each term. It was noted by most interviewed teachers that MAP leads to waste of instructional time; however, according to NWEA (2013), MAP allows teachers to save wasting time on concepts students already know when they use MAP data to have knowledge on what students actually need and focus their instruction on those needs. According to NWEA (2013), a MAP test takes about an hour; however, in the school where the research is taking place a MAP test is taking not less than three hours due to technical issues that occur from NWEA’s side sometimes and the school’s side at other times. It was made clear by students and teachers as well as leaders involved in this study that so much time is used for MAP testing which is depriving students of the time allotted for their learning. “….. the test wastes my times and deprives me of enjoying learning collaboratively with my colleagues.” Analysis of data gathered through different methods provided evidence of gaps in terms of resources and infrastructure in the school where the study took place. Almost all interviewed teachers expressed concerns about the testing atmosphere. The following graph demonstrates the teachers’ responses in terms of the challenges that they find hindering the impact on students’ achievement. 62 Figure 8. Testing Atmosphere Analysis Consequently, to benefit from the utility and all aspects of MAP test, the school should take corrective measures in terms of strengthening the internet, availing new/in good condition computers/laptops, and providing all needed resources for the test and for practicing for the test. 5.3. Summary of the Chapter All in all, the chapter interpreted all gathered data and drew conclusions about the targeted topic in this study. Besides, this chapter answered the research questions and highlighted the areas for improvement that have paved the way for some potential recommendations that will help this school and other schools improve their students’ results in MAP. MAP test was found to be unreliable and to be having no impact on students’ learning in this specific school due to the weak practices and accumulating challenges in the same school. As almost all stakeholders seemed to understand the importance of the test for meeting the National Agenda Targets and for improving the students’ achievement by tracking their academic growth, the school should provide teachers with professional learning to make use of MAP from all sides. 63 VI. Chapter 6: Conclusion 6.1. Chapter Overview This chapter gives a brief of the dissertation. It summarizes the findings and concludes the answers to the questions of the research. Besides, the chapter highlights the implication of the study and puts emphasis on the limitations. The researcher, additionally, suggests some potential recommendations that would benefit education in general and educational institutions and authorities in particular. 6.2 Implications of the Study This study paves the way towards more studies that investigate the reliability of MAP testing in different contexts, such as different American Curriculum schools in Dubai and outside Dubai. Benchmark tests are important due to the fact that they are designed to notify teachers that they need to teach different students differently (Olah, Lawrence, and Riggan, 2008) based on their individual needs. Accordingly, investigating the ways to make effective use of those assessments will support schools identify the gaps and bridge them to benefit from those mandated assessments. Moreover, the study at hand is important to the management of the school in order to take drastic measures to bridge the gaps in MAP assessment. It is also significant to policy makers to review and update their policies according to the needs of teachers, students, leaders, and the school overall. Policy makers and higher authorities need to consider the finding in this dissertation and make all needful to serve education in general and schools in particular. 64 6.3. Key Findings and Conclusion The existing dissertation aimed at investigating the utility, reliability, and impact of MAP testing in a school in Dubai. The researcher intended to identify the challenges that are hindering the targeted school from achieving the targets set by the KHDA for students’ attainment and progress. Through this, specific areas for improvement would be determined and plans of action would be put in place accordingly. The study concluded that MAP testing was not utilized effectively in the studied school. Consequently, students’ scores remained inconsistent and showed instability, which led to considering the test unreliable in this specific school. Data from surveys and interviews showed some knowledge of the students, teachers, and leaders about the importance of MAP testing and its impact on students’ learning. However, despite this knowledge, there appeared to be lack of utilization of the data of the test as should be based on what its purposes suggest. Additionally, class observations supported the lack of effective use of data. 90 % of teachers did not use MAP data to modify the curriculum and adapt instruction to cater to students’ needs, and when they did, it all remained on paper and was not implemented in classrooms. Conclusively, the test was found to be not reliable nor effective in the school where the study was done. It had no impact on students’ learning as their progress remained weak overtime. Their attainment and progress in lessons was not overall good due to the nonexistence of use of data to inform planning. 65 This, hence, calls for immediate actions to correct the ineffective practices and make professional use of this test that is mandated by the higher authorities because of its importance in the students’ education and for their academic growth. 6.4. Limitations The study involved a single site only and a small sample size compared to the actual number of students taking the test in American curriculum schools in Dubai and outside. It took place in a school in Dubai, and the findings were based on this American Curriculum School. Consequently, the findings remained specific to one context and could not be generalized. Accordingly, the extension of the research findings was limited in this study. The impact of MAP or other benchmark assessments on student outcomes had been examined by few studies only (Cordray, Pion, Brandt, Molefe, and Toby, 2012). This is also one of the weaknesses that require further work from the educators. Moreover, There is no clear evidence on the effectiveness of benchmark assessments on student outcome; this is also the case in the United States where MAP assessment is widely used in different school districts with no experimental evidence on its impact on students outcome (Cordray, Pion, Brandt, Molefe, and Toby, 2012). Another concern also arises here; one should dig deeper and investigate the reason behind assigning MAP testing for American Curriculum Schools in Dubai when this test is not done in most schools in different states in the USA. Additionally, it was also found out in this study that senior students did not feel MAP testing would be relevant to their university studies and after school education. This again raises another area for research and investigation; why is MAP testing 66 mandated in schools if it does not lead to continuity and progression in the transition to education beyond school. Educationists should raise those concerns and conduct more inquiry and research to prove the reliability of MAP assessments and improve its utility across all phases of education. Although MAP assessments are very much aligned to American Curriculum standards, it still lacks accuracy and full alignment. It also should not be used to judge students’ attainment and progress and accordingly the schools’ performance. The researcher faced limitation in finding studies and literature about MAP testing. There were only few studies that targeted MAP in few States in the USA. Also, the researcher could find informative documents constructed by NWEA to give guidance and information about what the test is and why it is used. Besides, very few articles analyzing and criticizing MAP assessments were found, but they were mostly positive as they were written by NWEA themselves. Lastly, the researcher was unable to find any studies that targeted MAP testing in Dubai schools or any articles that explained why that test was mandated in Dubai private schools. 6.5. Recommendations Educational leaders should be clear about the purpose of MAP assessments, and consequently they should assess the sources of current data (Finnerty, 2018) like the instructional time in specific. MAP testing provides valuable information about students’ learning (Simpson, 2016) and has had positive effects on instructions and curriculum modification to meet the needs of students. It has had a positive impact on students’ learning as they are exposed to a good level of challenge that reinforces their academic performance. However, there are negative effects that need to be minimized (Simpson, 2016). 67 Students should not feel stressed out and pressured to perform well due to the requirements of the higher authorities from schools (Simpson, 2016). Educators and parents should work on comforting students and encouraging them and avoid connecting performance in MAP testing with consequences (Simpson, 2016) so that students do not feel anxious but rather believe in themselves and their capabilities. Additionally, it should be made easier on teachers as well; stressed out and anxious teachers tend to deliver messages to students that would also make them feel under pressure. Accordingly, risks associated with benchmark testing need to be minimized by the KHDA as the higher authority for private schools in Dubai. Based on NWEA’s scale alignment studies, there is a link between the MAP assessments and the standardized tests of each state (Cordray, Pion, Brandt, Molefe, and Toby, 2012). However, MAP assessments in Dubai as required by the KHDA are not linked to other standardized nor external assessments. The researcher believes that there should be links between MAP assessments and other assessments and should be utilized more effectively for other purposes to ensure this type of benchmark assessment is given the quality it actually weighs. It is recommended by the researcher that better procedures be in place to ensure effective administration of MAP testing and better utilization of MAP data by school administrators and teachers. In the interviews, it was noted that teachers were not provided with the needed training on how to use MAP data effectively to differentiate instructions and improve students’ learning. Teachers were only trained on how to administer the tests and generate reports which are only two things of the many other things that the MAP training offers. MAP training can also train teachers on interpreting outcomes reports at different levels; additionally, teachers can be trained through MAP on using online resources and report results to determine students’ readiness and differentiate instruction accordingly (Cordray, Pion, Brandt, Molefe, and Toby, 2012). 68 A study by Lazarin (2014), concluded that regular testing at different times in a year burden students, parents, and educators. Consequently, the researcher believes that MAP testing is a good measure of progress. It should be administered at the beginning and end of the year. It should be added to a standardized testing program to ensure that attainment is properly measured. Besides, Teachers are able to understand and explain how testing affects students positively or negatively (Ledesma, 2011); thus, teachers should be consulted about the efficiency of the adopted test and accordingly decide whether to use it or eliminate it (Lazarin, 2014). 69 REFERENCES ‘A Parent ’ s Guide to MAP This guide was created as a resource to help you and your family’ (no date). Akaranga, S. I. and Makau, B. K. (2016) ‘Ethical Considerations and their Applications to Research: a Case of the University of Nairobi’, Journal of Educational Policy and Entrepreneurial Research, 3, pp. 1–9. AKBEN (2019) ‘No 主観的健康感を中心とした在宅高齢者における 健康関連指標に関す る共分散構造分析Title’, (May 2018), pp. 1–19. Akca, F. (2011) ‘The relationship between test anxiety and learned helplessness’, Social Behavior and Personality, 39(1), pp. 101–111. doi: 10.2224/sbp.2011.39.1.101. Anon, (2019). [online] Available at: https://health.usnews.com/health-news/patient- advice/articles/2015/11/16/standardized-tests-making-our-students-and-teachers-sick [Accessed 16 Oct. 2019]. Benjamin, A. S. and Pashler, H. (2015) ‘The Value of Standardized Testing: A Perspective From Cognitive Psychology’, Policy Insights from the Behavioral and Brain Sciences, 2(1), pp. 13–23. doi: 10.1177/2372732215601116. Bergan, J. R. et al. (2009) ‘Benchmark Assessment in Standards-Based Education : by’. Bjorklund-young, A. and Borkoski, C. (2016) ‘The Validity Research on i-Ready and MAP That We Have 1’, (November), pp. 1–6. Boman, J. et al. (2017) ‘Overview of Decoding across the Disciplines’, New Directions for Teaching and Learning, 2017(150), pp. 13–18. doi: 10.1002/tl.20234. Brown, R. S. and Coughlin, E. (2007) ‘The Predictive Validity of Selected Benchmark Assessments Used in the Mid-Atlantic Region. Issues & Answers. REL 2007-No. 017’, Regional Educational Laboratory Mid-Atlantic, pp. 1–35. Available at: http://ezphost.dur.ac.uk/login?url=https://search.proquest.com/docview/62051157?accountid=14 533%0Ahttp://openurl.ac.uk/ukfed:dur.ac.uk?genre=report&issn=&title=The+Predictive+Validit y+of+Selected+Benchmark+Assessments+Used+in+the+Mid-Atlantic+Region.+Issues. Business Insider. (2019). This is the psychological effect standardized testing has on children. [online] Available at: https://www.businessinsider.com/effect-of-standardized-testing-on- children-2016-5 [Accessed 16 Oct. 2019]. Cordray, D. et al. (2012) ‘The Impact of the Measures of Academic Progress (MAP) Program on Student Reading Achievement. Final Report. NCEE 2013-4000’, National Center for Education Evaluation and Regional Assistance. Available at: http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED537982&site=ehost-live. Core, F. C. (no date) ‘NWEA MAP overview’. Demie, F. (2013) ‘Using Data to Raise Achievement - Good Practice in Schools’, pp. 1–57. Available at: 70 https://www.lambeth.gov.uk/rsu/sites/lambeth.gov.uk.rsu/files/Using_Data_to_Raise_Achievem ent-Good_Practice_in_Schools_2013.pdf. edarabia.com. (2019). Education Policy Reform in the UAE: Building Teacher Capacity. [online] Available at: https://www.edarabia.com/education-policy-reform-uae-teacher-capacity/ [Accessed 2 Oct. 2019]. En.wikipedia.org. (2019). Knowledge and Human Development Authority. [online] Available at: https://en.wikipedia.org/wiki/Knowledge_and_Human_Development_Authority [Accessed 2 Oct. 2019]. Finnerty, D. C. (2018) ‘The Predictive Utility and Longitudinal Student Growth of NWEA MAP Interim Assessments in Two Pennsylvania Schools’. Gawthrop, J. (2014) ‘Measuring Student Achievement: A Study of Standardized Testing and its Effect on Student Learning’, p. 42. Golfashani, N. (2003) ‘Understanding Reliability and Validity in Qualitative Research’, The Qualitative Report, 8(4), pp. 597–607. Available at: https://nsuworks.nova.edu/tqr/vol8/iss4/6. Harris, Phillip, et al. (2012) ‘Standardized Tests Do Not Effectively Measure Student Achievement’, Greenhaven Press. doi: 10.1016/j.memsci.2016.05.019. Heale, R. and Twycross, A. (2015) ‘Validity and reliability in quantitative studies’, Evidence- Based Nursing, 18(3), pp. 66–67. doi: 10.1136/eb-2015-102129. Herman, J. L., Osmundson, E. and Dietel, R. (2010) ‘Benchmark Assessment for’. 10.1192/bjp.112.483.211-a. Ies.ed.gov. (2019). The Impact of the Measures of Academic Progress on Differentiated Instruction and Student Achievement. [online] Available at: https://ies.ed.gov/ncee/edlabs/projects/rct_245.asp?section=ALL [Accessed 2 Oct. 2019]. ‘In Reply: BEHAVIOUR THERAPY’ (1965) The British Journal of Psychiatry, 111(479), pp. 1009–1010. doi: 10.1192/bjp.111.479.1009-a. In Reply: BEHAVIOUR THERAPY (1966a) The British Journal of Psychiatry. doi: 10.1192/bjp.112.483.211-a. In Reply: BEHAVIOUR THERAPY (1966b) The British Journal of Psychiatry. doi: 10.1192/bjp.112.483.211-a. In Reply: BEHAVIOUR THERAPY (1966c) The British Journal of Psychiatry. doi: Jabbarifar, T. (2009) ‘the Importance of Classroom Assessment and Evaluation in Educational System’, Proceedings of the 2nd International Conference of Teaching and Learning (ICTL 2009), (Ictl), pp. 1–9. Johnson, R. B. and Onwuegbuzie, A. J. (2004) ‘Mixed Methods Research: A Research Paradigm Whose Time Has Come’, Educational Researcher, 33(7), pp. 14–26. doi: 10.3102/0013189X033007014. Kennedy, M. M. (2016) ‘How Does Professional Development Improve Teaching?’, Review of 71 Educational Research, 86(4), pp. 945–980. doi: 10.3102/0034654315626800. Khda.gov.ae. (2019). About Us. [online] Available at: https://www.khda.gov.ae/en//about-us [Accessed 2 Oct. 2019]. ‘KHDA INSPECTION FRAMEWORK-converted’ (no date). Lazarín, M. (2014) ‘Testing Overload in America’s Schools’, (October 2014), p. 30 p. doi: ED419696. Ledesma, P. (2019). Do Standardized Tests Reflect Student Learning in Schools?. [online] Education Week - Leading From the Classroom. Available at: http://blogs.edweek.org/teachers/leading_from_the_classroom/2011/03/do_standardized_tests_re flect_student_learning_in_schools.html [Accessed 16 Oct. 2019]. Malone, M. O. (no date) ‘School District , NY How a New York school district improved achievement by supporting growth’. Mühl, J. K. (2014) ‘Research methodology’, Contributions to Management Science, pp. 75–100. doi: 10.1007/978-3-319-04069-1_4. Munhall, P. L. (1988) ‘Ethical Considerations in Qualitative Research’, Western Journal of Nursing Research, 10(2), pp. 150–162. doi: 10.1177/019394598801000204. Nelson, H. (2013) ‘Testing More, Teaching Less’, A Union Of Professionals, pp. 1–34. Available at: http://blogs.edweek.org/edweek/teacherbeat/TestingMore_TeachingLess.pdf. Nisbet, S. (2004) ‘Using Assessment Data to Inform Planning and Teaching: The Case of State- Wide Benchmark Numeracy Tests’, pp. 26–38. NWEA (2013) ‘Myths about NWEA’s MAP - January 2013 newsletter’, pp. 1–4. Available at: http://www.edweek.org/media/nweamyths-blog.pdf. NWEA (2017) ‘Dubai , UAE International School Effectively Implements MAP Growth — and Sees Student Growth’. Oláh, L. N., Lawrence, N. R. and Riggan, M. (2010) ‘Learning to learn from Benchmark assessment data: How teachers analyze results’, Peabody Journal of Education, 85(2), pp. 226– 245. doi: 10.1080/01619561003688688. Progress, A. (2014) ‘Measures of Academic Progress The MAP Learning Continuum The fast , flexible way for MAP users to create customized learning paths for individual students and small groups’, (September). Reading Eagle. (2019). Standardized tests having ill effects on students. [online] Available at: https://www.readingeagle.com/voices/article/standardized-tests-having-ill-effects-on-students [Accessed 16 Oct. 2019]. Rossman, G. B. and Wilson, B. L. (1985) ‘Quantitative Qualitative Large-Scale Study’, Sage Publications, 9(5), pp. 627–643. doi: 10.1177/0193841X8500900505. Ruddell, P. W. (2012) ‘The Impact of the No Child Left Behind Act and School Choice on Student Achievement’, ProQuest LLC, (April). Available at: 72 http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED548126&site=ehost-live http://gateway.proquest.com/openurl?url_ver=Z39.88- 2004&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&res_dat=xri:pqm&rft_dat=xri:pqdiss:3514 709. ‘Running head: TEST STRESS AND TEACHING EFFICACY Teacher stress, teaching- efficacy, and job satisfaction in response to test-based educational accountability policies Nathaniel P. von der Embse’ (2016), (November 2015), pp. 308–317. Simpson, C. (2016) ‘Effects of Standardized Testing on Students’ Well-being’, (May), pp. 1–20. Shaw, L. (2019). Educators debate validity of MAP testing. [online] The Seattle Times. Available at: https://www.seattletimes.com/seattle-news/educators-debate-validity-of-map- testing/ [Accessed 16 Oct. 2019]. Suval, L. (2019). The Psychological Effects of Too Much Testing. [online] World of Psychology. Available at: https://psychcentral.com/blog/the-psychological-effects-of-too-much- testing/ [Accessed 16 Oct. 2019]. Tabari, R. (2014) ‘Working Paper 07 Education Reform in the UAE: An Investigation of Teachers’ Views of Change and Factors Impeding Reforms in Ras Al Khaimah Schools’, (July). Talmage, V. (2019). And You Thought Standardized Tests Were Bad.. [online] Save Maine Schools. Available at: https://emilytalmage.com/2017/09/16/and-you-thought-standardized-tests- were-bad/ [Accessed 16 Oct. 2019]. Teach. Learn. Grow. (2019). Top 10 MAP Reports for Teachers - #edchat #MAPtest. [online] Available at: https://www.nwea.org/blog/2016/top-10-map-reports-teachers/ [Accessed 16 Oct. 2019]. Tong, A., Sainsbury, P. and Craig, J. (2007) ‘Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups’, International Journal for Quality in Health Care, 19(6), pp. 349–357. doi: 10.1093/intqhc/mzm042. The Classroom | Empowering Students in Their College Journey. (2019). The Effects of Standardized Tests on Teachers and Students. [online] Available at: https://www.theclassroom.com/effects-standardized-tests-teachers-students-10379.html [Accessed 16 Oct. 2019]. Uaecabinet.ae. (2019). National Agenda. [online] Available at: https://uaecabinet.ae/en/national- agenda [Accessed 2 Oct. 2019]. Vey, L. D. (University of C. (2005) ‘Enhancing the Relationship Between Learning and Assessment’, (August). Walberg, H. J. (2012) ‘Standardized Tests Effectively Measure Student Achievement Good Measures of Student Performance’. Webb, E. and Weick, K. E. (1979) ‘Copyright © 1979. All rights reserved.’, Administrative Science Quarterly. Wiliam, D. (2013) ‘Assessment : The Bridge between’, Voices from the Middle, 21(2), p. 40. 73 Available at: file:///D:/semester 6/off to thesis/VM0212Assessment.pdf. YouTube. (2019). Schools with perfect MAP scores demand excellence. [online] Available at: https://www.youtube.com/watch?v=dVG4NOH1QYU [Accessed 2 Oct. 2019]. YouTube. (2019). Seattle Teacher Uprising: Boycotting Standardized MAP Tests. [online] Available at: https://www.youtube.com/watch?v=N5ODEoqZZHs [Accessed 2 Oct. 2019]. YouTube. (2019). The MAP Test: How it works. [online] Available at: https://www.youtube.com/watch?v=jTaY3AdiB8M [Accessed 2 Oct. 2019]. Zahran, R., Khaimah, R. Al and Pettaway, L. D. (2016) ‘Educational Leadership : Challenges in United Arab Emirates’, Global elearning journal, 5(1), pp. 1–8. Курбонов, К. М. et al. (2014) ‘No TitleМини-инвазивные вмешательства под ультразвуковым контролем при эхинококковом абсцессе печени’, Актуальные Вопросы Гепатобилиарной Хирургии: Материалы Xxi Международного Конгресса Ассоциации Гепатобилиарных Хирургов Стран Снг. - Пермь, p. 97. 寺嶋, 浩介; 林朋美 (2006) ‘No Titleルーブリックの構築により自己評価を促す問題解決学 習 の開発’, 3, pp. 77–101. 74 APPENDICES Appendix A: KHDA Standards and Indicators to Measure MAP and Other External Assessments  Element 1.1.2 of Indicator 1.1 of Standard 1 in the Inspection Framework 2015-2016: Measuring Attainment against National and International Standards (KHDA, 2015): The aforementioned element targets evaluating students’ attainment in national and external examination. Consequently, students’ results and scores in MAP, PISA, TIMSS, PIRLS, PBTS, EMSAT, TOEFL, IELTS, and SAT are analyzed and students’ attainment against the set targets and criteria is evaluated.  Element 1.1.4 of Indicator 1.1 of Standard 1 in the Inspection Framework 2015-2016: Measuring Attainment Over Time against National and International Standards (KHDA, 2015): This element measures trends in students’ attainment over three years in internal and external assessments.  Element 1.2.1 of Indicator 1.2 of Standard 1 in the Inspection Framework 2015-2016: Measuring Students’ Progress in internal and external assessments, including MAP test, against starting point and over time (KHDA, 2015): This element measures students’ progress in relation to their starting point and against curriculum standards.  Element 3.1.4 of Indicator 3.1 of Standard 3 in the Inspection Framework 2015-2016: Using Strategies and Modifying Instruction to Meet the Needs of Individual and Group of Students (KHDA, 2015): The above-mentioned element aims at assessing teaching and learning in terms of using assessment data from internal and external examinations to modify instruction and vary teaching strategies to cater to the needs of different students.  Element 3.2.2 of Indicator 3.2 of Standard 3 in the Inspection Framework 2015-2016: Benchmarking Students’ Results in a Set of National, International, and External Prospects (KHDA, 2015):  Element 3.2.3 of Indicator 3.2 of Standard 3 in the Inspection Framework 2015-2016: Analyzing Assessment Data to Track Students’ Progress and Inform Planning Accordingly (KHDA, 2015):  Element 3.2.4 of Indicator 3.2 of Standard 3 in the Inspection Framework 2015-2016: Using Assessment Data to Modify Strategies and Inform Instruction to Meet the Needs of Individual and Groups of Students (KHDA, 2015):  Element 3.2.5 of Indicator 3.2 of Standard 3 in the Inspection Framework 2015-2016: Personalizing Students’ Learning based on Teachers’ Knowledge of their Students’ Strengths and Weaknesses (KHDA, 2015): All of the above elements in the Inspection Framework aim at evaluating assessment in the school in terms of understanding data, analyzing it, and then rigorously using it to construct and implement plans that promote differentiation and personalize learning and regularly track students’ progress.  Element 4.1.5 of Indicator 4.1 of Standard 4 in the Inspection Framework 2015-2016: Using Assessment Data to Monitor Students’ Achievement and Review and Develop Curriculum Accordingly (KHDA, 2015): This element evaluates the schools’ use of assessment data, including that from external and benchmark tests, to review the school’s curriculum.  Element 6.1.2 of Indicator 6.1 of Standard 6 in the Inspection Framework 2015-2016: Evaluating the Knowledge of Leadership in Curriculum and Best Practices in Teaching, Learning, and Assessment and the Focus of Leadership on Improving Students’ Achievement (KHDA, 2015): This element aims at evaluating leadership in terms of their knowledge of the educational standards and their efforts to improve students’ achievement in internal and external exams, with specific focus on the external exams and benchmark tests targeted in the UAE National Agenda. 75 Appendix B: Classroom Observation Rubrics Elements Outstanding Very Good Good Acceptable Weak Very Weak 3.1.2 Teacher plan imaginative lessons, provide Teacher plan engaging lessons, provide Teacher plan purposeful lessons, provide Teacher plan lessons, manage time and use Teacher’s planning, time management Teacher’s planning, time inspiring learning environments and use time motivating learning environments and use interesting learning environments and use resources appropriately to provide and use of resources are variable. management and use of 2. Lesson planning, the and resources creatively to enable all groups of time and resources skilfully to enable all time and resources effectively to enable environments where students can meet Teacher do not consistently provide resources are ineffective and learning environment and students to learn very successfully. groups of students to be very successful students to be successful learners. learning expectations. environments that encourage learning. the learning environments the use of time and learners. are bleak and uninspiring. resources 3.1.3 Students’ teacherinteractions, ensure that Teacher’s interactions with students Teacher’s interactions with students Teacher’s interactions with students Teacher’s interactions with students Teacher’s interactions with students are always active and focused. ensure that they are keen to learn. ensure that they are engaged learners. ensure that they are willing learners. result in disinterest. Questioning is not students result in 3. Teacher– student Questioning challenges students’ thinking and Questioning promotes higher level thinking Questioning promotes thought and Questioning and dialogue engages students sufficiently challenging, and dialogue demotivation and interactions including the promotes insightful responses. Dialogue engages and critical responses. Dialogue engages considered responses. Dialogue engages in meaningful discussions. does not engage students effectively. disengagement. Questioning use of questioning and students in insightful discussions and reflection. students in thoughtful discussions and students in meaningful discussions and and dialogue are ineffective dialogue reflection. reflection. 3.1.4 Teacher uses strategies that very successfully Teacher uses strategies that are highly Teacher uses strategies that are effective in Teacher uses strategies that adequately Teacher does not use strategies that Teacher has low 4. Teaching strategies to meet the individual needs of students. Teacher effective in meeting the individual needs of meeting the individual needs of the meet the needs of groups of students. meet the needs of groups of students. expectations. Teacher lack meet the needs of has high expectations of all groups of students. the students. Teacher consistently provide students. Teacher provide appropriate Teacher provide challenge and support Teacher does not provide appropriate the knowledge and individuals and groups of Teacher provides very challenging work and specific levels of challenge and support. levels of challenge and support. generally, but this is not always sufficiently challenge and support. understanding of how to students. excellent support. personalised. meet the needs of students. Teacher do not provide any challenge or support. 3.1.5 Teacher skilfully develop students’ critical Teacher purposefully develop students’ Teacher systematically develop students’ Teacher sometimes develop students’ Teacher rarely develop students’ critical Teacher do not develop 5. Teaching to develop thinking, problem-solving, innovation and critical thinking, problem-solving, critical thinking, problem-solving, critical thinking, problem-solving, thinking, problem-solving, innovation students’ critical thinking, critical thinking, problem- independent learning skills. innovation and independent learning skills. innovation and independent learning skills. innovation and independent learning skills. and independent learning skills. problem-solving, innovation solving, innovation and and independent learning independent learning skills skills. 1.1.1 Most students attain The large majority The majority of students Most students attain Less than threequarters Few students attain 10. Attainment as levels that are above of students attain attain levels that are levels that are in of students levels that are in measured against curriculum standards. levels that are above above curriculum line with curriculum attain levels that are line with curriculum authorised and licensed curriculum standards. standards. standards and a few at least in line with standards. curriculum standards are above. curriculum standards. 1.2.2 In lessons, most students make better than In lessons, a large majority of students In lessons, the majority of students make In lessons, most students make expected In lessons, only a majority of students In lessons, only a few expected progress in relation to appropriate make better than expected progress in better than expected progress in relation progress in relation to appropriate learning make expected progress in relation to students make expected 11. Progress in lessons learning objectives aligned with the expected relation to appropriate learning objectives to appropriate learning objectives aligned objectives aligned with the expected appropriate learning objectives aligned progress in relation to curriculum standards. aligned with the expected curriculum with the expected curriculum standards. curriculum standards and a few make with the expected curriculum standards. appropriate learning standards. better progress objectives aligned with the expected curriculum standards. 3.2.4 Assessment information Assessment information Assessment information Assessment information Assessment information Assessment information Use of assessment is used skilfully and is used very effectively is used effectively to is used adequately to is not used adequately is not used to inform information to influence effectively to influence to influence teaching influence teaching and inform teaching and to inform teaching or teaching or curriculum teaching, the curriculum teaching and the and the curriculum the curriculum in order curriculum planning in curriculum planning. planning and the needs and students’ progress curriculum in order to in order to meet the to meet the learning order to meet the needs Consequently, the of students are not met. meet the learning needs learning needs of all needs of all groups of of groups of students. needs of groups of all groups of students groups of students students and enhance of students are not and to optimise their and to enhance their their progress. adequately met. progress. progress. 3.2.5 Teacher has in-depth knowledge of the strengths Teacher has very good knowledge of the Teacher has good knowledge of strengths Teacher has reasonable knowledge of the Teacher has insufficient knowledge of Teacher’s knowledge of the 14. Teacher’s knowledge of and weaknesses of individual students. Teacher strengths and weaknesses of individual and weaknesses of individual students. strengths and weaknesses of the individual the strengths and weaknesses of strengths and weaknesses of and support for students’ provides excellent personalised challenge and students. Teacher provides personalised Teacher provides well-focussed challenge students. Teacher provides some challenge students. Students are not given enough students is very limited. learning support. challenge and support. and support. and support. challenge and support. Students are given little challenge and support. 76 ogress of students, including those Appendix C: Students’ Survey Questions MAP Testing Survey This survey has only statistical purposes. All your answers will remain confidential, and you are free to keep your survey anonymous. The main purpose of this survey is to draw conclusions about the reliability of MAP tests and their impact on students’ learning in order to consider any possible improvement. Grade: ________ Survey Indicators Strongly Strongly Agree Disagree Agree Disagree 1. I understand what MAP testing is. 2. I am aware of the importance of MAP testing. 3. I am convinced that MAP testing has an impact on my learning. 4. My academic level has improved since I started sitting for MAP testing. 5. I feel MAP tests are linked to the curriculum I am studying. 6. MAP sets realistic targets for improvement. 7. The score set for each grade level by NWEA is appropriate for my level. 8. The level of the test questions meets my expectations. 9. MAP test results reflect my actual grade level. 10. The testing environment is appropriate for me. 11. The allotted time for the test is suitable. 4. Do you think students should sit for MAP Tests? Explain why or why not? ________________________________________________________________________________________________________________________ ________________________________________________________________________________________ 5. What challenges do you face during the test? ________________________________________________________________________________________________________________________ ________________________________________________________________________________________ 6. What kind of support would you request from your teachers to improve your MAP scores? ________________________________________________________________________________________________________________________ ________________________________________________________________________________________ 77 Appendix D: Teachers’ Interview Questions 1. How long have you been using MAP at your school? 2. Why have you decided to use MAP? 3. What are you using MAP data for? 4. How are you relating MAP data to curriculum, teaching and learning, and assessment? 5. How are your students feeling about MAP testing? 6. Does MAP testing have impact on students’ learning? 7. How is MAP helping you as a teacher? 8. What are the challenges that your students face in MAP testing? 9. What are the challenges that you face as a teacher whose students sit for MAP testing? 10. What are the positives and negatives of MAP Testing? 11. Do you recommend MAP testing in schools? Why or Why not? 78 Appendix E: Leaders’ Interview Questions 1. How long have you been using MAP at your school? 2. Why have you decided to use MAP? 3. What are you using MAP data for? 4. How are you relating MAP data to curriculum, teaching and learning, and assessment? 5. How are your students feeling about MAP testing? 6. Does MAP testing have impact on students’ learning? 7. How is MAP helping in improving students’ attainment and progress? 8. What are the challenges that your students face in MAP testing? 9. What are the challenges that your teachers face in MAP testing? 10. What challenges do you face as leaders in ensuring that MAP testing is done professionally and that MAP data is used consistently? 11. What are the positives and negatives of MAP Testing? 12. Do you recommend MAP testing in schools? Why or Why not? 79 Appendix F: Samples of Transcribed Interview Sample 1 80 Sample 2 81 Sample 3 82