Blog

Evaluating A Representative State Sample of Connecticut Seventh-grade Students’ Ability to Critically Evaluate Online Information

by Elena Forzani and Cheryl Maykel
University of Connecticut

Authors’ Note: The results presented in this paper are part of a larger study with Donald J. Leu, University of Connecticut; Jonna Kulikowich, The Pennsylvania State University; Nell Sedransk, National Institute of Statistical Sciences; and Julie Coiro, University of Rhode Island, and are based upon work supported by the U.S. Department of Education under awards R305G050154 and R305A090608. Opinions expressed herein are solely those of the authors and do not necessarily represent the position of the U.S. Department of Education. The authors also thank the following individuals for their valued assistance with this study: Greg McVerry, Ian O’Byrne, Lisa Zawilinski, Heidi Everett-Cacopardo, Mike Hillinger, Mark Lorah, Donna Bone, and an exceptional team of undergraduates at the Neag School of Education at the University of Connecticut.

Abstract

This study investigated the extent to which a sample of seventh grade students (n = 591) in Connecticut critically evaluated online information both within and across three different assessment formats. The formats included Closed (simulated Internet environment requiring constructed responses), Open (actual, unrestricted Internet environment requiring constructed responses), and Multiple Choice. Results indicated that critical evaluation was more difficult for students than the three other online reading and research skill areas assessed (i.e., Locate, Synthesize, and Communicate) in all three formats combined, and was one of the most difficult of the skill areas within each of the three formats. Additionally, among the four critical evaluation tasks assessed (e.g., finding out the author of a website, determining if that author is an expert, evaluating the author’s point of view, and evaluating the overall reliability of a website), evaluating the author’s expertise and evaluating the overall reliability of a website was the most difficult for students. Finally, students performed better on critical evaluation tasks in the Multiple Choice format than they did in either of the two performance-based formats. Findings suggest that critical evaluation persists as one of the most difficult online comprehension and research skills for students, especially when measured in a performance-based format.

Evaluating A Representative State Sample of Seventh-grade Students’ Ability to Critically Evaluate Online Information

The new Common Core State Standards (2012) that Connecticut has adopted call for students to “assess the credibility and accuracy” of a variety of digital information sources (p. 41). This means that today’s students must become proficient not just at gathering information sources and using them to produce writing, but also at evaluating them first to determine their relevancy and accuracy for the task at hand. As more and more of the texts students read and use move online, this skill becomes increasingly important for readers.

Digital information sources, like the Internet, have necessitated the use of new literacy skills as well as new ways of thinking about traditional literacy skills (Coiro, Knobel, Lankshear, & Leu, 2008; Lankshear & Knobel, 2006), such as source evaluation. Students today must learn how to conduct online research and comprehend various types of online texts if they are to be successful both with the Common Core standards and in today’s digital world (Common Core State Standards Initiative, 2012; Organisa-tion for Economic Co-operation and Development & the Centre for Educational Research and Innovation, 2010). When using online information, higher-level skills, such as critical evaluation (CE), become especially important (Goldman, Braasch, Wiley, Graesser, & Brodowinska, 2012), since anyone can publish to the Internet (Cope & Kalantzis, 2000; Fabos, 2008). Therefore, the reader, rather than a publisher, bookseller, or other intermediary, becomes the first, and, in many cases, only judge of the accuracy and reliability of information.

Many may assume that today’s students are skilled at effectively collecting and communi- cating reliable online information, since they have, presumably, used the Internet for much of their lives. However, this assumption may not be accurate. Although adolescent “digital natives” (Prensky, 2001) may be skilled with texting, gaming, social networking, creating mash-ups from multiple media sources, and downloading video and MP3 files, they are not always as skilled with the use of online information, and especially with the CE of online information (Bennett, Maton, & Kervin, 2008; Sutherland- Smith, 2002; Wallace, Kupperman, Krajcik, & Soloway, 2000). In fact, adolescents often overgeneralize their ability to read and research online information effectively because they are skilled with other online and tech-related tasks (Grimes & Boening, 2001; Kuiper, 2007).

Success in conducting research online often is dependent on the reader’s evaluation of the information found (Goldman et al., 2012; Wiley et al., 2009). As students search for and synthesize information from various sources, CE skills help guide their decisions about the accuracy of that information (Goldman, Braasch, Wiley, Graesser, & Brodowinska, 2012). A recent study showed how valuable CE skills are to an online research task. Goldman and colleagues (2012) found that students who were more skilled in CE were more focused and efficient (Goldman et al., 2012). When students searched for and incorporated information from various sources, CE skills guided their decisions about the accuracy of that information, helped them to determine what information to use from each source, and informed them of what to look for next (Goldman et al., 2012). As the Internet becomes increasingly central to full participation in today’s society, the critical evaluation of information found online becomes more important for both students and educators to understand.

Perspectives and Theoretical Background

The current study is framed by both a dual level theory of New Literacies (Coiro, Knobel, Lanskshear, & Leu, 2008; Leu, O’Byrne, Zawilinski, McVerry, & Everett-Cacopardo, 2009) and by perspectives on the critical evaluation of online information, especially the reliability of sources. It builds on previous work to investigate how well students in Connecticut critically evaluated online information both within and across three different assessment formats.

New Literacies: A Dual Level Theory

As the pace of technology change accelerates, so too does the pace with which literacies change. The literacies we use in our everyday and working lives are thus always continuously new. This poses a challenge for educators, who must keep up with the many new literacies available to their students. Some (Leu, O’Byrne, Zawilinski, McVerry, & Everett-Cacopardo, 2009; Leu, Kinzer, Coiro, Castek, & Henry, 2013) have thus proposed a dual level theory of New Literacies to address the challenge of conceptualizing literacies that are constantly changing. This theory conceives of literacy as having two interacting levels: an uppercase New Literacies and a lowercase new literacies. Uppercase New Literacies are broader, more stable, and consist of multiple, integrated perspectives. Lowercase new literacies are more rapidly changing and are comprised of more specific tools, such as text messaging (e.g., Lewis & Fabos, 2005), or of focused disciplinary areas, such as the semiotics of multimodality in online media (e.g., Kress,
2003). The frequent changes occurring within new literacies are guided by the broader, uppercase New Literacies, just as New Literacies are expanded upon and informed by changes within the specific contexts of the lower case literacies.

A commonality across uppercase New Literacies is that the Internet facilitates the advent of new online social practices (lowercase new literacies) that use lower case technologies, such as instant messaging, wikis, blogs, email, search engines, and social networks (Greenhow, Robelia, & Hughes, 2009; Lankshear & Knobel, 2006). The assessment used in this study was situated within a social network environment that required students to interact with student avatars through instant messages, emails, and wikis in the process of completing a research task. The assessment was thus informed by the uppercase concept that acknowledges the importance of online social practices while at the same time utilizing many lower case new literacies and acknowledging that online social practices occur with the use of many different tools.

The new literacies of online research and comprehension (Coiro, 2003; Leu, et al., 2011) is one of many lowercase theories. This theory seeks to describe what happens when we conduct research and read online. It suggests that at least five processing practices occur during online research and comprehension with a complex layering of both traditional and new skills and strategies that appear in several areas: 1) reading to define important questions or problems (Leu, Kinzer, Coiro, & Cammack, 2004); 2) reading to locate information (Bilal, 2000; Guinee, Eagleton, & Hall, 2003); 3) reading to evaluate information (Sanchez, Wiley, & Goldman, 2006); 4) reading to synthesize information (Goldman, Wiley, & Graeser, 2005; Leu et al., 2013; Jenkins, 2006); and 5) reading and writing to communicate information (Greenhow, Robelia, & Hughes, 2009). Within these five areas reside the skills, strategies, and dispositions that are both important for offline reading comprehension and also distinctive to online research and comprehension. This creates an interaction of both old and new literacies that we are still seeking to fully understand.

In the current study, we used both levels of New Literacies theory to frame our investigation. An uppercase theory of New Literacies suggests that new, online social practices have become important. Online research and comprehension, one of several lower case theories of new literacies, suggests that locating, evaluating, synthesizing, and communicating information are important areas to consider when we conduct research and read online. Thus, we evaluated students’ ability to locate, evaluate, synthesize and communicate information within an online research task that required students to engage in several social practices using text messaging, wikis, email, search engines, and a social network. We focused particular attention in this study on the evaluation of online information, specifically the evaluation of author, point of view, and reliability of source.

Critical Evaluation

The critical evaluation of online information is one of the most important skill sets required by readers today (Goldman, et al., 2012; Wiley et al., 2009). Yet, it is often the area of online research and comprehension with which students struggle the most (Kuiper & Volman, 2008). Lower-level skills, such as locating information on the Internet, may be easier for students to master than higher-level skills, such as evaluating the source and reliability of information. Thus, students may acquire and use information without having the skills to effectively evaluate its accuracy (Grimes & Boening, 2001). Moreover, students may overestimate their ability to critically evaluate online sources (Grimes & Boening, 2001). Students who are less skilled at determining the quality of information and who merely locate information without strategically evaluating it may end up falling behind their more savvy peers, who have the skills to effectively evaluate information before deciding whether and how to use it.

Research on critical evaluation has focused on a variety of information quality markers (e.g., accuracy, authority, comprehensiveness, coverage, currency, objectivity, reliability, and validity), but it often condenses these markers to credibility and relevance as the two main constructs (Judd, Farrow, & Tims, 2006; Kiili, Laurinen & Marttunen, 2008). This study focused on the credibility of the author or source of a website, defined in terms of expertise (Bråten, Strømsø, & Britt, 2009; Judd, Farrow, & Tims, 2006; Rieh & Belkin, 1998), and on the evaluation of the reliability of information (Goldman, et al., 2012; Kiili, Laurinen, & Marttunen, 2008; Sanchez, Wiley, & Goldman, 2006).

Much of the previous research on critical evaluation has focused on college students’ abilities (Bråten, Strømsø, & Britt, 2009; Goldman, et al., 2012; Sanchez, Wiley, & Goldman, 2006). This research has had an important impact, leading to critical evaluation and higher-level thinking becoming important components of the recent Common Core State Standards (2012) in the U.S. This research also has had a similar impact on frameworks for K-12 education in other nations such as the recent Australian Curriculum (Australian Curriculum Assessment and Reporting Authority, n.d.). While our understanding of college-aged students’ ability to evaluate information, especially online information, has gained greatly from this work, we know much less about younger students’ ability to critically evaluate online sources. Given that this is now part of many nations’ curriculum frameworks, it is an important area of inquiry. Teachers need to know students’ current capabilities as they begin to plan for and teach these important aspects of curriculum.

Thus, this study sought to determine how well students in Connecticut performed on a measure of critical evaluation compared to three other online research and comprehension skills: locating, synthesizing, and communicating online information. This study also evaluated how well students performed in four different aspects of critical evaluation. Two of these were related to the credibility of the author or source of a website, defined in terms of expertise (Judd, Farrow, & Tims, 2006; Rieh & Belkin, 1998), and two were related to the evaluation of the reliability of information (Bråten, Strømsø, & Britt, 2009; Goldman, et al., 2012; Kiili, Laurinen, & Marttunen, 2008; Sanchez, Wiley, & Goldman, 2006).

Specifically, this study evaluated seventh grade students on their ability to: 1) identify the author of a webpage; 2) evaluate the author’s expertise; 3) identify the author’s point of view; and 4) evaluate the overall reliability of the webpage. This study also sought to determine how well students performed on critical evaluation in three separate assessment formats, including a closed Internet assessment context (Closed, a simulated Internet environment), an open Internet assessment context (Open, the actual, unrestricted Internet), and a multiple choice context. While all three formats followed similar research scenarios, only the Closed and Open formats were perform- ance-based, and most directly represented an actual online research experience.

Method

Participants

This study is part of a larger study that sampled seventh-grade students in two states in the northeastern United States. The present study, however, reports on the results of a representative sample of students’ performance from only one of these two states. A total of 19 school districts were included in the sample, with one participating school per district. In each school, one teacher with two classes of approximately 20 students participated. In a few smaller schools, it was necessary to include two teachers with one class of approximately 20 students each. Districts and schools were selected using stratified random sampling. The sampling plan stratified schools according to three factors: 1) district percentage of Free and Reduced Price Lunches, (a proxy measure of socioeconomic status); 2) performance on the state reading comprehension assessment; and 3) geographical location (rural, urban, and suburban). This was done while taking note of school size. Schools were randomly sampled within each of these strata.

Principals at each of the selected schools identified the English Language Arts teacher or teachers (in the case of smaller schools) whose students best represented the school population and who were willing to participate. Teachers then selected two of their classes that best fit this description. Students from the selected classrooms who had parental consent and who gave their assent were allowed to participate in the ORCA assessments. This included a total of 725 seventh graders. Each student was assigned to complete one assessment activity on each of two days. The majority of students completed both of the planned assessment activities. However, due to absences and a few system errors, 18.5 percent of the sample did not complete both activities. Thus, the final sample for the present study included 591 students.

Online Research and Comprehension Assessments (ORCAs)

Eight research scenarios were developed using eight different life science topics, all requiring students to read and conduct research online. Each of these scenarios was developed in three different formats that included Closed, Open, and Multiple Choice (see Table 1). The Closed format allowed students to conduct their research in a closed online environment. This environment was created so that students could search for, select, and use websites from the project’s search engine, “Gloogle,” which was only populated with a predetermined set of websites. The Open format allowed students to search for, select, and use websites from the actual, Open Internet using Google. The Closed and Open formats were thus largely performance-based measures. Finally, the Multiple Choice format confined students to selecting sites and answers from a set of four answer choices per question. Each question and answer set was accompanied by screenshots of the websites or other web tools (e.g. emails, wikis) that students needed to use in order to successfully answer the questions. Students could toggle between the different screenshots as needed by clicking on various links or tabs. The Multiple Choice format thus attempted to provide students with a richer context than traditional multiple choice assessments.

In all scenarios, students were presented with science research problems that focused on the domain of health and human body systems, an area common to many seventh grade science curricula, with each of the eight scenarios focusing on a different topic. All topics are listed in Table 1. The scenarios were framed around two types of research: “Learn More About (LMA)” and “Investigate Conflicting Claims (ICC).” Half of the scenarios presented the research problem to students via an email message from the school board president (LMA scenarios) and half via a class wiki with a message from the teacher (ICC scenarios). LMA scenarios asked students to learn more about the research topic and to form a main idea about what they learned. ICC scenarios, on the other hand, asked students to investigate two sides of an issue and to take a position (See Table 1).

Each scenario included items assessing students’ ability to locate, evaluate, and synthesize information during their research. The scenarios also included items assessing students’ ability to communicate the results of the research via either email or wiki. Each scenario, called a LESC, represented each of the four skills areas of Locate, Evaluate, Synthesize, and Communicate with 16 score points per LESC and 4 score points per skill area. Each score point evaluated an online research and comprehension skill identified both from previous research and from discussions with researchers in this area. Each skill area (Locate, Evaluate, Synthesize, and Communicate) included three process skills and one product skill, with one score point assessing each skill, for a total of four score points in each of the four LESC skill areas.

The LESC questions appeared within a Facebook-like environment through avatars named Brianna and Jordan, who were introduced as students from another school. The questions did not appear in a linear sequence according to skill area. Rather, a more natural and logical sequence was used according to the nature of the research task. Students were guided to engage in the four skill areas through their online research tasks via requests and questions from Brianna and Jordan.

Table 1: The Eight LESC Scenarios by Topic

Topic
Research Question
Type of Research
Communication Tool Used in the Research
Energy Drinks
How do energy drinks affect heart health?
Learn more about
Email
Heart-Healthy Snacks
How do snacks affect heart health?
Learn more about
Email
Volume Level
Can listening to volume levels on an MP3 player cause hearing loss?
Learn more about
Email
Ringtones
How well can adults hear mosquito ringtones?
Learn more about
Email
Third-hand Smoke
Is third-hand smoke dangerous to lung health?
Investigate conflicting claims
Wiki
Asthma
Can Chihuahua dogs cure asthma?
Investigate conflicting claims
Wiki
Contact Lenses
Do cosmetic contact lenses harm your eyes?
Investigate conflicting claims
Wiki
Video Games
Do video games harm your eyes?
Investigate conflicting claims
Wiki

Note: Each Topic was developed in three different formats that included Closed, Open, and Multiple Choice.

The four score points for CE related directly to three of the traditional critical evaluation criteria that include authority, objectivity, and accuracy (Judd, Farrow, & Tims, 2006; Rieh & Belkin, 1998; Bråten et al., 2009; Goldman, et al., 2012; Kiili, Laurinen, & Marttunen, 2008; Sanchez, Wiley, & Goldman, 2006). Students were prompted by Jordan to determine the author of a given website (authority), evaluate the author’s expertise (authority), identify the author’s point of view with a supporting detail (objectivity), and evaluate the overall reliability of the site using at least one piece of valid reasoning (accuracy). The responses for the four CE score points were obtained through an instant message conversation with the avatar Jordan, who prompted students to access a website at a provided link. From the website, students had the opportunity to navigate to the author biography page, which was hyperlinked to the given site. If students navigated to the biography page, they then had the opportunity to gather more information on the author to inform their responses.

However, students were not directly asked to navigate to the biography page, and the link appeared somewhat differently in different LESCs, depending on the site that was used. Therefore, not all students accessed the additional information, and responses varied greatly.

Scoring the ORCA

An auto-capture system recorded students’ responses for all score points for later scoring. Video screen captures recorded students’ performance as a backup for the auto-capture system, and to score search activities that occurred outside of the assessment system in the Open Internet format. Three process score points and one product score point were calculated for each of the four major skill areas (Locate, Evaluate, Synthesize, and Communicate) using a binary (1 or 0) score point system. Each student completed two LESCs, so each student’s final score was comprised of an overall total of 32 score points.

The Multiple Choice reports were scored automatically by the ORCA scoring system. However, the Closed and Open reports were hand- scored by a team of eight scorers, with one scorer assigned to one of the eight topics each. Scorers were trained by two expert scorers to a minimum inter-rater reliability level of 90% accuracy for each score point. Each scorer was then released to score his or her LESC topic. Throughout the scoring process, the scoring of each score point was checked using a random sample of 20 student reports by one of two expert scorers within each set of 100 reports scored (20% of all Closed and Open assessments). Scorers who did not continue to meet 90% accuracy for each score point, within each set, were retrained and retested to this level before continuing scoring.

Procedures

LESC Administration

The ORCAs were administered during two assessment days held at each school. Before testing began, students were assigned to assessment topics and formats following a specific assignment plan that was designed to ensure equal and random assignment of students from various schools across LESCs. They were then entered into the ORCA database and assigned a unique identification number by the system. On each assessment day, students were read brief, standardized instructions before beginning the ORCAs, which used an automated start-up sequence on a set of MacBook Airs. By entering their unique ORCA identification numbers into the login screen, students were brought directly to their assigned ORCA in the online system. Students who typically received accommodations in the classroom received the same accommo- dations during the ORCA assessments. The test administrators for the ORCA were two graduate students from the university who, together with the lead Investigator, developed a protocol for school set up and test administration.

Scoring Procedures

The operational definition for each score point was similar across all three formats of the ORCA: Closed, Open, and Multiple Choice. However, the scoring process differed slightly for each format. For the Closed and Open formats, score reports were generated by the data capture tool of the ORCA system for each completed LESC and were used to score the Closed and Open formats, with one exception. In the Open condition on Synthesis tasks, QuickTime videos were used to score the Locate questions since the auto capture system could not capture students’ searches on the open Internet.

Analysis

Analysis of variance (ANOVA) was used to answer all four of the present study’s research questions:

  1. How well do seventh-grade students, in all three formats combined (Closed, Open, and Multiple Choice), perform on critical evaluation compared to three other online research and comprehension skills (locating, synthesizing, and communicating)?;
  2. How well do seventh-grade students perform in four dimensions of critical evaluation, including identifying the author of a webpage, evaluating the author’s expertise, identifying the author’s point of view, and evaluating the overall reliability of the webpage?;
  3. How well do seventh-grade students perform in each format separately on critical evaluation compared to three other online research and comprehension skills, including locating, synthesizing, and communicating information?; and,
  4. How well do seventh-grade students perform on critical evaluation, comparatively, in each of the three formats?

Results

Prior to the statistical analysis, all data were examined and found to meet assumptions of analysis of variance (ANOVA), including repeated measures ANOVA. A bonferroni correction was used to control for Type I error when conducting all post-hoc comparisons. To investigate the first research question, a one-way repeated measures ANOVA was conducted to compare students’ scores in each of the four skill areas in all three formats combined. Multivariate statistics revealed that there was a significant effect for LESC skill area score, Wilks’ Lambda = .418, F (3, 588) = 272.76 p <.0005, multivariate partial eta squared = .582. An analysis of pairwise comparisons showed that there was a significant difference among each of the four skill areas and each other skill area (p < .05 for all pairwise comparisons). Students’ scores were highest in Synthesize (M = 6.07, SD = 1.81), followed by Locate (M = 4.52, SD = 2.21.), Communicate (M = 4.22, SD = 2.28) and, finally, by Evaluate (M = 3.61, SD = 1.88). Thus, students scored the lowest on Evaluate (Table 2).

Table 2: Student Performance by LESC Skill Area Within and Between Three Formats

Locate M (SD)
Evaluate M (SD)
Synthesize M (SD)
Statistical Test M (SD)
Effect Size M (SD)
All Three Formats**
4.52 (2.21)
3.61 (1.88)
6.07 (1.81)
4.22 (2.28)
F (3, 588) = 272.76
np2 = .58
Closed only
3.85 (2.27)
2.84 (1.54)
6.32 (1.76)
3.12 (1.86)
F (3, 191) = 327.44
multivariate np2 = .84
Open only
4.44 (2.32)
2.71 (1.43)
6.06 (1.86)
3.00 (1.74)
F (3, 167) = 209.14
multivariate np2 = .79
Multiple Choice only
5.15 (1.87)
4.95 (1.67)
5.87 (1.78)
6.07 (1.67)
F (3, 224) = 43.19
multivariate np2 =.37

Note: p < .05

To address the second research question, a second one-way repeated measures ANOVA was conducted to compare students’ scores on the four Evaluate skills. The means and standard deviations are presented in Table 3. Multivariate statistics showed a significant overall effect for all four Critical Evaluation score points, Wilks’ Lambda = .390, F (3, 588) = 306.950, p < .0005, multivariate partial eta squared = .61. An examination of pairwise comparisons showed that there was a significant difference in student performance between each of the four score points and each other score point (p < .0005 for each pairwise comparison), except between score point 2 (evaluating author expertise) and score point 4 (determining the overall reliability of a website). Score point 1 (determining the author of a website) had the highest mean (M = 1.62, SD = .61), followed by score point 3 (determining the author’s point of view and providing supporting evidence; M = .77, SD = .77), score point 2 (determining author expert status; M = .65, SD = .
74), and, finally, by score point 4 (evaluating the reliability of a website; M = .57, SD = .72). Thus, students’ scored significantly higher on score point 1 (determining the author of the website) than on score points 2, 3, and 4. Similarly, scores on score point 3 (author’s point of view) were significantly higher for students than on score point 4 (evaluating the reliability of a website). However, score point 2 (author expertise) and score point 4 (evaluating the reliability of a website) were not significantly different, meaning students performed at a similar level on these two different questions.

Table 3: Student Performance by Critical Evaluation Score Point Dimension in All Three Formats Combined

Score Point 1
Score Point 2
Score Point 3
Score Point 4
Statistical Test
Effect Size
Determining the author of the website
Evaluating the author’s expertise
Identifying the author’s point of view and one piece of evidence that supports that point of view
Evaluating the overall reliability of the site using one piece of evidence from the site
– – –
– – –
1.62 (.61)
.65 (.74)
.77 (.77)
.57 (.72)
F (3, 588) = 306.95
np2 = .61

 

To address the third research question, three repeated measures ANOVAs were used to compare mean differences in CE to mean differences in each of the other four LESC skill areas (Locate, Synthesize and Communicate), within each of the three formats (Closed, Open, and Multiple Choice). The means and standard deviations of these analyses are presented in Table 2. The first repeated measures ANOVA was conducted to compare scores in the Closed format. Multivariate results show that there was a significant overall effect for LESC Skill Area in the Closed format, Wilks’ Lambda = .163, F (3, 191) = 327.44, p < .0005, multivariate partial eta squared = .84. Follow up, post hoc analyses of pairwise comparisons showed that each LESC Skill Area was significantly different from each other LESC skill area (p < .005), except for Evaluate and Communicate. This indicated that student performance in these two skill areas was not statistically different in the Closed format. Students scored higher on synthesize (M = 6.32, SD = 1.76) than on Locate (M = 3.85, SD = 2.27), followed by Communicate (M = 3.12, SD = 1.86) and Evaluate (M = 2.84, SD = 1.54).

The second one-way repeated measures ANOVA was conducted to compare scores in each of the four skill areas in the Open format. These means and standard deviations are also presented in Table 2. There was a significant effect for LESC Skill Area, Wilks’ Lambda = .210, F (3, 167) = 209.14, p < .0005, multivariate partial eta squared = .79. Follow-up post hoc analyses of pairwise comparisons showed that each LESC Skill Area was significantly different from each other LESC skill area (p < .005), except Evaluate and Communicate, as was found in the Closed format. Synthesize (M = 6.10, SD = 1.74) scores averaged higher than Locate (M = 4.44, SD = 2.32), Communicate (M = 3.00, SD = 1.74), and Evaluate (M = 2.71, SD = 1.43) scores, with Locate scores ranking second highest. Communi- cate and Evaluate score averages were lowest.

The third one-way repeated measures ANOVA was conducted to compare scores in each of the four skill areas in the Multiple Choice format. The means and standard deviations are presented in Table 2. There was a significant effect for LESC Skill Area, Wilks’ Lambda = .63, F (3, 224) = 43.19, p < .0005), multivariate partial eta squared = .37. Additionally, post hoc analyses of pairwise comparisons showed that there was significant difference (p < 0005) between Locate and Synthesize, Locate and Communicate, Evaluate and Synthesize, and Evaluate and Communicate. Evaluate scores were significantly lower (M = 4.95, SD = 1.67) than both Synthesize (M = 5.87, SD = 1.78) and Communicate scores (M = 6.10, SD = 1.67), but not significantly lower than Locate (M = 5.15, SD = 1.87) scores.

To answer the fourth and final research question, a one-way, between groups ANOVA was conducted to evaluate whether there was a significant mean difference in CE scores between the three formats, including Closed, Open, and Multiple Choice. A one-way between groups ANOVA was conducted to determine how well CE performed in each of the three LESC formats. Means and standard deviations are presented in Table 4. There was a statistically significant difference at the p < .0005 level in CE scores for the three formats: F (2, 588) = 135.69, p = .000. The effect size, measured using eta squared, was .316. Post-hoc comparisons indicated that the mean score for CE in the Multiple Choice format (M = 4.95, SD = 1.67) was significantly different from mean scores of CE in both the Closed format (M = 2.84, SD = 1.54) and the Open format (M = 2.71, SD = 1.43). Students scored higher on CE in the Multiple Choice Format than in either the Closed or Open formats. There was no statistically significant difference for CE between the Closed and Open formats.

Table 4: Student Performance on Critical Evaluation in Each of the Three Formats: Closed, Open, and Multiple Choice M (SD)

Format Group
Format Group
Statistical Test
p
Multiple Choice 4.95 (1.67)
Closed 2.84 (1.54)
Open 2.71 (1.43)
F (2, 588) = 135.69
.000
.000

Note: Effect size: Eta squared = .316

Discussion

This study sought to determine how well seventh graders in a large, representative state sample (n = 591) critically evaluated online information. Specifically, this study examined students’ performance in overall CE compared to their performance in three other skill areas, both within and across three different assessment formats. It also evaluated how well students performed in CE in each of the three formats.

Comparing CE to Locate, Synthesize and Communicate in All Formats Combined

Results from the analysis of our first research question indicated that CE was the most difficult of the four skill areas for students in all three formats combined, though the difference between CE and Communicate in the Open and Closed formats was not statistically significant. This finding supports an existing body of research that shows online CE is one of the most difficult online reading comprehension skills. As with studies of CE among older, college-aged students (Bråten, Strømsø, & Britt, 2009; Goldman, et al., 2012; Sanchez, Wiley, & Goldman, 2006), in this study, CE persisted as one of the most difficult skill areas for this younger, seventh-grade student population. The current study also demonstrates that even within a performance-based enviro- nment like the Closed and Open formats that more closely mimics an authentic Internet context, CE was one of the most challenging of the four skill areas for students.

Therefore, CE is one of the five skill areas of the new literacies of online research and comprehension (Coiro, 2003; Leu, et al., 2011) that may warrant the most instructional attention.

However, additional research is needed to determine in what ways CE is more difficult than other online reading and research skills, and how teachers should approach instruction of these skills. We do not know, for example, the types of challenges CE poses for students, or the ways in which students typically understand CE and attempt to use it when gathering sources. A follow-up qualitative analysis of students’ responses to the four critical evaluation questions would be useful in adding to our understanding of this issue. Nevertheless, findings from the present study can inform both research and practice by helping to make us more aware of the significant difficulty students face when attempting to evaluate the information they find online.

Comparing the Four Dimensions of CE in All Formats Combined

Findings from the analysis of our second research question also can inform research and practice by showing us which online CE dimensions are most difficult for students and where there is a greater need to focus instruction. Students scored highest on score point one, identifying the author of the website (M = 1.62, SD =.61). This was followed in order of difficulty by score point three, or identifying the point of view of the author and a piece of evidence that supports that point of view (M = .77, SD = .77). There was no statistically significant difference in student performance between score points two and four, though score point two, evaluating the expertise of the author, had a higher mean score (M = .65, SD = .74) than score point four, evaluating the overall reliability of a site (M = .57, SD = .72).

These results show that score point two (determining author expert status) may have been more difficult for students than was score point three (providing author’s point of view). One reason for this could be that score point two measures a higher-level skill than score point three. Although the score points were designed to be increasingly challenging, it appears that students actually had more difficulty determining expert status, even though it came before evaluating point of view in the task. However, it may be useful for students to evaluate the author’s expertise prior to examining the author’s point of view. Students’ knowledge of author background and expertise may help to inform their evaluation of an author’s point of view. This raises questions about whether skills in an assessment of online comprehension and research should be ordered from lower to higher levels of difficulty, or if it is more important for the questions to follow the logical sequence of the task. It may also be that an assessment that mirrors the complexities of an authentic online research experience, one in which students are naturally and logically moving back and forth between lower- and higher-level skills, is the best kind of assessment to determine students’ actual capabilities.

Comparing CE to Locate, Synthesize, and Communicate in Each Format

When we investigated our third research question, we found a significant difference in the mean scores of Evaluate compared to the mean scores of Synthesize, in each of the three formats, with students scoring higher on Synthesize items than on Evaluate items. In the Closed format, the mean scores for Evaluate also were significantly lower than those for Locate. In the Multiple Choice format, the mean scores for Evaluate were significantly lower than those for Communicate. As the analyses that combined the three formats showed, the difficulty of CE persisted when we looked at its effect in each of the three formats separately, especially compared to Synthesize. Thus, CE was one of the most difficult of the four skill areas regardless of the format in which it was assessed.

In the Closed and Open formats, it may be that Communicate posed as great a challenge as Evaluate, since students had to know how to use the email or wiki communication tools in order to be successful. In the Multiple Choice format, these questions were simplified, as students did not have to perform these actions but simply had to choose from a set of answers. Thus, it makes sense that Evaluate would be significantly harder than Communicate in the Multiple Choice format.

That CE persisted as one of the most difficult of the four skills across all three formats may suggest that all three formats are valid ways of measuring students’ ability to critically evaluate information online. It may also suggest that CE is, in fact, one of the most difficult of the four online reading and research skill areas for seventh-graders, since students consistently scored lower in this skill area regardless of the format in which it was measured. Teachers should thus pay particular attention to both instruction and assessment of this important yet challenging skill.

Comparing CE in Three Formats: Closed, Open, and Multiple Choice

When we compared CE in the three formats to investigate our fourth and final research question, we found a significant difference in the mean scores of CE in the Multiple Choice format compared to both the Closed and Open formats, though there was no significant difference in mean scores between the Closed and Open formats. While the three formats were developed to be similar to one another, these results show that CE poses less of a challenge in the Multiple Choice format than it does in the other two formats.

One reason for this may be that the Multiple Choice format offers a time advantage to test takers that the other two formats do not. The four CE score points appear in a linear sequence about three quarters of the way into each assessment in all three formats. Students tended to finish the Multiple Choice sessions much more quickly than they finished the Closed or Open sessions. Thus, it is possible that students taking the Closed and Open formats were fatigued by the time they engaged in the four CE skills, while students taking the Multiple Choice format were not.

It is also important to consider that students taking the Multiple Choice test may have had a navigational advantage that students taking the other two formats did not. Students taking the Closed and Open formats had to click on a link in order to navigate to the website they were to evaluate. The CE website contained a hyperlink that students could click on in order to obtain information about the author on the author biography page. However, the student had to decide whether or not to access this page and to figure out how to access the page with additional information. In the Multiple Choice format, however, both the CE website and corresponding author biography page were presented to students alongside the question and answer choices. Thus, students taking the Multiple Choice format had a greater chance of reading both pages since they were guided to do so.

A third possible reason that CE performed better in the Multiple Choice format than in either the Closed or Open format may be that CE was measured somewhat differently in the Multiple Choice format. Because of the nature of multiple choice testing, it is possible that the presentation of the CE items may have been less complex in this format, and may therefore have required less cognitive demand for students than it did in the other two formats. Rather than generating a response to the four CE questions on their own, as they were required to do in the Closed and Open formats, students taking the Multiple Choice assessment only had to choose from four possible answers. Each question also was presented on its own with its own images to use as reference points, whereas in the Closed, students had to manage multiple windows and types of information, including a notepad, a search window, the social networking site, and the email or wiki window. The CE task was thus much more complex in the Closed and Open formats than in the MC format.

Non-performance based assessments, such as the Multiple Choice format used in the present study, may overestimate students’ critical evaluation abilities. While performance-based assessments such as the Closed and Open formats used in the current study may be more difficult and time consuming to construct and score than non-performance based formats, they may also more accurately estimate students’ abilities. Test creators of multiple choice assessments, and those using and interpreting test results, should therefore keep this in mind when examining test data and forming conclusions about students’ ability to critically evaluate online information.

Implications and Limitations

Findings from this study contribute to literacy research and teaching practices in several key ways. First, findings add to existing research on CE by expanding our knowledge of how students perform in CE when it is assessed in performance- based and non-performance based ways. This study is one of the first to evaluate adolescents’ use of CE in an online environment within a performance-based assessment. Thus, the findings from this study, especially those that compare the three formats, are particularly informative for understanding how students actually conduct research in an online context.

Second, findings contribute to a growing body of research on CE showing it is a difficult skill area for students. CE may be one of the five online reading comprehension skills that is the most difficult for students and thus warrants the most careful instructional attention. Findings can inform existing literature on how students perform in online CE to support future studies and practice. Findings thus inform thinking about on which online skill areas teachers should focus the most, given what many students currently are able to do. Additionally, results show with which dimensions of CE students struggle the most and thus can guide teachers to focus on teaching and assessing the most complex and nuanced skills involved in the already complex skill area of CE. This may be especially timely, as teachers will need to teach and assess these types of skills with the implementation of the new Common Core State Standards (2012) in 2014.

Finally, findings from this study raise important questions about how best to teach and assess the CE of online information. The analyses conducted in this study show that CE may be one of the most persistently difficult skills for students when reading and conducting research online. The analyses conducted in this study do not show what effective instruction that addresses deficits in CE skills might entail, and spending more time teaching CE skills will not necessarily result in increasing students’ ability to effectively evaluate online information. Additionally, teachers may not have adequate technological skills to begin teaching online CE to their students. Thus, more research needs to be conducted to determine what effective versus ineffective instruction in CE of online information entails and how teachers can prepare for this instruction.

Without knowing how to teach and assess CE, we risk students learning only lower-level digital literacies skills, such as locating information, without also learning the higher-level skills necessary for using that information effectively. As teachers begin to plan for and implement the Common Core State Standards, an important question for both researchers and practitioners to ask is: What is the best approach to teaching and assessing online CE skills, which may be the most difficult and yet also the most critical for students to learn when reading and conducting research online?

References

Australian Curriculum Assessment and Reporting Authority. (n.d.). The Australian Curriculum, v1.2. Retrieved from www.australiancurriculum.edu.au/Home

Bennett, S., Maton, K., and Kervin, L. (2008). The ‘digital natives’ debate: A critical review of the evidence. British Journal of Educational Technology, Vol. 39, No. 5., pps. 775-786.

Bilal, D. (2000). Children’s use of the Yahooligans! Web search engine: Cognitive, physical, and affective behaviors on fact- based search tasks. Journal of the American Society for Information Science, 51, 646–665.

Braten, I., Strømsø, H. I., & Britt, M. A. (2009). Trust matters: examining the role of source evaluation in students’ construction of meaning within and across multiple texts. Reading Research Quarterly, 44, 6-28.

Coiro, J. (2003). Reading comprehension on the Internet: Expanding our understanding of reading comprehension to encompass new literacies. The Reading Teacher, 56, 458-464.

Coiro, J., Knoble, M., Lankshear, C., & Leu, D. J., Jr. (Eds.) (2008). Handbook of research on new literacies. Lawrence Erlbaum, Mahwah, NJ.

Common Core State Standards Initiative (2012). Common Core State Standards Initiative: Preparing America’s Students for College and Career. Retrieved from http://www.corestandards.org.

Cope, B., & Kalantzis, M. (2000). Multiliteracies: Literacy learning and the design of social futures. London: Routledge.

Fabos, B. (2008). The price of information: Critical literacy, education, and today’s Internet. In J. Coiro, M. Knobel, D. Leu, & C. Lankshear (Eds.). Handbook of research on new literacies (pp. 839-870). Mahwah, NJ: Erlbaum.

Goldman, S., Braasch, J., Wiley, J., Graesser, A., & Brodowinska, K. (2012). Comprehending and learning from Internet sources: Processing patterns of better and poorer learners. Reading Research Quarterly, 47, 356-381. doi: 10.1002/RRQ.027

Goldman, S. R., Wiley, J., & Graesser, A. C. (2005, April). Literacy in a knowledge society: constructing meaning from multiple sources of information. Paper presented at American Educational Research Association. Montreal, Canada

Guinee, K., Eagleton, M. B., & Hall, T. E. (2003). Adolescents’ Internet search strategies: Drawing upon familiar cognitive paradigms when accessing electronic information sources. Journal of Educational Computing Research, 29, 363–374.

Greenhow, C., Robelia, B., & Hughes, J. (2009). Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38(4), 246–259.

Grimes, D. & Boening, C. (2001). Worries with the Web: A look at student use of Web resources. College & Research Libraries, 62(1) 11-22.

Jenkins, H. (2006). Convergence Culture: Where Old and New Media Collide. New York: New York University Press.

Judd, V.C., Farrow, L.I. and Tims, B.J. 2006. Evaluating public web site information: a process and an instrument. Reference Service Review, Vol. 34, No. 1, pp. 12-32.

Kiili, C., Laurinen, L., & Marttunen, M. (2008). Students evaluating Internet sources: From versatile evaluators to uncritical readers. Journal of Educational Computing Research, 39(1), 75–95.

Kress, G. (2003). Literacy in the new media age. London, UK: Routledge.

Kuiper, E., & Volman, M. (2008). The Web as a source of information for students in K–12 education. In J. Coiro, M. Knobel, C. Lankshear, & D. Leu (Eds.), Handbook of research on new literacies (pp. 241–246). Mahwah, NJ: Lawrence Erlbaum.

Lankshear, C., & Knobel, M. (2006). New literacies (2nd ed.). Maidenhead, UK: Open University Press.

Leu, D. J., O’Byrne, W. I., Everett-Cacopardo, H., McVerry, J. G., Zawilinski, L.. Kiili, C., Kennedy, C., Forzani, E., (2011, September). The New Literacies of Online Reading Comprehension: Expanding the Literacy and Learning Curriculum. Journal of Adolescent & Adult Literacy, 55(1), 5–14. doi: 10.1598/JAAL.55.1.1

Leu, D. J., O’Byrne, W. I., Zawilinski, L., McVerry, J. G., & Everett-Cacopardo, H. (2009). Expanding the new literacies conversation. Educational Researcher, 38, 264-269.

Leu, D.J., Jr., Kinzer, C.K., Coiro, J., Cammack, D. (2004). Toward a theory of new literacies emerging from the Internet and other information and communication technologies. In R.B. Ruddell & N. Unrau (Eds.), Theoretical Models and Processes of Reading, Fifth Edition (1568-1611). International Reading Association: Newark, DE.

Leu, D. J., Kinzer, C. K., Coiro, J., Castek, J., Henry, L. A. (2013). New literacies: A dual level theory of the changing nature of literacy, instruction, and assessment. In Alvermann, D. E., Unrau, N. J., & Ruddell, R. B. (Eds.), Theoretical Models and Processes of Reading, Sixth Edition. Newark, DE: International Reading Association.

Lewis, C., & Fabos, B. (2005). Instant messaging, literacies, and social identities. Reading Research Quarterly, 40, 470–501.

Organisation for Economic Co-operation and Development & the Centre for Educational Research and Innovation. (2010). Trends shaping education 2010. Paris: OECD Publishing.

Prensky, M. (October, 2001). Digital natives, digital immigrants. On the Horizon. 9 (5). MCB University Press. Available at: http://www.albertomattiacci.it/docs/did/Digital_Natives_Digital_Immigrants.pdf

Rieh, S. Y., & Becklin, N. J. (1998). Understanding judgment of information quality and cognitive authority in WWW. Journal of the American Society for Information Science, 35, 279-289.

Sanchez, C. A., Wiley, J., & Goldman, S. R. (2006). Teaching students to evaluate source reliability during Internet research tasks. In S. A. Barab, K. E. Hay, & D. T. Hickey (Eds.), Proceedings of the seventh international conference on the learning sciences (pp. 662-666). Bloomington, IN: International Society of the Learning Sciences.

Sutherland-Smith, W. (2002). Weaving the literacy Web: Changes in reading from page to screen. The Reading Teacher, 55, 662–669.

Wallace, R. M., Kupperman, J., Krajcik. J., & Soloway, E. (2000). Science on the Web: Students Online in a Sixth-Grade Classroom. Journal of the Learning Sciences, 9, 75-104.

Wiley, J., Goldman, S., Graesser, A., Sanchez, C., Ash, I., & Hemmerich, J. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46, 1060–1106. doi:10.3102/0002831209333183

Creating a Culture of Literacy: Research and Recommendations for Teachers and Educational Leaders

By Dr. Penelope L. Lisi and Dr. Catherine Kurkjian
Central Connecticut State University

Historically, the teaching of reading has been one of the most critical, and perhaps challenging responsibilities of educators in schools around the world. In the U.S., 30% of all students are not graduating from high school, and 75% of all students with literacy problems in the third grade will still experience literacy difficulties in the ninth grade. One response to the challenges of developing literacy in U.S. schools has been the creation of standards or expectations of what students will know and be able to do. For many years, individual states have been responsible for the development of standards in a variety of content areas. Districts and schools have been expected to support educators in developing an awareness of state standards, who then work to align and implement the standards in curriculum and instruction. Implementation has been uneven and consequently, literacy levels have continued to remain, for the most part, stagnant.

Since 2010, a promising educational reform initiative in the United States has been the development of the Common Core State Standards (CCSS) in the English Language Arts, Mathematics, and other content areas. This reform is a state-led initiative organized by the National Governors Association (NGA) Center for Best Practices and the Council of Chief State School Officers (CCSSO). The common K-12 standards are intended to define knowledge and skills so that upon graduation students are college- and career-ready. A description on the shared website sponsored by NGA and the CCSSO states the standards are aligned with college and work expectations; are clear, understandable and consistent; include rigorous content and applica- tion of knowledge through high-order skills; build upon strengths and lessons of current state standards; are informed by other top performing countries, so that all students are prepared to succeed in our global economy and society; and are evidence-based (CCSS, 2010).

An important goal of this reform is to develop and implement common standards and invite collaboration across states as well as to utilize a common metric in terms of assessment. Thus far, forty-five states, the District of Columbia, four territories, and the Department of Defense Education Schools have adopted the Common Core State Standards. In Connecticut, the context of this research, the strategic plan calls for transitioning from state standards to the CCSS standards, from state assessments to CCSS aligned assessment, and then to the Smarter Balance CCSS Assessments.

Interestingly, much direction for implementation of the CCSS at the local level is coming from the Connecticut State Department of Education (CSDE) as illustrated in Table 1. Implementation work in Connecticut has focused on transitioning from the state standards to the CCSS standards, aligning curriculum, creating Professional Learning Communities (PLCs) at the state and district level, and developing and piloting a practice common core aligned state assessment. Most notable is encouragement for the creation and use of professional learning communities as a strategy to support implementation. According to the CSDE strategic plan, in spring 2013 the CSDE was in the process of further organizing district PLCs, aligning and making available model curriculum, providing exemplar student work and professional learning and assessment tools, and piloting CCSS aligned assessments.

The CCSS English Language Arts Standards are a departure from what has been promoted in the past, and they represent shifts in thinking about teaching and learning. In a recent EPE report three major shifts in the English Language Arts include the following:

Informational Text: Building knowl- edge through content-rich nonfiction and informational texts. At the elementary level, the standards call for a 50-50 balance between informational texts and literature. They shift the emphasis to 55 percent informational by middle school, and 70 percent by high school. Such reading includes content-rich nonfiction in history/social studies, science, and the arts. Informational text is seen as a way for students to build coherent general knowledge, as well as reading and writing skills.

Citing Evidence: Reading and writing grounded in evidence from text. The standards place a premium on students’ use of evidence from texts to present careful analyses and well-defended claims. Rather than asking students questions they can answer solely from their prior knowledge or experience, the standards envision students’ answering questions that depend on reading texts with care. The standards also require the cultivation of narrative writing throughout the grades. The reading standards focus on students’ ability to read carefully and grasp information, arguments, ideas, and details based on evidence.

Complex Text: Regular practice with complex text and its academic vocabulary. The standards build a “staircase” of increasing text complexity to prepare students for the types of texts they must read to be ready for the demands of college and careers. Closely related to text complexity, and inextricably connected to reading comprehension, is a focus on academic vocabulary: words that appear in a variety of content areas (such as “ignite” and “commit”). (Moving Forward: A National Perspective on States’ Progress In Common Core State Standards Implementing Planning, 2013, February, p.13).

The International Reading Association (IRA) supports the development and implementation of the standards. In a recent paper, the IRA International Reading Association Common Core State Standards Committee (2012) identifies areas that will present challenges to implementation of the standards, and provides guidelines and clarification to state and local leaders, teachers, principals, professors, and others who will implement the ELA standards. The guidelines call for extensive professional development. They note:

Changes this significant are not likely to occur successfully without equally significant investments in the knowledge and skills of educators along with necessary material supports (e.g., texts, technology). There are many things that teachers must do to try to help students reach the expectations detailed in the CCSS….States and schools will need to support such efforts with appropriate and timely professional development for teachers. (p.4)

Table 1: Strategic Plan. Adapted from the Connecticut State Department Common Core State Standards Strategic Plan. (Pp 21-23) Retrieved from http://www.sde.ct.gov/sde/lib/sde/pdf/ccss/ccss_strategic_plan_sbe_120512.pdf

Leadership from CSDE
Leadership from Districts
Dec 2012-Feb 2013
  • Exploring and analyzing the possibility of providing a new “Practice” Common Core-aligned state assessment in Spring 2013
  • Providing coordinated and consistent communication through the Common Core District Teams; setting Common Core District Team meetings
  • Aligning, making available model curriculum practices and resources & exemplar student work
  • Creating a Common Core District Team with guidance from the CSDE and attending the Common Core District Team meetings
  • Engaging with the CSDE in content specific Professional Learning Communities (PLCs)
  • Nominating educators to become Common Core Coaches
March-July 2013
  • Providing coordinated and consistent communication through the Common Core
    District Teams; setting Common Core District Team meetings
  • Organizing ELA and Math PLCs for districts to share best practices, lessons learned
  • Aligning and making available model curriculum practices and resources & exemplar student work, professional learning, & pilot assessments
  • Participating in the Common Core District Team meetings
  • Participating in the ELA and Math PLCs
  • Planning and training for statewide implementation of the Common Core and new Common Core- aligned assessment in August 2013-2014
August 2013-August 2014
  • Providing coordinated and consistent communication through the Common Core District Teams
  • Organizing ELA and Math PLCs for districts to learn and share best practices, lessons learned
  • Aligning and making available model curriculum practices and resources & exemplar student work, professional learning, assessment tools, & assessments
  • Participating in the ELA and Math PLCs
  • Implementing Common Core and new Common Core-aligned assessment
  • Participating in the pilot Technology Plan
  • Inviting CSDE to visit and view implementation of Common Core in classrooms

 
Clearly, the successful implementation of the CCSS necessitates the creation of a culture of literacy in schools in which all stakeholders, including teachers and leaders, are working together to improve the teaching of reading in PK-12 grades. The primary goal of this research project is to enhance our knowledge of leadership practices in support of a culture of literacy in ways that address significant literacy achievement challenges. In particular, we are interested to learn about how educators and educational leaders are addressing the reform initiative that requires the implementation of a new set of learning standards in schools across the nation.

This is the first part of a three-part study designed to provide insights into the perceptions of Connecticut teachers and administrators regarding the implementation of the Common Core State Standards (CCSS). The first phase of the investigation serves as a pilot study during which the survey tool was tested with a group of graduate students at Central Connecticut State University. Our findings will provide a preliminary view into the creation of the culture of literacy.

Conceptual Framework for the Study

Why is there a need to create a culture of literacy? In 2005, the National Association for Secondary School Principals (NASSP) published Creating a Culture of Literacy: A Guide for Middle and High School Principals, a document that describes the major deficit in the literacy achievement of United States’ secondary students. Unfortunately, direct literacy instruction that might address this glaring deficit ends, in most cases, at the third grade. Literacy instruction must not end when students enter middle school. And this necessitates strong and effective leadership. This study is guided by the literature and research about leadership for school improvement, as well as effective instructional practice.

Leadership for School Improvement

The literature is clear about the need for effective leadership as an essential ingredient in educational reform (Wahlstrom, Seashore Louis, Leithwood, and Anderson, 2010). Richard Elmore (2004) lists five principles for leadership that supports major instructional improvement efforts. These princi- ples are that: 1) educational leadership must be focused on the improvement of instruction; 2) instructional improvement requires opportunities for on-going individual and group learning of teachers; 3) leaders must model for teachers what they expect them to do; 4) leadership roles and activities emanate from the expertise needed for learning; and 5) leaders and teachers must be held mutually accountable for outcomes.

The literature on creating a culture of literacy that supports high levels of academic achievement indicate that the following principles must be in place: literacy is the top priority in the school; educators are committed to impacting student learning; educators maintain high expectations for students; and faculty and admin- istrators maintain a strong academic press (Murphy, 2004). Further, time is managed productively and opportunities exist for staff to engage in professional learning through powerful professional learning communities. According to the report, Creating a Culture of Literacy (2005), the Literacy Leader engages teachers in a variety of key activities, including: establishing specific and measurable goals for literacy; aligning curriculum with standards; ensuring that content-area literacy strategies are used daily; and evaluating the use of literacy strategies through formal and informal observations.

The literature in support of school improve- ment indicates that strong leadership is essential (Creating a Culture of Literacy, 2005; Murphy, 2004). Critical strategies for the literacy leader include: development of a Literacy Leadership Team (LLT); shared faculty commitment to improve achievement; creation of a collaborative environment in which teachers learn from and with each other; use of assessment data to identify specific learning needs; development of a school wide plan to address professional development needs of teachers; a curriculum that is aligned with standards; content-area literacy strategies that are used daily in classroom instruction; and development of an understanding of research- based literacy strategies (Creating a Culture of Literacy, 2005; Murphy, 2004; Reeves, 2004; Schmoker, 2006; Wahlstrom, Seashore Louis, Leithwood, and Anderson, 2010).

Unfortunately, a particularly problematic issue facing classroom teachers around the world is isolation (Short and Greer, 2002). Experienced teachers are often isolated from each other and not provided with significant opportunities for learning from and with each other. Some results of teachers working in isolation are feelings of inadequacy, insecurity, and lack of recognition. Recent research indicates that effectively designed professional development can counteract these feelings. In particular, “Professional development activities that take place at regular intervals and involve teachers in a rather stable social and collaborative context (i.e. networks or mentoring) have a significantly stronger association with teaching practices than regular workshops and courses” (OECD, 2009, p. 117).

Specific to supporting the professional development of teachers in a culture of literacy, the leader should: work closely with the Literacy Leadership Team (LLT) to determine professional learning needs of teachers; identify and use staff members’ skills and interests to support ongoing, job-embedded professional learning; implement coaching for teachers to learn and immerse literacy strategies within content classes; encourage “professional talk” among staff and provide time for discussions; provide resources for professional learning; use classroom observations to identify and support ongoing professional development (Creating a Culture of Literacy, 2005).

Purpose of the Study and Primary Research Questions

This pilot study, the first part of a three-year plan, will lay the groundwork for the second and third phases of our investigation to ascertain perceptions over time of Connecticut teachers and administrators regarding the implementation of the Common Core State Standards (CCSS). In years two and three, data will be collected from a random sample of teachers and administrators in the state using the refined survey to examine perceptions of the implementation of the CCSS over time.

While the study in all of its phases will not directly benefit participants, the perspectives on implementation of Common Core State literacy standards will inform university literacy and educational leadership professors as to how to enhance university-level curriculum related to the CCSS in a way that addresses needs with models of best practice. The study will inform the knowledge base on how leaders can better support large-scale changes.

Research questions that guide this study are as follows:

  • What is the level of awareness on the part of teachers and administrators in Connecticut of the Common Core State Standards (CCSS) in literacy?
  • What supports are provided by leaders for implementation of standards-based
    literacy instruction?
  • What types of changes in classroom practice have resulted from the implementation of standards-based literacy instruction?
Methodology

This is a descriptive study. Data are collected using a survey administered over three years. The survey is a 48-item instrument that has been adapted from the Common Core Feedback Loop and is used with permission from the U.S. Education Delivery Institute. Two mirror versions of the instrument were developed: one for educators, and one for educational leaders. Each version has the same number of items, yet the language has been altered slightly to reflect the respondents.

The pilot survey was disseminated in spring 2013 to graduate students in programs in the departments of Educational Leadership and Reading and Language Arts, either in paper format, or as a link to an online version of the survey using SelectSurvey.NET. Twenty-eight graduate level leadership students (who are also teachers and who have some leadership responsibilities) responded to the survey designed for educational leaders.

Sixty-six teachers enrolled in the Masters Degree Program in Reading and Language Arts Department responded to a survey designed for teachers. Eighty percent of the teachers (53) were primary grade school level spanning K-5. The remaining 20% were middle school teachers (9%), high school teachers (4%) or teachers who spanned elementary to middle school (6%). Most teachers (79%) worked in traditional public schools; another 15% worked in Magnet Schools or Charter Schools. The remaining 6% of the teacher worked in other special school settings (Expeditionary learning school, Montessori School and a Dual Language and International Baccalaureate). One teacher was unemployed. The setting in which teachers worked spanned a range of socioeconomic levels. Data are analyzed using descriptive statistics.

Preliminary Findings

Leadership in Support of the CCSS

In this first year of the study, preliminary data from responses by leaders and teachers provide some useful information to support the investigation of knowledge of leadership practices in support of a culture for literacy in our schools. In particular, preliminary data has been collected about how educators and educational leaders are addressing the reform initiative that requires the implementation of a new set of learning standards in schools in Connecticut.

Awareness of Standards. In terms of survey questions related to Awareness of Standards, the data indicate agreement with the following: respondents have read the new standards (81%, Leaders; 99%, Teachers); they have comprehensive knowledge (34%, Leaders; 38%, Teachers); Leaders (76%) agree that they are somewhat prepared to support school educators to teach the CCSS; Teachers (75%) agree they are somewhat prepared to teach the CCSS; both groups agree or strongly agree that the CCSS will lead to improved learning for the majority of students (93% Leaders; 83% Teachers).

The top three reasons provided by leadership students and teachers for why the
CCSS will benefit the majority of their schools’ students were the same and are as follows:

  1. They believe the standards will give students the opportunity to master key competencies, rather than just superficial exposure (75%, Leaders; 59%, Teachers).
  2. They indicate that the CCSS will help school systems ensure standards are vertically-aligned from kindergarten through grade 12 (Leaders, 71%; Teachers, 58%).
  3. They believe that standards will help educators focus on what’s most important (Leaders, 53%; Teachers, 43%). Each group demonstrated a different preference for their fourth top list of benefits. Fifty percent of the Leaders believe standards will provide students a clearer understanding of what they must know to succeed, while 43% of the Teachers believe that the standards will help focus educators on what is most important. (please see Table 2).

Among the Teachers (14%) who do not think that the Common Core will improve learning for all of their students the provide the following reasons:

  1. The standards are a “one size fits all” approach (10%).
  2. They are too rigorous for their students (7%).
  3. The standards do not provide flexibility for students who are not on grade level.
  4. The current state standards are better (3%).

While teachers have concerns, it appears that at least half of the Leader and Teacher group see important benefits as a result of the CCSS.

Table 2: Responses to Survey Questions related to Awareness of Standards

Key Concept
Leader response
Teacher response
(6) Knowledge of state’s transition to the CCSS
34% comprehensive knowledge
54% some knowledge
38% extensive knowledge
54% some knowledge
(7) Have read CCSS
81% yes
19% no
99% yes
1% no
(8) Level of agreement that the CCSS will lead to improved student learning for majority of students in my school
25% strongly agree
68% agree
22% strongly agree
61% agree
(9) Reasons for belief why CCSS will benefit majority of students
75 % believe they will give students the opportunity to master key competencies, rather than just superficial exposure.
71% believe they will help school system ensure standards are vertically-aligned from kindergarten through grade 12.
50% believe they will help educators focus on what’s most important.
50% believe they will provide students a clearer understanding of what they must know to succeed.
46% believe they will help educators better prepare students for college.
59% believe they will give students the opportunity to master key competencies, rather than just superficial exposure.
58% believe they will help school system ensure standards are vertically-aligned from kindergarten through grade 12
43% believe they will help educators focus on what’s most important.
35% believe they will provide students a clearer understanding of what they must know to succeed.
48% believe they will help educators better prepare students for college.
(11) Differences between state’s standards and CCSS
75% believe CCSS are more demanding and raise expectations for student learning.
84% believe CCSS are more demanding and raise expectations for student learning.
(12) Feel prepared to support school’s educators to teach the CCSS (Leader Survey
12) Feel prepared to teach the CCSS
75% feel somewhat prepared
11% do not feel prepared
11% feel completely prepared
76% feel somewhat prepared
9% do not feel prepared

Leaders n=28 Teachers n=66

Understanding of Standards. Leadership and Teacher groups were asked about their beliefs in terms of providing certain types of learning environments that are consistent with ideas embedded in the CCSS. Both surveys revealed that there is an accurate understanding regarding the three key areas that are consistent with the CCSS:

  1. structuring opportunities for students to have conversations and develop arguments based on the texts they’ve read (86%, Leaders; 87%, Teachers);
  2. creating learning experiences that build knowledge using informational texts, not just literature (89%, Leaders; 94%, Teachers);
  3. providing instruction in academic vocabulary to support students’ understanding of complex text (75%, Leaders; 83%, Teachers)

Misconceptions are evident in both groups regarding the importance placed on:

  1. providing students with ongoing opportunities to write creatively, drawing from personal experiences (65%, Leaders; 60%, Teachers), and
  2. utilizing pre-reading strategies to help all students fully understand a text through discussions and/or overviews of context, vocabulary (78%, Leaders; 66%, Teachers). These two learning opportunities are not closely aligned with the CCSS (see Table 3).

Table 3: Responses to Survey Questions related to Understanding of Standards

Key Concept
Leader response
Teacher response
(14a) Extent to which it is important to provide students ongoing opportunities to write creatively drawing from personal experiences
77% very important or important
21% somewhat important
90% very important or important
9% somewhat important
(14b) Extent to which it is important to give students opportunities for conversations and develop arguments based on texts they’ve read (CCSS Aligned)
100% very important or important
100% very important or important
(14c) Extent to which it is important to utilize pre- reading strategies to help all students fully understand a text through discussions and/or overviews of context,
vocabulary
86% very important or important
15% said somewhat important or unimportant
92% very important or important
8% somewhat important or unimportant
(14d) Extent to which it is important to create learning experiences that build knowledge using informational texts (CCSS Aligned)
93% very important or important
7% said somewhat important
100% very important or important
(14e) Extent to which it is important to provide instruction in academic vocabulary to support student understanding of complex text (CCSS Aligned)
92% very important or important
7% somewhat important
95% very important or important
5% somewhat important
(15a) Extent to which this practice is aligned with CCSS: Providing students ongoing opportunities to write creatively drawing from personal experiences
65% very significantly aligned or very aligned
25% somewhat aligned or insignificantly aligned
11% don’t know
60% significantly aligned or very aligned
35% somewhat aligned or insignificantly aligned
5% don’t know
(15b) Extent to which this practice is aligned with CCSS: Structuring opportunities for students to have conversations and develop arguments based on the texts they’ve read (CCSS Aligned)
86% very significantly aligned or very aligned
4% said somewhat aligned
87% very significantly aligned or very aligned
8% said somewhat aligned or unimportant
(15c) Extent to which this practice is aligned with the CCSS: Utilizing pre-reading strategies to help all students fully understand a text through discussions and/or overviews of context, vocabulary
78% very significantly aligned or very aligned
14% somewhat aligned or insignificantly aligned
11% don’t know
66% very significantly aligned or very aligned
25% somewhat aligned
8% said insignificantly aligned
1% don’t know
(15d) Extent to which this practice is aligned with the CCSS: Creating learning experiences that build knowledge using
informational texts
(CCSS Aligned)
89% very significantly aligned or very aligned
11% don’t know
94% very significantly aligned or very aligned
5% somewhat aligned
1% don’t know
(15e) Extent to which this practice is aligned with CCSS: Providing instruction in academic vocabulary to support students’ understanding of complex text
(CCSS Aligned)
75% very significantly aligned or very aligned
11% somewhat aligned
14% I don’t know
83% very significantly aligned or very aligned
11% somewhat aligned
1% not aligned
5% don’t know

Leaders n=28 Teachers n-66

Leader Support. While the implementation of the Common Core State Standards in classroom teaching and learning is a very new initiative, respondents were able to comment on leader support for the implementation of standards-based literacy instruction. In response to the question about the availability of different types of activities and resources, respondents reported availability of the following: collaborative planning time for deconstructing the CCSS (43%, Leaders; 48%, Teachers); collaborative planning time to align curriculum to the CCSS (39%, Leaders; 48%, Teachers); content-focused trainings on the CCSS (36%, Leaders; 43%, Teachers); resources on research/best practice in CCSS implementation (32%, Leaders; 25%, Teachers); job-embedded training or coaching focused on CCSS (29%, Leaders; 25%, Teachers); professional learning community focused on CCSS (29%, Leaders; 30%, Teachers). Additionally, 58% of the Leaders indicated there was a staff member who serves as a CCSS resource, while 22% of the Teachers indicated that this was so.

When asked about challenges to the implementation of the CCSS, the following needs were reported: more quality professional development (54%, Leaders; 41%, Teachers), more time to collaborate with colleagues (39%, Leaders; 29%, Teachers). Teachers (52%) also reported that they needed more aligned textbooks and materials.

According to Leaders, the following changes were made to the ways in which educators are supported in their understanding and use of the CCSS: they are sharing information and resources with educators about CCSS (61%); they are placing more emphasis on vertical alignment between grade levels (54%); they are creating more opportunities for collaboration among educators on CCSS (43%), and; and they are providing professional development opportunities that support CCSS (46%) (please see Table 4).

Thus far, nearly three fourths of leadership and teacher groups have received professional development on the implementation of the CCSS, and most participants agreed or strongly agreed that it was of high quality. The challenges facing them intersect. While they indicate availability of professional development and time for collaboration, they request that they need more.

Table 4: Responses To Survey Questions Related To Leader Support For Implementation Of Standards- Based Literacy Instruction

Key Concept
Leader response
Teacher response
(18) What activities or resources have been made available to teachers
43% collaborative planning time for deconstructing the CCSS
39% collaborative planning time to align curriculum to the CCSS
36% content-focused trainings on the CCSS
32% resources on research/best practice in CCSS implementation
29% job-embedded training or coaching focused on CCSS
29% professional learning community focused on CCSS
49% collaborative planning time for deconstructing the CCSS
48% collaborative planning time to align curriculum to the CCSS
43% said content-focused trainings on the CCSS
25% resources on research/best practice in CCSS
implementation;
25% job-embedded training or coaching focused on CCSS
30% professional learning community focused on CCSS
(19) Have you participated in professional development on
the CCSS?
67% yes
33% no
70% yes
30% no
(20) What type of PD opportunities have you had?
46% said one-day training
29% said Job-embedded training or coaching
21% said multi-day training
32 % of the teachers report they received job embedded training,
30% a one-day training opportunity
22% of the teachers report multi-day training.
23% report the formation of Professional Learning Communities,
6% report training in the form of a webinar or video, and
13% report that the Common
Core Standards are a focus in their university classrooms.
(21) Who provided the training?
46% staff member from my district
25% someone from outside the district
54% staff member from my district
22% training brought in from outside of district
4% said Department of
Education
6% said independent professional provider
12% university instructor
(23) Is there a staff
58% yes
35% don’t know
22% yes
43% don’t know
(29) Challenges to implementing CCSS
54% need more quality professional development
39% need more time to collaborate with colleagues
36% student knowledge
52% need more aligned textbooks and materials
43% need more time to collaborate with colleagues
41% need more quality professional development
29% need more formative assessments aligned to the Common Core
25% student knowledge
(32) What changes are you making to the ways you are supporting educators as a result of the CCSS? (Leader Survey)
32) What changes are you making to your teaching as a result of the CCSS? (Teacher Survey)
46% creating more opportunities for collaboration among educators focused on CCSS
21% ensuring curricular materials reflect CCSS expectations
21% sharing information and resources related to CCSS
51% incorporating curricular materials and instructional strategies into their teaching
51% structuring opportunities
for more students to develop and solve their own problems
49% asking students more questions and encouraging them to develop answers independently
22% increasing collaboration with colleagues within their schools and in other schools.
(35) Changes to ways educators are supported in understanding and using CCSS
61% sharing information and resources with educators about CCSS
54% placing more emphasis on vertical alignment between grade levels
43% creating more opportunities for collaboration among educators on CCSS
46% Providing professional
development opportunities that support CCSS
36% using classroom observations as opportunities to provide feedback on CCSS
41% ensuring curricular materials reflect CCSS expectations
39% providing more professional development opportunities on the Common Core
35% placing more emphasis on vertical alignment between grade levels
36% said creating more opportunities for collaboration among educators focused on CCSS
32% said sharing information and resources related to CCSS

Leaders n=28 Teachers n=66

Changes in classroom practice. The data from the Leadership and the Teacher Survey indicate that there are changes in teacher practice as a result of implementation of the CCSS. When asked if their school’s educators had incorporated the standards into their teaching expectations and practice, 75% (Leaders) and 70% (Teachers) agreed that some have incorporated them; 7% (Leaders) and 17% (Teachers) agree that all have fully incorporated them, and: 18% (Leaders) and 2% (Teachers) agree that they don’t know.

According to the Teacher Survey, the following kinds of CCSS-aligned changes are being made by teachers: 51% are structuring opportunities for more students to develop and solve their own problems; 49% are asking students more questions and encouraging them to develop answers independently; 22% are increasing collaboration with colleagues within their schools and in other schools. However, in regards to differentiation of instruction there appears to be some concern about differentiation of instruction among Leaders in comparison to the Teachers. Fifty-six percent (Leaders) and 77% (Teachers) report that the support provided to educators is helping them to differentiate instruction, while 46% (Leaders) and 17% (Teachers) disagreed with this statement.

Sixty percent of the leader respondents said they were confident in their ability to identify instructional practices that reflect the CCSS during classroom observations, though 28% disagreed or strongly agreed with this statement. Eighty-nine percent of leaders agreed that the CCSS will help them know what content should be taught and the sequence in which it should be taught. Leaders are mixed on their agreement as to whether the CCSS will improve their ability to identify the most effective educators (36% agree,
28% agree or strongly agree, and 25% do not know) (see Table 5).

Table 5: Responses to Survey Questions related to Changes in Classroom Practice as a Result of Implementation of the Common Core State Standards

Key Concept
Leader response
Teacher response
(31) My school’s educators have incorporated CCSS into their teaching
expectations and practice
7% agree that all have fully incorporated them
75% agree that some have
incorporated them
18% agree that they don’t know
17% indicate they have fully incorporated the Common Core into their teaching.
70% of the teachers have incorporated the CCSS in some areas of their teaching,
2% agree that they do not know
(34a) In my school, the CCSS and support provided to educators help them
differentiate instruction to meet unique
learner needs. (Leader Survey)
Effective practices to teach the Common Core will help me to differentiate instruction (Teacher Survey)
54% agree or strongly agree
46% disagree or strongly disagree
77% agree or strongly agree
17% disagree or strongly disagree
(34b) The CCSS will require that my school’s educators incorporate instructional technology into classroom learning. (Leader Survey)
The CCSS will require that I change the way I incorporate instructional technology into classroom learning. (Teacher Survey)
86% agree or strongly agree
11% disagree or strongly disagree
69% agree or strongly agree
12% disagree or strongly disagree
(34c) I feel confident about my ability to identify instructional practices that
reflect the CCSS during my classroom observations.
60% agree or strongly agree
28% disagree or strongly disagree
11% don’t know
No comparable question
(34d) The CCSS will improve my ability to identify the most effective educators
in my building.
36% agree or strongly agree
28% disagree or strongly disagree
25% don’t know
No comparable question
(34f) The CCSS will help me know what content should be taught, and in what sequence it should be taught in order for them to master key competencies.
89% agree or strongly agree
7% disagree or strongly disagree
69% agree or strongly agree
24% disagree or strongly disagree

Leaders n=28 Teachers n=66

Discussion

Research Question #1

In considering data that addresses Research Question #1 (What is the level of awareness on the part of teachers and administrators in Connecticut of the Common Core State Standards in Literacy?), responses by educational leader students indicate that the CCSS initiative is not intended to be something in which educators are “tinkering” around the edges of what impacts students directly in the classroom. The focus of this initiative at the national, state, and local level is on teaching and learning in the classroom. This is clearly in line with Elmore’s (2004) first principle for leadership that supports major instructional improvement, and that is that leadership must focus on the improvement of instruction. Educational leaders appear to be growing in their awareness of this initiative, which is a critical first step in deep and meaningful change. It is very difficult to support improvement in teaching and learning if the key stakeholders do not understand or do not know about the essential ideas and concepts in the initiative.

Similarly, teachers are growing in their awareness, yet nearly three fourths of the teachers report that they are only somewhat prepared to implement the CCSS. These findings on levels of preparedness are consistent with a recent national survey conducted by the Hewlett Foundation (2012) in which 92% of participating teachers indicated that they were at least slightly prepared, and with one third of the respondents indicating that they were very familiar with the standards. In this same survey teachers indicated that they are less confident in their ability to implement the standards with certain student groups such as English Language Learners and students with disabilities, and with low-income students. (Gewertz, 2012). At the time of this survey there is a seemingly limited, but emerging preparedness among the participants in this study.

Research Question #2

When reflecting on data that addresses Research Question #2 (What supports are provided by leaders for implementation of standards-based literacy instruction?), preliminary data appear to indicate that leaders are engaging teachers in a variety of activities. This is consistent with the description of the Literacy Leader (Creating a Culture of Literacy, 2005). There does seem to be an effort to align the standards with the curriculum in many instances. And consistent with another of Elmore’s (2004) key principles, leader students indicate there are some opportunities for collaborative activity related to the CCSS implementation.

It appears as though the leader students have been educated deeply enough themselves in the standards such that they can observe class- room practice and make sure that content-area literacy strategies are used daily and evaluate the use of literacy strategies through formal and informal observations. This is consistent with the stipulations of the Literacy Leader outlined in Creating a Culture of Literacy (2005).

Not necessarily apparent from the data are the following critical literacy leader strategies: use of assessment data to identify specific learning needs; development of a school wide plan to address professional development needs of teachers; use of a curriculum that is aligned with standards; use of content-area literacy strategies daily in classroom instruction; and development of an understanding of research-based literacy strategies (Creating a Culture of Literacy, 2005; Murphy, 2004; Reeves, 2004; Schmoker, 2006; Wahlstrom, Seashore Louis, Leithwood, and Anderson, 2010). It will be interesting to discern over time whether or not the leader also works to integrate these strategies.

Seventy percent of the teachers received professional development in one or multiple formats, with 67% reporting that the professional development provided was of high quality. At this point in time 30% of our sample have been afforded the support of professional learning communities. Collaboration among teachers is one of the most prevalent supports for the purposes of understanding the standards and learning how to implement them. It would be expected that over time opportunities to collaborate in professional learning communities would increase since these supports are key components of the State Department of Connecticut’s strategic plan.

One of the prevalent concerns that teachers reported was the need for Common Core aligned materials. This is not surprising since teachers are accountable for day-to-day implementation of the Common Core and the focus on the CCSS to a great degree revolves around the use of exemplar texts, with a focus on nonfiction. This is likely to be a change for some teachers. Teachers also indicate that they need more professional development. These findings are consistent with the findings from a national survey of Teacher Perspectives on the Common Core (2013) regarding challenges that teachers face in implementing the Common Core.

Surveys revealed that most teachers and leaders are aware of and support three of the major shifts in the CCSS: 1) structuring opportunities for students to have conversations and develop arguments based on the texts they’ve read; 2) Creating learning experiences that build knowledge using informational texts, not just literature; and 3) Providing instruction in academic vocabulary to support students’ understanding of complex text.

The data also revealed that both teachers and leaders have misconceptions regarding practices that are now being downplayed by the CCSS. About two thirds of each group believe in the importance of personal response through writing and drawing and the practice of front-loading pre-reading strategies prior to having students read a text. Respondents from each survey indicated that they think that these practices are aligned with the CCSS. Clearly, professional development will be needed to clarify the shift and to help leaders and teachers decide the conditions under which these practices are most appropriate.

Our findings regarding misconceptions are supported in the literature by a myriad of articles clarifying the shifts and misconceptions surrounding the Common Core (Gewertz, 2013; Short, 2013; IRA, 2012; Strasser & Dobberton, 2012a, 2012b). As leaders prepare educators and as teachers implement the standards with students, it is essential that professional development address the standards beyond the declarative and procedural knowledge level. The implementation of the CCSS will require that both leaders and teachers develop conditional knowledge as to how the Common Core Standards will impact teaching and learning. For example, under what circumstances and for whom is it appropriate to spend time building background, and when does this become less productive in terms of allowing students to problem solve while reading? This kind of knowledge will require Professional Learning Communities that work together to study the standards, read professional literature, generate questions and systematically examine the impact of their teaching within the wide range of diversity that exists within the classroom and at the school level.

Research Question #3

When looking at responses that address Research Question #3 (What types of changes in classroom practice have resulted from the implementation of standards-based literacy instruction?), the preliminary data appear to indicate that opportunities to collaborate on aspects of the CCSS have been put in place, yet teachers still consider this to be an area of need. Both the Leadership Survey and Teacher Survey concur that changes have been made to incorporate the CCSS in some areas of teaching. It appears that teachers are making a shift towards a more rigorous curriculum in alignment with the CCSS. At least half of the teachers report posing more evidence- based questions and requiring their students to answer them independently. Similarly, they report that they are structuring opportunities for more students to generate and answer their own questions.

The literature on creating a culture of literacy that supports high levels of academic achievement indicates that the following principles must be in place: literacy is the top priority in the school; educators are committed to impacting student learning; educators maintain high expectations for students; and faculty and administrators maintain a strong academic press (Murphy, 2004). Furthermore, time is managed productively and opportunities exist for staff to engage in professional learning through powerful professional learning communities. While there is movement towards supporting teachers in implementing the Common Core Standards, in light of what was learned from Research Questions 1 and 2, the work of forming powerful professional learning communities to support the implementation of the CCSS initiative has only just begun.

Recommendations

As leaders continue to work to develop a culture of literacy in light of the new standards reform initiative, the preliminary data from this study may provide insights into what leaders might do. Leaders in support of a culture of literacy are encouraged to:

  • Continue to support the development of PLCs during which educators can share best practice and learn from and with each other.
  • Have a clear professional development plan in place that includes job-embedded learning opportunities and time for collaboration.
  • Ensure that both leaders and teachers have
    a deep and conditional understanding of the shifts that the CCSS are requiring and that this understanding addresses the need to modify and differentiate instruction to meet the wide range of diversity existing at the classroom and school level.
  • Provide a range of resources to implement the shifts particular to nonfiction, along with other CCSS aligned materials and assessments to inform instruction.
Summary

There is strong consensus within the education community that American schools need to prepare students to participate in a global society. In particular, there is an especially strong focus on the need to address literacy challenges. In light of recent reform initiatives, most notably the Common Core State Standards, are leaders creating a culture or environment for enhancement of literacy? Preliminary data from the current study point to the fact that schools and school leaders do seem to be headed in a positive direction. There is still much room for additional and extensive support in order for this initiative to take deep root.

References

EPE Research Center, (2013). Findings from a National Survey of Teacher Perspectives on the Common Core (Bethesda, MD: EPE Research Center, 2013). Retrieved from http://www.edweek.org

EPE Research Center. (2013) Moving Forward: A National Perspective on States’
Progress in Common Core State Standards Implementing and Planning (Bethesda, MD: EPE Research Center, 2013). Retrieved from http://www.edweek.org

Elmore, R. (2004). School Reform from the Inside Out: Policy, Practice, and Performance. Boston, MA: Harvard Education Press.

Fleischman, H.L., Hopstock, P.J., Pelczar, M.P., and Shelley, B.E. (2010). Highlights
From PISA 2009: Performance of U.S. 15-Year- Old Students in Reading, Mathematics, and Science Literacy in an International Context (NCES 2011-004). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

Gewertz, C. (2011, September). Common standards implementation slow going, study finds. Education Week 31(4) from http://www.edweek.org/ew/articles/2011/09/14/04cep.h31.html

Gewertz.C. (2013, January) Interpretations Differ on Common Core’s Nonfiction
Rule. Education Week 32(19) from http://www.edweek.org/ew/articles/2013/01/
30/19nonfiction_ep.h32.html?qs=common+core+misunderstandings

International Reading Association Common Core State Standards (CCSS) Committee. (2012). Literacy implementation guidance for the ELA Common Core State Standards [White paper]. Retrieved from http://www.reading.org/Libraries/association-documents/ira_ccss_guidelines.pdf

Murphy, J. (2004). Leadership for literacy: Research-based practice, PreK-3. Thousand Oaks, CA: Sage.

National Association of Secondary School Principals. (2005). Creating a culture of literacy: A guide for middle and high school principals. Reston, VA: NASSP.

National Governors Association Center for Bestm Practices, Council of Chief mState School Officers. (2010). Common Core State Standards. National Governors Association Center for Best Practices, Council of Chief State School Officers, Washington D.C. Retrieved from http://www.corestandards.org/about-the-standards

Pryor, S., Boberge-Wentzell, D.,Ullman, D. Byrne, E. (December 2012). Connecticut
State Department Common Core State Standards Strategic Plan. Retrieved from http://www.sde.ct.gov/sde/lib/sde/pdf/ccss/ccss_strategic_plan_sbe_120512.pdf

Reeves, D. (2004). Accountability for learning: How teachers and school leaders can
take charge. Alexandria, VA: ASCD.

Schmoker, M. (2006). Results now: How we can achieve unprecedented improvements in teaching and learning. Alexandria VA: ASCD.

Short, K. (2013, February). Common Core State Standards: Misconceptions about Text Exemplars. World of Words Blog. University of Arizona from http://wowlit.org/blog/2013/01/07/common-core-state-standards-misconceptions-about-text-exemplars/

Short, P. M. & Greer, J.T. (2002). Leadership in empowered schools: Themes from
innovative efforts. Upper Saddle River, NJ: Merrill Prentice Hall.

Strasser, D. , Dobbertin, C. (2012a, July). Four myths about the ELA Common-Core
Standards. Education Week from http://www.edweek.org/tm/articles/2012/07/10/tln_strasserdobbertin.html

Strasser, D., Dobbertin, C. (2012b, April). Common Standards ignite debate over Prereading from Common Standards Ignite Debate Over Prereading

Torgesen, J., Houston, D., & Rissman, L. (2007). Improving literacy instruction in middle and high schools: A guide for principals. Portsmouth, NH: RMC Research Corporation, Center on Instruction.

Wahlstrom, K.L., Seashore Louis, K., Leithwood, K., & Anderson, S.E. (2010). Learning from leadership project: Investigating the links to improved student learning. University of MN: Center for Applied Research and Educational Improvement.

The Literacy Looking Glass

Adrienne Chasteen Snow
CARR Secondary Reading Chair, Secondary Reading Department Chair, Enfield Public Schools, Adjunct Instructor, CCSU Reading and Language Arts Department, and Adjunct Instructor, Asnuntuck CC, English Department

“Take care of the sense and the sounds will take care of themselves.” – Lewis Carroll, Alice’s Adventures in Wonderland & Through the Looking Glass

If only reading were truly that easy for our students! We could spend our Reading Intervention students’ time (a mere three-quarters of an hour for my students) working on the “sense”: metacognition, vocabulary, and the process of creating meaning as readers make their way through a text. Knowing, meanwhile, that the “sounds”: the word analysis, the automaticity, and the prosody would just naturally happen. Such a lovely fantasy; but we know that “literacy” doesn’t quite work that way.

Indeed, my secondary level Reading Intervention students come with a variety of issues, from lack of schema to difficulty inferring. And it is rare that a student needs only a “quick fix”. Instead, it is the Interventionist’s job to accelerate learning; to use data and progress monitoring in conjunction with a knowledge base from years of study in the field of Reading to help each student be a competent reader. I like to think of my end goal as one of ensuring that my students will be, as the RAND study described, (Snow, 2002) skillful adult readers. They will have the skills necessary to read a great mix of materials for a variety of purposes with adequate to good comprehension. Probably the most talked about hot topic goal is to make the student, as the Common Core State Standards (C.C.S.S., 2010) put it, college and career ready. Quite the task, yet I feel it is one we can face with confidence and strength knowing we have the skills, strategies, and tools, to push forward.

The ideas presented in Mesmer, Cunningham, & Heibert’s (2012) work challenge me to think about a model of text complexity for the upper grades. In the essay, the authors search for a framework to support the heavy emphasis on text complexity brought on by CCSS. How would a secondary level model differ from the model for the early grades? The model used by the researchers, developed by the RAND Reading Study Group (Snow, 2002), is most intriguing in that it combines four variables: the reader, the activity, and the text, surrounded by the sociocultural context. I can see this applying to the middle and high school struggling readers that I work with and am curious how the pieces of the components of the individual text might differ. Word, Syntax, and Discourse Structures are the breakdown of individual text in this model, all of which apply beyond the initial phase of learning to read and thus secondary reading. I see the need for more emphasis to be put on vocabulary development (the syntax of this model) and explicit understandings about text (the discourse structures) might need to be weighted heavier than the word component if we were apply this model to our secondary students.

The CCSS brings with it two key focus areas for Reading Teachers, Language Arts Consultants, Literacy Coaches, and other educational specialists whose responsibilities lie primarily in the area of preparing students to be the most literate individuals possible: close reading of text and text-dependent questions. I have spent the past few years trying to implement S.R.B.I. with fidelity and diagnosing the specifics of a student’s reading abilities; then using that diagnosis to identify strengths and weaknesses that I will use to plan intervention.

From the outside close reading of text and text-dependent questions seem like such higher order tasks and thus so far above my instruction. Yet, I realize that I do use the text as the center of instruction and questioning, either my own or those developed by the students. I feel that by using a collection of strategies I have found to be especially successful with secondary level students, my students are tiptoeing towards close reading of text.

A discovery as I worked on some Units of Study for my school district this Summer, was that my units for Reading Intervention are meant to repeat and be delivered in a student-based level of depth, differing from the Language Arts exemplars put out by the state that are structured to be delivered in 20-30 days. By examining the Smarter Balanced Assessment Consortium’s English Language Arts Item and Task Specifications, we can see that students are asked to go beyond multiple choice questions to include constructed response and performance tasks that measure critical thinking and problem solving. We see a leveling of the types of thinking that students will be asked to do on the Cognitive Rigor Matrix, a correlation between Webb’s Depth of Thinking and Bloom’s Type of Thinking.

The pressure is on those of us who work with secondary level students. With college and career readiness a tangible goal, we must plan, develop, and work with intent and purpose (just as we teach our students to). The following strategies will be ones that I will continue to implement in an effort to make my past practice match up with the more rigorous standards of the Common Core. I am careful to choose strategies that can be used across content areas and become a repertoire that can be applied in many reading contexts, not just my Reading class. It is not meant to be an exhaustive list, rather one that I will refer to as the school year progresses and to which I can look for research-based methods to reach my students. As always, the Gradual Release of Responsibility model is the framework on which I tack my instruction. I scaffold using Think Alouds as I move through the strategy itself, guide students in activating their schemas, and set an authentic purpose for reading. To illustrate I will use “Crime and Punishment” (Smith, 2012), an article about the recent action of the Supreme Court that struck down mandatory life without parole for juveniles.

List/Group/Label

This strategy helps and challenges students prepare to read an instructional level text by sorting and categorizing words and terms they will read about before they engage with the text (Readence, Moore, & Rickelman, 2000).

For example, if I am going to have my students read “Crime and Punishment”, I would give them the words parole, mandatory, rehabilitation, culpability, capacity, horrific, heinous, susceptible, and ebb and ask them to sort them into at least 2 categories. We would discuss their ideas and brainstorm what our reading for the day would be about. By activating their background knowledge in this way, they are preparing to set a purpose for their reading and read actively.

Anticipation Guide:

Encourages active reading, the Anticipation Guide (Herber & Herber,1993) consistently works with my secondary students who love to share their ideas and opinions. I use statements from or related to the content of the text with which students can agree or disagree and state their reasoning both before and after reading.

For example, with the article “Crime and Punishment” I would use the statements:

  • There are some truly horrible crimes committed by 17-year olds, and those crimes deserve life without parole.
  • Young people are more susceptible to peer pressure than adults and their personalities are not fully formed, making them less morally culpable and more capable of change.
  • To make a decision to lock up a person for the rest of his life on the basis of something he has done when he’s 13 just doesn’t make sense.

Text Coding

A way to mark one’s metacognition based on Chris Tovani’s (2000) work, Text Coding helps students to keep track of their own thinking during reading. Students mark the text and record what they are thinking either in the margins (if it is their copy) or on post-it notes (if I need it back). Some codes I use are:

Symbol
Meaning
V
Visualize or make a picture in my head
P
Predict
I
Infer
T-T
Text-to-text connection
T-W
Text-to-world connection
?
Question: I wonder; I don’t understand; or Puzzles me
+
New information (clarify)
X
I disagree

 

Using Context Clues

I will often use a Think Aloud to model this strategy for my students. Using explicit instruction, students learn to use signal words in conjunction with a variety of context clues to find the meaning of Tier 2 words and/or Academic Vocabulary that can be used in multiple content areas. We tend to encounter mostly the Example-Illustration type and the Logic/Inference type of
context clues (Vacca, 2002) in our secondary-level readings.

For example, for the article “Crime and Punishment” I would use a Think Aloud to model my thought process as I figure out the word “ebb” using the Synonym type of Context Clue with the sentences, “Nearly as suddenly, violent crime began to ebb across the country. The reasons for the drop-off are debated.” I would explain that drop-off is used as a synonym to ebb and that by recognizing the difficult word ebb and then paying special attention to the text immediately after it, I could find a word that means about the same thing.

QAR (Question Answer Relationships)

By identifying the type of questions they are being asked, Taffy Raphael’s strategy (Raphael, 1982) helps students have a better idea of what their answers might be. We talk about Right There, Think and Search, Author and Me and On My Own as the types of questions and that the answers come from the reader “In my head”, the text “In the book”, or a combination of the reader
and the text (Inference).]

The article “Crime and Punishment” could lead to questions such as:

Right There: What are the two harshest sentences that the Supreme Court has whittled away over the past decade?
Think and Search: How does the issue of human rights affect adolescent criminals?
Author and Me: Use the text and your life experiences to agree or disagree with professor William Otis who says there is little doubt that one reason for the decline in violent crime is that “the people who have been committing these crimes are now in jail”.
On My Own: Rebecca Falcon faults her choice of friends as a key component in making one of the worst decisions of her life. How important are friends in your decision making process?

SQUARE

SQUARE (Herczog & Porter, 2010) is an acronym that I like to use with nonfiction articles. I let students choose four out of the six letters to complete in partnerships or triads. I find it encourages higher-order thinking and problem solving.

Summarize– Identify and paraphrase the most important points in the text.
Question– Ask clarifying questions about the text to uncover points that are unclear.
Use– Use the information in a meaningful way by providing an example.
Apply– Use the concept in a new situation; make a connection to a current event.
Review– Reflect on your new interpretation by reviewing information from the text.
Express– Demonstrate your understanding in a creative way.

For example, with “Crime and Punishment”, students might act out the opinions of some of the Supreme Court justices that are highlighted in the article as the “Express”. They might research the case of one of the nine prisoners in Connecticut serving life without parole for crimes they committed when they were 17 or younger as the “Use”.

Use Graphic Organizers with Informational Text

Recognizing text structure is a powerful a key to comprehension; an especially important step in understanding the process of writing an effective summary (Armbruster, Anderson, & Ostertag, 1987). Use graphic organizers to plot and organize:

Problem/Solution
Cause/Effect
Chronological Order
Sequence
Compare/Contrast
Main Idea and Details

The article “Crime and Punishment” follows the Main Idea and Details text structure and I would ask students to identify which text structure it best fits into and then plot the Main Idea and Details into either a web or a triangle shaped template to demonstrate understanding.

Comprehension Strategies

Because my end goal is for my students not to need me, I use the Reciprocal Teaching (Brown & Palinscar, 1987) model as the underpinnings of my comprehension strategy instruction. I analyze the reading material and consider the learner(s) in order to plan an explicit focus on one or more of the following:

Previewing
Skimming/Scanning/Searching
Fixing-Up/Monitoring/Clarifying
Predicting
Visualizing
Questioning
Inferring
Analyzing/Evaluating/Making Connections
Organizing
Information/Summarizing/Visually
Representing

With the article “Crime and Punishment”, Visualizing would be a very effective strategy to focus on. Asking students to make a mental image of a mistake they have made going incredibly wrong and imaging the most serious consequences being applied to such, would help students to be able to Infer how Rebecca Falcon feels and Analyze, Evaluate, and Make Connections to the situation for over 2,000 people whose dire mistake when they were the same age led to life without parole.

RAT (Read Around the Text)

RAT is a strategy that encourages students to examine the whole text before just jumping in and reading. It is a series of six prompts that guides the reader to really notice those text features that will allow students to activate their background knowledge, make predictions, and set a purpose for reading. The steps are:

  1. Look at the pictures. What ideas are being presented?
  2. Look at the captions and read them.
  3. Look at the maps, charts, and graphs. Discuss what information they present.
  4. Look at the titles and headings. What is the big idea?
  5. Read the first and last lines of each paragraph for more information.
  6. Ask questions. Give yourself a reason to read.

DRTA (The Directed Reading Thinking Activity)

The steps developed by Russell Stauffer (1969) of activating schema, finding connections to what they know, making predictions, and setting purposes for reading are all a build up to my focus this year- using information in the text to form ideas and make arguments. Writing responses will be an extra step that I will incorporate to help them meet the more rigorous reading standards set by the CCSS.

With “Crime and Punishment” I would use List, Group, Label to activate schema, Visualizing to find connections to what they know, and the RAT strategy to guide students in making predictions and setting purposes for reading.

References

Armbruster, B.B., Anderson, T.H., & Ostertag. J. (1987). “Does text structure/summarization instruction facilitate learning from expository text?” Reading Research Quarterly, 22, 331–346.

Brown, A.L. & Palincsar, A.S. (1987). Reciprocal teaching of comprehension strategies: A natural history of one program for enhancing learning. In J. Day & J. Borkowski (Eds.), Intelligence and exceptionality: New directions in theory, assessment and instructional practices (pp. 81-132). Norwood, NJ: Ablex.

Carroll, L. (1865). Alice’s adventures in wonderland. New York: MacMillan.

Common Core State Standards Initative. (2010). Common core state standards for Engish language arts and literacy in history/social studies, science, and technical subjects. Washington, DC: National Governor’s Association Center for Best Practices and the Council of Chief State School Officers.

Herber, H.L. & Herber, J.N. (1993). Teaching in content areas with reading, writing and reasoning. Boston: Allyn & Bacon.

Herczog, M.M. & Porter, P. (2010). Strategies for struggling readers: A teacher resource guide. In We the people: The citizen and the constitution, Level 2. Center for Civic Education.
Retrieved August 7, 2012, from www.civiced.org/pdfs/books/2010bkwtplitguidelvl2MR.pdf

Mesmer, H.A., Cunningham, J.W., & Hiebert, E.H. (2012). Toward a theoretical model of text complexity for the early grades: Learning from the past, anticipating the future. Reading Research Quarterly, 47(3), 235-258.

Raphael, T.E. (1982). Questioning-answering strategies for children. The Reading Teacher, 34, 186-190.

Readance, J.D., Moore, D., Rickelman, R. (2000). Prereading activities for content area reading and learning. Newark: DE: International Reading Association.

Smith, P. (2012). Crime and punishment. The New York Times Upfront 145(2), 8-11.

Snow, C. (Ed.). (2002). Reading for understanding: Toward an R & D program in reading comprehension. Santa Monica, CA: Rand.

Stauffer, R.G. (1969). Directing reading maturity as a cognitive process. New York: Harper & Row.

Tovani, C. (2000). I read it, but I don’t get it: Comprehension strategies for adolescent readers. Portland, ME: Stenhouse Publishers.

Vacca, R.T. (2002). Making a difference in adolescents’ school lives: Visible and invisible aspects of content area reading. In A.E. Farstrup & S.J. Samuels (Eds.). What Research Has To Say About Reading Instruction (3rd ed., pp. 184–204). Newark, DE: International Reading Association.

Connecticut Association For Reading Research Summary Report of the Connecticut State Board of Education Meeting (CSBE) August 9, 2012

Dr. Ann Marie Mulready

The Connecticut State Board of Education convened a special meeting to approve plans for the four turnaround schools in the Commissioner’s Network—Curiale School in Bridgeport, the Academy at Milner School in Hartford, High School in the Community in New Haven, and the John Stanton School in Norwalk.

Public Participation

  • Mary Galucci, a parent of children in the Windham School system, questioned the funding of the inter-district magnet school, while other schools in the system experience significant deterioration of the buildings. She related her attempts to express her concerns to Steven Adamoski, the current Special Master, without response.
  • Wm Morrison, a teacher, questioned the use of CMT/CAPT data for comparing cohort achievement and evaluating teachers, noting that it violated sound statistical principles. In addition, he questioned the costs with respect to time and money.
  • Tim Nolan, the chair of the Region 19 reapportionment committee and a member of Board of Education, reported on the process for adjusting the weighted scoring to determine Board representation for the towns of Mansfield, Willington, Ashford. The current system for adjusting Board representation was successful and that representation was adjusted to reflect the growing Mansfield population.

Consent Agenda

The consent agenda was accepted and approved.

Commissioner’s Report

Commissioner Stefan Pryor reported on the general process for creating turnaround plans. Each school completed audits, determined needs, created a Turnaround Committee that included school personnel, a member of the school governance committee, a member of the Board of Education, and the superintendent. Each school is in a high priority district and performed in the lowest 10% of CMT or CAPT measures. These four schools are the first of 25 to be authorized over the next three years. Each school commits to three years of participation, with the aim of creating sustainable change within those years.

Curiale School.

Dr. Sandy Kase, Chief Administrative Officer for the Bridgeport Public Schools presented the Curiale turnaround plan. The plan includes:

  • An extended school day created through flexible scheduling and staggered staffing. This plan adds 88 minutes per day, resulting in an additional week in the school year.
  • Extended time for core literacy and mathematics instruction.
  • Partnerships with nearby community services to provide medical, dental, and psychological services.
  • New core curriculum aligned with the Common Core State Standards
  • Alignment of the after school, Lighthouse program, with student needs
  • Increased instructional services for struggling students in literacy and mathematics
  • Adoption of the UConn Gifted and Talented, School Wide, Enrichment Model
  • Structured opportunities for parent engagement
  • Extensive professional development and increased instructional resources
  • An emphasis on early childhood
  • A partnership with the Bridgeport police to provide safe corridors for children going to and from schools
  • The retention of teachers rated proficient and the filling of vacancies with teachers rated as excellent

In response to questions by the Board, Dr. Kase reported that a new principal and assistant principal had been appointed, and teacher evaluations were being aligned with the new state evaluation system, Also, once impending changes were defined, most teachers have chosen to remain in the turnaround school and approximately ⅓ of the staff are teachers of color. Dr. Kase also explained that in the past the school had been hampered by poor decision making related to budgeting and that the funding provided by the State Board would help underwrite the plan while working toward sustainable change.

Milner Academy.

Kelvin Rodan, Chief of Instutional Advancement Officer, and Dr. Michael Sharp, CFO of the Jamoke Charter Management Organization, presented the plan for the Milner Academy. Jamoke, a charter group, is working with Milner to achieve the turnaround. The highlights of the proposal include,

  • Strong family and community connections
  • Effective leadership and faculty
  • Effective use of curriculum and instruction
  • Effective use of time
  • Effective use of data to inform instruction and the use of benchmarks to measure progress
  • A decrease in the staffing ratio of adults to children from 12::1 to 8::1 through academic assistants
  • A teacher development plan
  • The overall establishment of a teacher driven school in collaboration with parents and the community
  • An extended day of ½ hour for the first year and another 25 minutes in the second year, along with 12 Saturday academies, resulting in 34 extra days of instruction by the second year.

The Commissioner informed the Board that changes were required in the plan, though the changes will be established in a Memo of Understanding, between the Jamoke organization and the Hartford Public schools and subject to the Commissioner’s approval. In particular revisions may be necessary for the 2013/2014, especially with respect to a collaborative method for assembling a quality faculty, staffing quality and stability, the establishment of a three year commitment by faculty, and the method for training of faculty.

In response to Board questions, the relationship between Milner and the Jamoke organization was explained as a partnership with Milner remaining as a Hartford public school. Sandra Ward, Director of Community Schools, explained that the after school program is supported by Catholic Charities and the Hartford Foundation for Public Giving. With respect to students with special needs, including English Language Learners, the staff serving those populations will be doubled.

With respect to benchmarks, specific SPI benchmarks will be detailed in the Memo of Understanding. The Jamoke organization will be expected to show the same rates of academic growth. Four per cent growth is expected and the goal is to outpace the state. In terms of safety, significant dollars will be invested in the physical aspects of the building, including alarms at entrances and exits, and climate control improvements for heat and cold.

Lastly, the Board questioned the replicability of the model, in particular the reduction of class size and the closing of enrollment on October 1st. Charles Jaskiewicz recommended to Chairman Taylor that language to address the variations that may occur in the population be added to the memo of understanding. A description of the responsibilities of academic assistants was requested by Chairman Taylor and Dr. Sharpe responded that they are are not paraprofessionals. They will have at least two years of college and may assist the teacher with small group instruction to provide differentiation and modeling of school expectations.

The Milner plan was approved unanimously.

High School in the Community (HSC).

The plan for the High School in the Community in New Haven, which has long been a teacher run school, will partner with the New Haven Federation of Teachers (NHFT) to take it to the next level. The plan was presented by Garth Harries, Assistant Superintendent and Dave Low, VP of NHFT. A key component in New Haven school reform has been portfolio development in school turnaround efforts. Two additional components of the effort have been to take aggressive action in the areas where a student needs it most and to emphasize to all stakeholders that it is the school unit that matters in achievement. Other highlights of the HSC turnaround include:

  • The development of teacher leadership and professional community
  • Shifting the academic focus to a competency based instructional model and determining what true personalized learning looks like
  • Developing a deep sense of community with students and parents

With respect to teacher leadership and teacher excellence, members of the staff have attended the National Association for Academic and Teaching Excellence. All teachers must reapply to the school, ⅔ of the staff will be rehired, and the remaining ⅓ will be new. (This is substantially completed.). In order to shift the academic focus, extra time will be added to the day, resulting 11 extra days in the calendar. In addition there is work aimed at developing professional collaborative time and the school is a member of the League of Innovative Schools, a group of schools transitioning to a mastery based (vs. time spent) model.

Mastery based learning begins with emphasis on core skills and the student report card will identify performance as Mastery, Exemplary Mastery and Not Yet Mastered. Instead of a four year model, the attainment of skills will be individually managed and at the end of a module students will defend their learning. Once they can defend the learning, credit is assigned and they move on to the next module. The final phase is a capstone project that may include mentoring by outside mentors. Graduation is dependent on completion of the curriculum, not the time spent, and the student may graduate whenever the requirements have been met.

A family outreach specialist and a health professional will be hired to increase community and family involvement and to support the physical and mental health of the students.

The motion to approve the plan was approved.

John Stanton School.

The plan for the Stanton school was presented by Abby Dolliver, superintendent of the Norwich Public Schools, and Tom St George, a middle school social studies teacher.

Turnaround support from the state will help with the search for talented staff already in progress. In addition, support for the classroom teacher will be provided by the presence of two other adults in each classroom in an effort to provide more individualized attention, since the size of the building limits the ability to reduce class size substantially. Administrative support will also be provided through the addition of 092 certified personnel.

Instructional time will be increased by 300 hours in 3 years, beginning with an additional hour in the school day this year. In September and October, the extra hour will be used for extensive professional development and the students will have additional instructional time beginning in November.

A summer program will offer differentiated intervention and an accelerated academy for students. New Family liaison positions will be added to meet the needs of non English speaking families. More deliberate partnerships with community child and family agencies, including the after school program, will be established to meet student and family social and emotional needs.

The process for developing this plan was not limited to Stanton School stakeholders, but all district faculties were invited to meetings, including the CEA representative, to contribute to the brain storming process.

In response to questions, Dolliver noted that 50% of the staff has changed and that Teaching Residents will serve to support the teachers on a rotating basis. The Resident may not be a certified teacher, but will be a part of the bargaining unit.

The motion to approve the John Stanton turnaround plan was passed.

Theresa Hopkins-Staten noted that there were common themes in all the plans–leadership, safety, and cultural competency. She asked the Commissioner whether or not the State Department of Education was working to address these issues across all these schools. Commissioner noted that the turnaround process was structured around seven categories. Based on these categories there will be ways to address the recurring themes and Michelle Rosato and Charlene Russell-Tucker are leading staff to address those.

Other Business

In other business, the Board approved the permanent appointment of Diane Ullman as the Chief Talent Officer.

Paul Vallas and Bob Trefry, Interim Superintendent and Board Chair, respectively, reported on the status of the Bridgeport Schools, currently operating with state oversight.

The process of change in the Bridgeport district began with assessing the needs of the district, as perceived by all the stakeholders in the district. Three specific goals were established to be accomplished within the year of oversight.

  • Close the budget deficit and establish a financial plan to provide stability
  • Establish a long term school improvement and close the achievement gap
  • Build the human infrastructure

Through city support, the forgivable loan, outside agencies in kind services, and budget cuts, the deficit was closed without significant staff losses–only 9 teachers lost jobs. The size of the central office was reduced by a third. A comprehensive PK-12 school instructional audit resulted in a strongly aligned PK-12 curriculum and investments have been made in books and materials.

The Board and the Commissioner praised the progress and the meeting was adjourned.

Effects Of Three Vocabulary Strategies On Kindergarten Tier 2 Word Recognition

Lisa Mello Handfield
Andover Elementary School, Andover, CT

Abstract

This action research project sought to determine the effects of three vocabulary strategies on Kindergarten Tier 2 word recognition. In a sample of convenience, 16 Kindergartners were pre-tested on their expressive and receptive knowledge of three target words with an investigator created assessment tool. Students were then assigned to small groups for a book sharing activity: Group I received incidental exposure vocabulary instruction,Group X received extended vocabulary instruction, and Groups XR and XR-R received extended instruction with re-reading vocabulary instruction. All groups read Bon-Bon the Downtown Cow (Appleton-Smith, 2003) and were post-tested with the same assessment tool. Overall data analysis results indicated that all groups increased their word knowledge. However, differential percentages of word recognition growth between groups, based on instructional strategies,were noted.

Introduction

In looking at the five main components (phonemic awareness, phonics, vocabulary, fluency and comprehension) of quality reading instruction, phonemic awareness and phonics are at the roots of the early levels of learning, where sounds and subsequent letters that match those sounds are being mastered. There are numerous programs and assessments that determine a child’s growth and current ability in these phonological areas. However, determining a young child’s oral vocabulary level, and subsequent instruction and assessment, is more difficult.

When educators delve into vocabulary acquisition, two major areas of influence become apparent for early learners. Educators need to be aware that a large part of word knowledge relies on incidental word learning. Incidental word learning stems from wide reading and rich oral language experiences, where adults have conversations with children and engage in book sharing experiences on a daily basis. Extensive reading also promotes vocabulary development as repeated exposures to words, word concepts and word contexts help widen a young child’s knowledge base. However, these key components may not occur in a youngster’s home or even before entering school.

Accordingly, educators also need to be cognizant of the value of direct vocabulary instruction. The school setting is where explicit instruction of vocabulary becomes most feasible. Specific word instruction relies on expanding a young child’s vocabulary with skill building activities that centers on word analysis, context clues, and reference use. Also, the Armbruster, Lehr & Osborn (2003) found that intentional instruction of vocabulary items is required for specific texts and the dependence on a single vocabulary instructional method will not result in optimal learning. Rich and robust vocabulary instruction allows children to actively engage in using and thinking about word meanings and in creating relationships among words.

Under the umbrella of the effective teaching cycle, I decided to explore and implement research-based vocabulary strategies suitable for young learners and formulate an assessment tool based on specific word instruction. My action research is based on the notion that extended and repeated instruction promotes active engagement with vocabulary, which leads to improved word learning (Armbruster, Lehr & Osborn, 2003) as my intention is to determine what vocabulary instruction method is most beneficial, at the Kindergarten level, for increasing target word recognition.

Review Of Related Literature

Vocabulary is simply the knowledge of words and word meanings. Components of effective vocabulary instruction include incidental word learning and intentional vocabulary teaching. In the first years of school, particularly Kindergarten, teachers utilize read-alouds and book sharing to expose children to oral language. Emphasis has conventionally been placed on the visual part of each page, particularly in the use of picture books and predictable text when reading to children (Justice, Pullen & Pence, 2008). Also, word concept instruction at the lowest grades has included context-specific vocabulary and child-friendly definitions. Key vocabulary instruction strategies consisted of oral and listening language skill building such as teacher think-alouds, questioning, and summarizing.

There is an abundance of research (McGee & Schickedanz, 2007; Lane & Allen, 2010) that supports the effectiveness of rich and robust vocabulary instruction which identifies key words, builds upon an already familiar concept and then allows children to cognitively stretch beyond the original context of word acquisition. Extended instruction is teacher-directed and explicit. Such an approach includes definitional and contextual elements of word knowledge. Repetition and multiple exposures to vocabulary items are important (Armbruster, Lehr & Osborn, 2003). Extended, or dialogic, reading instruction is a strategy which has recently received substantial merit in research findings (Coyne, McCoach, Loftus, Zipoli & Kapp, 2009; Coyne, McCoach and Kapp, 2007).

More specifically, Beck, McKeown and Kucan (2002) promote the use of the Text Talk strategy, with selected Tier 2 vocabulary from a story, to encourage dialogic reading interactions between the teacher and students. Vocabulary instruction involves learning new words for familiar concepts, as well as learning new concepts for familiar words. Early childhood educators need to draw on the higher listening levels that children possess in the pre-literacy stages of reading development to choose vivid words and dramatic storybooks (Newton, Rasinski, & Rasinski, 2008). Educators must orally reinforce child-provided connections between words and meanings (Beck, McKeown and Kucan, 2002). Vocabulary instruction needs to be purposeful, but also flexible, in responding to the individual and egocentric thoughts that young children possess.

Methodology

Participants.

Participants were a sample of convenience from my Kindergarten classroom. There were 16 children used for this vocabulary action research project. Ten children were boys and six children were girls. Four participants were identified as remedial/intervention readers who receive daily, supplemental instruction with a Remedial Reading teacher. Fifteen students are Caucasian and one student is of African-American descent. All participants were of Christian faith. No participants received free or reduced lunches. One child is an English Language Learner, with Polish as her first language.

Intervention.

Non-remedial readers (12 students) were randomly assigned to three groups (Group I, X, or XR, which had four students each, while the four remedial readers were kept together to form their own group (Group XR-R). All groups read the book Bon-Bon the Downtown Cow. This book was selected because it was unfamiliar to the children and written to be a high quality, decodable text that could serve numerous instructional purposes. I also chose this book because it was not a picture book so that the children could concentrate on the oral vocabulary, and would not rely heavily on the visual art, to tell the story. The vocabulary strategies employed during the reading the story varied:

Group I (random, non-remedial students) was given incidental exposure instruction, whereby the teacher read the book to them in a quiet, small group setting.

Group X (random, non-remedial students) were given extended instruction, whereby children received pre-reading exercises to expose them to the target words, listened for target words, had the teacher read the book to them in a quiet, small group setting and then discussed target words with postreading exercises.

Group XR (random, non-remedial students) were given extended instruction and rereading of text, whereby children received pre-reading exercises to expose them to the target words, listened for target words, had the teacher read the book to them in a quiet, small group setting and then discussed target words with post-reading exercises. The next day, the teacher re-read the book to them in a quiet, small group setting.

Group XR-R (remedial students) were given extended instruction and re-reading of text, whereby children received pre-reading exercises to expose them to the target words, listened for target words, had the teacher read the book to them in a quiet, small group setting and then discussed target words with post-reading exercises. The next day, the teacher re-read the book to them in a quiet, small group setting.

As part of the extended instruction vocabulary strategy (Groups X, XR and XR-R), the teacher provided oral, pre-reading activities to familiarize the children with the three target words. The participants repeated the target words and were instructed to raise their hand when they heard the words in the story. Upon hearing the words, the teacher re-read the sentence, provided a short definition for the target word and then reread the sentence with the child-friendly definition inserted. When the story was completed, teacherdirected vocabulary activities included answering literal questions, making text to self connections and decontextualized use of the target words. The specific scripts that were followed for pre-reading vocabulary activities were adapted from research by Coyne, McCoach & Kapp (2007).

Assessments.

Through consideration of Armbruster, Lehr & Osborn’s (2003) findings, which assert that specific vocabulary growth is best assessed through researcher-developed measures as they are more sensitive to gains achieved through instruction than are standardized tests, I developed my own pre-/post-instruction assessment. I chose three Tier 2 words that I believed Kindergartners would not normally encounter in their oral or receptive vocabularies. All children were asked individually by a teacher about each word, in turn. In this quiet location, each child was asked if they knew what the target word meant (EXP-DEF, Expressive definition). Each Kindergartner was prompted to answer two yes or no questions to clarify the meaning of the target word (EXP-Y/N, Expressive yes or no question). Both of these assessments parts were to provide a baseline for expressive vocabulary knowledge. Lastly, each Kindergartner was shown four pictures and asked to point to the target word. Distractors that were phonetically similar were included in this picture assessment. This assessment piece was to provide a baseline for receptive vocabulary knowledge (RECEP, Receptive definition). The specific scripts utilized for pre-/post-instruction assessments, and adapted from research by Coyne, McCoach and Kapp (2007).

Data Results/Analysis

Specific instructional strategies results.

In looking at post-instruction assessment data, several more and specific observations can be noted based on the instructional strategy utilized by the teacher. Group I, which received incidental exposure, showed growth in their expressive and receptive vocabulary knowledge. Simply reading a book aloud to children made measurable increases in vocabulary knowledge.

Group X, which received the extended vocabulary instruction strategy showed more than moderate gains in word knowledge. Introducing target vocabulary, discovering these vocabulary words in context and exploring words through decontextualized questions pointed to measurable increases in word knowledge.

Group XR, which received the extended vocabulary instruction strategy and a re-reading of the book, showed moderate growth in the area of word recognition. The overall percentage of preinstruction scores for Group X and Group XR were similar. However, the overall percentage of post-instruction scores varied. Group XR-R consisted of the remedial students who receive daily supplemental services from the Reading teacher. Overall, the gains for these students were the most modest. Better word recognition, after the vocabulary strategies of extended instruction and re-reading of the book, became apparent. Most notable was the fact that none of the students in this group could define any of the target words prior to the intervention; however, 17% could define the words during the post-instruction assessment. Providing two supportive word knowledge strategies (extended vocabulary instruction and re-reading) was the logical choice for these remedial students and these children showed growth when dual vocabulary methods were employed.

Discussion

Review of the overall post-instruction data illustrated that students had gains in word recognition skills, especially when this sample of convenience made measurable gains in orally defining target words. In addition, it is worthy of note that the overall percentage of definition knowledge growth increased greatly from 14% to 50% while the overall percentage of receptive knowledge growth increased moderately from 52% to 77%.

Conclusions

For me, as an educator, it was valuable to look back at the pre- and post-instruction data from this action research project to discover and analyze the levels of word knowledge in my classroom. Word knowledge can be identified on the following continuum: (a) no knowledge, (b) general sense of the word, (c) narrow, context-bound knowledge and (d) some knowledge of the word, but not being able to recall it readily enough to use it in appropriate situations and (e) rich, decontextualized knowledge of a word’s meaning (Beck, McKeown and Omanson, 1987). This project sought to increase word knowledge and when analyzing the overall findings, the children could answer more post-instruction expressive and receptive questions correctly for the new, target words. Oral vocabulary knowledge increased regardless of type of vocabulary instruction received.

I also discovered that simply reading to children introduces vocabulary and incidental exposure can provide a context for these new words. Extended instruction also exposes children to new vocabulary and can provide a more structured format in which to absorb contextual and decontextualized concepts for these new words. The added benefit of re-reading a story, along with extended instruction, was not evident in my research. However, Armbruster, Lehr & Osborn (2003) determined that how vocabulary is assessed and evaluated can have differential effects on instruction. Further study of the statistical significance of vocabulary gains for extended instruction, with or without re-reading, versus the incidental exposure technique is in  order. Such repeated administrations or replications of this study, using the incidental exposure, extended instruction and extended instruction with re-reading strategies, could better pinpoint for teacher-researchers the effects of each.

References

Appleton-Smith, L. (2003). Bon-Bon the Downtown Cow. Lyme, NH: Flyleaf Publications.

Armbruster, Bonnie B., Lehr, Fran & Osborn, Jean (2003). Put reading first: The research building blocks for teaching children to read: Kindergarten through grade 3. Jessup, MD: National Institute for Literacy.

Beck, I., McKeown M. & Kucan L. (2002). Developing vocabulary in the earliest grades. In Bringing words to life: robust vocabulary instruction (pp. 47-71). New York, NY: Guilford Press.

Beck, I.L., McKeown, M.G., & Omanson, R.C. (1987). The effects and uses of diverse vocabulary instructional techniques. In M.G. McKeown & M.E. Curtis (Eds.), The nature of vocabulary acquisition (pp. 147-163). Hillsdale, NJ: Erlbaum.

Coyne, M., McCoach D., & Kapp, S. (2007). Vocabulary intervention for kindergarten students: Comparing extended instruction to embedded instruction and incidental exposure. Learning Disability Quarterly 30(2), 74-88.

Coyne, M., McCoach D., Loftus, S., Zipoli R. & Kapp, S. (2009). Direct vocabulary instruction in kindergarten: Teaching for breadth versus depth. The Elementary School Journal 110(1), 1-18.

Justice, L., Pullen, P., & Pence, K. (2008). Influence of verbal and nonverbal references to print on preschoolers’ visual attention to print during storybook reading. Developmental Psychology 44(3), 855-866.

Lane, H. & Allen, S. (2010). The vocabulary-rich classroom: Modeling sophisticated word use to promote word consciousness and vocabulary growth. The Reading Teacher 63(5), 362–370.

McGee, L. & Schickedanz, J. (2007). Repeated interactive read-alouds in preschool and kindergarten. The Reading Teacher 60(8), 742-751.

Newton, E., Rasinski, D. & Rasinski, T. (2008). Evidence-based instruction in reading: a professional development guide to vocabulary. Boston, MA: Pearson Education.