Spotlight on Data

Mon, 04/07/2014 - 14:12 -- Emily Johnson

In our last post, we wrapped up our series exploring the subject areas for which CRAs are available. For this post, we would like to highlight some of the data that support using CRAs to improve college and career readiness. In CRAfT’s efforts to design lessons that would be both useful for educators and effective in preparing students for college and career, we collected and analyzed several pieces of data so that you could feel confident in bringing CRAs into your classroom.

During the 2012-2013 school year, field testers across the state of Texas implemented and reviewed the CRAs. More than 75 educators in 18 institutions, including high schools, community colleges, and four-year institutions, representing more than 3,500 students participated in the field test. The results of the field test were overwhelmingly positive. The great news is CRAs were shown to be effective in increasing students’ college and career readiness, and educators found them helpful and usable.

Teams from The University of Texas at Austin and Texas A&M University collected the following data:

  • Surveys of field testers about Professional Development
  • Surveys of field testers about their experiences implementing CRAs
  • Surveys of field testers’ perceptions of students’ college readiness
  • Surveys of educators’ desires/needs from CRAfT website
  • Classroom observations of CRA implementation
  • Interviews of individual field testers
  • Scoring sheets completed by field testers assessing student work
  • Student work products
  • Scoring sheets completed by higher education reviewers to compare to field tester ratings

Effectiveness

How does CRAfT know the CRAs were effective in preparing students for college and career? The primary measure we used to determine this was the College Readiness Mindset Survey. This form asked educators to evaluate each of their students in a number of readiness domains, including key content knowledge in the subject, ability to persevere when presented with unknown subject matter, intellectual curiosity, and research skills.

Educators evaluated students before implementing CRAs in their classrooms and after students completed CRAs, and these data showed gains in 19 critical skill areas, 16 of which were statistically significant:

In addition to the College Readiness Mindset Surveys, CRAfT also asked field testers to tell us in their own words what they thought about CRAs.  Here are some of the responses we received:

  • “I believe the CRAs have the ability to help students see the difference in college work vs. high school level work.”
  • “This experience has been invaluable for my students. They have had the opportunity after each CRA to evaluate their performance according to the college-readiness standards. This process has resulted in a better understanding of where their weaknesses lie and the motivation to address them.”
  • “The lesson felt like something I would teach anyway. It did slow me down, but in a good way. I think I generally take it for granted that my students are better at critical thinking than they are.”

Usability

The CRAfT team knows that while there are many approaches to education that help prepare students, none of them work unless educators feel they are helpful for instruction and can reasonably be put to use in their classrooms.

Most field testers reported a positive impact of the CRAs on several elements. While they expressed that CRAs encouraged students to be responsible for their own learning, they also noted positive differences the CRAs made to their own teaching practice. In particular, field testers said that their ideas about planning and instruction shifted to be more student centered.

Here’s what some of our field testers said about how CRAs helped them prepare students:

  • “This CRA . . . has given me a greater understanding of what it means to be college ready and ways I can cultivate college-ready skills into my students. As a result, I have included more assignments that have a writing, researching, and presentation component into my curriculum.”
  • “I really liked the way the student handout put the learning in the hands of the students and have used the same approach now for other assignments I’ve created.”

2014 Follow-up

The field test of the CRAs ended in May of 2013, and the CRAfT team spent the summer of 2013 revising the CRAs and scoring guides according to the field testers’ feedback. We recently went back and surveyed the faculty who field-tested the CRAs to determine whether they are still using them, as well as to elicit any additional feedback they have.

  • 52% of faculty respondents say they are still actively using CRAs in their classrooms
  • Of the 48% who are not currently using CRAs, 65% of them are no longer in the classroom or they teach courses for which they believe CRAs are not a good match (e.g., alternative schools).
  • Nearly all of the remaining instructors cite time restrictions due to standardized testing as the primary reason they are not currently using CRAs, but they indicated they still believe CRAs to be high quality, challenging assignments.
  • The top 15 most-used CRAs among past field testers are:
    • Choosing the Best Websites to Support Your Argument
    • Evaluating Art: What's Your Favorite Movie?
    • Exploring a College Textbook
    • Function Families
    • Functions
    • Interrogating the Text: Reading Closely, Reading Critically
    • Japanese-American Internment Revisited
    • Reader's Analysis: Author, Purpose, Audience, and Meaning
    • Rhetorical Analysis I: Understanding Speeches
    • Rhetorical Analysis II: Understanding Editorials
    • Speaking to the Occasion
    • The Climate of College: Planning for Your Future
    • The Silken Tent: Metaphors in Life and Literature
    • Words, Words, Words: Learning and Using New Vocabulary
    • Writing to Inspire or Advise

These data show that CRAs are valuable tools for teachers to include in their curriculum. Remember, many of the CRAs are very flexible and, regardless of the suggested topic, can be adapted to address the content you are covering in your class. 

We would love to hear from you about your usage of CRAs and any suggestions you have. Please take a few minutes and share your experience here.

For a printable one-page summary data highlights, click here.

In our next post, on April 21, we will focus on Career Readiness.