Our Data

Data Support Using CRAs to Improve Readiness

College Readiness Assignments for Texas (CRAfT) is a multi-year research initiative, supported by a grant from the Texas Higher Education Coordinating Board. The CRAfT team led the development and revision of College Readiness Assignments (CRAs) by faculty from high schools, community colleges, and universities and the subsequent statewide field test of the CRAs to assess their effectiveness and usability.

During the 2012-2013 school year, field testers across the state of Texas implemented and reviewed the CRAs. More than 75 educators in 18 institutions, including high schools, community colleges, and four-year institutions, representing more than 3,500 students participated in the field test. The results of the field test were overwhelmingly positive. CRAs were shown to be effective in increasing students’ college and career readiness, and educators found them helpful and usable. Below, we present a brief summary of the results, which support the CRAs’ effectiveness, usability, and their ability to signal the levels and types of readiness needed for college and career success.

Effectiveness

Data collected:

  • Surveys of educators’ perceptions of students’ college readiness
  • Interviews of individual educators

Findings:

  • College Readiness Mindset Survey (Pre/Post) measured gains in 19 critical skill areas, 15 of which were statistically significant:
    • Mastery of big ideas; Studying; Research; Trying multiple strategies to construct an answer; Accurate self awareness of academic abilities; Data; Transferring knowledge to new or different contexts; Technology; Time management; Perseverance; Intellectual curiosity; Writing; Reasoning; Quantitative problem solving; Working collaboratively; Working independently; Academic integrity; Reading; Key content knowledge
  • Quote from educator: “I believe the CRAs have the ability to help students see the difference in college work vs. high school level work.”
  • Quote from educator: “This experience has been invaluable for my students. They have had the opportunity after each CRA to evaluate their performance according to the college-readiness standards. This process has resulted in a better understanding of where their weaknesses lie and the motivation to address them.”

 

Usability

Data collected:

  • Surveys of educators about their experiences implementing CRAs
  • Classroom observations of CRA implementation
  • Interviews of individual educators

Findings:

  • Most educators reported a positive impact of the CRAs on several elements: the importance of encouraging students to be responsible for their own learning, their ideas about instructional planning, and shifting their overall instruction to be more student centered.
  • CRAs were very helpful, and faculty appreciated their standalone nature, inclusion of TEKS and CCRS, challenging assignments, and thorough instructions.
  • Quote from educator: “This CRA . . . has given me a greater understanding of what it means to be college ready and ways I can cultivate college-ready skills into my students. As a result, I have included more assignments that have a writing, researching, and presentation component into my curriculum.”
  • Quote from educator: “I really liked the way the student handout put the learning in the hands of the students and have used the same approach now for other assignments I’ve created.”

 

Degrees of Readiness

Data collected:

  • Surveys of educators about their experiences implementing CRAs
  • Scoring sheets completed by educators assessing student work
  • Student work products
  • Scoring sheets completed by higher education reviewers to compare to field tester ratings

Findings:

  • Higher education reviewers independently scored the submitted work products according to the same scoring sheets used by the field testers; secondary educators tended to rate student work as more college ready than higher education reviewers.
  • On the 1-4 scale (1= Initiating College Ready, 2 = Approaching College Ready, 3 = College Ready, 4 = Exceeding College Ready), the average rating from educators across all disciplines was 2.70 (close to the College Ready category), while the average rating from higher education reviewers was 2.05 (Approaching College Ready).
  • While secondary educators and higher education reviewers rated ‘Content Knowledge’ items similarly, secondary educators tended to rate the areas of ‘Key Cognitive Skills’ and ‘Foundational Skills’ up to a full point higher than higher education reviewers. This underscores the notion that secondary educators tend to underestimate the level of readiness needed in Cross-Disciplinary skills (such as critical thinking, reasoning, and research).
  • Quote from educator: “It showed me that the students really need more basic reading skills to help them with reading assignments in college texts.”
  • Quote from educator: “The lesson felt like something I would teach anyway. It did slow me down, but in a good way. I think I generally take it for granted that my students are better at critical thinking than they are.”

 

For a printable PDF of this information, click here.