ࡱ > g / bjbjVV b r< r< ' p p D 4 E ^ : < 6 r D D D D D D D $ F I f D ^ D F D w w w v D w D w w n > B PZ ? qD D 0 E @ J J @ B B J B X " w D D E J p : 6-22-12 WAC Assessment Plan: GOAL: Assess effectiveness of WAC across departments and over several years. Several forms of indirect measures have been developed for the WAC program: Faculty survey (developed but not yet administered) Evaluation of faculty training workshops Quantitative data collected (how many faculty trained per year, number of WAC courses developed, number of WAC workshops, brown bag lunches, and special events hosted each year, number of departments developing upper division WAC initiatives, departmental teams receiving WAC grants and the final reports of these grants. There are two direct measures of assessment for the WAC program: a stratified random sample of student writing assessed using a twelve trait rubric with a four point scale and the annual syllabus review process that certifies new WAC courses and recertifies existing WAC courses on a three year rotating basis. 1. Faculty Survey This survey was developed several years ago but was never instituted. 2. Evaluation of faculty training workshops. Data is available for the past 8 years and is available upon request. This data is in the process of being summarized. 3. Quantitative Data: Faculty Training In September 2008, the WAC Committee suggested and UUPC concurred that all faculty who teach WAC courses should receive training. At that time, 53 faculty had been trained in summer seminars over the course of four years. When the mandate was instituted, abbreviated training sessions were added fall and spring terms to provide more training to more colleagues. The Summer Seminar was reserved for faculty who were designing new courses. 279 faculty have been trained by the WAC Director since 2003 when training began. There was a peak of training in 2007 when the WAC program was officially adopted. 154 faculty were trained over those three years. Over the past three years, 83 additional faculty have been trained, about 20 each year. Training will continue in the fall. (See Appendix A below for a graph of all training results.) As the university gears up for the QEP on research, the WAC program is planning to provide additional training for departments across the university. WAC Courses Approved 104 Courses have been WAC approved since 2003. 38 of these courses have been approved in the past three years: 7 Courses were approved during the 2009 - 2010 school year 6 Courses were approved during the 2010 - 2011 school year 25 Courses were approved during the 2011 - 2012 school year WAC Sections Taught Each Term During the Past 3 Years: The number of sections designated as Writing Across the Curriculum have been steadily increasing over the past three years. The chart below provides semester-by-semester data and annual totals. School YearFallSpringSummerTotal2009 - 2010268239705772010 - 2011274258746062011 - 201231729670683 Number of WAC Workshops [brown bag lunches, and special events hosted each year, number of departments developing upper division WAC initiatives, departmental teams receiving WAC grants and the final reports of these grants. (still working on this material] 3. Syllabus review process: All new WAC syllabi are reviewed by at least two WAC Committee members using a rubric check list keyed to the two levels of WAC courses that we certify. WAC Committee members are faculty who have taken the WAC seminar and have volunteered to serve on the Committee. Members have represented a broad range of departments from chemistry, social work, English, and history to sociology, public policy, nursing, and business. WAC courses can be replacements for second semester first-year writing, ENC 1102, or any 2000-4000 level course that meet the WAC guidelines. Guidelines for both courses may be found at HYPERLINK "http://www.fau.edu/wac" www.fau.edu/wac. All certified syllabi are reviewed every three years on a rotating basis. Faculty are notified near the beginning of the school year and asked to submit copies of syllabi for each WAC course they teach. If they teach multiple sections of a WAC course, only one representative syllabus is submitted. Each syllabus is reviewed by a WAC Committee member. If concerns are found, the report is sent to the faculty member and asked to revise the syllabus. The process is repeated until all syllabi are recertified. All History and Philosophy WAC syllabi (1st year) All English WAC syllabi (2nd year) All remaining WAC syllabi (3rd year) Student Paper Assessment What is the process? Random samples of essays are collected from all WAC courses. How will random sampling process work? chose random sample of classes making sure that there is a stratified random sample of classes across class days and times (computer generated). A statement will be provided to students that reads something like the following: Language for consent form and IRB approval? The aim of this study is to provide information to the university about the effectiveness and sustainability of the WAC program and will not be used to assess individual student or faculty performance. Your class has been randomly selected to participate in the assessment process. The information you provide will be kept strictly confidential by removing all personal identifiers that you offer and replacing them with numbered codes, which will only be seen by the staff and faculty who prepare the data for analysis. All personal data and papers that you provide will be automatically downloaded from the internet, password protected, and then secured on a computer that has no internet connection. How will papers be designated? All students from randomly selected sections will be asked to submit papers. Papers from those sections will be randomly selected. Typically, these papers will be substantial end-of-term projects/papers that have been revised. Papers should be single authored. First and second drafts should be submitted. Students will estimate how many hours they spent working on each draft: 1-3; 3-6; 6-9; 10 or more How will papers be collected? The Director of Assessment on campus, OIT, and the WAC Director collaborated to develop an online interface for students to submit papers. The interface allows students to upload their papers and fill out the self-efficacy survey. It also provides an interface for raters to score the papers and for administrators to manage the data collection and rating processes. Office of Institutional Effectiveness also processes the data from the student surveys and compares that data to student information such as the following to identify any significant correlations. # of credit hours completed toward graduation and class designation (sophomore, etc.) What classes student is currently taking Coded number so that the submission and data sheet can be correlated without using student names Name also in a hidden field Course # Gender Age Transfer and if so, from where? 2yr or 4yr? Receiving financial aid? Employed? For how many hours? Does student speak a different primary language at home than at school? If so, which one(s)? Were you born and raised in US? If not, how long have you lived in the US? Major or intended major? The WAC program assistant review the uploads and filters them to make sure all identifying information is removed except for the data sheet info. How will the papers be assessed? 15-20 trained reviewers read and rate papers Reviewers are normed and paid for their efforts. Norming generally includes 10-12 hours of norming before rating begins, additional norming sessions across the five days of rating, and spot norming with individual raters as scores come in and show discrepancies between raters. What rubric(s) will be followed for what disciplines? The WAC Committee has developed a University-wide rubric on which papers are evaluated. The rubric is used for program review and comparison of twelve traits of thesis-driven writing on a four point scale. (See HYPERLINK "http://www.fau.edu/wac/assessmentrubric.doc" http://www.fau.edu/wac/assessmentrubric.doc) How will the data be processed? Three raters read each paper and score each of the 12 traits on a four-point scale. Scores are processed for modes and averages; if two scores are identical the third score is removed; if three scores are sequential (eg. 2, 3, & 4), they are averaged. If three scores are different but not sequential, the two closest scores are averaged. This system provides consistently accurate data. Who will be the readers? Paid instructors/faculty who have taught WAC courses at FAU. How will they get compensated? Money is allocated in our annual budget for assessment process. How will we include indirect measures like writing efficacy scales (surveys of student & faculty confidence), and student profile survey to correspond with papers collected? Students complete a self-efficacy survey when they submit their papers each term. How often is the assessment cycle repeated? (outside consultant review assessment plan) Paul Anderson visited FAU to review the development of the WACprogram and assessment process Spring 2006 Year 1: review Year 2: review (outside consultant review program) Irv Peckham was brought to campus to review and monitor the assessment process. His report is available upon request. Year 3: review Year 4: review Year 5: review (only certain rubric items) Year 6: review (only certain rubric items) Year 7: review (outside consultant review assessment program) The first papers were collected in Spring 2007. Data from the first three years was not usable because of problems encountered in the rating interface and norming processes. Current data is available for 2010-2011. Data for 2011-2012 is being processed now. Appendix A NOTE: WAC Summer training for 2012 was cancelled because the Director of WAC was on sick leave. Appendix B EMBED Excel.Sheet.12 This table represents the WAC courses that have been developed under each departmental course heading. [I need to format this data more effectively] ! ( n o p _ i x y 2 g t ԿȿĎ h- h- 5hyH h h b h b h 5h b h#m 5h b hyH 5 ht7z 5 h- 5h- ht7z h6b&