Institutional Planning, Assessment & Research 2010 Institutional Planning, Assessment & Research Assessment Review Committee Report HCAS – Support/Foundations George Bailey June 4, Institutional Planning, Assessment, and Research

Mentoring/Review Process Established four coordinators to work with reviewers. Coordinators met together numerous times as needed. 10 faculty reviewed 8 centers, one office, two support programs, and 25 Foundations assessment units. The full committee met once for training. Reviews were conducted individually with assistance from coordinators as needed Institutional Planning, Assessment & Research

ARCS were rigorously trained. Initial training was provided by IPAR staff Institutional Planning, Assessment & Research

Component Data DevelopingAcceptableProficient Outcome Means of Assessment Criteria for Success Results Actions Taken Follow-Up to Actions Taken Institutional Planning, Assessment & Research

Data Visualization Institutional Planning, Assessment & Research

Best Practices – Coordinators Receiving Feedback From ARCS Institutional Planning, Assessment & Research

OUTCOME STATEMENTS OCCASIONALLY WERE CHALLENGING TO CRITIQUE– TYPICAL ARC COMMENT ‘Its Greek to me!”: Example Student Learning Outcome One: ο ὐ δένα πώποτε τ ῶ ν πολιτ ῶ ν, ὦ ἄ νδρες Ἀ θηνα ῖ οι, ο ὔ τε γραφ ὴ ν γραψάμενος ο ὔ τ ᾽ ἐ ν ε ὐ θύναις λυπήσας, ἀ λλ ᾽ ὡ ς ἔ γωγε νομίζω μέτριον ἐ μαυτ ὸ ν πρ ὸ ς ἕ καστα τούτων παρεσχηκώς, ὁ ρ ῶ ν δ ὲ τήν τε πόλιν μεγάλα βλαπτομένην ὑ π ὸ Τιμάρχου τουτου ὶ δημηγορο ῦ ντος παρ ὰ το ὺ ς νόμους, κα ὶ α ὐ τ ὸ ς ἰ δί ᾳ συκοφαντούμενος （ὃ ν δ ὲ τρόπον, προϊόντος ἐ πιδείξω το ῦ λόγου ）ο ὐ δέναπώποτετ ῶ νπολιτ ῶ ν ὦ ἄ νδρες Ἀ θηνα ῖ οιο ὔ τεγραφ ὴ νγραψάμενοςο ὔ τ ᾽ ἐ νε ὐ θύναιςλυπήσας ἀ λλ ᾽ ὡ ς ἔ γωγενομίζωμέτριον ἐ μαυτ ὸ νπρ ὸ ς ἕ καστα τούτωνπαρεσχηκώς ὁ ρ ῶ νδ ὲτήντεπόλινμεγάλαβλαπτομένην ὑ π ὸΤιμάρχου τουτου ὶδημηγορο ῦ ντοςπαρ ὰτο ὺ ςνόμουςκα ὶα ὐ τ ὸ ς ἰ δί ᾳσυκοφαντούμενοςὃ ν δ ὲτρόπονπροϊόντος ἐ πιδείξωτο ῦλόγου Institutional Planning, Assessment & Research

Occasionally, ARCS founds “means of assessment” statements somewhat less than fully transparent: The General formula for Assessing Student Learning is as follows:-[ ( l / L) ^2] + [ ( v / C )^2 ] + [ ( a / A )^2 ] + ……… = One Measure of Improvement, where l is the ‘unit outcome’ that can be considered on a learning continuum and L is the maximum modulus that can be considered in that continuum, v is the rate of change of that outcome with respect to ‘demonstrated actions,’ namely, learning and C is the maximum modulus that can be considered in that continuum, a is the rate of change of learning with respect to ‘effort’ in that continuum and A is the maximum modulus fina/ly obtained by assessment. Generally, value l can be considered as an element of learning involving effort such that l = [( x ) ^2 + ( y ) ^2 + ( z ) ^2] ^ Institutional Planning, Assessment & Research This was addressed via one on one exchanges with UACS.

Exchanges generally were well received: Institutional Planning, Assessment & Research

In some cases, medieval criteria for success wanted restatement in postmodern terms: Institutional Planning, Assessment & Research

ARCS were able to get UACS to implement the necessary revisions, as illustrated below: An increase in student learning by one percentage point will prompt the unit to revise pedagogy or curriculum by more than one percent (specifically, by, the sum of the two coefficients obtained by the assessment method). Since the actual student learning success rate is (approximately) the nominal success rate minus deflation due to instructor variation, when inflation rises, the real success rate increases and student learning improves, and decreases otherwise Institutional Planning, Assessment & Research

The most challenging form taken by results reported is illustrated below: Results: Addressing this required creative thinking by ARCS: Institutional Planning, Assessment & Research

Raising the dead: Institutional Planning, Assessment & Research

Which finally brings us to the review of actions taken: Institutional Planning, Assessment & Research

All in all, the team is happy that the format is revised for ! Institutional Planning, Assessment & Research

Rubric and Review Process Feedback Most helpful to reviewers was direct discussion with a coordinator. Coordinators benefited from regularly meeting together to be clear on the advice to give individual reviewers. The biggest difficulty was overcoming different views regarding what was OK and what was not. After coordinators identified a concern and decided on a response, this was communicated to the reviewer. All team members worked creatively to assist individual UACs to understand and make necessary improvements Institutional Planning, Assessment & Research