The Annual Assessment Cycle
The Office of Institutional Research supplies the various units of the University of Wollongong in Dubai with extensive data entailing an assortment of direct and indirect measures regarding academic and administrative services and programs. The assessment cycle employed by the University is the following standardised method:
Plan: identify and define what we want to achieve, and develop an approach for achieving those objectives;
Act: develop and implement specific strategies and actions for achieving our objectives;
Review: identify outcomes and monitor how we are progressing towards achieving our objectives; and,
Improve: identify changes that need to be made so that we can better meet our objectives and, possibly, reconsider the appropriateness of our objectives.
The four steps of the quality cycle are integrated in the planning processes undertaken at the university-wide level, by the faculties and departments that comprise the institution, and by individual staff members. As such, assessment is not a process completed by one unit of the institution resulting in a report to be distributed and acted upon; instead, it is an ongoing interaction between the staff member and their superior, between a manager and an executive director, or the institution and its board of trustees.
“Closing the Loop” – whereby sets of data are analysed by the Office of Institutional Research are acted upon by the relevant parties at the University – is primarily accomplished through the distribution of reports by the Office to relevant stakeholders at UOWD. The formal process entails tabling survey or other analytical reports to a standing committee for official recognition via minutes, allocating responsibilities to committee members or their units, and documenting follow-up activities according to formal actioning. Less formally, findings from the Office of Institutional Research are also distributed directly to relevant manager and deans as full reports either by bound printout or .pdf versions via e-mail.
Measuring Institutional Performance
Academic quality, departmental performance, and progress towards the goals set forth in the University’s Strategic Plan are monitored using a system of integrated information collection and reporting arrangements. The main mechanisms used for this purpose can be broadly categorised as internal to UOWD and external. Measures can be further characterised as direct or indirect.
Internal Measurement – Direct Measures
Direct measures of institutional performance drawn from internal sources are generally based upon counts of student and staff activities and performance. The registration database contains a significant portion of this material, particularly the data which show the number of students enrolled in each program, class sizes, and numbers of new and continuing students for each semester by program. This same database allows for the analysis of retention and attrition rates, trends in weighted average marks, progression rates, and – potentially – direct measures of learning outcomes via course embedded assessments. Through these datasets and their analysis, the Office of Institutional Research employs a variety of performance indicators in the following ways:
- Count of new students: Statistical data on number of new students per program, unique student count per level and unique student count overall.
- Full Time Student Equivalence: Statistical data by program per semester.
- Percentage of International students: Data on number of students recruited from outside the UAE.
- Conversion rates: This data set includes Enquiry Yield (number of applications/number of enquires), Admissions Yield (number of offers/number of applications) and Matriculant Yield (number of newly enrolled students/number of offers).
- Student Retention: Statistical data of student cohorts, by program.
- Academic Progression: Trend data on student performance (WAMs) by student cohorts, by program.
- Graduation Rates: Statistical data by student cohorts, by program.
- Effectiveness of Language Preparatory Courses: Comparative statistics for student cohorts by program.
- Faculty Student Ratio: Number of full time equivalent (FTE) students divided by the FTE Faculty.
Additional internally derived direct measures can come from a diverse range of sources such as usage rates of co-curricular and extra-curricular resources; number of teaching staff, their qualifications, and teaching loads; research activities and publication counts by researchers; etc. As a set, these measures directly measure aspects of UOWD and are necessary components of comparative and trend analyses of performance. Many of these direct measures are included in the UOWD Factbook, an annual publication compiled by the Office of Institutional Research.
Internal Measurement – Indirect Measures
A second set of internally derived measures are the surveys conducted and analysed by the Office of Institutional Research. Demographic and other questions related to respondent characteristics are combined with evaluations of a variety of services, activities, and infrastructure as well as general opinions, preferences, and recommendations. The evaluations and surveys conducted by the Office of Institutional Research are:
- The Language Studies Centre Student Experience Survey – Conducted twice a year, this survey of non-credit language program students covers a variety of demographic, academic, and general experience topics covering university facilities and services.
- Graduate Exit Survey – Conducted twice a year, this survey concentrates on academic aspects of the students’ experiences at UOWD and features program-specific questions. Reports are released annually on a calendar year basis.
- Student Experience Survey – An annual survey of student opinions regarding their experiences at the University with a focus on co-curricular and extra-curricular services and facilities. Reports cover academic year periods.
- Incoming Student Survey – Taken by incoming students at time of initial enrolment. Topics include marketing, the admissions process, and the orientation session for new students. Reports are issued each semester.
- Subject/Instructor Evaluations – Every credit and non-credit course is evaluated by students every term that it is offered at OWD. These evaluations are an important element in monitoring the teaching and course delivery quality at the University.
- The Language Studies Centre New Student Survey – Conducted every term, this survey of non-credit language program students contains a range of demographic and marketing questions.
- Alumni Survey – Conducted every two years, the alumni survey asks past graduates to evaluate their UOWD academic program in the context of their subsequent work experience.
- Employer Survey – Conducted every two years, this survey seeks input from the employers of UOWD graduates on the levels of skill and performance of UOWD alumni and general questions about academic programs and their fit to the job market.
- Staff Survey – Conducted through the Australian-based Voice employment survey, this solicits opinions and measures satisfaction among the teaching, administrative, and support staff at UOWD.
- Student Library Survey – Every two years the student population is asked to evaluate the services, facilities, resources, and timings of the UOWD library.
- Faculty Library Survey – Every year, faculty members are asked to evaluate the library services and resources in reference to both the needs of their students and their own requirements as researchers.
Other surveys are also conducted to monitor CASPD workshops, ULT modules, and Professional Institute courses.
External Measurement – The Academic Quality Assurance Process
The University of Wollongong in Dubai has developed a formal review process with the University of Wollongong in Australia for monitoring academic quality. This external quality assurance process is designed to provide a systematic review for all programs and constituent subjects offered at UOWD and to encourage collaboration between staff members at the two universities on matters such as curriculum, syllabus, assessment tasks, and teaching methods.
Each subject offered at UOWD is reviewed once a year by an academic at UOW (Australia). The key elements of this QA process are (i) benchmarking of assessment tasks and student performance for each subject in each semester of offer, and (ii) an annual review of each subject, including a review of appropriateness of the subject learning outcomes especially in the context of the degree program(s) to which it contributes. Other elements reviewed include curricula content and assessment of learning outcomes. This process of benchmarking contributes to the verification and consistency of academic standards. An annual QA review of subject outlines will also be undertaken to ensure that learning outcomes, textbooks, and assessment tasks are appropriate in achieving the stated outcomes.
External Measurement – Benchmarking
The University of Wollongong in Dubai compares a variety of its key elements and performance indicators to those at other educational institutions both overseas and within the United Arab Emirates as part of its ongoing improvement process. These comparisons put the performance of UOWD and its constituent departments and faculties in a broader context and help University planners to focus on areas that are underperforming in a comparative sense. These activities are also the source of learning from other institutions and aid in the dissemination of best practices among institutions of higher education to the benefit of the region’s students.