TIRSA

A Tool for Institutional Research Self-Assessment

DPM Research is offering a standardized self-assessment tool to measure and assess the breadth and depth of higher education institutional research (IR or an equivalent office of a different name).  As an executive using this tool, you will be able to understand the development of current IR activities and identify concrete actions to improve and advance these projects to better serve your institution as a whole.

As a result, you will be in a better position to establish actionable priorities to advance and develop IR activities, identify investment gaps, and understand how your office, as a team, can work effectively and efficiently to promote a wide range of sophisticated products and services for your institution. 

What is TIRSA?

TIRSA was developed using a knowledge management framework. This framework unifies four dimensions of IR activity into one evaluation tool, refocusing the attention from IR reporting activities to identifying the competencies required to perform these duties. These competencies are the skills IR professionals must have to efficiently and effectively produce high-quality IR work. The advantages to studying IR using this process is that it gives institutions a multi-dimensional tool to understand the development and performance of key dimensions. 

These dimensions include:

  • Competencies required to carry out IR activities.
  • Resources that support these activities.
  • Organizational sectors of work.
  • Functions that use IR data.

About TIRSA diagram

For more information about each of these elements of this diagram, please consult the glossary, in the next tab.

Glossary

Definitions of functions

 
Institutional positioning

Refers to information about the relevance of the institution’s academic offer with respect to the needs of the country (both programs and institutional accreditation), to the status of demand, and to changes in supply and demand over time. It also includes information on the institution’s comparative status with peers, obtained through benchmarking and networking exercises, among others.

Internal operations

Refers to information used in the operational management of the institution, both administrative and academic, including management of applications, registration, and graduation activities. It also includes management of personnel and resources, and other activities that contribute to management control.

Quality assurance

Refers to information used for internal activities of quality management (processes of self-evaluation, evaluation of academic programs, evaluation of faculty), continuous improvement processes and external evaluations (accreditation that covers topics such as academic and institutional management), as well as any measurement of the impact of quality assurance processes.

Relations with groups of interest

Refers to information used for relations with the environment (government, accreditation agencies, media), with the production sector (private or public undertakings, employers), with other educational institutions, with alumni, as well as with internationalization activities. It also includes all activities involving interactions with the community, such as academic and cultural extensions, continuing education, and social service.

Strategic management and planning

Refers to information used for strategic management and planning such as scorecards, organizational goals and performance indicators.


Definitions of competencies

 

Analyzing information

It is defined as the ability to analyze data using quantitative or qualitative methodologies, which allows to describe a situation, to make comparisons or, at a more advanced level, to develop models (explanatory or predictive), of institutional phenomena, in order to contribute to the resolution of problems. This ability to analyze is usually related to team work, and the knowledge and experience of the participants.

Benchmarking

It consists in identifying ways to measure and compare the performance of different institutions using consistent comparison criteria. It is a distinctive competency given that it requires the compilation of external data not always pre-structured for comparison purposes. It starts with a basic level of simple awareness of the existence of this type of processes, through the implementation of existing studies, culminating in the development of new benchmarking initiatives. It can also refer to groups of institutions that wish to compare among themselves and excludes external rankings.

Communicating

It facilitates the use of the knowledge generated. At the most basic level, communication is reactive in providing basic information when it is requested. The competency evolves to a proactive level of distributing timely information on an ongoing basis. Communication skills includes the ability to identify the audience as well as the purpose of reports/studies to take them into account in the communication design. Eventually, it seeks to open new channels of communication, to reach different audiences and to generate greater impact.

Compiling information

It consists of the different skills needed to get the information necessary to prepare reports or studies. It includes the collection of data from surveys of different interest groups (ranging from a basic level where the IR professional keeps abreast of the existence of certain surveys, through the participation in ongoing surveys, and culminating in the elaboration and implementation of new instruments if necessary), the collection of data with other qualitative methods (focus groups, key informant interviews), or extracting data from administrative databases. Data from the host institutions or from other organizations may be compiled.

Designing reports

It is defined as the ability to create and produce reports from a basic level with static data on individual organizational areas, evolving to the production of reports with pre-established procedures, and culminating in the creation of reports on data that had not previously occurred. As IR advances, there is a progression to facilitate self-directed access to the data and to user-friendly designs and visualizations that facilitate the understanding of information.

Researching

Includes the development of academic and/or applied research skills. It seeks the development of innovative studies that generate new knowledge or generate innovative solutions to practical problems. It can refer both to the improvement and innovation of educational practices at the institution, as well as to academic studies presented at conferences or submitted to peer reviewed publications. The emphasis is not on repeating existing practices, but rather on innovating.


Definitions of organizational sectors

 

Academic sector

Includes information on undergraduate and graduate students, such as the income profile (e.g., type of school, socio-economic characteristics prior to entrance, demographic information), admission (e.g., admission average, academic career prior to admission, application and selection), adaptation programs (e.g., diagnostic and remedial), enrollment, attrition, academic achievement (e.g., attendance, qualifications) and obtaining the degree (e.g., graduation, time to graduate).

External relations

Includes all information about activities involving the community or through which the institution contributes to society, as well as all information pertaining to the analysis and understanding of its environment. It also includes information on relations between the institution and other organizations (partnerships and relations with the government, the accreditation agencies, or the media), linkages with the production sector (private or public undertakings, employers), international relations, relations with other institutions (other higher education or secondary schools), follow-up of alumni (job placement, surveys) as well as academic, cultural and sports extension activities, continuing education, and social services.

Financial management

Includes information on budgets and budgetary control, program costing, financial statements, income sources, accounting, medium and long-term financial assessments, projections of revenues and expenses, management of strategic projects, as well as other sources of financial information of the institution.

Human resources

Includes information about faculty (e.g., academic degree, training and development, career and professional activities, performance and its evaluation, academic rank or categorization, career path, work plans [i.e., hours devoted to teaching, research, extension and management], recruitment, wages and benefits) as well as about the staff, such as managers, professionals, administrators and technicians (e.g. career path, hiring, salaries). It also includes information on the satisfaction of staff (surveys).

Infrastructure and equipment

Includes information about the facilities (rooms, laboratories, workshops or other specialized learning rooms, green spaces, sports or cultural areas), availability, habitability and ease of access of the facilities, the library (collection, access, computers), the technological infrastructure (information systems that support the management of data, infrastructure for distance education or other practical activities, bandwidth) as well as information about the equipment to support teaching, research, or other functions (availability, quality).

Research

Includes both academic and applied research, as well as innovation and experimental development activities. Information on research projects, internal funding, public or private funding, research centres and research products is considered (e.g., publications, patents, enterprises or other relevant products that enable the dissemination of knowledge).

Student services

Includes information on scholarships (internal or external, distribution of funds), co- and extra-curricular activities (sports, social, cultural), academic support services (peer students, psychoeducational support), food services, other services to students (funds for student entrepreneurship, clubs and organizations) as well as student surveys (satisfaction, participation).

The self-assessment questionnaire measures the development of the competencies, highlighting which ones can be strengthened and indicating potential areas for growth and development. We have developed a scale that measures the level of competency advancement inspired by Knowledge Management principles where the following five levels are identified:

STAGE 1

INITIATE. At this stage, the compilation, production, analysis, and dissemination of information and knowledge occurs in an ad hoc manner. This results in a limited impact on organizational performance.


STAGE 2

DEVELOP. At this stage, the institution establishes the first actions of a knowledge management strategy that must be intimately linked to the overall strategic objectives of the organization. The processes and procedures that are implemented need to be documented in order to make them repeatable. This ability to replicate is the most important characteristic of this stage.


STAGE 3

STANDARDIZE. The practices and processes identified in the previous stage begin to be used more comprehensively. Standardization is important because it allows the organization to benefit from economies of scale, to learn from experience, to establish common performance measures, and to emphasize the development of competencies that can be adapted to different circumstances. At this stage, there is also a clear indication of teamwork and collaboration.


STAGE 4

OPTIMIZE. At this stage the organization leverages the processes and approaches already standardized and adapts them to all sectors of the organization. The emphasis is on the search for mechanisms to optimize the activities that have been made repeatable and standardized.


STAGE 5

INNOVATE. Because of the standardization and optimization from the previous stages, which results in effective and efficient work, resources can now be released to invest in innovation. The innovation relates to the acquisition of new information or to the analysis of existing information in new ways to answer questions that still have not been addressed or resolved.

Levels

This framework offers a definition for the level of development for each competency in the form of descriptive statements. These statements provide the basis of the questionnaire, the objective measurement tool used to determine the level of development attained. This questionnaire assigns a level of development to each competency in each organizational sector. As an example, statements describing the level of development for the competency Designing reports are included in the table below. This scale measures how competencies are performing, ranging from initiation to innovation. 

Levels

Whether your office is in the developing stage of creating standardized, repeatable practices, or you are looking to innovate and continuously improve your designs, TIRSA is the right tool establish the priorities for your office to reach its goals.

NEW COMPONENT: STEWARDSHIP

Your IR office undoubtedly interacts frequently with other services/areas (e.g. IT, finance, Provost office) to identify issues of interest, collaborate in their analysis, and make recommendations. This requires the ability to build and maintain relationships within and across organizations to foster the coordination and dialogue necessary to work through complex issues. The ability to work well in this environment while supporting the IR office to achieve the best for itself and the rest of the institution is good stewardship. You are invited to participate in the addition of a stewardship scale to the existing TIRSA framework.