Applications created with and for users - about the UX approach to tools based on the example of the NIWUS application to support describing qualifications and assigning the level of the national  framework.

New applications - a new approach to user needs

Since the beginning of the development of the Integrated Qualifications Register (IQR), the perspective of its users has been essential. The information included in the public register is supposed to be not only reliable, but also useful to the public. A similar premise guides the tools under development - the applications to accompany the IQR. In order to achieve this goal, the Educational Research Institute (IBE) in recent months has adopted the principle of strong involvement of potential users of its IT tools at all stages of product development: from researching the needs and expectations of future users by their participation in the development of tool concepts and mock-ups to testing prototypes, verifying subsequent versions of the tool, providing feedback and reusing the tool. How, specifically, have these processes been organized for tools supporting the development of the Integrated Qualifications System (IQS)? What principles guide the inclusion of the general public in the work? Does this approach produce the expected results?

Cooperation with experts vs. user needs

In developing the IT tools that will support the Integrated Qualifications Register's operations, experts in the field in which the IT tool is being developed were invited to collaborate. It was important for these experts to have at least the general idea of the Integrated Qualification System (if they did not have such knowledge, the first meeting was always focused on introducing the basic elements of the IQS).

AIRA

In the case of the AIRA application, which is intended to become a tool for automatic support of HR departments and companies, work on the concept and solutions took place among programmers, graphic designers, experts in the area of employee selection and recruitment, and, at the same time, specialists in the implementation and promotion of the IQS. During the workshops conducted using the Design Thinking method, thanks to their knowledge and experience, the usability of the solutions for potential application users was verified on an ongoing basis, as well as the use of data from the IQR.

A good knowledge of the qualification system allowed for a creative approach to the development of the application functionalities while constantly aligning the solutions to their integration with the IQR.

A group of consultants was also invited to a two-stage qualitative study of the AIRA tool prototype. The IDI (in-depth interview) method was chosen for the study: in the first stage, the interviews focused on the recruitment and employee selection processes (difficulties, challenges, needs to be addressed by AIRA). In the second stage of the study, interviewees tested a clickable mockup of the AIRA application against the needs and expectations identified in the first stage.

Chatbot

Another application under development with the participation of the invited specialists is a virtual register assistant (chatbot). In this case, the so-called "expert panel" was established - a group of career counsellors (11 people) working with different clients (unemployed people, people who want to retrain, returnees from abroad, employers, high school students broken down by type of school, parents, etc). The team engaged at the conceptual stage was then asked to test the virtual assistant among other career counsellors and their clients. The group accompanied the chatbot development team at key points in the process, i.e. while creating solutions for career counsellors, selecting data sources, identifying risks, and testing the initial version of the chatbot.

NIWUS

The last IT tool, in the development of which special focus was placed on the needs of the users, is NIWUS - a tool for intelligent support of system stakeholders - a web application that supports the description of qualifications and the assignment of the PQF (Polish Qualifications Framework) levels. The development of the tool's concept was guided by the UCD (User Centred Design) methodology.  The work began with a user context analysis - a phase involving gathering information about the tool's potential users, their goals, technical and organisational environment. It was agreed that NIWUS would be targeted at

the representatives of the industry involved in describing qualifications, the representatives of the Educational Research Institute (IBE) providing methodological support for describing qualifications, the employees of ministries and the experts they appointed for assigning the PQF levels to qualifications.

At this stage, a survey of the needs and expectations of the application's users was conducted during which answers were attempted to the questions of what kind of automatic/semi-automatic support for their work is expected by those describing qualifications and assigning the PQF levels, what functionalities would be most valuable to them, what would make their work easier and what would only complicate it unnecessarily, what kind of AI-based microservices could be particularly valuable.

An additional step in the development of the application was the expert evaluation of foreign solutions supporting the description of qualifications and the assignment of national qualifications framework levels. The authors of the expert report covered 7 countries in depth; in addition, they conducted a survey in 10 countries. As a result, 3 countries/regions were identified whose work could inspire the development of the NIWUS tool: Flemish region of Belgium (with the AHOVOKS institution), France (Cereq) and South Africa (SAQA). The expert report also confirmed that the solutions adopted in the tool after the study, such as linking of similar qualifications, syntactic and semantic analyses of learning outcomes, are valuable also for users of foreign registers.

The design phase of the NIWUS tool was focused on weekly team meetings provisionally called "NIWUS mock-up development sessions," during which information structure, sketches and mockups were developed. Their final result was a mockup of the application, showing the different views of the interface.

User-friendly tools, that is?

Most of the experts working with the Educational Research Institute (IBE) on tool development emphasised the need to create applications that are intuitive and user-friendly.

The article presents several usability and visual solutions that were considered during the design of the NIWUS application to meet this expectation. The selection of solutions was guided by a set of principles - 10 heuristics formulated by Jakob Nielsen:

1. Show system status. Attention was paid to ensure that in all views where the user is involved, for example in the development of an application for the inclusion of a qualification, it is clear where he or she is; what has already been done and what is still to be done; clear and transparent navigation based on form cards is used.
2. Match the system and the real world. The principle is that the content and the descriptions used in the application should be clear to users - for example, due to the highly elaborate form of the application for inclusion of a qualification, it was decided to redesign the form so that its fields are grouped into sections arranged according to a logical sequence. Simplified names were developed for some form fields. Instructions for filling in the fields appear in tooltips.

Mock-up showing the screen view of the NIWUS application (in the module supporting filling out of the application form for inclusion of a qualification in the Integrated Qualifications System) with the Notepad tool pulled out and active tooltip

 

3. Give the user full control. Users should be able to manage their activities. NIWUS users are free to navigate through the application and take advantage of system-generated prompts. The order of filling in the sections and fields of the form is arbitrary (with the exception of the fields the filling of which leads to the prompt appearing in another field). For better organisation of one's own work, the user can switch between sections with information on which section he/she has not yet filled in completely. The user can explore the application without registering.
4. Stick to standards and maintain consistency. In designing the application, current market standards and users' habits of how the tools work were respected. Attention was paid to the visual consistency of the elements within the application.
5. Prevent errors. The application is designed in such a way that the user is informed about the effects of his/her actions, including the incorrect ones, which may cause changes in the way the application works. Some of the fields contain within them sample wording to help the user fill out a particular section of the application. They are placed there by default - the user can use them, can delete them or modify them at will.
6. Let the user choose instead of forcing them to remember. Don't make me think. The application is designed so that the user enters specific data only once. Once entered, the data is automatically processed and pulled to where it will be used in the further process. The user also has continuous access to auxiliary tools:

• notebook, where he/she can write any content and create his/her own bookmarks;
• toolbox, which provides hints and tips for the currently active section.

In addition, 4 microservices are planned to be implemented to support the description of a qualification. Three of them will be based on machine learning models:

• suggesting the PQF level,
• suggesting the type of validation methods for learning outcomes,
• showing similar qualifications on the basis of a full description of qualifications,
• searching for similar qualifications in the qualifications.gov.pl/k register based on a given phrase

 

Mock-up showing active Toolbox with prompts for the Learning Outcome field in the NIWUS application (in the module supporting filling out of the application for inclusion of a qualification in the Integrated Qualifications System)

 

5. Ensure flexibility and efficiency. Users of the application work in different ways, have their own habits, pace of work. The application allows within its environment to customise the interface so that anyone can work as efficiently as possible. It is possible to stop work at any time without losing the entered data.
6. Take care of aesthetics and moderation. When designing the application, care was taken to ensure that the solutions used were visually appealing and optimal in terms of the amount of information displayed on the screen at any given time.
7. Ensure effective error handling. When an error occurs in the application, the user is told what he or she needs to do to report it and what the next steps are.
8. Provide support and documentation. In developing the NIWUS application, a strong emphasis was placed on user onboarding: a process that facilitates learning the application during the first interaction with the product. It was decided that a short instructional video should be included on the product page and the prompts should be displayed as tooltips describing the behaviour of specific functionalities guiding the user to a sequence of actions supported by the application.

Mockup showing the selection and matching of descriptors from the PQF that best correspond to key learning outcomes. In this partially automatic way, a concordance table and a recommendation for assigning a PQF level to a qualification are produced.

Good applications are those created with users and for users

The resulting model of cooperation with people who are familiar with the idea of the IQS and are experts in the field brought many benefits during the development of the application, in particular, the following were appreciated:

Critical and creative evaluation of experts - experts were able to evaluate the concept and implementation of the products on an ongoing basis, and the ideas they generated concerning the functionalities corresponded to the needs of the target audience;

Verification and control - thanks to the constant "access" to experts, it was possible to find answers to emerging doubts in real time, but also to verify emerging solutions;

In-depth needs analysis - the designed research allowed for an in-depth analysis of the issues being tested (surveys of audience needs and expectations, mock-up tests);

Assistance in the interpretation of the research results - knowledge of the field and knowledge of the needs of the target audience of the application allows one to more accurately interpret the obtained research results;

Planning promotional activities and reaching out to users - knowledge of the specifics of the industry and knowledge of the IQR's dissemination strategy allows for better planning of the product's implementation; also, not infrequently, experts collaborating on a given application become its first users and promoters.

Although the involvement of experts and the creation of IT tools in the user-focused methodology may turn out to be time-consuming and requiring effort, it seems to be the right way and ensures not only usability, but also high quality of the resulting applications.

* The object of procurement is developed as part of the project “Operating and developing the Integrated Qualifications Register (stage 2)” implemented by the Educational Research Institute. The Institute is a research establishment that conducts interdisciplinary scientific research on the functioning and effectiveness of the education system in Poland.

 

Authors:

Małgorzata Barlik-Bokowy - front-end developer with interest in UI/UX design, professionally related to the fields of design and technology for over fifteen years, currently involved in developing AI tools and applications built around Integrated Qualifications System

Emilia Danowska-Florczyk - PhD, experienced qualifications systems expert and policy analyst, researcher; with 10 years of experience in the field of LLL and qualifications systems, especially in the field of describing qualifications and developing solutions for awarding bodies and validation institutions; nowadays participating in developing tools and applications based on natural language processing

Magdalena Kamieniecka - psychologist, trainer, coach. Focuses on the subject of occupational guidance in Poland and around the world, her professional interests revolve around the subject of decision-making processes related to the formation of educational and professional paths. Currently at the IBE, she is part of the team designing artificial intelligence applications for the stakeholders of the IQS under the project "Operating and Developing the Integrated Qualifications Register."