The youth Digital Skills Indicator: Report on the conceptualisation and development of the ySKILLS digital skills measure

This report presents the youth Digital Skills Indicator (yDSI), a unique, extensively crossnationally validated measurement tool with 31 items, distributed over digital skills and digital knowledge questions, that can be used for large-scale population research. The yDSI is the only measurement tool for youth digital skills that has been tested using the full range of validation practices. Over a period of six months, consultation with experts (face validity), cognitive interviews (content validity), pilot surveys (construct validity) and performance tests (criterion validity) with young people were conducted in a wide range of European countries.

A review of the literature led to a framework identifying four dimensions that constitute digital skills: (1) technical and operational skills; (2) information navigation and processing skills; (3) communication and interaction skills; and (4) content creation and production skills. Across all four dimensions a distinction should be made between being able to use the functionalities of information and communication technologies (ICTs) (functional aspects) and understanding why ICTs are designed and content is produced in certain ways and being able to use that knowledge in managing interactions in and with digital spaces (critical aspects).

Existing publications that report on survey instruments to measure digital skills, tend to cover technical and operational and information navigation and processing skills more than they do communication and interaction and content creation and production skills. Furthermore, functional aspects are more commonly measured than critical aspects of skills. Many studies that present survey items for the measurement of digital skills fall foul of seven “sins”. These studies (1) have basic survey item design flaws; (2) are solely PC-based; (3) are too vague or general; (4) measure outcomes instead of skills; (5) measure use instead of skills; (6) measure attitudes instead of skills; and (7) measure confidence instead of skills. Seven best practices for digital skills survey design are proposed to prevent the seven problematic practices from occurring: (1) ask participants “Can you do?” or “Do you know how to do?” (skill) rather than “Have you done?” or “Do you do?” (use); (2) avoid device-, app- or activity-specific items; (3) include (functional) digital skills and (critical) digital knowledge items; (4) at least half of the digital knowledge items should involve statements that are untrue; (5) items should ask “Do you know how to do?” (skill) rather than “How good are you at?” or “How do you rate yourself on?” (confidence); (6) items should use truth claims and emphasise the here and now to make the person evaluate their actual personal skills; and (7) answer options should be scale-based and include an option encouraging people to admit to a lack of understanding to avoid social desirability bias.

Cognitive interviews and performance tests showed that many young people did not master a range of skills, including critical information navigation and processing skills. Moreover, these were the hardest to measure cross-nationally. Knowledge around how content was created and produced was also lacking.

Analyses of skewness and kurtosis, confirmatory factor analysis, difficulty estimation and equivalence testing established that the final short version of the yDSI has overall high construct, convergent and discriminant validity. This means that the hypothesised four skills dimensions are clearly present in the yDSI, and that items measure variety within each dimension.