SoundLines: test protocol
SoundLines: development of an educational game accessible to children with visual impairments
Identify the most effective methods to suggest to a person with visual impairments how to connect two elements on a screen following a line. For this purpose, within a game for mobile devices structured in levels, different combinations of lines, interaction modes and sound cues (sonifications) are used.
Participant: Person of any age with visual impairment, who does not have other forms of disability.
Legal custodian: If the participant is a minor.
Supervisor: Experimenter of the research project.
Assistant: In charge of collaborating on the research project, support the supervisor.
The supervisor thanks the participant for the collaboration and explains the purpose of the experimentation: "The aim of this project is to understand which are the most effective and intuitive methods to help a person with visual disabilities to connect two elements on the screen of a mobile device following a line ”, Then the experimenter asks the following questions to the participant:
"How old are you?" Open answer.
"What kind of visual impairment do you have?" Open answer.
"Specifically what level of visual impairment do you have?": Open answer.
"At what age was the onset of your visual impairment?" Open answer.
"Do you use mobile devices?" Answer: yes, no. In case of positive answer:
"Which ones?": Open answer.
"Since when?": Open answer.
"Do you use assistive technologies for accessing mobile devices?" Answer: yes, no. In case of positive answer:
"Which ones?" Open answer.
"Since when?": Open answer.
"Have you ever explored tactile graphics?" Answer: yes, no. In case of positive answer:
"How do you explore the drawing to identify the elements present and understand their position?" Open answer.
"Have you ever explored a touch screen?" Answer: yes, no. In case of positive answer:
"How do you explore the screen to identify the elements present and understand their position?" Open answer.
If the two previous answers were affirmative, "Did you use the same techniques to explore a tactile design and a touch screen?" Answer: yes, no. If not,
"What are the differences between the techniques used?" Open answer.
"What features of a tactile design prevent you from using the same technique used for a touch screen?" Open answer.
"What features of a touch screen prevent you from using the same technique used for a tactile design?" Open answer.
“Have you ever used technologies or applications where sound is used to signal the position of certain elements or the direction to follow? Answer: yes, no. In case of positive answer:
"Do you think they are useful"? Answer: on a scale of 1 to 7
"Do you think the instructions provided are easy to understand?" Answer: on a scale of 1 to 7
"Do you think they are pleasant to use?" Answer: on a scale of 1 to 7
The supervisor reads the following instructions to the participant, to explain the functioning of the game and the accessibility tools used by the application: “The SoundLines application is a game divided into levels whose purpose is to identify a cat and a kitten on the screen and connect them. To locate the two cats you need to drag one or two fingers on the screen. To connect them you have to drag the kitten towards the cat following a line, using one or two fingers. When you are on the line a sound helps to understand if you are following the right path based on the position of your finger. If the two cats are connected the level ends. Before the start of each level, instructions are read and indications are provided on the number of fingers to be used and the type of sound which will be played. Each step of the game is also accompanied by instructions read by the application. If the movement of your fingers starts from one of the edges of the screen, system screens may open: in this case, do not hesitate to let us know so that we can intervene."
The supervisor and the assistant set up a mobile device necessary for the test. In particular, an iPhone X is used on which the SoundLines application has been installed and the VoiceOver voice assistant is enabled.
The supervisor places the device in landscape mode and opens the SoundLines application, then the supervisor hands the device over to the participant.
When opening the application, VoiceOver reads the contents of the home screen: "SoundLines. Help a kitten find its mother cat: you just need to connect them with a line! Ready?". The participant listens to the instructions and presses the "Start the game" button to continue.
A level is loaded. The following properties are associated with each level:
Line position (3): can be horizontal, vertical or diagonal. The two icons are located at the ends of the line.
Interaction mode (2): can be single-touch or multi-touch. In the first case, the participant must use a single finger to locate the elements and connect them. In the second case, the participant must use one finger to locate the cat and another to locate the kitten in the exploration phase, while connecting them, the participant must keep one finger on the cat and the other on the kitten, moving it towards the cat.
Sonification mode (4): a sound is played which varies according to the position of the finger on the line and whose volume, frequency of reproduction or timbre is changed, or no sound indication is given.
A level description screen opens. The application reproduces two short sounds, which suggest to the participant how to interact and sonify. The possible sound combinations (earcons) are:
Single-touch interaction - Volume modulation sonification
Single-touch interaction - Period modulation sonification
Single-touch interaction - Pitch modulation sonification
Multi-touch interaction - Volume modulation sonification
Multi-touch interaction - Period modulation sonification
Multi-touch interaction - Pitch modulation sonification
Subsequently, VoiceOver reads the contents of the screen: “The line is [horizontal / vertical / diagonal]. Use [one finger / two fingers] to find cat and kitten and to connect them. The sound will help you by varying its [volume / frequency / pitch] to let you know if you are following the right path or going out of the line. Press play to continue ". The participant listens to the instructions and presses the "Play" button to continue.
A game screen opens, with the cat icon displayed. The exploration phase begins, in which the participant must locate the icon by dragging a finger on the screen. VoiceOver says: "Use one finger to find the cat". If the participant raises their finger, VoiceOver says "Find the cat".
Once the cat is found, a kitten icon appears. The participant must locate the icon on the screen by dragging their finger. VoiceOver says: "You found the cat! Find the kitten "or" You found the cat! Keep one finger on the cat and use another finger to find the kitten ". If the participant raises their finger, VoiceOver says "Find the kitten" or "Keep one finger on the cat and use another finger to find the kitten".
A meow is played when the participant touches the cat or kitten icon.
Once both elements are found, a line appears and the connection phase begins, in which the participant must connect the two cats following the line to conclude the level. VoiceOver says "You found the kitten! Drag it towards the cat following the line, or "You found the kitten! Keep one finger on the cat and use the other to drag the kitten towards the cat following the line." When the participant places their finger inside the line, a continuous sound is played which suggests the right position inside the line and indicates if you are moving towards the edges of the line. If the participant leaves the line, VoiceOver says "You are out of the line. Retry!", If the participant raises their finger, VoiceOver says "Finger lifted. Find the kitten again and retry! " or "Finger lifted. Remember to place one finger on the cat and another on the kitten and retry! ". In both cases an error sound is played.
When the participant connects the two cats, the level ends, a meow is played and VoiceOver says “Great! Level completed".
At the end of the level a new description screen opens or, if the participant has completed all the levels, a conclusive screen, in which VoiceOver reads “Congratulations! You completed all the levels! Now kitten and cat can play together".
The supervisor thanks the participant for taking the test, then asks the following questions:
"On a scale of 1 to 5 how much ..."
"...did you find the application simple to use?"
"...did you find the instructions provided by the application clear?"
"...do you think most people can learn to use the application easily?"
"...do you think a child can learn to learn how to use the application?"
"Among the three different levels which ..."
"...one did you prefer?" Answer: horizontal line, vertical line, diagonal line.
"...one did you find easier to complete?" Answer: horizontal line, vertical line, diagonal line.
"Between the interaction modalities which ..."
"...one did you prefer?" Answer: single-touch, multi-touch.
"...one did you find easier to perform?" Answer: single-touch, multi-touch.
"...one did you find easier to find the destination?" Answer: single-touch, multi-touch.
"Among the modes of sonification which ..."
"...one did you prefer"? Answer: volume, frequency, pitch, no sonification.
"...one did you find easier to understand?" Answer: volume, frequency, pitch, no sonification.
"...one did you find more pleasant?" Answer: volume, frequency, pitch, no sonification.
Full results can be downloaded as a .csv file: here