Eye-tracking Project at Trinity Fields School 2014-2015.


The project:

New eye-tracking software allows us to easily record and playback where pupils are looking at on a screen. We are using Tobii Gaze Viewer which is around £250 for a year licence for one PC which isn't too bad considering what you can do with it.

This means that essentially we can see what our pupils are seeing and the importance of this should be obvious- it gives us an unique insight into the pupil's cognitive abilities. We can view their preferences and what gets their attention. We can watch how they read, watch how they watch videos and watch as they think through and complete tasks.

At Trinity Fields School we have put together a pilot project for 2014-15 with two general aims- to find out more about the pupils cognitive levels and also their receptive language level. The project is being completed in-house with a lead teacher, a project teacher and some extra staff time funded through our Pupil Deprivation Grant from the Welsh Government (PDG).

The project is split into two small groups of four pupils.

1: Project Baseline Results for PMLD group (Sept-Oct 2014):

4 pupils operating at P2-4. Aged 3-8. This group we are showing a variety of photographs and videos. These are initially of our choosing but will branch out in the second part as we refine from the baseline evidence.

Full baseline reports for the four pupils can be found in the links below.
Pupil A
Pupil B
Pupil C
Pupil D

2: Project Baseline Results for ASD group (2014-2015)

4 mainly non-verbal pupils with ASD at a P5-P8 level. Aged 10-13. This group are also looking at photographs and videos. We are also reading through books with them and analysing what they are looking at when they have free time on the computer.

Pupil A
Pupil B
Pupil C

Eyetracking: Some General Information

Two methods of analysis are being used via the Tobii Gaze Viewer software- the heat map and the gaze plot. This is the difference between the two.

Heat Maps
This records where the pupil has looked and for how long- red is the most, yellow the middle and green the least. Used for analysing how pupils view still images - useful for a general overview of screen attention.

Gaze Plots
This is much more useful as it records using a sequence of lines and numbers where the pupil has looked, for how long and where they looked next. Used for videos, moving activities and sequential tasks, the bigger the number the longer the gaze at that point.
gaze plots.png

What can eyetracking tell us about our pupils?

These videos will give an example of the variety of great evidence you can get from this eyetracking software.

These videos are from a 7 year old with Rett Syndrome. She is non-verbal, has extremely limited physical control and has been assessed at a P3ii level.

Video 1:
She is watching a Frozen song. The gaze plot starts after 14 seconds (she is quite distractable!).

Key points:

All the way through she is looking straight at the girls' faces- there's no doubt about that.
21 seconds in: You can see her quickly tracking the girl walking up the stairs (gaze plots 10-15) - in fact she then seems to anticipate where the girl is going by looking ahead of her to the top of the stairs (gaze plot 16 & 17) before the scene cuts.
26 seconds in: You can also see, as the camera pans away from the main girl that she looks across to the right side of the screen before the other girl appears in shot- this shows that she is anticipating someone else appearing there because the main girl is moving out of the main shot (gaze plot 24-27).
After gaze plot 33 she gets distracted and looks away as someone walks in the room.

We know from observation that this girl is a people watcher in class- now we have evidence of that and much more.
We now have evidence to show that not only can she can track someone walking but anticipate where they are going by following the angle as they are walking up a flight of stairs- she can even quickly calculate and predict where that person will end up when she gets to the top. We also have evidence that she has learnt and can anticipate where someone out of shot will appear on a screen using quite discrete visual clues.

Video 2:

A video of a dog walking along a wall.

Key Points:

She recognises quickly the main subject of the video- she scans around the dog's body very quickly while it is moving (gaze plots 2-16) and also glances at some of the things in the background.
Then - perhaps she doesn't like dogs- she looks away for the rest of the video after 10 seconds (gaze plot 20)- completely uninterested in it (it takes a while for the old gaze plots to fade).
So from this we know she is using her visual skills effectively to quickly identify what is on the screen (a dog) and then looks away uninterested. This shows quite acute visual skills as the video isn't a cartoon- it's a greyish moving dog against a greyish background. We can speculate that she looks away not because she doesn't know what is happening in the video and doesn't identify it as a dog- we have seen her look directly at and around the main subject- but because she has no interest in it- so perhaps she is not a dog fan!

Still Pictures:

In keeping with the above when shown a series of still images she looks at the faces- note that the tracking is a little out and I think she is actually looking slightly to the left of where the gaze points are recording.
Sometimes she looks away- she misses Thomas the Tank for example- but remember she has Rett Syndrome and she has difficultly with fine and gross motor movements.

Key Points:

20 seconds in- even in more complex photographs she looks quickly from the face of the girl to the boy's face (gaze plot 55-58)
27 seconds in- with the crying girls face she looks from one eye to the other rapidly (gaze plot 76-80).
34 seconds in- in a very visually messy photograph she looks from the lead boys face to the girl wearing purple (gaze plot 95-97)
38 seconds in- when the bright shiny smiling face comes on she looks right at it's eyes (gaze plot 120-126).

This confirms academic work already done on pupils with Rett Syndrome.

Example of reading a book:

This is the same pupil reading through a book with the teacher.

Note how she is effectively looking at all the characters in the story, scanning all their faces and even spotting mums face in the window (plot 133). She does not look at the text indicating this has no meaning for her yet. There are also indications that her receptive language skills are good- about 2-3 seconds after the following questions she fleetingly looks at the subjects of those questions:

Can you see the little dog? Plot 52
Where's the big dog? Plots 82-86 with a vocalisation in response (this pupil vocalises when she is engaged and excited!)
Where's Floppy the dog? Plot 134

Now she does miss some other questions and her responses to the ones above are fleeting and time delayed, but the small time delay does fit in with what we have observed with her before- and more evidence over time will confirm the levels of her receptive understanding- with more simpler choices offered.

Choice Making:

A video showing how this same pupil is now able to choose independently with Look to Learn from Smartbox AT.

As you can see she is now independently choosing from a menu of 5 options, then from 8 options (after scanning them) and then she presses the Start button herself before engaging with the activity. For a P3 pupil (she will be re-assessed in May and has just achieved most of her remaining Routes for Learning targets with eyegaze) this is remarkable.

Mr Tumble!

This video is a gaze plot of another pupil- a non-verbal 4 year old with SLD working at a P3 level- watching Mr Tumble singing his Hello Song:

Key Points:

Gaze plots 1-6 are the teacher switching the sound on the video.
Gaze plots 15-21 are the pupil looking straight at Mr Tumble's face.
Gaze plots 23-27 show the pupil tracking Mr Tumble as he moves into shot (5 individual tracking gaze plots in two seconds show how sharp his responses are).
Gaze plots 27-28 show him focusing on Mr Tumble's face as he sings. The pupil then looks at the face of the teacher working with him.
Gaze Plots 29-32 show him returning to Mr Tumble- then he looks back at the teacher again.

So again- it's good to know that this pupil is looking at Mr Tumble's face- he's not distracted by the blue border or the stars or Mr Tumble's clothes- and he's not looking at the signing hands either. As we know the pupil and were present at the session we can also say that he was looking off screen at the teacher because he was expecting her to sing along with the song!

For a pupil assessed at P3 only last month this is important information to discover.

Checking understanding of a task:

This video shows an 8 year boy with quadriplegic cerebral palsy (P3) and how he is effectively tracking falling snowflakes (well it is nearly Christmas!)

He tracks individual flakes (plots 44-54). Around plot 70 you can see how he is effectively sweeping to hit the falling snowflakes- he is definitely not randomly doing this and so it shows an understanding of the task and that he is using his own methodology to make the effects happen. Plots 104/105 are when he notices that the snowman is growing- then he goes off to hit some more! Again at plot 139 he notices the snowman's new hat and clearly acknowledges the star reward at plot 178.

As eyegaze works so well we need to be aware that pupils are understanding the tasks that they are doing and not just randomly swiping the screen attracted by the graphics or the music (See Pupil C for evidence of this)- I think this video demonstrates an understanding of the demands of this task and an appropriate response to the rewards.

Take a look at the evidence reports for more detailed evidence from individual pupil studies.