The data is human before it is numerical
The AutomaViC study contains coded interactions from nonspeaking autistic participants working through different typing-training activities.
There are prompts from the system, prompts from communication partners, participant interactions, and the sequence connecting them.
On paper, that sounds like a dataset. In practice, it is a record of people interacting with each other. Before thinking about models, I first have to decide what those interactions mean in a form that can actually be analysed.
A prompt is not just a sentence
One of the things I am interested in is communication-partner prompting.
The same words can mean different things depending on what the participant was doing, what happened immediately before, and what the partner seems to be responding to.
That makes the problem more interesting than simply embedding prompt text and clustering it. The surrounding interaction state matters too.
Where NLP fits
The project is still ongoing, so I do not have a final model or result to present.
Right now I am working through how to represent prompt content, participant state, and sequence information in a way that is useful for analysis.
NLP is part of that, but I do not want to use it just because the data contains text. The representation has to help answer the actual research question.
What I am trying to avoid
The easy version of this project would be to throw embeddings at every prompt and call the clusters insight.
I am trying to be more careful than that.
Before modelling, I want to understand what the coded events actually represent, what information is lost when I simplify them, and what a useful comparison would look like.
The project has made me appreciate how much of research happens before the model is even chosen.
This note covers the project as described in my portfolio. The linked source and demo are the best places to look at the implementation.