Consistency As The Strength Of AI In Keywording

After teaching the AI, the subject indexing was done again by the machine and the results were evaluated. There were an average of 3.9 index terms given by a person per application. The machine found an average of 1.3 similar index terms and an average of 3.6 different index terms. In total, the artificial intelligence gave an average of 4.9 words per application, which is slightly more than the number of words given by people.

Another summary of the number of index terms was made, this time using limited data. Only 66% of the written applications were included in the calculation (17% from both extremes of the dispersion were filtered out). The number of index terms found by the machine increased to 5.7 words on average. Of these, an average of 1.7 words corresponded to terms chosen by people and 4.0 words were different.

The effect of teaching on the result was therefore clear. When the number of words found before the teaching was compared to those found after the teaching, the average effectiveness of the teaching on the index terms chosen by the artificial intelligence was 1.1 words. However, it is worth noting that even without training, the artificial intelligence found a bigger number of relevant keywords in the applications than a human had found.

Index terms open to interpretation

In connection with the training of the machine, it became apparent that a surprisingly large proportion of the words given by humans were imprecise in one way or another in relation to the description of the project. At the same time, the words given by the machine turned out to be of more uniform quality than the original ones, because the machine intuitively cannot add meanings that are not directly found in the description of the project.

The person knows the context around the project under consideration and knows how to combine things relevant to the description from outside the description. On the other hand, people classify things not only through the text, but also through their own worldview. This can lead to a situation where a project presenting cultural history and art might have women as its index term, although the desired target group would in reality be wider.

Imprecise terms or terms open to interpretation could have been chosen for many reasons. An imprecise index term may be based on information held by the project handler, which is not recorded in the project description. It’s also possible that the index term is based on a conscious or unconscious presupposition that the processor has about the project’s audience, implementation or result. Even if a person’s intuitive choices may be very apt and true, they can still be problematic when looking at the uniformity of the subject indexing.

“It seems to be essential for the uniformity of the subject indexing that there is a shared understanding of how broadly or narrowly the words are understood in this context.”
–Essi Helander, Linguistics Scientist, Headai

A particularly interesting discussion with the client was sparked by an example, where the scope of the word used as index term had not been precisely defined, which led to imprecise results. This got us wondering, does every person handling the project applications have the same point of view on, for example, equality? Is it clear what equality means as the Finnish Food Authority’s index term, or can everyone attach their own meanings into it? It seems to be essential for the uniformity of the subject indexing that there is a shared understanding of how broadly or narrowly the words are understood in this context.

In terms of uniformity, the support of artificial intelligence can be very significant, as it always selects index terms unambiguously and reproducibly based on the description, and does not know how to look for broader meanings outside the description. The machine does not offer far-fetched index terms, but chooses based on exactly what is mentioned in the application.