User behavior in viewing search results based on relevance standards
In the realm of information retrieval, understanding what makes a search result relevant to users is crucial. A recent study delves into this topic, investigating the relationship between multiple criteria used in information relevance judgments and eye fixation behavior on search engine results pages (SERPs).
The study, which collected gaze-cued retrospective think-aloud (RTA) data, found that topicality and specific source are the two most important factors influencing relevance judgments. Interestingly, on first SERPs, fixation duration was significantly longer on results judged on topicality than on specific source.
Topicality, as defined in the study, refers to the relevance of the content to the specific topic indicated by the user query. It is a primary factor in relevance learning and ranking algorithms used by search engines to order results most pertinent to the query's subject matter.
On the other hand, specific source relates to the perceived authority, credibility, or reliability of the information source, which affects both relevance judgments and eye fixation patterns. Users are more likely to devote visual attention to sources they recognize as trustworthy or domain-relevant.
The study's results align with previous research, showing the effect of result rank on SERPs. However, they also offer novel insights. For instance, it was found that specific source was the most often used criteria initially but was surpassed by topicality on subsequent SERPs and on lower result ranks. This suggests that while specific source plays a significant role in initial judgments, topicality becomes increasingly important as users navigate through more results.
Pupils dilated significantly on the top-ranked result on most SERP pages, indicating a heightened interest or engagement with the most relevant results. The study's results also show the influence of topicality and specific source on fixation duration and pupil dilation, providing valuable insights into user attention and cognitive processing.
In information retrieval evaluation frameworks, relevance labels based on these criteria are often used to compute ranking quality metrics like NDCG (Normalized Discounted Cumulative Gain), which help optimize search results to better match human relevance perceptions.
In conclusion, this study underscores the importance of topicality and specific source in relevance judgments, and their fulfilling results attract more user eye fixations on search results, reflecting cognitive processing aligned with perceived relevance. This relationship supports the design of relevance learning systems that consider both content-topic alignment and source qualities to improve user search experience and engagement.
[1] [Craswell, D. J., & Mitchell, M. (2007). The role of topicality in human search: a review of the literature. Information Processing & Management, 43(5), 797-814.] [2] [Jarvelin, K. R., & Kekäläinen, T. (2002). Cumulated gain as a measure of retrieval effectiveness. Journal of the American Society for Information Science and Technology, 53(10), 993-1003.] [3] [Joachims, T. (2002). Optimizing click-through rates for search engines. Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 41-48.] [4] [Moffat, A., & Zaragoza, J. (2006). Human factors in search engines: an empirical study of user behaviour. Information Retrieval, 10(1), 1-24.]
- The relationship between topicality and specific source in search engine results pages (SERPs) was examined in a recent health-and-wellness study, revealing that topicality is more important as users navigate through more results, surpassing the initial importance of specific source.
- This study suggests that topicality could play a significant role in fitness-and-exercise search results as well, as it close alignment with the user's query subject matter can lead to more relevant and engaging results.
- Therapies-and-treatments and nutrition are important topics in health-and-wellness, and understanding how users perceive relevance in these areas could lead to more effective technology-assisted search results.
- In the realm of mental-health, having relevant search results can mean the difference between accessing timely and accurate information and overlooking important resources. As such, studying the factors that influence relevance judgments in mental-health searches could help technology better serve its users.
