On the accuracy of objective image and video quality models: New methodology for performance evaluation - Nantes Université Access content directly
Conference Papers Year : 2016

On the accuracy of objective image and video quality models: New methodology for performance evaluation

Abstract

There are several standard methods for evaluating the performance of models for objective quality assessment with respect to results of subjective tests. However, all of them suffer from one or more of the following drawbacks: They do not consider the uncertainty in the subjective scores, requiring the models to make certain decision where the correct behavior is not known. They are vulnerable to the quality range of the stimuli in the experiments. In order to compare the models, they require a mapping of predicted values to the subjective scores, thus not comparing the models exactly as they are used in the real scenarios. In this paper, new methodology for objective models performance evaluation is proposed. The method is based on determining the classification abilities of the models considering two scenarios inspired by the real applications. It does not suffer from the previously stated drawbacks and enables to easily evaluate the performance on the data from multiple subjective experiments. Moreover, techniques to determine statistical significance of the performance differences are suggested. The proposed framework is tested on several selected metrics and datasets, showing the ability to provide a complementary information about the models' behavior while being in parallel with other state-of-the-art methods.
Fichier principal
Vignette du fichier
Qomex2016_paper.pdf (696.9 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01395440 , version 1 (10-11-2016)

Identifiers

Cite

Lukáš Krasula, Karel Fliegel, Patrick Le Callet, Miloš Klíma. On the accuracy of objective image and video quality models: New methodology for performance evaluation. 8th International Conference on Quality of Multimedia Experience (QoMEX), Jun 2016, Lisbonne, Portugal. ⟨10.1109/QoMEX.2016.7498936⟩. ⟨hal-01395440⟩
457 View
1001 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More