We consider the design of proper scoring rules, equivalently proper losses, when the goal is to elicit some function, known as a property, of the underlying distribution. We provide a full characterization of the class of proper scoring rules when the property is linear as a function of the input distribution. A key conclusion is that any such scoring rule can be written in the form of a Bregma...

The purpose of this paper is to investigate the properties of partitioning scoring rules in the bipartite ranking setup. We focus on ranking rules based on scoring functions. General sufficient conditions for the AUC consistency of scoring functions that are constant on cells of a partition of the feature space are provided. Rate bounds are obtained for cubic histogram scoring rules under mild ...

What are the natural loss functions or fitting criteria for binary class probability estimation? This question has a simple answer: so-called “proper scoring rules”, that is, functions that score probability estimates in view of data in a Fisher-consistent manner. Proper scoring rules comprise most loss functions currently in use: log-loss, squared error loss, boosting loss, and as limiting cas...

in assessing foreign language writing, holistic and analytic scoring can be used to measure a variety of discourse and linguistic features. this study aimed to investigate the possible significant effect of analyt- ic and holistic assessments on improving writing skill among iranian efl learners. for this purpose, two groups of intermediate efl learners, after being homogenized, were divided in...

The effects of solvation and entropy play a critical role in determining the binding free energy in protein-ligand interactions. Despite the good balance between speed and accuracy, no current knowledge-based scoring functions account for the effects of solvation and configurational entropy explicitly due to the difficulty in deriving the corresponding pair potentials and the resulting double c...

Œe BM25 ranking function is one of the most well known query relevance document scoring functions and many variations of it are proposed. Œe BM25F function is one of its adaptations designed formodeling documentswithmultiple fields. Œe Expanded Span method extends a BM25-like function by taking into considerations of the proximity between term occurrences. In this note, we combine these two var...