Package com.gengoai.hermes.ml

Class CoNLLEvaluation

java.lang.Object

com.gengoai.hermes.ml.CoNLLEvaluation

All Implemented Interfaces:

Evaluation, SequenceLabelerEvaluation

public class CoNLLEvaluation
extends Object
implements SequenceLabelerEvaluation

Evaluation used in CoNLL for Named Entity Recognition. Defined as: "Precision is the percentage of named entities found by the learning system that are correct. Recall is the percentage of named entities present in the corpus that are found by the system. A named entity is correct only if it is an exact match of the corresponding entity in the data file"

Sang, Erik F. Tjong Kim and Fien De Meulder. “Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition.” ArXiv cs.CL/0306050 (2003): n. pag.

Author:

David B. Bracewell

Constructor Summary

Constructors

Constructor	Description
CoNLLEvaluation(String outputName)	Instantiates a new CoNLLEvaluation.

Method Summary

Modifier and Type	Method	Description
double	accuracy()	Calculates the accuracy of the system as defined as:
void	evaluate(@NonNull Model model, @NonNull DataSet dataset)
double	f1(String label)	Calculates the F1 measure of the system as defined as:
double	macroF1()	Calculates the macro F1-measure or the arithmetic mean of the per-class F1-scores.
double	macroPrecision()	Calculates the macro precision or the arithmetic mean of the per-class precision-scores.
double	macroRecall()	Calculates the macro recall or the arithmetic mean of the per-class recall-scores.
void	merge(@NonNull SequenceLabelerEvaluation evaluation)
double	microF1()	Calculates the micro F1-measure which is the F1-measure over the micro-precision and micro-recall
double	microPrecision()	Calculates the micro-precision of the system as defined as:
double	microRecall()	Calculates the micro-recall of the system as defined as:
void	output(@NonNull PrintStream printStream, boolean printConfusionMatrix)
double	precision(String label)	Calculates the precision of the system for a given label defined as:
double	recall(String label)	Calculates the recall of the system for a given label defined as:

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.gengoai.apollo.ml.evaluation.Evaluation

output, report, report

Methods inherited from interface com.gengoai.apollo.ml.evaluation.SequenceLabelerEvaluation

output, output

Constructor Detail

CoNLLEvaluation

public CoNLLEvaluation(String outputName)

Instantiates a new CoNLLEvaluation.

Parameters:

outputName - the output name of the model to evaluate

Method Detail

accuracy

public double accuracy()

Calculates the accuracy of the system as defined as:

 
    accuracy = correct / (correct + incorrect + missed)
 
 

Returns:

the accuracy of the model

evaluate

public void evaluate(@NonNull
                     @NonNull Model model,
                     @NonNull
                     @NonNull DataSet dataset)

Specified by:

evaluate in interface Evaluation

f1

public double f1(String label)

Calculates the F1 measure of the system as defined as:

 
    F1 = (2 * precision * recall) / (precision + recall)
 
 

Returns:

the F1 measure of the model

macroF1

public double macroF1()

Calculates the macro F1-measure or the arithmetic mean of the per-class F1-scores.

Returns:

the macro F1-measure

macroPrecision

public double macroPrecision()

Calculates the macro precision or the arithmetic mean of the per-class precision-scores.

Returns:

the macro precision

macroRecall

public double macroRecall()

Calculates the macro recall or the arithmetic mean of the per-class recall-scores.

Returns:

the macro recall

merge

public void merge(@NonNull
                  @NonNull SequenceLabelerEvaluation evaluation)

Specified by:

merge in interface SequenceLabelerEvaluation

microF1

public double microF1()

Calculates the micro F1-measure which is the F1-measure over the micro-precision and micro-recall

Returns:

the micro F1-measure

microPrecision

public double microPrecision()

Calculates the micro-precision of the system as defined as:

 
    micro-precision = sum(correct) / ( sum(correct) + sum(incorrect) )
 
 

Returns:

the micro-precision of the model

microRecall

public double microRecall()

Calculates the micro-recall of the system as defined as:

 
    micro-recall = sum(correct) / ( sum(correct) + sum(missed) )
 
 

Returns:

the micro-recall of the model

output

public void output(@NonNull
                   @NonNull PrintStream printStream,
                   boolean printConfusionMatrix)

Specified by:

output in interface SequenceLabelerEvaluation

precision

public double precision(String label)

Calculates the precision of the system for a given label defined as:

 
    precision = correct / ( correct + incorrect )
 
 

Returns:

the precision of the model

recall

public double recall(String label)

Calculates the recall of the system for a given label defined as:

 
    precision = correct / ( correct + missed )
 
 

Returns:

the recall of the model