Morpho Challenge -> Results

Click the column headers to sort the rows.

Results for English. The 1st column shows the name of the method or author and the year of evaluation. The 2nd column shows the type of the algorithm: semi-supervised (S), unsupervised (U), or unsupervised with supervised parameter tuning (P). The 3rd column shows wheter the algorithm returned a segmentation. The 4th column (#a/w) shows the average number of analyses per word, the 5th column (#m/w) the average number of morphemes per word, and the 6th column (#lex) the size of the morpheme lexicon. The rest of the columns give the results of the different evaluations.
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Allomorfessor 2008 U 1.00 1.62 180813 - - - - - 83.31 15.84 26.61 87.22 62.38 72.74 98.51 64.80 78.18 85.54 7.70 14.10 95.99 30.20 45.93 85.28 7.71 14.12 95.51 30.70 46.45
Allomorfessor 2009 U 1.00 2.59 23741 38.31 - - - - 68.98 56.82 62.31 79.01 76.48 77.72 89.20 79.89 84.28 64.57 44.22 52.45 73.79 58.34 65.14 63.85 42.31 50.85 73.05 55.99 63.38
Bernhard 1 2007 U x 1.00 2.61 55490 38.88 - 81.63 73.02 77.08 75.61 57.87 65.56 83.67 79.32 81.44 91.12 82.64 86.67 69.15 55.22 61.38 77.16 66.86 71.63 68.58 52.67 59.55 76.67 63.93 69.71
Bernhard 2 2007 U x 1.00 2.90 52582 39.39 - 75.03 76.11 75.56 67.42 65.11 66.24 79.09 80.93 80.00 86.29 84.73 85.50 55.41 65.06 59.82 64.51 74.28 69.03 54.93 62.00 58.22 64.10 70.48 67.11
Bordag 5 2007 U 1.00 1.97 190094 34.11 - - - - 61.81 34.63 44.39 82.72 70.09 75.88 92.10 72.37 81.05 59.91 30.89 40.72 80.50 48.14 60.22 59.13 30.53 40.22 79.98 46.57 58.83
Bordag 5a 2007 U 1.00 1.97 189568 34.20 - - - - 61.75 35.21 44.84 82.78 70.11 75.91 91.98 72.44 81.04 59.71 30.90 40.69 80.32 48.16 60.19 58.94 30.55 40.20 79.81 46.59 58.81
Can 2009 U x 1.00 2.09 150097 - - 59.14 63.54 61.25 58.52 44.82 50.76 72.10 73.35 72.72 83.43 78.70 80.99 56.12 42.62 48.42 68.47 57.41 62.44 55.39 40.69 46.88 67.67 54.89 60.60
DEAP MDL-CAT 2010 S 4.20 3.41 418915 24.09 - - - - 49.83 75.08 59.90 25.79 85.03 39.57 27.29 88.53 41.71 36.42 82.34 50.48 43.33 87.08 57.85 12.12 75.89 20.90 21.01 81.76 33.43
DEAP MDL-NOCAT 2010 S x 1.71 3.42 216528 29.16 - 49.80 87.05 63.35 51.44 80.95 62.91 52.33 81.59 63.76 57.25 89.16 69.72 35.99 83.31 50.24 42.60 87.55 57.29 32.20 76.98 45.39 39.53 81.34 53.19
DEAP PROB-CAT 2010 S 4.63 2.79 790355 21.86 - - - - 55.00 56.61 55.79 22.29 77.30 34.60 23.72 90.65 37.59 41.69 83.05 55.50 48.29 88.57 62.49 10.04 76.46 17.74 18.71 83.11 30.55
DEAP PROB-NOCAT 2010 S x 1.69 2.91 301086 31.41 - 57.49 80.13 66.95 58.52 63.06 60.70 56.18 73.14 63.55 62.01 81.39 70.39 40.78 84.23 54.94 47.10 89.15 61.63 36.38 75.51 49.09 43.82 81.14 56.90
Lignos 2009 U 1.00 1.74 198546 - - - - - 83.49 45.00 58.48 92.40 73.33 81.77 94.38 79.79 86.47 73.98 50.08 59.71 85.13 61.77 71.59 73.04 47.83 57.79 84.55 58.50 69.14
Lignos Aggressive Comp. 2010 U 1.00 2.10 153688 37.84 - - - - 71.45 52.31 60.40 82.17 75.88 78.90 89.45 80.99 85.01 61.86 52.73 56.90 72.72 64.52 68.36 61.16 50.37 55.21 72.11 61.30 66.24
Lignos Base Inference 2010 U 1.00 1.92 168208 38.32 - - - - 80.77 53.76 64.55 87.44 76.82 81.78 92.79 82.28 87.22 71.91 58.98 64.79 80.46 69.29 74.45 71.11 56.54 62.98 79.87 65.79 72.14
Lignos Iterative Comp. 2010 U 1.00 1.91 171964 37.76 - - - - 80.27 52.76 63.67 87.36 76.41 81.52 92.94 81.78 87.00 71.14 57.65 63.67 80.23 68.35 73.80 70.38 55.22 61.87 79.62 64.93 71.52
MAGIP 2010 S 1.00 3.76 93086 22.57 - - - - 29.88 70.65 42.00 48.76 70.83 57.75 56.12 86.64 68.10 22.62 74.66 34.72 24.07 81.92 37.19 22.36 71.80 34.09 23.82 77.94 36.47
McNamee 3 2007 U 1.00 1.00 15212 30.35 - - - - 43.47 22.73 29.85 87.05 45.77 59.99 90.86 59.84 72.15 45.56 1.73 3.33 90.62 23.56 37.39 45.47 1.75 3.36 90.26 24.31 38.30
McNamee 4 2007 U 1.00 1.00 98475 35.72 - - - - 75.96 18.17 29.32 97.73 51.27 67.26 99.04 58.87 73.84 78.23 1.22 2.40 99.04 23.26 37.67 78.23 1.24 2.43 98.95 24.04 38.67
McNamee 5 2007 U 1.00 1.00 243578 34.32 - - - - 92.34 9.88 17.84 98.81 51.92 68.07 99.91 58.36 73.68 94.58 0.64 1.26 99.91 23.00 37.39 94.58 0.64 1.26 99.87 23.79 38.42
MetaMorph 2009 U x 1.00 1.44 330990 - - 61.46 38.33 47.21 69.29 26.00 37.81 84.48 68.05 75.38 90.19 75.63 82.27 59.96 38.37 46.79 74.13 53.99 62.47 59.30 36.37 45.08 73.58 51.33 60.47
Morfessor Baseline 2010 U x 1.00 2.17 61508 38.35 - 87.40 60.91 71.78 81.39 41.70 55.14 86.86 72.26 78.89 95.09 76.38 84.71 77.00 34.33 47.43 86.49 50.53 63.77 76.21 32.64 45.65 85.82 49.17 62.50
Morfessor Categories-MAP 2007 U x 1.00 2.07 137973 37.50 - 86.72 56.65 68.53 80.57 36.31 50.06 86.55 70.62 77.78 94.60 73.85 82.94 73.92 27.28 39.81 87.30 45.10 59.46 73.47 26.37 38.77 86.86 44.36 58.72
Morfessor Categories-MAP 2008 U x 1.00 2.07 137973 36.80 - 86.72 56.65 68.53 80.57 36.31 50.06 86.55 70.62 77.78 94.60 73.85 82.94 73.92 27.28 39.81 87.30 45.10 59.46 73.47 26.37 38.77 86.86 44.36 58.72
Morfessor Categories-MAP 2010 U x 1.00 2.01 166078 37.54 - 95.79 51.75 67.19 86.84 30.03 44.63 92.09 68.15 78.33 96.89 72.47 82.92 79.25 25.31 38.34 91.37 43.14 58.60 78.77 24.29 37.10 91.12 42.42 57.88
Morfessor S+W 2010 S x 1.00 2.65 32196 37.76 - 76.66 78.94 77.78 65.62 69.28 67.40 79.39 81.76 80.56 86.11 85.81 85.96 58.76 68.57 63.27 67.13 76.75 71.61 58.10 65.75 61.67 66.58 72.98 69.62
Morfessor S+W+L 2010 S 1.00 2.59 45427 38.20 - - - - 67.87 66.43 67.14 82.08 84.50 83.27 87.37 87.84 87.60 67.05 77.45 71.86 74.15 83.86 78.70 66.47 74.06 70.04 73.61 79.66 76.50
Morfessor U+W 2010 P x 1.00 2.80 14555 36.90 - 66.56 69.40 67.94 60.33 59.55 59.94 73.13 76.73 74.89 84.09 80.59 82.30 56.14 46.24 50.67 65.79 59.93 62.70 55.57 44.00 49.07 65.16 57.47 61.05
MorphAcq 2010 U 1.00 1.72 227131 36.80 - - - - 67.83 53.43 59.78 78.51 77.59 78.04 86.66 82.29 84.42 64.67 55.83 59.91 73.53 66.74 69.96 64.13 53.07 58.06 73.00 63.04 67.65
MorphoNet 2009 U 1.00 1.75 211439 35.16 - - - - 65.08 47.82 55.13 72.03 74.30 73.15 86.66 79.51 82.93 58.29 49.19 53.34 68.43 61.65 64.86 57.45 47.03 51.71 67.77 58.52 62.80
ParaMor 2007 U 2.42 1.88 233981 28.19 - - - - 67.60 55.72 61.09 38.40 67.82 49.03 46.47 72.29 56.57 54.73 60.31 57.37 58.96 70.78 64.31 28.58 57.53 38.17 34.92 68.20 46.17
ParaMor 2008 U x 1.27 1.75 252997 39.42 - - - - 63.32 51.96 57.08 73.93 75.63 74.77 81.95 82.59 82.26 54.69 58.34 56.43 64.00 68.54 66.17 54.17 55.91 55.00 63.63 64.64 64.11
ParaMor Mimic 2009 P x 1.00 3.04 188716 38.21 - 57.08 70.21 62.96 53.13 59.01 55.91 67.93 78.27 72.73 76.06 85.60 80.54 43.19 68.42 52.93 49.64 76.76 60.27 42.78 65.58 51.76 49.28 72.73 58.73
ParaMor-Morfessor 2007 U 3.42 1.93 386257 26.93 - - - - 70.09 67.38 68.71 24.28 61.80 34.85 32.71 71.25 44.84 53.57 66.98 59.52 56.92 75.81 65.00 23.09 62.59 33.72 28.59 71.36 40.82
ParaMor-Morfessor 2008 U 2.27 1.89 378364 40.43 - - - - 69.59 65.57 67.52 40.08 68.24 50.40 50.42 80.28 61.94 53.77 65.81 59.17 61.06 74.74 67.20 40.71 60.17 48.55 43.14 68.93 53.06
ParaMor-Morfessor Mimic 2009 P x 1.00 2.96 166310 36.90 - 63.99 79.21 70.78 54.80 60.17 57.36 69.62 81.05 74.90 79.35 85.22 82.17 49.14 69.54 57.56 57.41 77.82 66.05 48.71 66.40 56.16 57.03 73.66 64.26
ParaMor-Morfessor Union 2009 P x 1.00 2.87 120148 38.79 - 63.07 78.76 70.04 55.68 62.33 58.82 69.62 80.95 74.86 79.24 85.03 82.02 49.87 66.89 57.12 57.79 75.76 65.55 49.42 63.83 55.69 57.39 71.71 63.74
Promodes 2 2009 U 1.00 3.63 47456 - - - - - 32.24 61.10 42.21 47.79 61.82 53.91 60.17 75.13 66.82 21.89 32.78 26.21 32.88 49.40 39.46 21.68 31.74 25.72 32.33 48.25 38.70
Promodes 2009 U 1.00 3.28 107111 - - - - - 36.20 64.81 46.46 61.18 75.61 67.64 68.34 88.14 76.98 33.56 82.50 47.70 36.68 88.37 51.84 33.24 78.71 46.73 36.38 83.56 50.68
Promodes 2010 P x 1.00 3.80 72811 32.86 - 43.91 68.21 53.42 39.59 64.72 49.13 56.58 73.82 64.06 67.38 82.22 74.05 33.39 60.07 42.89 41.46 70.51 52.20 32.98 56.78 41.69 40.88 67.30 50.84
Promodes committee 2009 U 1.00 3.63 47456 - - - - - 32.24 61.10 42.21 47.79 61.82 53.91 60.17 75.13 66.82 21.89 32.78 26.21 32.88 49.40 39.46 21.68 31.74 25.72 32.33 48.25 38.70
Promodes-E 2010 P x 1.00 3.09 109050 33.79 - 49.48 53.72 51.50 49.69 46.22 47.90 65.73 69.77 67.69 77.56 74.69 76.10 41.54 32.79 36.62 57.53 49.91 53.44 41.18 31.26 35.51 56.82 48.81 52.50
Promodes-H 2010 P x 1.00 4.39 45043 20.34 - 22.23 54.80 31.62 26.84 63.69 37.77 40.16 62.98 49.04 53.36 78.91 63.66 27.62 47.25 34.83 30.85 61.06 40.97 27.35 45.51 34.14 30.56 58.23 40.06
RALI-ANA 2009 U x 1.00 2.10 166826 - - 66.03 49.64 56.66 64.61 33.48 44.10 76.94 67.92 72.15 85.51 74.09 79.39 47.63 32.67 38.72 65.84 49.40 56.44 47.11 30.51 37.00 65.37 47.27 54.85
RALI-COF 2009 U 1.00 1.91 145733 - - - - - 68.32 46.45 55.30 79.31 73.28 76.17 91.67 77.41 83.93 71.27 41.92 52.75 81.78 56.06 66.50 70.48 39.89 50.91 80.97 53.77 64.61
RePortS 2007 U x 1.00 1.57 211475 36.03 - 86.84 60.93 71.61 78.22 43.44 55.86 76.83 64.13 69.90 80.46 68.49 73.99 76.46 60.28 67.40 84.13 67.44 74.86 75.61 55.66 64.10 83.45 62.98 71.77
UNGRADE 2009 U 1.00 3.87 123634 - - - - - 28.29 51.74 36.58 44.55 68.94 54.12 60.31 78.32 68.14 22.67 41.06 29.18 26.27 56.80 35.91 22.34 39.39 28.49 25.82 54.40 35.00
Zeman 1 2008 U 3.18 1.74 905251 - - - - - 67.13 46.67 55.06 45.56 61.37 52.29 51.95 69.85 59.58 41.60 50.41 45.57 51.25 63.23 56.60 30.23 46.25 36.56 43.91 59.36 50.47
Zeman 2007 U 3.18 1.74 905251 - - - - - 67.13 46.67 55.06 45.56 61.37 52.29 51.95 69.85 59.58 41.60 50.41 45.57 51.25 63.23 56.60 30.23 46.25 36.56 43.91 59.36 50.47
Zeman 3 2008 U 1.08 1.37 319982 - - - - - 77.32 9.69 17.23 88.39 58.28 70.24 93.49 64.04 76.01 65.52 5.63 10.36 91.45 27.78 42.61 57.23 5.51 10.04 88.41 28.27 42.83
letters - x 1.00 9.10 28 - - 15.13 91.13 25.95 4.41 99.88 8.44 9.42 29.53 14.29 25.96 94.15 40.68 3.48 99.74 6.73 3.34 99.80 6.46 3.26 95.67 6.30 3.28 94.63 6.35