Morpho Challenge -> Results

Click the column headers to sort the rows.

Results for Finnish. The 1st column shows the name of the method or author and the year of evaluation. The 2nd column shows the type of the algorithm: semi-supervised (S), unsupervised (U), or unsupervised with supervised parameter tuning (P). The 3rd column shows wheter the algorithm returned a segmentation. The 4th column (#a/w) shows the average number of analyses per word, the 5th column (#m/w) the average number of morphemes per word, and the 6th column (#lex) the size of the morpheme lexicon. The rest of the columns give the results of the different evaluations.
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Allomorfessor 2008 U 1.00 1.86 486096 - - - - - 92.64 8.65 15.83 79.78 43.87 56.61 98.45 45.36 62.11 85.90 2.14 4.17 96.60 8.88 16.25 85.53 2.18 4.25 96.37 9.16 16.71
Allomorfessor 2009 U 1.00 2.46 70228 45.68 26.80 - - - 86.51 19.96 32.44 71.69 49.05 58.25 95.42 50.23 65.81 75.10 5.23 9.77 87.85 12.07 21.21 74.66 5.23 9.77 87.36 12.26 21.48
Bernhard 1 2007 U x 1.00 3.16 87590 47.99 - 70.33 52.06 59.82 78.11 29.39 42.71 66.14 55.00 60.06 89.71 56.72 69.49 66.76 13.85 22.92 72.48 20.15 31.51 66.22 13.53 22.45 71.81 19.89 31.13
Bernhard 2 2007 U x 1.00 3.89 87915 49.07 - 65.46 62.76 64.07 63.92 44.48 52.45 61.74 60.51 61.11 80.06 65.13 71.82 44.46 33.19 37.98 48.05 38.06 42.45 44.09 31.29 36.57 47.68 36.10 41.06
Bordag 5 2007 U 1.00 2.84 517091 43.10 - - - - 72.80 26.36 38.71 66.09 52.17 58.31 90.16 53.19 66.91 66.87 8.33 14.80 77.98 14.82 24.90 66.07 8.33 14.79 77.08 14.96 25.05
Bordag 5a 2007 U 1.00 2.84 514670 42.83 - - - - 72.45 27.21 39.56 66.21 52.25 58.41 89.99 53.33 66.97 66.78 8.40 14.93 77.84 14.89 24.99 65.98 8.41 14.91 76.93 15.03 25.13
DEAP MDL-CAT 2010 S 5.31 3.23 2549280 37.25 25.90 - - - 57.43 66.59 61.67 21.01 64.26 31.66 27.74 72.43 40.10 33.40 63.85 43.85 34.17 66.51 45.14 16.94 51.85 25.53 17.96 55.10 27.08
DEAP MDL-NOCAT 2010 S x 3.29 3.41 1753045 41.60 25.62 36.85 86.59 51.69 56.03 70.71 62.52 29.60 66.45 40.95 38.16 75.62 50.72 31.11 68.96 42.86 31.84 71.23 44.00 21.25 57.56 31.04 22.39 60.17 32.63
DEAP PROB-CAT 2010 S 5.31 2.17 7289719 28.43 25.89 - - - 51.93 42.41 46.69 21.06 50.95 29.79 27.81 62.27 38.45 35.75 50.05 41.69 36.93 53.95 43.83 17.04 45.08 24.73 18.33 49.10 26.69
DEAP PROB-NOCAT 2010 S x 2.35 2.31 3203926 31.74 25.55 46.22 61.16 52.65 56.66 44.79 50.03 39.36 56.22 46.29 49.07 67.20 56.72 34.22 52.48 41.41 35.33 56.16 43.36 28.66 46.54 35.46 30.22 50.31 37.75
Lignos Aggressive Comp. 2010 U 1.00 3.45 356834 49.14 25.97 - - - 63.07 39.98 48.94 66.91 60.04 63.29 81.88 65.48 72.76 45.59 34.69 39.39 52.33 39.54 45.04 45.16 32.75 37.96 51.95 37.58 43.61
Lignos Base Inference 2010 U 1.00 2.58 560186 41.51 26.19 - - - 78.98 24.46 37.35 79.90 56.05 65.88 90.41 62.70 74.05 66.28 29.56 40.87 70.74 34.89 46.72 65.93 28.40 39.69 70.38 33.68 45.54
Lignos Iterative Comp. 2010 U 1.00 2.74 603566 42.29 26.05 - - - 75.32 26.70 39.43 77.33 56.92 65.57 88.89 63.44 74.04 61.94 30.50 40.85 66.80 35.70 46.51 61.60 29.18 39.58 66.44 34.36 45.27
MAGIP 2010 S 1.00 4.42 751715 38.71 25.33 - - - 36.25 53.73 43.29 43.49 53.64 48.03 59.38 66.70 62.82 22.91 48.82 31.18 25.02 52.88 33.96 22.32 45.37 29.91 24.74 49.30 32.93
McNamee 3 2007 U 1.00 1.00 20063 32.48 - - - - 45.53 11.79 18.73 86.21 29.45 43.90 88.83 42.29 57.30 38.92 0.43 0.85 88.67 6.07 11.35 38.88 0.43 0.85 88.47 6.29 11.74
McNamee 4 2007 U 1.00 1.00 137629 36.04 - - - - 68.09 8.32 14.82 98.41 33.51 50.00 98.63 41.54 58.46 59.23 0.15 0.29 98.67 5.87 11.08 59.17 0.15 0.30 98.65 6.10 11.49
McNamee 5 2007 U 1.00 1.00 435814 37.00 - - - - 86.69 5.03 9.51 99.77 33.97 50.68 99.83 41.23 58.35 82.71 0.08 0.15 99.83 5.82 10.99 82.71 0.08 0.16 99.81 6.05 11.40
MetaMorph 2009 U x 1.00 1.87 2036496 - 25.72 19.44 4.61 7.44 37.17 15.15 21.53 69.59 40.27 51.02 79.92 45.07 57.63 25.82 1.96 3.64 55.45 9.11 15.64 25.45 1.93 3.58 54.82 9.26 15.82
Morfessor Baseline 2010 U x 1.00 2.17 175658 42.35 26.65 82.95 38.22 52.32 90.60 14.39 24.83 76.14 47.42 58.44 96.50 49.16 65.13 79.31 4.59 8.67 91.05 11.31 20.11 78.74 4.61 8.70 90.57 11.53 20.45
Morfessor Categories-MAP 2008 U x 1.00 2.83 223412 46.74 - 75.98 50.63 60.76 79.76 24.95 38.02 67.13 52.09 58.66 91.78 53.14 67.31 66.41 8.19 14.57 78.66 14.95 25.11 65.93 8.05 14.33 78.09 14.92 25.04
Morfessor Categories-MAP 2010 U x 1.00 2.88 238837 47.54 26.34 76.43 50.63 60.90 80.31 29.51 43.16 70.05 54.25 61.14 91.16 55.73 69.17 66.77 10.73 18.47 77.61 16.99 27.86 66.25 10.49 18.10 77.01 16.89 27.68
Morfessor S+W 2010 S x 1.00 4.20 14066 47.50 25.94 66.94 77.30 71.74 57.59 55.21 56.38 59.14 65.33 62.08 74.95 69.50 72.11 47.56 39.33 43.01 49.69 43.74 46.48 46.96 37.49 41.66 49.12 41.82 45.13
Morfessor S+W+L 2010 S 1.00 4.27 19175 44.65 25.82 - - - 58.38 63.35 60.76 67.17 75.73 71.19 77.29 77.69 77.48 54.74 56.15 55.41 56.29 59.46 57.80 54.28 53.48 53.85 55.83 56.70 56.24
Morfessor U+W 2010 P x 1.00 3.99 3677 40.42 25.61 56.91 58.24 57.56 56.97 42.98 49.00 53.58 55.71 54.62 72.96 59.31 65.43 46.97 15.97 23.82 50.56 21.98 30.62 46.34 15.42 23.13 49.85 21.34 29.87
MorphoNet 2009 U 1.00 2.53 984581 38.75 25.56 - - - 63.35 22.62 33.34 59.53 53.27 56.22 85.03 58.93 69.61 48.80 26.66 34.45 58.36 31.61 40.98 48.32 25.33 33.21 57.93 30.24 39.72
ParaMor 2008 U x 1.00 2.62 1123572 38.28 - 48.87 50.93 49.86 49.97 37.64 42.93 56.24 54.95 55.58 70.50 65.17 67.72 30.79 36.62 33.42 38.67 41.20 39.87 30.50 34.36 32.28 38.39 38.87 38.60
ParaMor Mimic 2009 P x 1.00 3.30 1149382 39.05 25.53 47.94 52.12 49.94 47.15 40.50 43.57 55.21 55.73 55.46 68.82 66.77 67.77 29.29 40.71 34.03 35.70 45.23 39.87 29.04 38.16 32.94 35.42 42.64 38.66
ParaMor-Morfessor 2008 U 2.00 2.72 1359325 45.42 - - - - 65.21 50.43 56.87 30.56 48.71 37.53 51.17 63.49 56.66 32.35 39.94 35.72 35.78 44.49 39.63 28.57 33.28 30.72 29.94 37.10 33.11
ParaMor-Morfessor Mimic 2009 P x 1.00 4.24 323561 44.46 25.54 58.18 74.65 65.39 51.75 45.42 48.38 53.01 63.27 57.69 72.57 67.28 69.82 32.88 38.52 35.45 38.51 43.22 40.71 32.57 36.26 34.29 38.17 40.95 39.49
ParaMor-Morfessor Union 2009 P x 1.00 4.02 215118 47.13 25.44 54.41 77.22 63.83 47.89 50.98 49.39 49.76 63.49 55.79 69.39 67.89 68.63 29.67 42.33 34.87 33.19 46.78 38.81 29.37 39.74 33.76 32.86 44.10 37.64
Promodes 2 2009 U 1.00 5.64 97558 - - - - - 33.51 61.32 43.34 37.91 53.17 44.26 54.90 66.45 60.11 23.52 38.13 29.06 26.32 42.51 32.47 23.17 35.75 28.08 25.95 40.04 31.45
Promodes 2009 U 1.00 4.88 296010 - - - - - 35.86 51.41 42.25 43.37 58.69 49.88 60.85 69.30 64.79 27.03 40.17 32.29 27.53 44.76 34.06 26.63 37.68 31.18 27.13 42.17 32.98
Promodes 2010 P x 1.00 5.46 235837 37.21 25.64 43.99 65.56 52.65 41.59 47.41 44.31 44.93 59.47 51.18 64.87 65.74 65.30 32.32 36.03 34.06 34.26 40.91 37.27 31.87 34.04 32.90 33.83 38.84 36.15
Promodes committee 2009 U 1.00 4.94 199700 - - - - - 41.20 48.22 44.44 42.45 51.87 46.69 62.47 59.42 60.90 26.83 21.26 23.69 32.27 26.87 29.29 26.43 20.18 22.86 31.82 25.70 28.40
Promodes-E 2010 P x 1.00 5.21 261874 32.13 25.79 39.34 55.61 46.08 43.26 45.73 44.46 44.23 55.55 49.25 64.74 62.22 63.46 36.22 27.06 30.97 38.89 32.45 35.37 35.71 25.71 29.88 38.33 31.00 34.27
Promodes-H 2010 P x 1.00 5.93 192720 31.09 25.35 29.54 52.34 37.76 35.08 53.45 42.36 35.89 52.73 42.71 55.84 64.03 59.65 27.52 32.83 29.92 28.98 37.77 32.78 27.10 30.90 28.86 28.63 35.70 31.76
RALI-ANA 2009 U x 1.00 2.05 1217550 - - 59.11 27.68 37.69 60.06 10.33 17.63 69.56 45.24 54.82 84.30 50.74 63.34 35.86 7.59 12.51 63.03 13.95 22.81 35.48 7.18 11.93 62.66 13.56 22.27
RALI-COF 2009 U 1.00 2.39 723171 - - - - - 74.76 26.20 38.81 74.61 55.96 63.94 90.27 58.73 71.15 64.06 16.60 26.35 76.02 22.14 34.27 63.72 16.05 25.62 75.65 21.66 33.65
UNGRADE 2009 U 1.00 4.60 790814 - - - - - 40.78 33.02 36.49 41.48 51.64 46.00 64.19 59.13 61.55 25.30 21.47 23.21 27.46 27.58 27.49 24.89 20.14 22.24 27.02 26.09 26.52
Zeman 1 2008 U 3.61 1.81 5379817 - - - - - 72.02 21.90 33.59 45.24 41.57 43.32 52.23 52.66 52.44 42.60 26.36 32.53 50.48 31.76 38.94 30.95 23.28 26.53 38.49 28.22 32.52
Zeman 2007 U 3.62 1.81 5434453 - - - - - 71.88 22.40 34.16 44.52 41.51 42.96 51.22 52.81 52.00 42.69 26.81 32.90 49.95 32.17 39.10 31.16 23.66 26.87 37.75 28.55 32.49
Zeman 3 2008 U 1.21 1.62 1830751 - - - - - 71.94 3.84 7.30 82.19 41.10 54.80 88.30 47.07 61.40 55.33 3.88 7.25 84.53 10.02 17.91 46.84 3.74 6.92 80.03 10.07 17.89
letters - x 1.00 13.78 32 - - 19.75 92.74 32.56 5.04 99.89 9.60 9.29 30.46 14.24 31.61 88.69 46.59 5.13 99.76 9.76 4.30 99.79 8.25 4.24 92.53 8.11 4.26 92.56 8.15