Morpho Challenge -> Results

Click the column headers to sort the rows.

Results for German. The 1st column shows the name of the method or author and the year of evaluation. The 2nd column shows the type of the algorithm: semi-supervised (S), unsupervised (U), or unsupervised with supervised parameter tuning (P). The 3rd column shows wheter the algorithm returned a segmentation. The 4th column (#a/w) shows the average number of analyses per word, the 5th column (#m/w) the average number of morphemes per word, and the 6th column (#lex) the size of the morpheme lexicon. The rest of the columns give the results of the different evaluations.
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Allomorfessor 2008 U 1.00 1.83 334851 - - - - - 87.82 8.54 15.57 84.89 43.12 57.19 96.89 47.62 63.85 77.62 5.96 11.07 92.34 14.98 25.76 77.47 5.71 10.63 92.17 15.05 25.87
Allomorfessor 2009 U 1.00 2.63 43609 43.93 30.09 - - - 77.78 28.83 42.07 77.24 51.57 61.84 88.34 57.54 69.68 58.79 20.03 29.86 71.59 28.18 40.42 57.54 18.42 27.89 70.63 26.83 38.87
Bernhard 1 2007 U x 1.00 3.12 56173 45.01 - - - - 66.82 42.48 51.94 71.64 54.68 62.02 80.63 64.71 71.79 41.83 39.70 40.67 52.76 45.99 49.09 40.53 34.45 37.18 51.63 41.20 45.78
Bernhard 2 2007 U x 1.00 3.72 53497 44.93 - - - - 54.02 60.77 57.20 64.27 57.65 60.78 71.63 71.20 71.41 26.76 59.05 36.80 36.02 63.43 45.92 25.97 50.95 34.36 35.29 55.89 43.23
Bordag 5 2007 U 1.00 2.99 267680 41.87 - - - - 62.48 42.77 50.78 71.80 57.32 63.74 81.81 65.40 72.68 45.36 34.17 38.94 57.40 40.94 47.76 44.22 30.60 36.14 56.45 37.61 45.11
Bordag 5a 2007 U 1.00 2.99 266924 41.47 - - - - 62.22 43.81 51.42 71.96 57.34 63.82 81.65 65.57 72.73 45.42 34.22 39.00 57.30 40.99 47.76 44.29 30.64 36.19 56.35 37.66 45.12
Can 1 2009 U x 1.00 2.27 320971 - - - - - 73.16 15.27 25.27 71.92 46.68 56.61 92.84 48.11 63.37 63.02 5.59 10.26 81.51 15.45 25.97 62.31 5.29 9.75 80.86 15.26 25.67
Can 2 2009 U x 1.00 2.56 274334 - - - - - 57.67 42.67 49.05 63.56 55.01 58.97 76.45 66.46 71.10 36.10 40.67 38.21 47.85 47.13 47.46 35.55 36.98 36.22 47.28 43.62 45.35
Lignos 2009 U 1.00 2.08 515357 - - - - - 78.90 21.35 33.61 85.88 45.45 59.44 89.62 58.02 70.43 53.20 27.88 36.55 74.66 34.22 46.90 52.08 25.36 34.08 73.99 32.09 44.74
Lignos Aggressive Comp. 2010 U 1.00 3.10 263886 43.79 29.80 - - - 59.41 37.21 45.76 68.75 51.17 58.67 77.85 63.86 70.16 35.84 39.26 37.43 48.43 45.42 46.85 34.68 34.74 34.67 47.46 41.20 44.08
Lignos Base Inference 2010 U 1.00 2.87 333093 44.34 29.52 - - - 66.38 35.36 46.14 74.98 50.84 60.59 80.11 65.21 71.89 40.74 43.80 42.20 52.10 49.60 50.80 39.51 39.35 39.42 51.04 45.43 48.05
Lignos Iterative Comp. 2010 U 1.00 2.93 337128 42.00 29.85 - - - 62.13 34.70 44.53 71.55 50.35 59.10 79.03 63.76 70.57 38.89 40.68 39.73 50.62 46.84 48.63 37.75 36.19 36.91 49.62 42.66 45.85
McNamee 3 2007 U 1.00 1.00 16633 27.82 - - - - 45.78 12.92 20.15 87.60 30.39 45.12 89.67 41.14 56.40 39.05 0.55 1.09 89.50 8.91 16.20 39.01 0.56 1.11 89.38 9.25 16.77
McNamee 4 2007 U 1.00 1.00 130802 32.60 - - - - 75.62 9.67 17.15 96.78 33.61 49.89 98.66 40.37 57.29 67.54 0.29 0.58 98.66 8.73 16.03 67.54 0.30 0.61 98.64 9.08 16.62
McNamee 5 2007 U 1.00 1.00 434152 33.97 - - - - 90.92 6.13 11.48 97.62 33.92 50.34 99.70 39.93 57.02 84.77 0.17 0.34 99.72 8.65 15.91 84.77 0.18 0.35 99.71 9.00 16.50
MetaMorph 2009 U x 1.00 1.75 1113773 - 29.44 - - - 39.91 19.81 26.47 66.38 42.17 51.58 77.69 47.83 59.21 28.46 5.39 9.06 49.61 15.58 23.70 28.08 4.98 8.46 49.05 15.30 23.32
Morfessor Baseline 2010 U x 1.00 2.20 121830 43.91 29.94 - - - 82.80 19.77 31.92 81.28 48.06 60.40 91.36 54.48 68.25 63.54 15.58 25.00 79.09 23.85 36.62 62.42 14.48 23.47 78.34 23.09 35.64
Morfessor Categories-MAP 2007 U x 1.00 3.10 166963 44.25 - - - - 70.73 38.82 50.13 76.51 57.00 65.33 85.10 63.58 72.78 49.81 29.29 36.88 62.94 36.36 46.08 48.93 27.58 35.26 62.20 34.86 44.66
Morfessor Categories-MAP 2008 U x 1.00 3.10 166963 46.50 - - - - 70.73 38.82 50.13 76.51 57.00 65.33 85.10 63.58 72.78 49.81 29.29 36.88 62.94 36.36 46.08 48.93 27.58 35.26 62.20 34.86 44.66
Morfessor Categories-MAP 2010 U x 1.00 2.95 201889 46.57 30.08 - - - 72.70 35.43 47.64 78.26 55.83 65.17 86.94 62.01 72.38 52.58 25.63 34.45 66.81 32.98 44.15 51.75 24.19 32.95 66.16 31.75 42.90
Morfessor U+W 2010 P x 1.00 3.21 10814 46.66 29.86 - - - 58.55 44.94 50.85 65.71 55.60 60.23 76.58 62.57 68.86 41.54 29.83 34.70 49.61 37.07 42.41 40.37 26.38 31.88 48.49 33.81 39.82
MorphAcq 2010 U 1.00 1.47 917276 38.62 29.86 - - - 69.97 25.29 37.15 76.11 51.26 61.25 86.32 57.45 68.98 51.14 21.38 30.13 67.27 29.14 40.64 50.09 18.80 27.31 66.45 26.69 38.06
MorphoNet 2009 U 1.00 1.57 816818 33.12 29.46 - - - 67.41 30.19 41.71 71.95 51.93 60.32 82.76 60.97 70.21 45.35 39.89 42.42 59.24 45.31 51.33 44.51 34.76 39.01 58.56 40.50 47.86
ParaMor 2007 U 1.91 1.72 1001441 31.42 - - - - 68.69 33.28 44.83 48.24 46.60 47.40 56.45 59.33 57.85 41.33 47.38 44.13 50.41 52.71 51.52 26.81 41.97 32.70 37.51 47.79 42.01
ParaMor 2008 U x 1.25 1.65 908556 36.24 - - - - 56.98 42.10 48.42 68.34 49.69 57.54 73.10 67.34 70.10 31.68 54.03 39.92 44.84 58.40 50.71 30.72 46.76 37.06 44.03 51.53 47.47
ParaMor Mimic 2009 P x 1.00 2.74 726045 37.45 29.77 - - - 50.81 47.68 49.20 65.21 50.24 56.75 69.03 70.50 69.75 27.55 62.13 38.15 36.51 66.01 46.98 26.42 53.72 35.40 35.51 58.09 44.04
ParaMor-Morfessor 2007 U 2.91 2.20 1191842 38.98 - - - - 69.96 55.42 61.85 28.68 49.59 36.34 40.14 60.79 48.35 39.70 58.36 47.24 43.80 62.85 51.61 24.13 49.08 32.34 26.12 54.04 35.21
ParaMor-Morfessor 2008 U 2.25 2.30 1094322 46.90 - - - - 64.06 61.52 62.76 38.93 53.17 44.92 50.58 67.13 57.69 32.20 63.88 42.79 38.07 67.75 48.73 28.68 51.10 36.73 31.67 55.58 40.35
ParaMor-Morfessor Mimic 2009 P x 1.00 3.87 206380 44.97 29.48 - - - 51.07 57.79 54.22 62.36 60.20 61.26 70.44 71.29 70.86 28.56 58.22 38.31 35.18 62.90 45.11 27.59 50.91 35.77 34.25 55.98 42.48
ParaMor-Morfessor Union 2009 P x 1.00 3.63 156981 44.02 29.64 - - - 52.53 60.27 56.14 62.75 60.30 61.50 70.40 71.63 71.00 29.89 59.40 39.75 35.97 63.90 46.01 29.03 51.63 37.14 35.17 56.50 43.33
Promodes 2 2009 U 1.00 4.97 94057 - - - - - 36.11 50.52 42.12 48.67 50.12 49.38 59.21 61.67 60.40 20.33 28.76 23.74 28.99 36.04 32.07 19.76 27.28 22.84 28.31 34.60 31.07
Promodes 2009 U 1.00 3.67 326224 - - - - - 49.88 33.95 40.40 63.88 51.48 57.01 76.23 59.78 67.01 33.42 30.97 32.11 47.34 38.51 42.43 32.41 27.10 29.48 46.41 34.87 39.78
Promodes committee 2009 U 1.00 4.13 183007 - - - - - 48.48 34.61 40.39 58.88 49.06 53.51 73.37 53.91 62.14 28.53 13.66 18.41 46.92 22.53 30.40 28.04 13.23 17.92 46.30 22.14 29.92
RALI-ANA 2009 U x 1.00 1.93 695995 - - - - - 61.39 15.34 24.55 72.12 43.84 54.52 82.18 52.06 63.73 32.64 14.41 19.93 60.16 22.71 32.93 31.74 13.05 18.44 59.53 21.61 31.67
RALI-COF 2009 U 1.00 2.17 429399 - - - - - 67.53 34.38 45.57 73.22 51.57 60.51 83.23 63.44 71.99 39.76 32.11 35.47 59.08 38.76 46.77 38.88 29.87 33.73 58.39 36.82 45.12
UNGRADE 2009 U 1.00 4.17 497947 - - - - - 39.02 29.25 33.44 45.58 49.98 47.68 64.68 55.17 59.54 23.10 18.61 20.58 27.68 27.86 27.74 22.29 16.19 18.73 26.83 25.45 26.10
Zeman 1 2008 U 4.11 1.80 4054397 - - - - - 66.86 29.73 41.16 45.31 41.52 43.32 52.23 52.98 52.60 32.42 40.62 36.04 45.75 46.83 46.26 21.56 35.63 26.86 36.76 42.05 39.21
Zeman 2007 U 4.15 1.81 4094228 - - - - - 66.81 29.83 41.25 45.46 41.51 43.39 52.04 53.12 52.56 32.46 40.63 36.07 45.75 46.85 46.27 21.50 35.62 26.81 36.65 42.08 39.17
Zeman 3 2008 U 1.12 1.43 1053275 - - - - - 74.19 7.10 12.95 81.63 39.67 53.39 87.89 47.92 62.02 47.83 8.96 15.08 83.28 17.10 28.36 38.03 8.45 13.82 79.53 16.97 27.96
letters - x 1.00 13.39 29 - - - - - 2.80 99.92 5.45 8.34 23.97 12.37 16.16 86.61 27.22 2.81 99.89 5.46 2.37 99.91 4.64 2.19 86.95 4.28 2.22 87.64 4.34