Morpho Challenge -> Results

Click the column headers to sort the rows.

Results for Turkish. The 1st column shows the name of the method or author and the year of evaluation. The 2nd column shows the type of the algorithm: semi-supervised (S), unsupervised (U), or unsupervised with supervised parameter tuning (P). The 3rd column shows wheter the algorithm returned a segmentation. The 4th column (#a/w) shows the average number of analyses per word, the 5th column (#m/w) the average number of morphemes per word, and the 6th column (#lex) the size of the morpheme lexicon. The rest of the columns give the results of the different evaluations.
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Method Type Seg. #a/w #m/w #lexicon IR MAP SMT BLEU BPR Pre BPR Rec BPR F MC Pre MC Rec MC F EMMA Pre EMMA Rec EMMA F EMMA-2 Pre EMMA-2 Rec EMMA-2 F CoMMA-B0 Pre CoMMA-B0 Rec CoMMA-B0 F CoMMA-B1 Pre CoMMA-B1 Rec CoMMA-B1 F CoMMA-S0 Pre CoMMA-S0 Rec CoMMA-S0 F CoMMA-S1 Pre CoMMA-S1 Rec CoMMA-S1 F
Allomorfessor 2008 U 1.00 1.76 183297 - - - - - 93.16 9.56 17.35 71.39 32.69 44.84 98.73 32.46 48.85 94.05 1.39 2.73 97.69 8.73 16.01 93.33 1.45 2.85 97.17 9.50 17.29
Allomorfessor 2009 U 1.00 2.37 29193 - - - - - 85.89 19.53 31.82 63.07 34.63 44.71 94.77 34.53 50.62 82.30 2.89 5.58 88.13 10.21 18.29 80.54 2.92 5.63 86.39 10.77 19.13
Bernhard 1 2007 U x 1.00 2.48 86490 - - 79.63 35.99 49.57 81.32 16.37 27.25 63.16 34.76 44.84 93.07 34.91 50.76 77.92 3.35 6.41 84.80 10.61 18.84 75.74 3.24 6.22 82.68 11.04 19.46
Bernhard 2 2007 U x 1.00 2.73 87637 - - 78.80 40.76 53.72 77.00 20.77 32.72 63.02 36.97 46.60 87.99 37.48 52.56 69.76 6.75 12.29 76.07 13.78 23.32 65.16 5.71 10.50 71.87 13.21 22.30
Bordag 5 2007 U 1.00 2.24 219488 - - - - - 81.19 23.44 36.38 63.87 36.53 46.47 92.99 36.25 52.16 81.79 5.02 9.46 87.77 12.17 21.36 76.05 4.62 8.71 82.85 12.39 21.54
Bordag 5a 2007 U 1.00 2.24 219864 - - - - - 81.06 23.51 36.45 63.66 36.40 46.31 92.97 36.22 52.12 81.75 5.00 9.42 87.74 12.16 21.34 76.00 4.60 8.66 82.82 12.38 21.52
Can 1 2009 U x 1.00 1.93 328129 - - 70.38 21.50 32.91 73.38 8.80 15.71 70.05 32.36 44.27 92.41 33.51 49.17 61.90 3.67 6.92 85.75 10.82 19.19 59.25 2.66 5.09 84.54 10.24 18.25
Can 2 2009 U x 1.00 3.81 214912 - - 43.44 37.54 40.27 42.18 36.69 39.24 50.37 35.03 41.32 61.71 45.37 52.28 29.53 29.00 29.20 39.72 34.29 36.75 26.17 21.31 23.44 36.84 26.88 31.04
DEAP MDL-CAT 2010 S 4.88 2.59 891260 - - - - - 72.75 53.49 61.65 23.90 55.64 33.44 35.29 56.05 43.31 50.26 39.03 43.93 51.31 43.73 47.21 26.83 31.64 29.03 27.23 37.28 31.47
DEAP MDL-NOCAT 2010 S x 3.17 2.67 631500 - - 49.00 79.99 60.77 68.93 56.12 61.87 35.90 55.54 43.60 49.33 54.29 51.69 46.24 42.05 44.03 47.22 46.49 46.84 32.81 31.82 32.30 33.96 37.18 35.49
DEAP PROB-CAT 2010 S 4.68 2.30 1169357 - - - - - 68.86 55.81 61.65 24.17 52.06 33.01 35.71 52.70 42.57 43.60 46.84 45.14 44.64 50.92 47.55 25.03 35.47 29.34 25.94 40.66 31.67
DEAP PROB-NOCAT 2010 S x 3.49 2.34 1023903 - - 43.39 72.74 54.35 65.71 58.04 61.63 32.03 52.23 39.71 45.05 51.83 48.20 40.69 49.15 44.50 41.66 53.03 46.64 28.34 35.73 31.59 29.60 40.67 34.26
Lignos Aggressive Comp. 2010 U 1.00 3.08 212291 - - - - - 55.51 34.36 42.45 57.09 37.33 45.13 74.11 45.39 56.29 37.01 29.20 32.60 47.63 34.60 40.04 33.26 20.70 25.49 44.85 26.63 33.39
Lignos Base Inference 2010 U 1.00 2.00 285196 - - - - - 72.81 16.11 26.38 71.31 35.30 47.22 84.93 39.96 54.35 52.40 12.36 20.00 65.10 18.92 29.30 48.02 9.77 16.24 61.80 16.57 26.13
Lignos Iterative Comp. 2010 U 1.00 2.42 248172 - - - - - 68.69 21.44 32.68 66.07 36.02 46.62 81.63 41.52 55.03 51.40 16.10 24.50 61.78 22.39 32.84 47.60 12.13 19.32 58.79 18.84 28.51
MAGIP 2010 S 1.00 4.99 99549 - - - - - 32.00 65.80 43.06 34.79 35.41 35.09 47.81 51.53 49.60 24.62 61.32 35.12 24.27 64.40 35.23 22.23 40.59 28.71 22.42 44.49 29.80
McNamee 3 2007 U 1.00 1.00 19389 - - - - - 65.00 16.23 25.98 88.91 25.77 39.95 91.78 29.96 45.17 61.45 0.41 0.80 92.61 7.49 13.86 60.70 0.44 0.87 92.04 8.21 15.06
McNamee 4 2007 U 1.00 1.00 102515 - - - - - 85.49 10.31 18.40 95.96 27.63 42.91 98.96 29.15 45.03 84.28 0.13 0.26 99.16 7.29 13.57 83.80 0.14 0.28 99.03 8.01 14.81
McNamee 5 2007 U 1.00 1.00 226698 - - - - - 94.80 5.20 9.87 95.93 27.66 42.93 99.76 28.70 44.57 94.47 0.04 0.08 99.89 7.22 13.46 94.47 0.05 0.09 99.85 7.94 14.70
MetaMorph 2009 U x 1.00 1.99 470080 - - 3.90 7.77 5.16 39.14 29.45 33.61 54.86 30.28 39.01 70.16 33.76 45.58 26.69 2.87 5.17 38.57 10.15 16.05 23.47 2.37 4.31 34.64 9.88 15.34
Morfessor Baseline 2010 U x 1.00 2.14 53473 - - 84.04 36.88 51.26 89.68 17.78 29.67 65.43 34.30 45.00 96.38 34.22 50.50 87.27 2.69 5.23 91.95 10.00 18.03 85.95 2.73 5.30 90.65 10.61 18.99
Morfessor Categories-MAP 2008 U x 1.00 2.76 107431 - - 82.19 50.56 62.61 77.36 33.47 46.73 63.07 40.36 49.22 85.87 40.80 55.31 68.88 13.52 22.58 74.30 20.04 31.54 62.67 10.52 18.01 68.92 17.59 28.00
Morfessor Categories-MAP 2010 U x 1.00 2.64 114834 - - 83.53 48.82 61.62 79.38 31.88 45.49 64.68 40.04 49.46 87.99 40.16 55.14 71.99 11.91 20.42 77.43 18.53 29.88 65.92 9.49 16.58 72.19 16.63 27.01
Morfessor S+W 2010 S x 1.00 3.58 9398 - - 77.52 62.04 68.91 65.71 47.15 54.90 57.34 44.25 49.94 76.81 45.86 57.42 62.57 21.51 32.00 64.60 27.52 38.59 57.91 16.94 26.21 59.98 23.22 33.48
Morfessor S+W+L 2010 S 1.00 3.56 14713 - - - - - 71.69 59.97 65.31 75.32 57.79 65.39 82.16 59.26 68.85 76.31 38.71 51.35 77.64 43.39 55.65 74.78 32.79 45.58 76.04 37.74 50.43
Morfessor U+W 2010 P x 1.00 4.57 595 - - 43.79 48.09 45.83 40.71 46.76 43.52 29.59 26.27 27.83 48.15 42.43 45.09 32.09 20.38 24.90 32.77 26.36 29.17 28.00 15.13 19.63 28.64 21.19 24.32
MorphAcq 2010 U 1.00 2.31 180709 - - - - - 79.02 19.78 31.64 63.10 37.44 46.99 88.74 37.82 53.03 76.18 8.95 16.02 81.69 15.81 26.49 71.61 7.14 12.99 77.68 14.42 24.31
MorphoNet 2009 U 1.00 3.45 180103 - - - - - 61.75 30.90 41.19 45.46 42.46 43.90 81.62 43.79 57.00 53.34 19.42 28.45 58.90 25.62 35.68 48.78 15.77 23.81 54.69 22.16 31.52
ParaMor 2008 U x 1.00 2.89 245737 - - 61.05 51.18 55.67 57.35 45.75 50.90 54.04 43.55 48.23 71.71 47.52 57.15 41.54 39.08 40.23 47.38 43.83 45.50 38.59 25.93 30.97 44.60 31.37 36.80
ParaMor Mimic 2009 P x 1.00 3.69 218334 - - 54.77 53.55 54.15 49.54 54.77 52.02 49.87 44.86 47.22 65.77 50.37 57.04 34.20 48.75 40.15 37.42 52.75 43.73 31.64 32.93 32.22 34.94 37.70 36.22
ParaMor-Morfessor 2008 U 2.00 2.83 354280 - - - - - 66.78 57.97 62.07 36.23 47.06 40.93 61.78 47.04 53.40 40.61 44.20 42.29 43.51 48.52 45.85 39.53 28.93 33.39 43.29 34.44 38.34
ParaMor-Morfessor Mimic 2009 P x 1.00 4.05 150775 - - 60.38 62.01 61.18 48.07 60.39 53.53 49.19 45.84 47.45 66.56 50.43 57.38 35.69 51.58 42.16 39.71 55.36 46.23 33.17 33.91 33.50 37.26 38.76 37.97
ParaMor-Morfessor Union 2009 P x 1.00 3.86 104868 - - 59.27 62.01 60.61 47.25 60.01 52.88 48.19 45.21 46.65 65.10 50.23 56.69 35.00 50.26 41.22 38.22 54.15 44.77 32.45 33.19 32.76 35.71 37.94 36.76
Promodes 2 2009 U 1.00 5.12 14364 - - - - - 35.36 58.70 44.14 33.96 33.74 33.84 48.68 48.53 48.59 26.99 36.66 31.05 28.52 41.42 33.73 23.91 26.18 24.96 25.42 31.37 28.05
Promodes 2009 U 1.00 5.16 62869 - - - - - 32.22 66.42 43.39 34.66 37.91 36.21 48.42 52.27 50.26 25.30 48.95 33.31 25.43 52.75 34.27 22.67 35.05 27.49 22.80 39.32 28.82
Promodes 2010 P x 1.00 4.50 42277 - - 52.67 54.22 53.43 46.59 51.67 49.00 44.71 41.17 42.86 61.13 47.43 53.40 37.90 32.86 35.17 39.95 37.99 38.91 34.02 23.89 28.04 36.08 29.45 32.40
Promodes committee 2009 U 1.00 3.38 113584 - - - - - 55.30 28.35 37.48 52.48 35.45 42.31 74.54 37.42 49.82 39.72 6.34 10.93 54.54 13.37 21.45 36.56 5.41 9.42 51.52 12.74 20.40
Promodes-E 2010 P x 1.00 4.97 20512 - - 49.64 56.60 52.89 40.75 52.39 45.84 37.22 36.64 36.93 54.52 45.17 49.40 36.70 32.01 34.17 37.63 37.23 37.39 33.56 22.06 26.60 34.51 27.71 30.71
Promodes-H 2010 P x 1.00 4.25 58742 - - 46.75 43.49 45.05 47.88 39.37 43.21 41.23 35.81 38.33 61.07 40.77 48.89 39.90 16.86 23.69 43.40 23.23 30.24 35.99 12.18 18.18 39.45 18.74 25.39
RALI-ANA 2009 U x 1.00 2.35 193643 - - 67.43 30.15 41.67 69.52 12.85 21.69 64.35 35.08 45.41 83.81 36.37 50.72 50.50 6.80 11.97 65.94 13.86 22.88 45.90 5.67 10.09 62.29 13.03 21.54
RALI-COF 2009 U 1.00 3.68 83624 - - - - - 48.43 44.54 46.40 46.93 42.50 44.61 67.85 43.74 53.19 48.10 23.98 31.99 51.51 29.87 37.78 44.21 16.62 24.15 47.52 23.06 31.03
UNGRADE 2009 U 1.00 3.75 211236 - - - - - 46.67 30.16 36.64 41.38 37.01 39.07 63.47 40.52 49.45 32.74 13.68 19.27 36.71 20.31 26.12 28.58 10.86 15.72 32.59 17.54 22.77
Zeman 1 2008 U 3.24 1.76 1205970 - - - - - 77.48 22.71 35.13 46.68 37.92 41.84 59.87 38.41 46.79 46.90 17.10 25.05 56.24 23.23 32.87 38.51 14.98 21.56 47.99 21.73 29.91
Zeman 2007 U 3.24 1.76 1205970 - - - - - 77.48 22.71 35.13 46.68 37.92 41.84 59.87 38.41 46.79 46.90 17.10 25.05 56.24 23.23 32.87 38.51 14.98 21.56 47.99 21.73 29.91
Zeman 3 2008 U 1.15 1.52 501154 - - - - - 72.28 3.87 7.34 76.95 30.72 43.91 90.65 33.00 48.38 52.46 1.91 3.68 85.92 9.15 16.52 45.09 1.71 3.29 82.82 9.65 17.27
letters - x 1.00 9.99 33 - - 23.93 67.27 35.30 8.66 99.13 15.93 11.21 20.92 14.59 30.60 59.59 40.41 9.09 98.24 16.63 8.04 98.40 14.86 7.28 64.40 13.08 7.31 66.20 13.16