Laboratory of Computer and Information Science / Neural Networks Research Centre CIS Lab Helsinki University of Technology

Competition 2: English


Table: The obtained average precision (AP%) in the information retrieval task for the submitted segmentations in English (Competition 2 participants in bold and reference methods in normal font). Indexing is performed using Okapi (BM25) weighting for all morphemes except the most common ones (stoplist) with frequency higher than 150,000 (right).
AUTHOR METHOD AP%
snowball porter 0.4081
Monson Paramor+Morfessor 0.3989
TWOL first 0.3957
Monson Paramor 0.3928
TWOL all 0.3922
Morfessor baseline 0.3861
grammatical first 0.3734
Morfessor MAP 0.3713
Monson Morfessor 0.3637
McNamee five 0.3630
McNamee four 0.3566
McNamee lcn5 0.3563
grammatical all 0.3542
dummy - 0.3293

Return to the result page

Last year's results (Morpho Challenge 2007)


Table: For reference, the results for Morpho Challenge 2007. There were some changes in the reference methods in 2008. The obtained average precision (AP%) in the information retrieval task for the submitted segmentations in English (Competition 2 participants in bold and reference methods in normal font). Indexing is performed using Tfidf (BM25) weighting for all morphemes (left) and Okapi (BM25) weighting for all morphemes except the most common ones (stoplist) with frequency higher than 150,000 (right).
Tfidf BM25 weighting for all morphemes Okapi BM25 weighting and 150,000 stoplist
METHOD WORDLIST AP% METHOD WORDLIST AP%
porter withnew 0.3052 porter withnew 0.4083
McNamee 5 withoutnew 0.2888 Bernhard 2 withnew 0.3943
McNamee 5 withnew 0.2885 Bernhard 2 withoutnew 0.3922
morfessor baseline withnew 0.2863 Bernhard 1 withnew 0.3900
morfessor baseline withoutnew 0.2851 morfessor baseline withnew 0.3882
McNamee 4 withoutnew 0.2842 Bernhard 1 withoutnew 0.3881
McNamee 4 withnew 0.2838 morfessor baseline withoutnew 0.3869
tepper withoutnew 0.2784 grammatical first withoutnew 0.3774
dummy withnew 0.2783 grammatical first withnew 0.3756
morfessor catmap withnew 0.2782 tepper withoutnew 0.3728
Bernhard 1 withoutnew 0.2781 Monson morfessor withoutnew 0.3721
Bernhard 1 withnew 0.2777 morfessor catmap withnew 0.3716
morfessor catmap withoutnew 0.2774 morfessor catmap withoutnew 0.3714
Bernhard 2 withnew 0.2682 Monson morfessor withnew 0.3703
Monson morfessor withoutnew 0.2676 Pitler withoutnew 0.3652
Bernhard 2 withoutnew 0.2673 Pitler withnew 0.3648
Monson morfessor withnew 0.2667 grammatical all withoutnew 0.3621
Pitler withoutnew 0.2666 grammatical all withnew 0.3592
Pitler withnew 0.2639 McNamee 4 withoutnew 0.3577
Monson paramor-m. withnew 0.2628 McNamee 4 withnew 0.3576
Monson paramor-m. withoutnew 0.2624 McNamee 5 withoutnew 0.3438
grammatical all withoutnew 0.2619 Monson paramor-m. withnew 0.3435
grammatical first withoutnew 0.2612 McNamee 5 withnew 0.3433
grammatical all withnew 0.2602 Bordag 5 withoutnew 0.3427
grammatical first withnew 0.2599 Monson paramor-m. withoutnew 0.3426
Monson paramor withnew 0.2400 Bordag 5 withnew 0.3421
Monson paramor withoutnew 0.2390 Bordag 5a withoutnew 0.3409
Zeman withoutnew 0.2297 Bordag 5a withnew 0.3395
Bordag 5 withoutnew 0.2210 dummy withnew 0.3123
Bordag 5 withnew 0.2202 McNamee 3 withoutnew 0.3047
Bordag 5a withoutnew 0.2169 McNamee 3 withnew 0.3030
Bordag 5a withnew 0.2165 Monson paramor withnew 0.2835
McNamee 3 withoutnew 0.1695 Monson paramor withoutnew 0.2821
McNamee 3 withnew 0.1677 Zeman withoutnew 0.2674

Return to the result page

HOME | RULES | SCHEDULE | DATASETS | EVALUATION | WORKSHOP | RESULTS | FAQ | CONTACT

You are at: CIS → Unsupervised Morpheme Analysis -- Morpho Challenge 2007

Page maintained by webmaster at cis.hut.fi, last updated Thursday, 10-Jul-2008 09:29:07 EEST