The book is intended for students and researchers interested in statistical approaches to Natural Language Processing. He works on problems at the intersection of natural language processing and machine learning.
About this book The impact of computer systems that can understand natural language will be tremendous. Eugene Charniak Brown University. Show all. Show next xx. In Figure 2 , it is not clear how the syntactic arguments e. An ordinary solution is the conversion of PTB trees into some form of dependency-based representations. This article adopts three representations that can be converted from PTB trees. Johansson and Nugues, It should be noted, however, that this conversion cannot work perfectly with automatic parsing, because the conversion program relies on additional information function tags and empty categories of the original Penn Treebank, which are not produced by the parsers listed above.
HD: dependency trees of syntactic heads Fig. We then convert lexicalized trees into dependencies between lexical heads. This format can represent dependency relations similar to CoNLL, although relation types are not sufficient to identify important grammatical relations. For example, in Figure 3 , the subject and the object relations are assigned the same relation types, NP, and are not distinguishable. SD: the Stanford dependency format Fig. This format was originally proposed for extracting dependency relations useful for practical applications de Marneffe et al.
A program to convert PTB is attached to the Stanford parser. Although the concept looks similar to CoNLL, this representation does not necessarily form a tree structure, and is designed to express more fine-grained relations, such as apposition.
- Theory and Applications of Natural Language Processing!
- Publications — Christoph Teichmann.
- La storia segreta di Gesù (Religioni e Misticismo Vol. 8) (Italian Edition).
Research groups for biomedical NLP recently adopted this representation for corpus annotation Pyysalo et al. Deep parsing aims to compute in-depth syntactic and semantic structures based on syntactic theories, such as HPSG Pollard and Sag, Recent research developments have allowed for efficient and robust deep parsing of real-world texts Miyao and Tsujii, PAS is a graph structure that represents the relations among words Fig. The concept is therefore similar to CoNLL dependencies, though PAS expresses deeper relations, such as long distance dependencies, and may include shared structures.
In addition to the PAS format, the PTB format can also be created from Enju's output by using tree structure matching Matsuzaki and Tsujii, , but this conversion is imperfect because the forms of PTB and Enju's output are not entirely compatible. We can also obtain the CoNLL. That is, five parse representations are available for the Enju parser. In our approach to parser evaluation, we measure the accuracy of a PPI extraction system, in which the parser output is embedded as statistical features of a machine learning classifier. We run the classifier with features of every possible combination of a parser and a parse representation, by applying conversions between representations when necessary.
PPI extraction is an information extraction task to identify protein pairs that are mentioned as interacting in biomedical papers. Because the number of biomedical papers is growing rapidly, it is becoming difficult for biomedical researchers to find all papers relevant to their research; thus, there is an emerging need for reliable text mining technologies, such as automatic PPI extraction from texts.
Figure 6 shows two sentences that include protein names: the former sentence mentions a protein interaction, while the latter does not. From this dependency tree, we can extract the dependency path shown in Figure 7 , which appears to be a strong clue in knowing that these proteins are mentioned as interacting.
Two types of features are incorporated in the classifier. The first is bag-of-words features, which are regarded as a strong baseline for PPI extraction systems. Lemmas of words before, between and after the pair of target proteins are included, and a linear kernel is used for these features. This kernel is included in all our models. The other type of feature is parser output features.
Because a tree kernel measures the similarity of trees by counting common subtrees, it is expected that the system finds effective subsequences of dependency paths. For the PTB representation, we directly encode phrase structure trees. This experiment indicates differences or overlaps in the information conveyed by two different parsers or parse representations.
Coarse-to-Fine Natural Language Processing | Slav Petrov | Springer
It is widely believed that the choice of the representation format for parser output may greatly affect the performance of applications, although this has not been extensively investigated. We should, therefore, evaluate the parser performance in multiple parse representations. In this article, we create multiple parse representations by converting each parser's default output into other representations when possible. This experiment can also be considered to be a comparative evaluation of parse representations, thus providing an indication for selecting an appropriate parse representation for similar information extraction and text mining tasks.
Table 1 lists the formats for parser output used in this work, and Figure 9 shows our scheme for representation conversion. Although only CoNLL is available for dependency parsers, we can create four representations for the phrase structure parsers, and five for the deep parsers. Dotted arrows in Figure 9 indicate imperfect conversion, in which the conversion inherently introduces errors, and may decrease the accuracy. We should, therefore, take caution when comparing the results obtained by imperfect conversion. The domain of our target text is different from the Wall Street Journal WSJ portion of the Penn Treebank, which is the de facto standard data for parser training.
Since all these parsers have programs for training with a PTB-style treebank, we use those programs for retraining with default parameter settings. In preliminary experiments, we found that dependency parsers attain higher dependency accuracy when trained only with GENIA. Tsuruoka et al.
In addition to investigating the impact of different parsers and different syntactic representations on PPI identification accuracy, we also examine how the parse accuracy of a single parser affects the PPI accuracy. To this end, we retrain one of the parsers KSDEP with varying amounts of training text, resulting in several different versions of the same parser, having different levels of accuracy. This allows us to establish a relationship between the accuracy of the parser and the amount of training data used to create the parser. When the parser is used as a component in the PPI identification system, we can determine the relationship between the size of the dataset used to train the parser, the parser's accuracy, and the overall PPI system's accuracy.
This provides a rough guide for what level of accuracy to expect in the PPI task when a new parser is used, as long as the accuracy of the parser is known.
The data consists of biomedical paper abstracts sentences , which are sentence-split, tokenized and annotated with proteins and PPIs. We use the gold protein annotations given in the data, and multi-word protein names are concatenated and treated as single words. The accuracy is measured by abstract-wise fold cross-validation and the one-answer-per-occurrence criterion Giuliano et al.
A prediction threshold for the support vector machine SVM is moved to adjust the balance of precision and recall, and the maximum f -score is reported for each experiment. Table 2 shows the time used by each parser for parsing the entire AIMed corpus, and the PPI accuracy obtained by using the output from each parser with different parse representation. Table 2 clearly shows that all the parsers achieved better results than the baseline, demonstrating contributions of these parsers to PPI extraction. Differences among parsers are relatively smaller than the differences from the baseline, proving that dependency parsing, phrase structure parsing and deep parsing perform equally well in this task.
While the accuracy level of PPI extraction is similar, parsing speed differs considerably for different parsing frameworks.
The 4 Biggest Open Problems in NLP
The dependency parsers are much faster than the other parsers, while the phrase structure parsers are relatively slower, and the deep parsers are in between. It is noteworthy that the dependency parsers achieved comparable accuracy with the other parsers, while they are more efficient. The experimental results also demonstrate that the PTB format is worse than the other representations with respect to contributions to accuracy improvements. The conversion from PTB to dependency-based representations is, therefore, desirable for this task, although it is possible that better results might be obtained with PTB if a different feature extraction mechanism is used.
Among dependency-based representations. HD is slightly worse, indicating that surface syntactic relations are insufficient for this task. This might be a reason for the high performances of the dependency parsers that directly compute CoNLL dependencies. This result implies that, deep relations, such as long-distance dependencies, might contribute to accuracy improvements, although this does not necessarily mean the superiority of PAS to CoNLL, because two imperfect conversions, i.
Interestingly, the accuracy improvements are observed even for ensembles of different representations from the same parser. This indicates that a single parse representation is insufficient for expressing the true potential of a parser. Effectiveness of combining two parsers is also attested by the fact that it resulted in larger improvements. Further investigation of the sources of these improvements will illustrate the advantages and disadvantages of these parsers and representations, leading us to better parsing models and a better design for parse representations.
This figure demonstrates that increasing the size of the parser training set contributes to increasing parse accuracy. Training the parser with only sentences results in parse accuracy of about Accuracy rises sharply with additional training data until the size of the training set reaches about sentences about From there, accuracy climbs consistently, but slowly, until Parser training set size number of sentences versus parse accuracy and PPI extraction accuracy f -score.
Figure 10 also shows the relationship between the amount of parser training data and the accuracy of PPI extraction. This shows that the accuracy of PPI extraction generally increases with the use of more sentences to train the parser.
AAAI 12222 Highlights: Dialogue, reproducibility, and more
Although it may appear that further increasing the training data for the parser may not improve the PPI extraction accuracy, we see that the two curves match each other to a large extent. This is supported by the strong correlation between parse accuracy and PPI accuracy, shown in Figure PPI extraction experiments on AIMed have been reported repeatedly, although the figures cannot be compared directly because of the differences in data preprocessing and the number of target protein pairs Airola et al.
Table 4 compares our best result with previously reported accuracy figures. Among these, Giuliano et al. Bunescu and Mooney applied SVMs with subsequence kernels to the same task, although they provided only a precision—recall graph, and its maximum f -score is around Since we did not run experiments on protein-pairwise cross-validation, our system cannot be compared directly to the results reported by Erkan et al. We have presented our attempts to evaluate contributions of natural language parsers and their representations to PPI extraction.
The basic idea is to measure the accuracy improvements of the PPI extraction task by incorporating the parser output as statistical features of a machine learning classifier. Experiments showed that state-of-the-art parsers improved PPI extraction accuracy, and the obtained accuracy is better than previously reported accuracy on the same data. These parsers attain accuracy levels that are on par with each other, while parsing speed differs considerably. A shortcoming of our experiments is that there is no guarantee that the results obtained with our PPI extraction system can be generalized to other dataset and tasks.
Such evaluations are indispensable for a more general understanding of the performance characteristics of different parsers in specific applications in bioinformatics, and our methodology provides a template for how these evaluations may be conducted. Oxford University Press is a department of the University of Oxford.
Related Coarse-to-Fine Natural Language Processing (Theory and Applications of Natural Language Processing)
Copyright 2019 - All Right Reserved