Neural Text Summarization
Piji Li
NLP Center, Tencent AI [email protected]
Paper Reading, Sep.6, 2018
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 1 / 63
Table of Contents
1 Introduction
2 Methods
3 Conclusion
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 2 / 63
Table of Contents
1 Introduction
2 Methods
3 Conclusion
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 3 / 63
IntroductionText Summarization
The goal of automatic text summarization is to automatically producea succinct summary, preserving the most important information for asingle document or a set of documents about the same topic (event).
7/11/2017 mogren.one/graphics/illustrations/mogren_summarization.svg
http://mogren.one/graphics/illustrations/mogren_summarization.svg 1/1
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 4 / 63
IntroductionText Summarization - Categories
Input:
Single-Document Summarization (SDS)Multi-Document Summarization (MDS)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 5 / 63
IntroductionSingle-Document Summarization
Cambodian leader Hun Sen on Friday rejected opposition parties 'demands for talks outside the country , accusing them of trying to ``internationalize '' the political crisis .Government and opposition parties have asked King NorodomSihanouk to host a summit meeting after a series of post-electionnegotiations between the two opposition groups and Hun Sen 's party toform a new government failed .Opposition leaders Prince Norodom Ranariddh and Sam Rainsy , citingHun Sen 's threats to arrest opposition figures after two alleged attemptson his life , said they could not negotiate freely in Cambodia and calledfor talks at Sihanouk 's residence in Beijing .Hun Sen , however ,rejected that .``I would like to make it clear that all meetings related to Cambodianaffairs must be conducted in the Kingdom of Cambodia , '' Hun Sentold reporters after a Cabinet meeting on Friday .`` No-one shouldinternationalize Cambodian affairs .It is detrimental to the sovereignty of Cambodia , '' he said .Hun Sen 'sCambodian People 's Party won 64 of the 122 parliamentary seats inJuly 's elections , short of the two-thirds majority needed to form agovernment on its own .Ranariddh and Sam Rainsy have charged thatHun Sen 's victory in the elections was achieved through widespreadfraud .They have demanded a thorough investigation into their electioncomplaints as a precondition for their cooperation in getting thenational assembly moving and a new government formed …….
Cambodian government rejects opposition's call for talks abroad
Document
Summary
Figure 1: Single-document summarization.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 6 / 63
IntroductionMulti-Document Summarization
Fingerprints and photos of two men who boarded the doomed Malaysia Airlines passenger jet are
being sent to U.S. authorities so they can be compared against records of known terrorists and
criminals. The cause of the plane's disappearance has baffled investigators and they have not said
that they believed that terrorism was involved, but they are also not ruling anything out. The
investigation into the disappearance of the jetliner with 239 passengers and crew has centered so
far around the fact that two passengers used passports stolen in Thailand from an Austrian and an
Italian. The plane which left Kuala Lumpur, Malaysia, was headed for Beijing. Three of the
passengers, one adult and two children, were American. ……
(CNN) -- A delegation of painters and calligraphers, a group of Buddhists returning from a
religious gathering in Kuala Lumpur, a three-generation family, nine senior travelers and five
toddlers. Most of the 227 passengers on board missing Malaysia Airlines Flight 370 were Chinese,
according to the airline's flight manifest. The 12 missing crew members on the flight that
disappeared early Saturday were Malaysian. The airline's list showed the passengers hailed from 14
countries, but later it was learned that two people named on the manifest -- an Austrian and an
Italian -- whose passports had been stolen were not aboard the plane. The plane was carrying five
children under 5 years old, the airline said. ……
Vietnamese aircraft spotted what they suspected was one of the doors belonging to the ill-fated
Malaysia Airlines Flight MH370 on Sunday, as troubling questions emerged about how two
passengers managed to board the Boeing 777 using stolen passports. The discovery comes as
officials consider the possibility that the plane disintegrated mid-flight, a senior source told Reuters.
The state-run Thanh Nien newspaper cited Lt. Gen. Vo Van Tuan, deputy chief of staff of Vietnam's
army, as saying searchers in a low-flying plane had spotted an object suspected of being a door
from the missing jet. It was found in waters about 56 miles south of Tho Chu island, in the same
area where oil slicks were spotted Saturday. ……
…
Flight MH370, carrying 239
people vanished over the
South China Sea in less than
an hour after taking off from
Kuala Lumpur, with two
passengers boarded the
Boeing 777 using stolen
passports. Possible reasons
could be an abrupt breakup of
the plane or an act of
terrorism. The government
was determining the "true
identities" of the passengers
who used the stolen passports.
Investigators were trying to
determine the path of the
plane by analysing civilian
and military radar data while
ships and aircraft from seven
countries scouring the seas
around Malaysia and south of
Vietnam.
Documents Summary
Figure 2: Multi-document summarization for the topic “Malaysia AirlinesDisappearance”.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 7 / 63
IntroductionText Summarization - Categories
Input:
Single-Document Summarization (SDS)Multi-Document Summarization (MDS)
Output:
ExtractiveCompressiveAbstractive
Machine learning methods:
SupervisedUnsupervised
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 8 / 63
IntroductionText Summarization - History
Since 1950s:
Concept Weight (Luhn, 1958), Centroid (Radev et al., 2004), LexRank(Erkan and Radev, 2004), TextRank (Mihalcea and Tarau, 2004),Sparse Coding (He et al., 2012; Li et al., 2015)Feature+Regression (Min et al., 2012; Wang et al., 2013)
Most of the summarization methods are extractive.
Abstractive summarization is full of challenges. Some indirectmethods employ sentence fusing (Barzilay and McKeown, 2005) orphrase merging (Bing et al., 2015).
The indirect strategies will do harm to the linguistic quality of theconstructed sentences.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 9 / 63
IntroductionText Summarization - History
Before the neural summarization era...silent
2012
2015 (Rush et al., 2015)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 10 / 63
Table of Contents
1 Introduction
2 Methods
3 Conclusion
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 11 / 63
MethodsEssential Idea
Salience Detection (Words, Sentences)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 12 / 63
MethodsInspiration from DBN, DNN, CNN
Liu, Yan, Sheng-hua Zhong, andWenjie Li. “Query-OrientedMulti-Document Summarizationvia Unsupervised Deep Learn-ing.” In AAAI. 2012.
Denil, Misha, Alban Demiraj, NalKalchbrenner, Phil Blunsom, andNando de Freitas. “Modelling,visualising and summarising doc-uments with a single convolu-tional neural network.” arXivpreprint arXiv:1406.3830 (2014).
Figure 3: Visualization of Parameters.Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 13 / 63
MethodsBetter Semantic Representations
Since 1950s:
Concept Weight (Luhn, 1958), Centroid (Radev et al., 2004), LexRank(Erkan and Radev, 2004), TextRank (Mihalcea and Tarau, 2004),Sparse Coding (He et al., 2012; Li et al., 2015)
Bag-of-Words (BoWs)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 14 / 63
MethodsBetter Semantic Representations
Word2vec (Mikolov et al., 2013), Paragraph Vector (Le and Mikolov,2014), RNN-Sent (Tang et al., 2015), CNN-Sent (Kim, 2014)
Improve the performance of PageRank and Data Reconstructionbased models.
Works:
Kageback, Mikael, Olof Mogren, Nina Tahmasebi, and Devdatt Dub-hashi. “Extractive summarization using continuous vector spacemodels.” In CVSC 2014.Yin, Wenpeng, and Yulong Pei. ”Optimizing Sentence Modeling andSelection for Document Summarization.” In IJCAI 2015.Li, Piji, Wai Lam, Lidong Bing, Weiwei Guo, and Hang Li. ”Cascadedattention based unsupervised information distillation for compres-sive summarization.” In EMNLP 2017.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 15 / 63
MethodsInspiration from NMT
Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. ”Neuralmachine translation by jointly learning to align and translate.”arXiv preprint arXiv:1409.0473 (2014). (citation:4300+)
Figure 4: Attention-based seq2seq framework. Figure from OpenNMT (Kleinet al., 2017)
.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 16 / 63
Methods
2015
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 17 / 63
MethodsA Neural Attention Model for Abstractive Sentence Summarization
Rush, Alexander M., Sumit Chopra, and Jason Weston. ”A neuralattention model for abstractive sentence summarization.”EMNLP (2015). (citation:570+)
Figure 5: (a) NNLM decoder with additional encoder element. (b) Attentionbased encoder.
.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 18 / 63
MethodsA Neural Attention Model for Abstractive Sentence Summarization
Rush, Alexander M., Sumit Chopra, and Jason Weston. ”A neuralattention model for abstractive sentence summarization.”EMNLP (2015). (citation:570+)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 19 / 63
MethodsLCSTS: A Large Scale Chinese Short Text Summarization Dataset
Hu, Baotian, Qingcai Chen, and Fangze Zhu. ”LCSTS: A LargeScale Chinese Short Text Summarization Dataset.” EMNLP(2015). (citation:49)
(a) (b)
Figure 6: (a) Encoder-Decoder. (b) Attention based Decoder.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 20 / 63
MethodsLCSTS: A Large Scale Chinese Short Text Summarization Dataset
Hu, Baotian, Qingcai Chen, and Fangze Zhu. ”LCSTS: A LargeScale Chinese Short Text Summarization Dataset.” EMNLP(2015). (citation:49)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 21 / 63
MethodsGenerating News Headlines with Recurrent Neural Networks
Lopyrev, Konstantin. ”Generating news headlines with recurrentneural networks.” arXiv preprint arXiv:1512.01712 (2015).(citation:28)
Investigations of several NMT models.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 22 / 63
Methods
2016
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 23 / 63
MethodsAbstractive sentence summarization with attentive recurrent neural networks
Chopra, Sumit, Michael Auli, and Alexander M. Rush. ”Abstractivesentence summarization with attentive recurrent neuralnetworks.” NAACL, pp. 93-98. 2016. (citation:138)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 24 / 63
MethodsAbstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, Ramesh, Bowen Zhou, Cicero dos Santos, Ca glar Gulcehre,and Bing Xiang. ”Abstractive Text Summarization usingSequence-to-sequence RNNs and Beyond.” CoNLL 2016 (2016):280. (citation:183)
3 pages version in Feb. 2016.
Many tricks (nmt, copy, coverage, hierarchical, external knowledge).
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 25 / 63
MethodsAbstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, Ramesh, Bowen Zhou, Cicero dos Santos, Ca glar Gulcehre,and Bing Xiang. ”Abstractive Text Summarization usingSequence-to-sequence RNNs and Beyond.” CoNLL 2016 (2016):280. (citation:183)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 26 / 63
MethodsAbstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, Ramesh, Bowen Zhou, Cicero dos Santos, Ca glar Gulcehre,and Bing Xiang. ”Abstractive Text Summarization usingSequence-to-sequence RNNs and Beyond.” CoNLL 2016 (2016):280. (citation:183)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 27 / 63
MethodsAbstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, Ramesh, Bowen Zhou, Cicero dos Santos, Ca glar Gulcehre,and Bing Xiang. ”Abstractive Text Summarization usingSequence-to-sequence RNNs and Beyond.” CoNLL 2016 (2016):280. (citation:183)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 28 / 63
MethodsAbstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, Ramesh, Bowen Zhou, Cicero dos Santos, Ca glar Gulcehre,and Bing Xiang. ”Abstractive Text Summarization usingSequence-to-sequence RNNs and Beyond.” CoNLL 2016 (2016):280. (citation:183)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 29 / 63
MethodsWhy Copy?
OOV
Extraction
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 30 / 63
MethodsCopy Mechanism
Vinyals, Oriol, Meire Fortunato, and Navdeep Jaitly. ”Pointer networks.”In NIPS, pp. 2692-2700. 2015. (citation:352)
Gulcehre, Caglar, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, andYoshua Bengio. ”Pointing the Unknown Words.” In ACL, vol. 1,pp. 140-149. 2016. (citation:126)
Gu, Jiatao, Zhengdong Lu, Hang Li, and Victor OK Li.”Incorporating Copying Mechanism in Sequence-to-SequenceLearning.” In ACL, vol. 1, pp. 1631-1640. 2016. (citation:192)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 31 / 63
MethodsCopy Mechanism
Figure 7: Pointer-generator model. (See et al., 2017)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 32 / 63
MethodsCopy Mechanism – Performance
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 33 / 63
MethodsWhy Coverage?
Diversity
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 34 / 63
MethodsCoverage Mechanism
Tu, Zhaopeng, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. ”ModelingCoverage for Neural Machine Translation.” In ACL 2016. (citation:187)
Chen, Qian, Xiaodan Zhu, Zhenhua Ling, Si Wei, and Hui Jiang.”Distraction-based neural networks for modeling documents.” InIJCAI 2016. (citation:28)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 35 / 63
MethodsCoverage Mechanism
Chen, Qian, Xiaodan Zhu, Zhenhua Ling, Si Wei, and Hui Jiang.”Distraction-based neural networks for modeling documents.” InIJCAI 2016. (citation:28)
Figure 8: Operation of coverage mechanism.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 36 / 63
MethodsCoverage Mechanism – Performance
Chen, Qian, Xiaodan Zhu, Zhenhua Ling, Si Wei, and Hui Jiang.”Distraction-based neural networks for modeling documents.” InIJCAI 2016. (citation:28)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 37 / 63
MethodsMore Works in 20161
Cheng, Jianpeng, and Mirella Lapata. ”Neural Summarization byExtracting Sentences and Words.” In ACL, 2016. (citation:108)
Cao, Ziqiang, Wenjie Li, Sujian Li, Furu Wei, and Yanran Li.”AttSum: Joint Learning of Focusing and Summarization with NeuralAttention.” In COLING, 2016.
Zeng, Wenyuan, Wenjie Luo, Sanja Fidler, and Raquel Urtasun.”Efficient summarization with read-again and copy mechanism.”arXiv preprint arXiv:1611.03382 (2016).
Miao, Yishu, and Phil Blunsom. ”Language as a Latent Variable:Discrete Generative Models for Sentence Compression.” In EMNLP.2016.
...
1https://github.com/lipiji/App-DL#text-summarizationPiji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 38 / 63
Methods
2017
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 39 / 63
Methods
Inspirations from the traditional summarization methods.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 40 / 63
Methods
Nallapati, Ramesh, Feifei Zhai, and Bowen Zhou. ”SummaRuNNer:A Recurrent Neural Network Based Sequence Model forExtractive Summarization of Documents.” In AAAI, pp.3075-3081. 2017. (citation:58)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 41 / 63
MethodsAbstractive document summarization with a graph-based attentional neural model
Tan, Jiwei, Xiaojun Wan, and Jianguo Xiao. ”Abstractivedocument summarization with a graph-based attentional neuralmodel.” In ACL 2017. (citation:24)
ACL Outstanding Paper.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 42 / 63
MethodsAbstractive document summarization with a graph-based attentional neural model
Tan, Jiwei, Xiaojun Wan, and Jianguo Xiao. ”Abstractivedocument summarization with a graph-based attentional neuralmodel.” In ACL 2017. (citation:24)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 43 / 63
MethodsSelective Encoding for Abstractive Sentence Summarization
Zhou, Qingyu, Nan Yang, Furu Wei, and Ming Zhou. ”SelectiveEncoding for Abstractive Sentence Summarization.” In ACL2017. (citation:24)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 44 / 63
Methods
Recall the Copy and Coverage Mechanism in 2016.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 45 / 63
MethodsSelective Encoding for Abstractive Sentence Summarization
See, Abigail, Peter J. Liu, and Christopher D. Manning. ”Get ToThe Point: Summarization with Pointer-Generator Networks.”In ACL 2017. (citation:114)Writing? Figures?
Figure 9: Pointer-Generator Networks.Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 46 / 63
Methods
Reinforcement Learning.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 47 / 63
MethodsA deep reinforced model for abstractive summarization
Paulus, Romain, Caiming Xiong, and Richard Socher. ”A deepreinforced model for abstractive summarization.” arXiv preprintarXiv:1705.04304 (2017). (citation:107)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 48 / 63
MethodsA deep reinforced model for abstractive summarization
Intra-attention modeling.
Reinforced learning trick.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 49 / 63
MethodsA deep reinforced model for abstractive summarization
Paulus, Romain, Caiming Xiong, and Richard Socher. ”A deepreinforced model for abstractive summarization.” arXiv preprintarXiv:1705.04304 (2017). (citation:107)
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 50 / 63
Methods
2018
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 51 / 63
MethodsReinforcement Learning based Methods
Celikyilmaz, Asli, Antoine Bosselut, Xiaodong He, and Yejin Choi.”Deep Communicating Agents for Abstractive Summarization.”In NAACL 2018.
Wu, Yuxiang, and Baotian Hu. ”Learning to Extract CoherentSummary via Deep Reinforcement Learning.” In AAAI 2018.
Wang, Li, Junlin Yao, Yunzhe Tao, Li Zhong, Wei Liu, and Qiang Du.”A reinforced topic-aware convolutional sequence-to-sequencemodel for abstractive text summarization.” In IJCAI 2018.
Chen, Yen-Chun, and Mohit Bansal. ”Fast AbstractiveSummarization with Reinforce-Selected Sentence Rewriting.”arXiv preprint arXiv:1805.11080 (2018).
Keneshloo, Yaser, Tian Shi, Chandan K. Reddy, and NarenRamakrishnan. ”Deep Reinforcement Learning For Sequence toSequence Models.” arXiv preprint arXiv:1805.09461 (2018).
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 52 / 63
MethodsCNN-seq2seq, Transformer
Wang, Li, Junlin Yao, Yunzhe Tao, Li Zhong, Wei Liu, and Qiang Du.”A reinforced topic-aware convolutional sequence-to-sequencemodel for abstractive text summarization.” In IJCAI 2018.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 53 / 63
MethodsRecent Works
Wojciech Kryscinski, Romain Paulus, Caiming Xiong, Richard Socher.”Improving Abstraction in Text Summarization .” arXiv preprintarXiv:1808.07913 (2018).
Zhang, Xingxing, Mirella Lapata, Furu Wei, and Ming Zhou. ”NeuralLatent Extractive Document Summarization.” arXiv preprintarXiv:1808.07187 (2018).
Sebastian Gehrmann, Yuntian Deng, Alexander M. Rush. ”Bottom-UpAbstractive Summarization.” arXiv preprint arXiv:1808.10792 (2018).
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 54 / 63
Methods
More:
https://github.com/lipiji/App-DL#text-summarization
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 55 / 63
Table of Contents
1 Introduction
2 Methods
3 Conclusion
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 56 / 63
Conclusion
Challenges:
Long text abstractive summarization.Abstractive multi-document summarization.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 57 / 63
Thanks a lot!Q & A
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 58 / 63
References I
Regina Barzilay and Kathleen R McKeown. Sentence fusion for multi-document news summarization. Computational Linguistics, 31(3):297–328, 2005.
Lidong Bing, Piji Li, Yi Liao, Wai Lam, Weiwei Guo, and Rebecca Passon-neau. Abstractive multi-document summarization via phrase selection andmerging. In Proceedings of the 53rd Annual Meeting of the Associationfor Computational Linguistics and the 7th International Joint Conferenceon Natural Language Processing (Volume 1: Long Papers), volume 1,pages 1587–1597, 2015.
Gunes Erkan and Dragomir R Radev. Lexrank: Graph-based lexical cen-trality as salience in text summarization. Journal of Artificial IntelligenceResearch, 22:457–479, 2004.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 59 / 63
References II
Zhanying He, Chun Chen, Jiajun Bu, Can Wang, Lijun Zhang, Deng Cai,and Xiaofei He. Document summarization based on data reconstruction.In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intel-ligence, pages 620–626. AAAI Press, 2012.
Yoon Kim. Convolutional neural networks for sentence classification. InProceedings of the 2014 Conference on Empirical Methods in NaturalLanguage Processing, pages 1746–1751, 2014.
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander MRush. Opennmt: Open-source toolkit for neural machine translation.arXiv preprint arXiv:1701.02810, 2017.
Quoc Le and Tomas Mikolov. Distributed representations of sentences anddocuments. In International Conference on Machine Learning, pages1188–1196, 2014.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 60 / 63
References III
Piji Li, Lidong Bing, Wai Lam, Hang Li, and Yi Liao. Reader-aware multi-document summarization via sparse coding. In The 24th InternationalJoint Conference on Artificial Intelligence, pages 1270–1276, 2015.
Hans Peter Luhn. The automatic creation of literature abstracts. IBMJournal of research and development, 2(2):159–165, 1958.
Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In Pro-ceedings of the 2004 conference on empirical methods in natural languageprocessing, 2004.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficientestimation of word representations in vector space. arXiv preprintarXiv:1301.3781, 2013.
Ziheng Lin Min, Yen Kan Chew, and Lim Tan. Exploiting category-specificinformation for multi-document summarization. The 21th InternationalConference on Computational Linguistics (COLING), pages 2903–2108,2012.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 61 / 63
References IV
Dragomir R Radev, Hongyan Jing, Ma lgorzata Stys, and Daniel Tam.Centroid-based summarization of multiple documents. Information Pro-cessing & Management, 40(6):919–938, 2004.
Alexander M Rush, Sumit Chopra, and Jason Weston. A neural attentionmodel for abstractive sentence summarization. In Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing, pages379–389, 2015.
Abigail See, Peter J Liu, and Christopher D Manning. Get to the point:Summarization with pointer-generator networks. In Proceedings of the55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers), volume 1, pages 1073–1083, 2017.
Duyu Tang, Bing Qin, and Ting Liu. Document modeling with gated re-current neural network for sentiment classification. In Proceedings of the2015 conference on empirical methods in natural language processing,pages 1422–1432, 2015.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 62 / 63
References V
Lu Wang, Hema Raghavan, Vittorio Castelli, Radu Florian, and ClaireCardie. A sentence compression based framework to query-focused multi-document summarization. In Proceedings of the 51st Annual Meeting ofthe Association for Computational Linguistics (Volume 1: Long Papers),volume 1, pages 1384–1394, 2013.
Piji Li (Tencent AI Lab) Neural Text Summarization Sep.6, 2018 63 / 63