HomeЛюди и блогиRelated VideosMore From: CodeEmporium

Build a Text Summarizer in Java

82 ratings | 6076 views
Get the Code here : https://github.com/ajhalthor/text-summarizer Follow me on Twitter : https://twitter.com/ajhalthor Take a look at the original by Shlomi Babluki : http://thetokenizer.com/2013/04/28/build-your-own-summary-tool/ TRANSCRIPT OVERVIEW ALGORITHM 1. Take the full CONTENT and split it into PARAGRAPHS. 2. Split each PARAGRAPH into SENTENCES. 3. Compare every sentence with every other. This is done by Counting the number of common words and then Normalize this by dividing by average number of words per sentence. 4. These intermediate scores/values are stored in an INTERSECTION matrix 5. Create the key-value dictionary - Key : Sentence - Value : Sum of intersection values with this sentence 6. From every paragraph, extract the sentences with the highest score. 7. Sort the selected sentences in order of appearance in the original text to preserve content and meaning. And like that, you have generated a summary of the original text. CLASSES IN JAVA PROJECT 1. Sentence : The entire text is divided into a number of paragraphs and each paragraph is divided into a number of sentences. 2. Paragraph : Every paragraph has a number associated with it and an Array List of sentences. 3. Sentence Comparitor : Compare Sentence objects based on Score 4. SentenceComparatorForSummary : Compare Sentence objects based on position in text. 5. SummayTool : akes care of all the operations from extracting sentences to generating the summary. HOW IS MY SUMMARIZER BETTER THAN THE ORIGINAL ? My text summarizer selects number of sentences from a paragraph depending on the length. This is an improvement over the original text summarizer implementation that only selects 1 sentence per paragraph regardless of length. So, If the author decides to crunch everything into 1 paragraph, then only one sentence would be chosen. In the current implementation, we set it to accept several sentences for larger paragraphs. It delivers cogent summaries for general essays, reviews and publications. RUN THIS PROGRAM $ javac -d bin improved_summary.java $ java -classpath bin improved_summary
Html code for embedding videos on your blog
Text Comments (21)
Umang Mavani (10 days ago)
well its working for normal passages .....but it fails to generate summary for passages with only dialogues in them.....so how do I generate summary for this type of passages?? plz help for this doubt of mine.
Roie Zemel (1 month ago)
Thank you! The video helped me a lot! But where did the "compareToIgnoreCase" method come from? And what does she do?
Roie Zemel (1 month ago)
Never mind, I got it :)
Nitika Katyal (2 months ago)
tu bhagvaan hai !!
apoorva pandey (6 months ago)
I need this project .
ASHWANI KUMAR (7 months ago)
how did you evaluate your summarization system?
kushal pokharel (10 months ago)
What is that '5' called ??? Term wise is it a thresold ???
Youssef lahia (10 months ago)
thanks bro it's working but i would like to extract the key Words
CodeEmporium (10 months ago)
Glad you liked it Youssef! (and thanks for subscribing!) Although I haven't created a video on it, I have an auto key phrase extractor in my github repository. Check it out here: https://github.com/ajhalthor/auto-keyphrase-extraction
nithya krish (1 year ago)
itz not working bro!
CodeEmporium (1 year ago)
Which file are you summarizing? Did you try the sample files provided?
nithya krish (1 year ago)
once i execute the code, the entire content of the input file display in the output konsole it won't summarize the content!
CodeEmporium (1 year ago)
Hey nithya. What's the error?
muzamil shah (1 year ago)
not working here
muzamil shah (1 year ago)
now working
CodeEmporium (1 year ago)
Try changing the path of the sample file nexus_6p.txt to the current directory and get rid of the path part of line 37. Not really much else to say here. The source of the error can only be that line.
muzamil shah (1 year ago)
even i have file in perfect locaiton
muzamil shah (1 year ago)
java.io.FileNotFoundException: samples/amazon/nexus_6p.txt (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at java.io.FileInputStream.<init>(FileInputStream.java:93) at SummaryTool.init(SummaryTool.java:37)
CodeEmporium (1 year ago)
What error do you get?
Manraj Singh (1 year ago)
hlo bro i want your help in programing plz contact me on my whatsapp no:+91 95017 49182 or send me your whatsapp no.
CodeEmporium (1 year ago)
You can mail me at : [email protected] It's a lot easier that way

Would you like to comment?

Join YouTube for a free account, or sign in if you are already a member.