Skip to content

Commit

Permalink
feedreader change. passing searchTerms= empty in config file returns …
Browse files Browse the repository at this point in the history
…fulll documents from the provided feed
  • Loading branch information
isanvicente committed Jan 24, 2020
1 parent 4ef1fe8 commit be83b64
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 8 deletions.
9 changes: 4 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,16 +105,15 @@ To install the module in the local maven repository, usually located in ~/.m2/,
mvn clean install
````

7. USING MSM
USING MSM
=========================


MSM Funcionalities
==========================
MSM provides 3 main funcionalities:
MSM provides 6 main funcionalities:
- **twitter**: Twitter Public stream crawling.
- **feed**: Syndication feed crawling (RSS, Atom, ...). Feed types supported by ROME tools (http://rometools.github.io/rome/)
- **influence**: looks for the influence of a given list of sources. Klout index for twitter users and PageRank for websites.
- **influence**: looks for the influence of a given list of sources. Klout index for twitter users and PageRank for websites. As of May 2018 Klout index is no longer available.

- **twtUser**: asks Twitter for the user profiles of a given list of Twitter users and return their follower and friend information.
- **langid**: Language detection for sentences. Used mainly to evaluate langid and optimaize.
- **geocode**: Geocoding wrapper for several geocoding APIs (access keys needed for some of them). Given a string it returns its geolocation coordinates.
Expand Down
13 changes: 10 additions & 3 deletions src/main/java/elh/eus/MSM/CLI.java
Original file line number Diff line number Diff line change
Expand Up @@ -291,9 +291,16 @@ public final void feedReader()
String kwrds = params.getProperty("searchTerms", "none");
if (!kwrds.equalsIgnoreCase("none"))
{
List<String> terms = Arrays.asList(params.getProperty("searchTerms").split(","));
kwrdList = Keyword.createFromList(terms,Arrays.asList(params.getProperty("langs", "all").split(",")));
System.err.println("MSM::TwitterStreamClient - retrieved "+kwrdList+" keywords from config file");
if (!kwrds.equalsIgnoreCase(""))
{
List<String> terms = Arrays.asList(params.getProperty("searchTerms").split(","));
kwrdList = Keyword.createFromList(terms,Arrays.asList(params.getProperty("langs", "all").split(",")));

System.err.println("MSM::FeedReader - retrieved "+kwrdList.size()+" keywords from config file");
}
else{
System.err.println("MSM::FeedReader - no keywords given , full articles will be returned");
}
}
// if no keyword is found in config file try to load them from the database as a last resort.
else{
Expand Down
4 changes: 4 additions & 0 deletions src/main/java/elh/eus/MSM/Keyword.java
Original file line number Diff line number Diff line change
Expand Up @@ -255,8 +255,12 @@ public static List<Keyword> getAnchors (List<Keyword> kwrds)
public static Set<Keyword> createFromList(List<String> keyList, List<String> langs)
{
Set<Keyword> result = new HashSet<Keyword>();
if (keyList.isEmpty()) {
return result;
}
for (String key : keyList)
{
//System.err.println("creating kword list: "+ key);
for (String l: langs)
{
Keyword k = new Keyword(key,l,false,false,true,key);
Expand Down

0 comments on commit be83b64

Please sign in to comment.