The contents of the module are the following:
+ pom.xml maven pom file which deals with everything related to compilation and execution of the module
+ src/ java source code of the module
+ Furthermore, the installation process, as described in the README.md, will generate another directory:
target/ it contains binary executable and other directories
Installing the elh-absa requires the following steps:
If you already have installed in your machine JDK7 and MAVEN 3, please go to step 3 directly. Otherwise, follow these steps:
- Install JDK 1.7
If you do not install JDK 1.7 in a default location, you will probably need to configure the PATH in .bashrc or .bash_profile:
export JAVA_HOME=/yourpath/local/java17
export PATH=${JAVA_HOME}/bin:${PATH}
If you use tcsh you will need to specify it in your .login as follows:
setenv JAVA_HOME /usr/java/java17
setenv PATH ${JAVA_HOME}/bin:${PATH}
If you re-login into your shell and run the command
java -version
You should now see that your jdk is 1.7
- Install MAVEN 3
Download MAVEN 3 from
wget http://apache.rediris.es/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
Now you need to configure the PATH. For Bash Shell:
export MAVEN_HOME=/home/myuser/local/apache-maven-3.0.5
export PATH=${MAVEN_HOME}/bin:${PATH}
For tcsh shell:
setenv MAVEN3_HOME ~/local/apache-maven-3.0.5
setenv PATH ${MAVEN3}/bin:{PATH}
If you re-login into your shell and run the command
mvn -version
You should see reference to the MAVEN version you have just installed plus the JDK 7 that is using.
- Get module source code
hg clone https://bitbucket.org/elh-eus/elh-crawler
- Installing using maven
cd MSM
mvn clean package
This step will create a directory called target/ which contains various directories and files. Most importantly, there you will find the module executable:
elh-MSM-1.0.jar
This executable contains every dependency the module needs, so it is completely portable as long as you have a JVM 1.7 installed.
To install the module in the local maven repository, usually located in ~/.m2/, execute:
mvn clean install
- USING MSM =========================
MSM provides 3 main funcionalities:
- twitter: Twitter Public stream crawling.
- feed: Syndication feed crawling (RSS, Atom, ...). Feed types supported by ROME tools (http://rometools.github.io/rome/)
- influence: looks for the influence of a given list of sources. Klout index for twitter users and PageRank for websites.
- twtUser: asks Twitter for the user profiles of a given list of Twitter users and return their follower and friend information.
- You can get general help by calling:
java -jar MSM-1.0.jar -help
- twitter: Twitter Public stream crawling. Call Example:
java -jar MSM-1.0.jar twitter -c config.cfg -s stout 2>> MSM-twitter.log
- feed: Syndication feed crawling (RSS, Atom, ...). Feed types supported by ROME tools (http://rometools.github.io/rome/)
java -jar MSM-1.0.jar feed -c config.cfg -u http://feeds2.feedburner.com/example
- influence: looks for the influence of a given list of sources. Klout index for twitter users and PageRank for websites.
java -jar MSM-1.0.jar influence -c config.cfg -db 2>> MSM-Influence.log
You can also generate the javadoc of the module by executing:
mvn javadoc:jar
Which will create a jar file core/target/elh-MSM-1.0-javadoc.jar
Iñaki San Vicente and Xabier Saralegi
Elhuyar Foundation
{i.sanvicente,x.saralegi}@elhuyar.com