Readme.md 4.27 KB
Newer Older
Bertrand Goupil's avatar
Bertrand Goupil committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
# Limsi-SourceExtractor

## Abstract

This is a code refactoring of Limsi's source extractor program in order to expose source extraction as a web service.
This is a Spring Boot application deployed in a Docker image. 

## Project requirement
This is a Java/Spring boot project. All dependencies are setup in a Maven pom.
On Macos X or Windows workstation, Docker should be install [https://www.docker.com/docker-toolbox](https://www.docker.com/docker-toolbox)

+ Java Oracle JDK8 [http://www.oracle.com/technetwork/java/javase/downloads/index.html](http://www.oracle.com/technetwork/java/javase/downloads/index.html)
+ Apache Maven [https://maven.apache.org/download.cgi](https://maven.apache.org/download.cgi)
+ Docker [https://www.docker.com/docker-toolbox](https://www.docker.com/docker-toolbox)



## Run and Build project
### Run localy
+ Change `application.yml` in `PROJECT_HOME/Limsi-SourceExtractor/src/test/configuration` with local lib and resources path (change PROJECT_HOME). Required absolute path.

    resource: 
     #Directory containing the librairies and models
     lib: PROJECT_HOME/Limsi-SourceExtractor/lib
     #Directory containing the language-dependent resources
     resources: PROJECT_HOME/Limsi-SourceExtractor/resources

+ `mvn clean install`
+ Goto to `target` directory
+ Run in a terminal `java -Dspring.config.location=../src/test/configuration/ -jar limsiSourceExtractor.jar` 

If running from IDE (eclipse), VM argument `java -Dspring.config.location=src/test/configuration/` must be added in the "Run configuration".

#### Troubleshooting
+ Application do not find `lib` and `resources` path at startup:
 
	=> check that application.yml set the correct path for lib and resources
	=> -Dspring.configuration path should end with /
39 40
+ Wapiti OS libraries may crash the webapp: see Wapiti build section below

Bertrand Goupil's avatar
Bertrand Goupil committed
41 42 43 44 45 46 47 48
#### Bug
+ Service stop on MacosX during extraction with wapiti's message `error: invalid model format` (This is not happening running inside eclipse ...)

### Run with Docker
####Use default resources
Models and resources are embedded by default in the docker image.
+ Build docker image `mvn clean package docker:build`
+ Start default image `docker run -p 8080:8080 -t limsi-source-extractor`
Bertrand Goupil's avatar
Bertrand Goupil committed
49
+ Start named default image `docker run --name limsi-extractor -p 8080:8080 -t limsi-source-extractor`
Bertrand Goupil's avatar
Bertrand Goupil committed
50

51 52 53
Note: Memory usage can be setup with JAVA_OPTS in the docker command line:
 `docker run --name limsi-extractor -p 8080:8080 -e JAVA_OPTS='-Xmx3g' -d limsi-source-extractor`
 
Bertrand Goupil's avatar
Bertrand Goupil committed
54
####Custom resources/lib
Bertrand Goupil's avatar
Bertrand Goupil committed
55
Build image can be used with new training set. In this case it is possible to overwrite default model configuration.
Bertrand Goupil's avatar
Bertrand Goupil committed
56 57 58 59

In the docker image volumes have been setup to mount new resources.
+ `lib` folder should be mount to `/configuration/lib`
+ `resources` folder should be mount to `/configuration/resources`
Bertrand Goupil's avatar
Bertrand Goupil committed
60
+ Configuration file from PROJECT_HOME/Limsi-SourceExtractor/configuration/ that define `application.yml`  with docker Volumes must be use.
Bertrand Goupil's avatar
Bertrand Goupil committed
61 62 63 64 65 66 67 68 69 70 71

+ Run docker image with mount resources:

    docker run -p 8080:8080 -v /PROJECT_HOME/Limsi-SourceExtractor/configuration:/configuration -v /PROJECT_HOME/Limsi-SourceExtractor/lib:/configuration/lib -v /PROJECT_HOME/Limsi-SourceExtractor/resources:/configuration/resources -t limsi-source-extractor

#### Docker cheat sheet

+ List images `docker images`
+ List containers `docker ps -a`
+ Stop container `docker stop <container_name>`
+ Start container `docker start <container_name>`
72
+ Show container logs `docker logs <container_name>`
Bertrand Goupil's avatar
Bertrand Goupil committed
73 74 75 76
+ Remove stop containers `docker rm $(docker ps -a -q)`
+ Remove image `docker rmi <image_name>`
+ Load image from file `docker load < image_file.tar`
+ Export image `docker export > image_file.tar`
77
+ Set environment variables  `docker run -e JAVA_OPTS=`
Bertrand Goupil's avatar
Bertrand Goupil committed
78

79

Bertrand Goupil's avatar
Bertrand Goupil committed
80 81 82 83 84 85 86 87 88 89 90
## Test
HTTP GET http://localhost:8080/sourceExtractor/

    =>HTTP 200 "Greetings from Spring Boot!"
    
HTTP POST http://localhost:8080/sourceExtractor/extractNewsML
Charset text/xml

    curl -XPOST -H "Content-Type: text/xml" -d @afp.com-20150123T153420Z-TX-PAR-SZS52.xml http://localhost:8080/sourceExtractor/extractNewsML


91 92 93 94 95 96 97 98
## Annex
### Third-party libraries

#### Wapiti build
Wapiti OS libraries may failed depending of your OS version.
You must then build the OS library from the Wapiti project: [https://github.com/kermitt2/Wapiti](https://github.com/kermitt2/Wapiti)