Initialising FREME

This article explains how to initialise the FREME database.

Prepare the database

The database connection is defined in the `application.properties` file, see FREME configuration options.

To add the default FREME entries to the tables, execute the following MySQL scripts at your FREME database.

Templates: add all FREME Templates
Sparql Converters (Filters): add all FREME sparql filters
XSLT Converters for H2 or MySQL database: add all FREME XSLT converters
Pipelines: add pipelines

Alternativ you can add the data to your database using Hibernate. Hibernate provides the possibility to autoload content into the database when the database schema is created. To autoload the FREME entities mentioned above, do the following:

Put the templates script, the sparql converters script, the xslt converters script for H2 database (or xslt converters script for MySQL database ) and this init-admin script into your FREME config folder inside target/freme-package. The init-admin script is necessary to meet the foreign key requirements of the other tables. Note that this overwrites `admin.username` and `admin.password` in your `application.properties` file. If you start FREME from within your IDE make sure that the three added files are on the classpath.

Add the following entries to your `application.properties` file if you use the H2 (default) database backend:

spring.jpa.properties.hibernate.hbm2ddl.import_files=init-admin.sql, init-sparqlconverter.sql, init-xsltconverter.h2.sql, init-template.sql
spring.jpa.properties.hibernate.hbm2ddl.import_files_sql_extractor=org.hibernate.tool.hbm2ddl.MultipleLinesSqlCommandExtractor
spring.jpa.hibernate.ddl-auto=create

or this, if you use MySQL as database backend:

spring.jpa.properties.hibernate.hbm2ddl.import_files=init-admin.sql, init-sparqlconverter.sql, init-xsltconverter.mysql.sql, init-template.sql
spring.jpa.properties.hibernate.hbm2ddl.import_files_sql_extractor=org.hibernate.tool.hbm2ddl.MultipleLinesSqlCommandExtractor
spring.jpa.hibernate.ddl-auto=create

Start FREME
Remove the added entries from your `application.properties` file to prevent hibernate from re-creating the databases (and dropping its previous content) when you restart it.

NOTE: Change the admin password to a secure one. The script init-admin.sql sets it to "admin". You can overwrite the password at startup via the application.properties, see the parameters admin.create, admin.username, admin.password at the FREME Configuration Options article, or by updating the password afterwards via a REST PUT request to /users, see How to change the password of a user.

Prepare the data needed for FREME-NER linking

If you want to use the linking method of FREME-NER in your local FREME installation, you have to provide the datasets for linking. They are stored in the Solr Server and the Virtuoso Triple Store. Further, we need to provide the metadata of the datasets in the MySql Database.

This paragraph explains how to upload the data when you have already installed Virtuoso and Solr. If you haven't done this, you can find some instructions in the FREME-NER article.

Load Metadata into the MySql Database

FREME standard distribution

    INSERT INTO `dataset_metadata` (`visibility`, `owner_name`, `creation_time`, `description`, `name`, `total_entities`) VALUES
(1,'admin',1435966058019,'DBpedia datasets for all languages','dbpedia',12382158),
(1,'admin',1471102054379,'Europeana.eu is an internet portal that acts as an interface to millions of books, paintings, films, museum objects and archival records that have been digitised throughout Europe.','europeana',1431968);

FREME full distribution

        INSERT INTO `dataset_metadata` (`visibility`, `owner_name`, `creation_time`, `description`, `name`, `total_entities`) VALUES
(1,'admin',1447055977285,'CORIS FP7 dataset that can be used for entity linking of entities representing projects. The dataset also can be used to link to people involved in those projects or organizations.','cordis-fp7',336762),
(1,'admin',1435966058019,'DBpedia datasets for all languages','dbpedia',12382158),
(1,'admin',1442404750691,'Organization Name Linked Data (http://www.lib.ncsu.edu/ld/onld/)','onld',5132),
(1,'admin',1442405451333,'Geopolitical ontology (http://www.fao.org/countryprofiles/geoinfo/en/)','geopolitical',4328),
(1,'admin',1442580539219,'A dataset of global airports featuring their names and unique IATA/ICAO identifiers.','global_airports',18578),
(1,'admin',1444922284386,'The VIAF® (Virtual International Authority File) dataset is a collection of multiple name authority files. (http://viaf.org/)','viaf',7260000),
(1,'admin',1444922402140,'The ORCID dataset provides persistent digital identifier for researchers (http://orcid.org/)','orcid',1320778);

Load data into the Virtuoso Triple Store

Stop virtuoso:

          /usr/local/virtuoso-opensource/bin/isql 1111 dba
          in mysql shell: shutdown();

Move the content of the virtuoso folder inside the standard (or full) unpacked Freme distribution folder to the database folder inside the virtuoso installation folder, e.g.:
```
mv ~/freme-install/downloads/virtuoso/* /usr/local/virtuoso-opensource/var/lib/virtuoso/db/
```

Start virtuoso from inside the db folder:

          cd /usr/local/virtuoso-opensource/var/lib/virtuoso/db/
          sudo nohup /usr/local/virtuoso-opensource/bin/virtuoso-t &

Now you should see data from dbpedia for instance when you run the default query in the web interface .

Load data into the Solr Server

Download one of the two FREME-distribtuions: FREME standard or FREME full (contains more preinitialized data):

 wget http://api.freme-project.eu/data/dumps/freme-standard.tar.gz

 wget http://api.freme-project.eu/data/dumps/freme-full.tar.gz

Create e-linker core:

unpack the data and go to the solr installation folder from where you can start the solr server, e.g.:

        tar xvf freme-standard.tar.gz
        cd ~/freme-install/solr-5.5.2
          /bin/solr start

create core, stop solr, move the elinker data inside the unpacked FREME distribution folder to the new core and restart the solr server:

          /bin/solr create_core -c elinker
          /bin/solr stop -all
          rm -rf server/solr/elinker/
          mv ~/freme-install/downloads/solr/elinker/ server/solr/elinker
          bin/solr start

It might take a few minutes until you can use the new core.
Add Tilde token to FREME config:

go to cloned freme-package folder and edit the application.properties file, e.g.:

        cd ~/freme-install/freme-package
        vim config/application.properties

for now you can use the default token, in the future you might need to use your own Tilde Token.

        tilde.translation.authentication=Basic RlJFTUU6dXxGcjNtM19zJGN1ciQ=
        tilde.terminology.authentication=Basic RlJFTUU6dXxGcjNtM19zJGN1ciQ=

Restart Freme.

        cd ~/freme-install/freme-package
        bin/stop_server.sh