If you want to simply test the system and quickly get going, please see the Quickstart section.

Customize preexisting VM's

A simple way to get an installation going is to clone and customize Martin's repos for your installation. These consist of three repos that are needed to set up the Archiver Appliance environment.
  1. vagrant_archiver_appliance
  2. puppet_module_archiver_appliance
  3. puppet_module_epics_softioc
Simply follow the rules in the README of the first repo and the other two repos will be pulled in automatically. The Puppet manifests are found in puppet_module_archiver_appliance.

Site specific installs

Han maintains a set of scripts for site specific installs. This is an excellent starting off point for folks who wish to build their own deployment bundles. This is tested against Debian/CentOS; but should be easily extensible for other distributions.

Using an install script

If you plan to have only one machine in the cluster, you can consider using the install_scripts/single_machine_install.sh install script that comes with the installation bundle. This install script accommodates installations with a "standard" set of parameters and installs the EPICS archiver appliance on one machine. In addition to the System requirements, the install_scripts/single_machine_install.sh will ask for
  1. Location of the Tomcat distribution.
  2. Location of the MySQL client jar - usually a file with a name like mysql-connector-java-5.1.21-bin.jar
  3. A MySQL connection string that looks like so --user=archappl --password=archappl --database=archappl that can be used with the MySQL client like so mysql ${MYSQL_CONNECTION_STRING} -e "SHOW DATABASES". This implies that the MySQL schema has already been created using something like
    mysql --user=root --password=*****
    CREATE DATABASE archappl;
    GRANT ALL ON archappl.* TO 'archappl' identified by 'archappl';
The install_scripts/single_machine_install.sh install script creates a couple of scripts in the deployment folder that can be customized for your site.
  1. sampleStartup.sh - This is a script in the fashion of scripts in /etc/init.d that can be used to start and stop the four Tomcat processes of your archiver appliance.
  2. deployRelease.sh - This can be used to upgrade your installation to a new release of the EPICS archiver appliance. The deployRelease.sh also includes some post install hooks to deploy your site specific content as outlined here.


For a finer control over your installation, installation and configuration consists of these steps. For the cluster
  1. Create an appliances.xml
  2. Optionally, create your policies.py file
In addition to installing the JDK, EPICS (see System requirements), for each appliance
  1. Install and configure Tomcat
    1. Compile the Apache Commons Daemon that is supplied with Tomcat.
  2. Install MySQL (or other persistence provider)
    1. Create the tables
    2. Create a connection pool in Tomcat
  3. Set up storage
  4. Create individual Tomcats for each of the WAR files using the provided python script that copies a single Tomcat installation into four individual Tomcats - one for each WAR.
  5. Deploy the WAR files into their respective containers - This is the deployment step that will be run when you upgrade to a new release.
  6. Stop/Start each of the Tomcats

Create an appliances.xml

The appliances.xml is a file that lists all the appliances in a cluster of archiver appliance. While it is not necessary to point to the same physical file, the contents are expected to be identical across all appliances in the cluster. The details of the file are outlined in the ConfigService javadoc. A sample appliances.xml with two appliances looks like

Create your policies file

The EPICS archiver appliance ships with a sample policies.py (from the tests site) that creates a three stage storage environment. These are
  1. STS - A datastore that uses the PlainPBStoragePlugin to store data in a folder specified by the environment variable ARCHAPPL_SHORT_TERM_FOLDER at the granularity of an hour.
  2. MTS - A datastore that uses the PlainPBStoragePlugin to store data in a folder specified by the environment variable ARCHAPPL_MEDIUM_TERM_FOLDER at the granularity of a day.
  3. LTS - A datastore that uses the PlainPBStoragePlugin to store data in a folder specified by the environment variable ARCHAPPL_LONG_TERM_FOLDER at the granularity of an year.
If you are using the generic build and would like to point to a different policies.py file, you can use the ARCHAPPL_POLICIES environment variable, like so.

export ARCHAPPL_POLICIES=/nfs/epics/archiver/production_policies.py

On the other hand, if you are using a site specific build, you can bundle your site-specific policies.py as part of the mgmt WAR during the site specific build. Just add your policies.py to the source code repository under src/sitespecific/YOUR_SITE/classpathfiles and build the war by setting the ARCHAPPL_SITEID during the build using something like export ARCHAPPL_SITEID=YOUR_SITE. In this case, you do not need to specify the ARCHAPPL_POLICIES environment variable.

Installing Tomcat and setting up Apache Commons Daemon

Installing Tomcat consists of
  1. Untar'ing the Tomcat distribution. It is best to set the environment variable TOMCAT_HOME to the location where the Tomcat distribution is expanded. Many of the following steps require a TOMCAT_HOME to be set.
  2. Editing the conf/server.xml file to change the ports to better suit your installation.
    1. By default, the connector port for the HTTP connector is set to 8080. Change this to the port used by the mgmt webapp for this appliance, in this example, 17665.
      <Connector connectionTimeout="20000" port="808017665" protocol="HTTP/1.1" redirectPort="8443"/>
    2. Remove/comment out the sections for the AJP connector.
    3. At the end, there should be two ports active in the conf/server.xml file, one for the HTTP connector and the other for the SHUTDOWN command.
  3. Setting the appropriate log4j configuration level by creating/editing the lib/log4j.properties. Here's a sample that logs exceptions and errors with one exception - log messages logged to the config namespace are logged at INFO level.
    # Set root logger level and its only appender to A1.
    log4j.rootLogger=ERROR, A1
    # A1 is set to be a DailyRollingFileAppender
    # A1 uses PatternLayout.
    log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
  4. To use Apache Commons Daemon, unzip the ${TOMCAT_HOME}/bin/commons-daemon-native.tar.gz and follow the instructions. Once you have built this, copy the jsvc binary to the Tomcat bin folder for convenience. Note, it's not required that you use Apache Commons Daemon especially, if you are already using system monitoring and management tools like Nagios or Hyperic.
    [ bin ]$ tar zxf commons-daemon-native.tar.gz
    [ bin ]$ cd commons-daemon-1.1.0-native-src
    [ commons-daemon-1.1.0-native-src ]$ cd unix/
    [ unix ]$ ./configure
    *** Current host ***
    checking build system type... x86_64-pc-linux-gnu
    [ unix ]$ make
    (cd native; make  all)
    [ unix ]$ cp jsvc ../../../bin/

Installing MySQL

The version of MySQL that is available from your distribution is acceptable; though this is completely untuned. Please look at the more than excellent chapters on MySQL optimization at the MySQL web site to tune your MySQL instance. In addition to various parameters, even something as simple as setting innodb_flush_log_at_trx_commit=0 (assuming you are ok with this) will go a long way in improving performace (especially when importing channel archiver configuration files etc). Each appliance has its own installation of MySQL. In each appliance,

Setting up storage

This is specific to the needs of your policies.py. However, if you are using the default policies.py that ships with the box or a variant thereof, you'll need to set up three stages of storage. A useful way to do this is to create a folder called /arch and then create soft links in this folder to the actual physical location. For example,
[ arch ]$ ls -ltra
total 32
lrwxrwxrwx    1 archappl archappl      8 Jun 21  2013 sts -> /dev/shm
lrwxrwxrwx    1 archappl archappl      4 Jun 21  2013 mts -> data
lrwxrwxrwx    1 archappl archappl     40 Feb 12  2014 lts -> /nfs/site/archappl/archappl01
drwxr-xr-x  195 archappl archappl    4096 Oct 15 15:05 data
We then set environment variables in the startup script that point to the locations within /arch. For example,

export ARCHAPPL_SHORT_TERM_FOLDER=/arch/sts/ArchiverStore
export ARCHAPPL_MEDIUM_TERM_FOLDER=/arch/mts/ArchiverStore
export ARCHAPPL_LONG_TERM_FOLDER=/arch/lts/ArchiverStore

Create individual Tomcat containers for each of the web apps

The mgmt.war file contains a script deployMultipleTomcats.py in the install folder that will use the information in the appliances.xml file and the identity of this appliance to generate individual Tomcat containers from a single Tomcat install (identified by the environment variable TOMCAT_HOME). To run this script, set the following environment variables
  1. TOMCAT_HOME - This is the Tomcat installation that you prepared in the previous steps.
  2. ARCHAPPL_APPLIANCES - This points to the appliances.xml that you created in the previous steps.
  3. ARCHAPPL_MYIDENTITY - This is the identity of the current appliance, for example appliance0. If this is not set, the system will default to using the machine's hostname as determined by making a call to InetAddress.getLocalHost().getCanonicalHostName(). However, this makes ARCHAPPL_MYIDENTITY a physical entity and not a logical entity; so, if you can, use a logical name for this entry. Note, this must match the identity element of this appliance as it is defined in the appliances.xml.
and then run the deployMultipleTomcats.py script passing in one argument that identifies the parent folder of the individual Tomcat containers.
[ single_machine_install ]$ export TOMCAT_HOME=/arch/single_machine_install/tomcats/apache-tomcat-9.0.20
[ single_machine_install ]$ export ARCHAPPL_APPLIANCES=/arch/single_machine_install/sample_appliances.xml
[ single_machine_install ]$ export ARCHAPPL_MYIDENTITY=appliance0
[ single_machine_install ]$ ./install_scripts/deployMultipleTomcats.py /arch/single_machine_install/tomcats
	tomcat installation at /arch/single_machine_install/tomcats/apache-tomcat-9.0.20
	to generate deployments for appliance appliance0
	using configuration info from /arch/single_machine_install/sample_appliances.xml
	into folder /arch/single_machine_install/tomcats
The start/stop port is the standard Tomcat start/stop port. Changing it to something else random - 16000
The stop/start ports for the new instance will being at  16001
Generating tomcat folder for  mgmt  in location /arch/single_machine_install/tomcats/mgmt
Commenting connector with protocol  AJP/1.3 . If you do need this connector, you should un-comment this.
Generating tomcat folder for  engine  in location /arch/single_machine_install/tomcats/engine
Commenting connector with protocol  AJP/1.3 . If you do need this connector, you should un-comment this.
Generating tomcat folder for  etl  in location /arch/single_machine_install/tomcats/etl
Commenting connector with protocol  AJP/1.3 . If you do need this connector, you should un-comment this.
Generating tomcat folder for  retrieval  in location /arch/single_machine_install/tomcats/retrieval
Commenting connector with protocol  AJP/1.3 . If you do need this connector, you should un-comment this.
[ single_machine_install ]$ 
This is the last of the steps that are install specific; that is, you'll execute these only on installation of a new appliance. The remaining steps are those that will be executed on deployment of new release, start/stop etc.

Deploy the WAR files onto their respective containers

Deploying/upgrading a WAR file in a Tomcat container is very easy. Each container has a webapps folder; all we have to do is to copy the (newer) WAR into this folder and Tomcat (should) will expand the WAR file and deploy the WAR file on startup. The deployment/upgrade steps are
  1. Stop all four Tomcat containers.
  2. Remove the older WAR file and expanded WAR file from the webapps folder (if present).
  3. Copy the newer WAR file into the webapps folder.
  4. Optionally expand the WAR file after copying it over to the webapps folder
    • This lets you replace individual files in the expanded WAR file (for example, images, policies etc) giving you one more way to do site specific deployments.
  5. Start all four Tomcat containers.
If DEPLOY_DIR is the parent folder of the individual Tomcat containers and WARSRC_DIR is the location where the WAR files are present, then the deploy steps (steps 2 and 3 in the list above) look something like

pushd ${DEPLOY_DIR}/mgmt/webapps && rm -rf mgmt*; cp ${WARSRC_DIR}/mgmt.war .; mkdir mgmt; cd mgmt; jar xf ../mgmt.war; popd;
pushd ${DEPLOY_DIR}/engine/webapps && rm -rf engine*; cp ${WARSRC_DIR}/engine.war .; mkdir engine; cd engine; jar xf ../engine.war; popd;
pushd ${DEPLOY_DIR}/etl/webapps && rm -rf etl*; cp ${WARSRC_DIR}/etl.war .; mkdir etl; cd etl; jar xf ../etl.war; popd;
pushd ${DEPLOY_DIR}/retrieval/webapps && rm -rf retrieval*; cp ${WARSRC_DIR}/retrieval.war .; mkdir retrieval; cd retrieval; jar xf ../retrieval.war; popd;

Stopping and starting the individual Tomcats

Running multiple Tomcats on a single machine using the same install requires two enviromnent variables
  1. CATALINA_HOME - This is the install folder for Tomcat that is common to all Tomcat instances; in our case this is $TOMCAT_HOME
  2. CATALINA_BASE - This is the deploy folder for Tomcat that is specific to each Tomcat instance; in our case this is
    • ${DEPLOY_DIR}/mgmt
    • ${DEPLOY_DIR}/etl
    • ${DEPLOY_DIR}/engine
    • ${DEPLOY_DIR}/retrieval
If you are using Apache Commons Daemon, then two bash functions for stopping and starting Tomcat instance look something like

function startTomcatAtLocation() {
    if [ -z "$1" ]; then echo "startTomcatAtLocation called without any arguments"; exit 1; fi
    export CATALINA_BASE=$1
    echo "Starting tomcat at location ${CATALINA_BASE}"
    pushd ${CATALINA_BASE}/logs
    ${CATALINA_HOME}/bin/jsvc \
        -server \
        -cp ${CATALINA_HOME}/bin/bootstrap.jar:${CATALINA_HOME}/bin/tomcat-juli.jar \
        ${JAVA_OPTS} \
        -Dcatalina.base=${CATALINA_BASE} \
        -Dcatalina.home=${CATALINA_HOME} \
        -cwd ${CATALINA_BASE}/logs \
        -outfile ${CATALINA_BASE}/logs/catalina.out \
        -errfile ${CATALINA_BASE}/logs/catalina.err \
        -pidfile ${CATALINA_BASE}/pid \
        org.apache.catalina.startup.Bootstrap start

function stopTomcatAtLocation() {
    if [ -z "$1" ]; then echo "stopTomcatAtLocation called without any arguments"; exit 1; fi
    export CATALINA_BASE=$1
    echo "Stopping tomcat at location ${CATALINA_BASE}"
    pushd ${CATALINA_BASE}/logs
    ${CATALINA_HOME}/bin/jsvc \
        -server \
        -cp ${CATALINA_HOME}/bin/bootstrap.jar:${CATALINA_HOME}/bin/tomcat-juli.jar \
        ${JAVA_OPTS} \
        -Dcatalina.base=${CATALINA_BASE} \
        -Dcatalina.home=${CATALINA_HOME} \
        -cwd ${CATALINA_BASE}/logs \
        -outfile ${CATALINA_BASE}/logs/catalina.out \
        -errfile ${CATALINA_BASE}/logs/catalina.err \
        -pidfile ${CATALINA_BASE}/pid \
        -stop \

and you'd invoke these using something like

stopTomcatAtLocation ${DEPLOY_DIR}/engine
stopTomcatAtLocation ${DEPLOY_DIR}/retrieval
stopTomcatAtLocation ${DEPLOY_DIR}/etl
stopTomcatAtLocation ${DEPLOY_DIR}/mgmt


startTomcatAtLocation ${DEPLOY_DIR}/mgmt
startTomcatAtLocation ${DEPLOY_DIR}/engine
startTomcatAtLocation ${DEPLOY_DIR}/etl
startTomcatAtLocation ${DEPLOY_DIR}/retrieval

Remember to set all the appropriate environment variables from the previous steps
  7. ARCHAPPL_LONG_TERM_FOLDER or equivalent
  8. JAVA_OPTS - This is the environment variable typically used by Tomcat to pass arguments to the VM. You can pass in appropriate arguments like so
    export JAVA_OPTS="-XX:+UseG1GC -Xmx4G -Xms4G -ea"
  9. LD_LIBRARY_PATH - If you are using JCA, please make sure your LD_LIBRARY_PATH includes the paths to the JCA and EPICS base .so's.
A sample startup script using these elements is available here. Please modify to suit your installation.

Other containers

It is possible to deploy the 4 WAR files of the archiver appliance on other servlet containers or to use other industry standard provisioning software to provision an appliance. The details outlined here are guidelines on how to provision an appliance using Tomcat as a servlet container. If you generate scripts for industry standard provisioning software and are willing to share them, please add them to the repositorty and contact the collaboration; we'll be happy to modify these documents to accomodate the same.