Package org.epics.archiverappliance.config

Interface ConfigService

  • All Known Implementing Classes:
    ConfigServiceForTests, DefaultConfigService
    public interface ConfigService
    Interface for appliance configuration. One gets to a config service implementation thru dependency injection of one kind or the other. In a servlet container, this is initialized by ArchServletContextListener (which is registered as part of the web.xml). ArchServletContextListener is also used in the unit tests that involve tomcat. Guice is a good option for this but it takes over the dispatch logic from tomcat and we'll need to investigate if that has any impact.
    Author:
    mshankar

    • Field Detail

      • ARCHAPPL_APPLIANCES

        static final String ARCHAPPL_APPLIANCES
        This is the environment variable that points to the file containing the various appliances in this cluster. This list of appliances is expected to be the same for all appliances in the cluster; so it is perfectly legal to place it in NFS somewhere and point to the same file/location from all appliances in the cluster. It is reasonably important that all appliances see the same list of cluster members or we tend to have split-brain effects (See wikipedia). The format of the file itself is simple XML like so 
         <appliances>
   <appliance>
     <identity>archiver</identity>
     <cluster_inetport>archiver:77770</cluster_inetport>
     <mgmt_url>http://archiver.slac.stanford.edu:77765/mgmt/bpl</mgmt_url>
     <engine_url>http://archiver.slac.stanford.edu:77765/engine/bpl</engine_url>
     <etl_url>http://archiver.slac.stanford.edu:77765/etl/bpl</etl_url>
     <retrieval_url>http://archiver.slac.stanford.edu:77765/retrieval/bpl</retrieval_url>
     <data_retrieval_url>http://archiver.slac.stanford.edu:77765/retrieval/</data_retrieval_url>
   </appliance>
 </appliances>
        Note that the appliance identity as defined by the ARCHAPPL_MYIDENTITY has to match the identity element of one of the appliances in the list of appliances as defined by the ARCHAPPL_APPLIANCES. Each appliance (which includes the mgmt, engine, etl and retrieval WAR's) must have a unique identity.
        If the ARCHAPPL_APPLIANCES is not set, then we look for a file called appliances.xml in the WEB-INF/classes of the current WAR using WEB-INF/classes/appliances.xml. The default build script places the site-specific appliances.xml into WEB-INF/classes/appliances.xml.
        See Also:
        Constant Field Values

      • ARCHAPPL_MYIDENTITY

        static final String ARCHAPPL_MYIDENTITY
        This is an optional environment variable that determines this appliance's identity. If this is not set, the archiver appliance uses InetAddress.getLocalHost().getCanonicalHostName() to determine the FQDN of this machine. This is then used as the appliance identity to lookup the appliance info in ARCHAPPL_APPLIANCES. To use this environment variable, for example, in Linux, set the appliance's identity using export ARCHAPPL_MYIDENTITY="archiver". Each appliance (which includes the mgmt, engine, etl and retrieval WAR's) must have a unique identity. To accommodate the multi-instance unit tests, if this environment variable is not set, we check for the existence of the java system property ARCHAPPL_MYIDENTITY. Typically, the multi-instance unit tests (which are incapable of altering the environment) use the java system property method. In environments that run the unit tests, leave the environment variable ARCHAPPL_MYIDENTITY unset so that the various multi-instance unit tests have the ability to control the appliance identity.
        See Also:
        Constant Field Values

      • ARCHAPPL_SITEID

        static final String ARCHAPPL_SITEID
        This is the environment variable that identifies the site (LCLS, LCLSII, slacdev, NSLSII etc) to be used when generating the war files. This is primarily a build-time property; the build.xml has various site specific hooks which let you change the appliances.xml, policies, images etc on a per site basis. The unit tests use the tests site which is also the default site if this environment variable is not specified. Files for a site are stored in the sitespecific/<site> folder.
        See Also:
        Constant Field Values

      • ARCHAPPL_CONFIGSERVICE_IMPL

        static final String ARCHAPPL_CONFIGSERVICE_IMPL
        This is an optional environment variable/system property that is used to identify the config service implementation.
        See Also:
        Constant Field Values

      • CONFIG_SERVICE_NAME

        static final String CONFIG_SERVICE_NAME
        This is the name used to identify the config service implementation in one of the existing singletons (like the ServletContext)
        See Also:
        Constant Field Values

      • ARCHAPPL_PERSISTENCE_LAYER

        static final String ARCHAPPL_PERSISTENCE_LAYER
        This is an optional environment/system.property that is used to identity the persistence layer If this is not set, we initialize MySQLPersistence as the persistence layer; so in production environments, you can leave this unset/blank Set this to the class name of the class implementing ConfigPersistence The unit tests however will set this to use InMemoryPersistence, which is a dummy persistence layer.
        See Also:
        Constant Field Values

      • ARCHAPPL_POLICIES

        static final String ARCHAPPL_POLICIES
        This is an optional environment/system.property that is used to specify the location of the policies.py policies file. If this is not set, we search for the policies.py in the servlet classpath using the path /WEB-INF/classes/policies.py
        See Also:
        Constant Field Values

      • ARCHAPPL_PROPERTIES_FILENAME

        static final String ARCHAPPL_PROPERTIES_FILENAME
        The name/path of the archappl.properties file. By default, we look for archappl.properties in the webapp's classpath - this will typically resolve into WEB-INF/classes of the webapp. However, you can override this using an environment variable (or java system property) of the same name. For example, export ARCHAPPL_PROPERTIES_FILENAME=/etc/mylab_archappl.properties should force the components to load their properties from /etc/mylab_archappl.properties
        See Also:
        Constant Field Values

      • DEFAULT_ARCHAPPL_PROPERTIES_FILENAME

        static final String DEFAULT_ARCHAPPL_PROPERTIES_FILENAME
        This is the name of the properties file that is looked for in the webapp's classpath if one is not specified using a environment/JVM property.
        See Also:
        Constant Field Values

      • ARCHAPPL_NAMEDFLAGS_PROPERTIES_FILE_PROPERTY

        static final String ARCHAPPL_NAMEDFLAGS_PROPERTIES_FILE_PROPERTY
        See Also:
        Constant Field Values

    • Method Detail

      • initialize

        void initialize​(javax.servlet.ServletContext sce)
         throws ConfigException
        If you have a null constructor and need a ServletContext, implement this method.
        Parameters:
        sce - ServletContext
        Throws:
        ConfigException -  

      • getWebInfFolder

        String getWebInfFolder()
        We expect to live within a servlet container. This call returns the full path to the WEB-INF folder of the webapp as it is deployed in the container. Is typically a call to the servletContext.getRealPath("WEB-INF/")
        Returns:
        String WebInfFolder  

      • postStartup

        void postStartup()
          throws ConfigException
        This method is called after the mgmt WAR file has started up and set up the cluster and recovered data from persistence. Each appliance's mgmt war is responsible for calling this method on the other components (engine, etl and retrieval) using BPL. Until this method is called on all the web apps, the cluster is not considered to have started up.
        Throws:
        ConfigException -  

      • isStartupComplete

        boolean isStartupComplete()
        Have we completed all the startup steps?
        Returns:
        boolean True or False

      • getTimeOfAppserverStartup

        long getTimeOfAppserverStartup()
        Get an approximate time in epoch seconds when the appserver started up.
        Returns:
        The time this app server started up.

      • getInstallationProperties

        Properties getInstallationProperties()
        An arbitrary list of name/value pairs can be specified in a file called archappl.properties that is loaded from the classpath.
        Returns:
        Properties  

      • getAppliancesInCluster

        Iterable<ApplianceInfo> getAppliancesInCluster()
        Get all the appliances in this cluster. Much goodness is facilitated if the objects are returned in the same order (perhaps order of creation) all the time.
        Returns:
        ApplianceInfo  

      • getMyApplianceInfo

        ApplianceInfo getMyApplianceInfo()
        Get the appliance information for this appliance.
        Returns:
        ApplianceInfo  

      • getAppliance

        ApplianceInfo getAppliance​(String identity)
        Given an identity of an appliance, return the appliance info for that appliance
        Parameters:
        identity - The appliance identify
        Returns:
        ApplianceInfo  

      • getAllPVs

        Collection<String> getAllPVs()
        Get an exhaustive list of all the PVs this cluster of appliances knows about Much goodness is facilitated if the objects are returned in the same order (perhaps order of creation) all the time.
        Returns:
        String AllPVs  

      • getAllExpandedNames

        void getAllExpandedNames​(Consumer<String> func)
        For automated PV submission, IOC engineers could add .VAL, fields, aliases etc. This method attempts to return all possible PV's that the archiver could know about. This is a lot of names; so we take in a consumer that potentially streams a name out as quickly as possible.
        Parameters:
        func - A consumer of pvNames

      • getApplianceForPV

        ApplianceInfo getApplianceForPV​(String pvName)
        Given a PV, get us the appliance that is responsible for archiving it. Note that this may be null as the assignment of PV's to appliances can take some time.
        Parameters:
        pvName - The name of PV.
        Returns:
        ApplianceInfo  

      • getPVsForAppliance

        Iterable<String> getPVsForAppliance​(ApplianceInfo info)
        Get all PVs being archived by this appliance. Much goodness is facilitated if the objects are returned in the same order (perhaps order of creation) all the time.
        Parameters:
        info - ApplianceInfo
        Returns:
        string All PVs being archiveed by this appliance

      • getPVsForThisAppliance

        Iterable<String> getPVsForThisAppliance()
        Get all the PVs for this appliance. Much goodness is facilitated if the objects are returned in the same order (perhaps order of creation) all the time.
        Returns:
        String All PVs being archiveed for this appliance

      • isBeingArchivedOnThisAppliance

        boolean isBeingArchivedOnThisAppliance​(String pvName)
        Is this PV archived on this appliance. This method also checks aliases and fields.
        Parameters:
        pvName - The name of PV.
        Returns:
        boolean True or False

      • getPVsForApplianceMatchingRegex

        Set<String> getPVsForApplianceMatchingRegex​(String nameToMatch)
        Get the pvNames for this appliance matching the given regex.
        Parameters:
        nameToMatch -  
        Returns:
        string PVsForApplianceMatchingRegex  

      • getTypeInfoForPV

        PVTypeInfo getTypeInfoForPV​(String pvName)
        Gets information about a PV's type, i.e its DBR type, graphic limits etc. This information is assumed to be somewhat static and is expected to come from a cache if possible as it is used in data retrieval.
        Parameters:
        pvName - The name of PV.
        Returns:
        PVTypeInfo  

      • updateTypeInfoForPV

        void updateTypeInfoForPV​(String pvName,
                         PVTypeInfo typeInfo)
        Update the type information about a PV; updating both ther persistent and cached versions of the information. Clients are not expected to call this method a million times a second. In general, this is expected to be called when archiving a PV for the first time, or perhaps when an appserver startups etc...
        Parameters:
        pvName - The name of PV.
        typeInfo - PVTypeInfo

      • removePVFromCluster

        void removePVFromCluster​(String pvName)
        Remove the pv from all cached and persisted configuration.
        Parameters:
        pvName - The name of PV.

      • getAggregatedApplianceInfo

        ApplianceAggregateInfo getAggregatedApplianceInfo​(ApplianceInfo applianceInfo)
                                           throws IOException
        Facilitates various optimizations for BPL that uses appliance wide information by caching and maintaining this information on a per appliance basis
        Parameters:
        applianceInfo - ApplianceInfo
        Returns:
        ApplianceAggregateInfo  
        Throws:
        IOException -  

      • addToArchiveRequests

        void addToArchiveRequests​(String pvName,
                          UserSpecifiedSamplingParams userSpecifiedSamplingParams)
        The workflow for requesting a PV to be archived consists of multiple steps This method adds a PV to the persisted list of PVs that are currently engaged in this workflow in addition to any user specified overrides
        Parameters:
        pvName - The name of PV.
        userSpecifiedSamplingParams - - Use a null contructor for userSpecifiedSamplingParams if no override specified.

      • updateArchiveRequest

        void updateArchiveRequest​(String pvName,
                          UserSpecifiedSamplingParams userSpecifiedSamplingParams)
        Update the archive request (mostly with aliases) if and only if we have this in our persistence.
        Parameters:
        pvName - The name of PV.
        userSpecifiedSamplingParams -  

      • getArchiveRequestsCurrentlyInWorkflow

        Set<String> getArchiveRequestsCurrentlyInWorkflow()
        Gets a list of PVs that are currently engaged in the archive PV workflow
        Returns:
        String ArchiveRequestsCurrentlyInWorkflow  

      • doesPVHaveArchiveRequestInWorkflow

        boolean doesPVHaveArchiveRequestInWorkflow​(String pvname)
        Is this pv in the archive request workflow.
        Parameters:
        pvname - The name of PV.
        Returns:
        boolean True or False

      • getInitialDelayBeforeStartingArchiveRequestWorkflow

        int getInitialDelayBeforeStartingArchiveRequestWorkflow()
        In clustered environments, to give capacity planning a chance to work correctly, we want to kick off the archive PV workflow only after all the machines have started. This is an approximation for that metric; though not a very satisfactory approximation. TODO -- Think thru implications of making the appliances.xml strict...
        Returns:
        - Initial delay in seconds.

      • getUserSpecifiedSamplingParams

        UserSpecifiedSamplingParams getUserSpecifiedSamplingParams​(String pvName)
        Returns any user specified parameters for the archive request.
        Parameters:
        pvName - The name of PV.
        Returns:
        UserSpecifiedSamplingParams  

      • archiveRequestWorkflowCompleted

        void archiveRequestWorkflowCompleted​(String pvName)
        Mark this pv as having it archive pv request completed and pull this request out of persistent store Can be used in the case of aborting a PV archive request as well
        Parameters:
        pvName - The name of PV.

      • getExtraFields

        String[] getExtraFields()
        Get a list of extra fields that are obtained when we initially make a request for archiving. These are used in the policies to make decisions on how to archive the PV.
        Returns:
        String ExtraFields  

      • getRuntimeFields

        Set<String> getRuntimeFields()
        Get a list of fields for PVs that are monitored and maintained in the engine. These are used when displaying the PV in visualization tools like the ArchiveViewer as additional information for the PV. Some of these could be archived along with the PV but need not be. In this case, the engine simply maintains the latest copy in memory and this is served up when data from the engine in included in the stream.
        Returns:
        String RuntimeFields

      • addAlias

        void addAlias​(String aliasName,
              String realName)
        Register an alias
        Parameters:
        aliasName -  
        realName - This is the name under which the PV will be archived under

      • removeAlias

        void removeAlias​(String aliasName,
                 String realName)
        Remove an alias for the specified realname
        Parameters:
        aliasName -  
        realName - This is the name under which the PV will be archived under

      • getAllAliases

        List<String> getAllAliases()
        Get all the aliases in the system. This is used for matching during glob requests in the UI.
        Returns:
        String AllAliases  

      • getRealNameForAlias

        String getRealNameForAlias​(String aliasName)
        Gets the .NAME field for a PV if it exists. Otherwise, this returns null
        Parameters:
        aliasName -  
        Returns:
        String RealNameForAlias

      • getPolicyText

        InputStream getPolicyText()
                   throws IOException
        Return the text of the policy for this installation. Gets you an InputStream; remember to close it.
        Returns:
        InputStream  
        Throws:
        IOException -  

      • computePolicyForPV

        PolicyConfig computePolicyForPV​(String pvName,
                                MetaInfo metaInfo,
                                UserSpecifiedSamplingParams userSpecParams)
                         throws IOException
        Given a pvName (for now, we should have a pv details object of some kind soon), determine the policy applicable for archiving this PV.
        Parameters:
        pvName - The name of PV.
        metaInfo - The MetaInfo of PV
        userSpecParams - UserSpecifiedSamplingParams
        Returns:
        PolicyConfig  
        Throws:
        IOException -  

      • getPoliciesInInstallation

        HashMap<String,​String> getPoliciesInInstallation()
                                                throws IOException
        Return a map of name to description of all the policies in the system This is used to drive a dropdown in the UI.
        Returns:
        HashMap  
        Throws:
        IOException -  

      • getFieldsArchivedAsPartOfStream

        List<String> getFieldsArchivedAsPartOfStream()
                                      throws IOException
        This product offers the ability to archive certain fields (like HIHI, LOLO etc) as part of every PV. The data for these fields is embedded into the stream as extra fields using the FieldValues interface of events. This method lists all these fields. Requests for archiving these fields are deferred to and combined with the request for archiving the .VAL. We also assume that the data type (double/float) for these fields is the same as the .VAL.
        Returns:
        String  
        Throws:
        IOException -  

      • getArchiverTypeSystem

        TypeSystem getArchiverTypeSystem()
        Returns a TypeSystem object that is used to convert from JCA DBR's to Event's (actually, DBRTimeEvents)
        Returns:
        TypeSystem  

      • getWarFile

        ConfigService.WAR_FILE getWarFile()
        Which component is this configservice instance.
        Returns:
        WAR_FILE  

      • getRetrievalRuntimeState

        RetrievalState getRetrievalRuntimeState()
        Returns the runtime state for the retrieval app
        Returns:
        RetrievalState  

      • getETLLookup

        PBThreeTierETLPVLookup getETLLookup()
        Return the runtime state for ETL. This may eventually be moved to a RunTime class but that would still start from the configservice.
        Returns:
        PBThreeTierETLPVLookup  

      • getEngineContext

        EngineContext getEngineContext()
        Return the runtime state for the engine.
        Returns:
        EngineContext  

      • getMgmtRuntimeState

        MgmtRuntimeState getMgmtRuntimeState()
        Return the runtime state for the mgmt webapp.
        Returns:
        MgmtRuntimeStat  

      • isShuttingDown

        boolean isShuttingDown()
        Is this appliance component shutting down?
        Returns:
        boolean True or False

      • addShutdownHook

        void addShutdownHook​(Runnable runnable)
        Add an appserver agnostic shutdown hook; for example, to close the CA channels on shutdown
        Parameters:
        runnable - Runnable

      • shutdownNow

        void shutdownNow()
        Call the registered shutdown hooks and shut the archive appliance down.

      • getEventBus

        com.google.common.eventbus.EventBus getEventBus()
        Get the event bus used for events within this appliance.
        Returns:
        EventBus  

      • getExternalArchiverDataServers

        Map<String,​String> getExternalArchiverDataServers()
        This product has the ability to proxy data from other archiver data servers. We currently integrate with Channel Archiver XMLRPC data servers and other EPICS Archiver Appliance clusters. Get a list of external Archiver Data Servers that we know about.
        Returns:
        Map ExternalArchiverDataServers

      • addExternalArchiverDataServer

        void addExternalArchiverDataServer​(String serverURL,
                                   String archivesCSV)
                            throws IOException
        Add a external Archiver Data Server into the system.
        Parameters:
        serverURL - - For Channel Archivers, this is the URL to the XML-RPC server. For other EPICS Archiver Appliance clusters, this is the data_retrieval_url of the cluster as defined in the appliances.xml.
        archivesCSV - - For Channel Archivers, this is a comma separated list of indexes. For other EPICS Archiver Appliance clusters, this is the string pbraw.
        Throws:
        IOException -  

      • removeExternalArchiverDataServer

        void removeExternalArchiverDataServer​(String serverURL,
                                      String archivesCSV)
                               throws IOException
        Removes an entry for an external Archiver Data Server from the system Note; we may need to restart the entire cluster for this change to take effect.
        Parameters:
        serverURL - - For Channel Archivers, this is the URL to the XML-RPC server. For other EPICS Archiver Appliance clusters, this is the data_retrieval_url of the cluster as defined in the appliances.xml.
        archivesCSV - - For Channel Archivers, this is a comma separated list of indexes. For other EPICS Archiver Appliance clusters, this is the string pbraw.
        Throws:
        IOException -  

      • getChannelArchiverDataServers

        List<ChannelArchiverDataServerPVInfo> getChannelArchiverDataServers​(String pvName)
        Return a list of ChannelArchiverDataServerPVInfos for a PV if one exists; otherwise return null. The servers are sorted in order of the start seconds. Note: this only applies to Channel Archiver XML RPC servers. For proxying external EPICS Archiver Appliance clusters, we do not cache the PV's that are being archived on the external system.
        Parameters:
        pvName - The name of PV.
        Returns:
        ChannelArchiverDataServerPVInfo  

      • refreshPVDataFromChannelArchiverDataServers

        void refreshPVDataFromChannelArchiverDataServers()
        For all the Channel Archiver XMLRPC data servers in the mix, update the PV info. This should help improve performance a little in proxying data from ChannelArchiver data servers that are still active. For proxying external EPICS Archiver Appliance clusters, since we do not cache the PV's that are being archived on the external system, this is a no-op.

      • getFailoverServerURLs

        Set<String> getFailoverServerURLs()
        Get a list of external archiver appliances configured for failover. Failover appliances are not used as proxies to minimize imposing the retrieval load of the this installation on a potentially less powerful appliance used principally for failover.
        Returns:

      • getFailoverApplianceURL

        String getFailoverApplianceURL​(String pvName)
        Get the first external archiver appliance that also archives this PV and is configured with a mergeDuringRetrieval query parameter.
        Returns:
        - Returns null if no failover appliance.

      • resetFailoverCaches

        void resetFailoverCaches()
        Each retrieval component in a cluster caches the PV's from remote failover appliances. These caches contain one entry for each PV in this appliance indicating if the PV is being archived in the remote appliance. This information is cached using a TTL to minimize the impact on the remote failover appliance. This method manually unloads this cache.

      • getPVNameToKeyConverter

        PVNameToKeyMapping getPVNameToKeyConverter()
        Implementation for converting a PV name to something that forms the prefix of a chunk's key. See @see{PVNameToKeyMapping} for more details.
        Returns:
        PVNameToKeyMapping  

      • getPausedPVsInThisAppliance

        Set<String> getPausedPVsInThisAppliance()
        Get a set of PVs that have been paused in this appliance.
        Returns:
        String  

      • getNamedFlag

        boolean getNamedFlag​(String name)
        Named flags are used to control various process in the appliance; for example, the ETL process in a PlainPBStoragePlugin Named flags are not persistent; each time the server starts up, all the named flags are set to false You can optionally load values for named flags from a file by specifying the ARCHAPPL_NAMEDFLAGS_PROPERTIES_FILE_PROPERTY property in archappl.properties. This method gets the value of the specified named flag. If the flag has not been defined before in the cluster, this method will return false.
        Parameters:
        name -  
        Returns:
        boolean True or False

      • setNamedFlag

        void setNamedFlag​(String name,
                  boolean value)
        Sets the value of the named flag specified by name to the specified value
        Parameters:
        name -  
        value -  

      • getNamedFlagNames

        Set<String> getNamedFlagNames()
        Return the names of all the named flags that we know about
        Returns:
        String