How do I ensure business continuity?
Last Updated: | Applicable Hub Versions: All
Answer
Planview Hub maintains information critical to organizational business processes, and therefore should be included in a comprehensive business continuity plan (aka a disaster recovery plan) that safeguards data and ensures business continuity in hardware and operational failure scenarios.
See the sections below for an overview of our Business Continuity & Disaster Recovery materials.
High Availability and Disaster Recovery
Business continuity can take many forms, and an organization’s choice of High Availability strategies must be strongly influenced by both its required uptime Service Level Agreement and the specific Planview Hub features it is using.
- High Availability (HA) refers to the continued availability of Planview Hub in the event of a system component failure.
- Disaster Recovery (DR) is related to HA but refers to a system’s resilience to data loss incurred as a result of disaster. Planview Hub DR strategies require the configuration of a reliable supported external database, such as Oracle, SQL Server, MySQL, or PostgreSQL. The database must be configured with sufficient redundancy to maximize uptime and reduce the probability of data loss to a level acceptable to the organization.
The necessity for a HA implementation and a DR plan depends on your organization’s policies. Detailed HA and DR planning is beyond the scope of this document; however, implications of downtime, some possible strategies, and important considerations will be presented.
Downtime Implications
If the Hub Server becomes unavailable the impact will vary depending on the type of integration:
- Synchronization integrations will not create or update artifacts in synchronized repositories.
- Enterprise Data Stream integrations will not record artifact changes from their integrated source repositories to their target databases, which may cause a loss of fidelity in reporting data.
- Gateway integrations cannot accept payloads from integrated gateway collections; this can result in data loss if the integrated tools cannot handle the downtime.
When a momentarily unavailable Hub Server becomes available once again, the effects are as follows:
- All Synchronization integrations will begin processing where they left off when the server became unavailable; there may be a backlog of changes to process, but no synchronizations will be lost.
- Enterprise Data Stream integrations will begin detecting artifact changes; any changes that occurred when service was unavailable will be detected, but multiple changes to the same field will have lost fidelity (only one change to that field will be reported).
- Hub will begin accepting Gateway collection payloads, and if the integrated repositories are configured correctly to retry payloads, they will be processed as usual without data loss. Otherwise, the data may be lost.
Disaster Recovery Strategy
Planview Hub Server’s operational database stores both configuration and operational data. A Disaster Recovery strategy will generally involve a form of database replication and backup.
Considerations
- Planview Hub instances require a single operational database schema per Hub instance. Under no circumstances may multiple instances of Hub access the same operational database schema simultaneously, as this would likely result in Hub creating duplicate data in the integrated repositories. As such, the server instance will detect and prohibit this configuration. When a Hub Server instance is started, it obtains a lock on the database schema; if there is already a lock on the database, it will steal the lock, rendering the formerly active instance inactive.
- Only one instance of Planview Hub may run at once. Multiple instances of Hub Server connecting to the same repositories will likely create duplicate data and could result in infinite update cycles.
- When Hub is upgraded, downtime for the Hub service is required in order to upgrade the database. Care must be taken to ensure that during an upgrade any failover monitoring scripts are disabled.
High Availability Strategies
Achieving High Availability involves clustering of Hub Server instances to support failover, where “instance” refers to a deployed but not necessarily running Hub Server. Two strategies are possible, but each is suited to a particular type of integration.
Active-Passive
Considerations
- Best suited for Synchronization integrations
- Also useful for Enterprise Data Stream and Gateway integrations
- Utilizes one active and one or more passive Hub Server instances, all configured to connect to the same operational database schema
- Failover is not instantaneous. Downtime is confined to the time it takes to detect the failed instance, shut it down gracefully if possible, and start the Failover instance.
- Implications upon failover:
- Synchronization integrations will not create or update artifacts in synchronized repositories during failover. This will not result in data loss since synchronization continues at the point the service was stopped.
- Enterprise Data Stream integrations will not record artifact changes from their integrated source repositories to their target databases. The lack of data for the downtime period will cause a loss of fidelity in reporting data due to the missing data points.
- Gateway integrations cannot accept payloads from gateway collections; this can result in data loss if the integrated tools cannot handle the downtime. Client attempts to access a gateway collection will result in an error during downtime.
Implementation
The strategy for Active-Passive is to have two deployed Tasktop Server instances configured to use the same database, where the passive (failover) instance is not currently running. Upon failure of the active instance, the passive instance can be started via a script. Before startup, there should be an attempt to gracefully shut down the failed instance if possible. The passive instance will then connect to the database and become the primary (active) instance, stealing the database lock from the previously active instance if it is still valid.
The single database schema should be configured for high availability according to the chosen database management system (DBMS) instructions; this will likely involve a form of replication.
Details
Planview Hub instances cannot be configured via the user interface to use an external database that has already been configured by another instance. Some system-level intervention is required.
To configure a primary and a secondary instance of Planview Hub:
- Deploy the primary instance and configure it to use an external database as required.
- Shut down the primary instance.
- Deploy a secondary instance of Planview Hub.
- Select the appropriate JDBC driver under Settings > Storage Settings > Use External Database. DO NOT configure credentials. Once selected, you can click “Cancel”, or alternatively, navigate away from the page. If you choose to navigate away from the page, you will see a warning that you have unsaved changes - this is OK. Click “leave”. If you return to this page you will see that the JDBC Driver section of the External Database storage settings shows that a driver has been selected.
- Configuring the secondary instance’s database credentials through the Hub UI is not supported, because this would cause Hub to attempt to transfer its existing database to the external database. This would fail immediately with an error message to avoid overwriting an active database.
- Instead, the database configuration file for the failover instance must be modified manually and the instance restarted.
- The install directory on the primary instance contains a file which holds the external database configuration for the primary instance: tasktop-db.json. Copy this file to the same location on the secondary instance - if a file of the same name exists on the secondary instance already, it should be backed up prior to overwriting it.
- Restart the secondary instance so that it will re-read its database configuration file and set up its database connection.
- The secondary instance’s web UI will display a message stating that its configured database cannot be reached, and will show the database settings with an empty password field.
Since the primary instance is shut down, you can safely connect the secondary instance to the external database by entering the database password, whereupon it will steal the database lock from the primary instance.
At this point the secondary instance is the active instance, and the primary instance is the inactive instance. Upon failure detection of the active instance, simply start the inactive instance. If necessary, it will steal the database lock from the formerly active instance to become the active instance itself.
Active-Active
Considerations
- Suited for: Gateway Collections (web hooks), and cases where a gateway collection is used in conjunction with Enterprise Data Stream.
- NOT TO BE USED for Synchronization integrations
- No downtime
- Gateway collections do not synchronize data; they are one-way only. Therefore artifact data is not required to be kept within a single database table. This allows multiple concurrent database schemas to be used: one for each active instance of Hub Server. An external load balancer will route incoming traffic to active instances as necessary, and if an instance fails, it will no longer receive traffic.
- This configuration will scale as necessary by adding additional instances and associated database schemas.
Implementation
The strategy for Active-Active is to have two (or more) deployed Hub Server instances where each instance is configured to connect to separate operational database schemas. Both instances are behind a single Load Balancer, which must be configured to use a strategy such as round-robin combined with health-check monitoring so that it can stop routing traffic to nodes that have failed.
The multiple database schemas should be configured for High Availability according to the chosen Database Management System (DBMS) instructions; this will likely involve a form of replication.
Complex High Availability
A larger Hub Server deployment may involve Synchronization integrations, Gateway integrations, and Enterprise Data Stream integrations. The two High Availability strategies outlined above can be combined to provide a high availability environment for the entire deployment as per the following diagram.
