Architecture (TNG)
From ControlTier
This document describes a new architecture for the ControlTier system addressing most of the issues raised in the roadmap wish list.
Contents |
Components
The diagram below describes the CTL system components in relation to each other. Centralized management is done via a single webapp called "CTL Center" utilizing a set of REST accessible services. These services in turn manage data in an RDB and files in a WebDAV server. Remote commands are executed via the CTL clients installed on hosts where processes are managed. External tools that manage host infrastructure (virtual or physical) can be integrated to share Node definition data to the CTL system.
Each of the components is described further in its own section.
Figure: System components
CTL Center
The "CTL Center" (say control center) serves as the graphical operations console to manage processes through the ControlTier infrastructure. Users log in and depending on their role and access navigate through a dashboard to one of several tools, either to view past and current activity, execute a command or process or administrate the system.
Functional areas
CTL Center's feature set is similar to those found across several webapps currently developed by ControlTier developers:
- Command execution: The Jobcenter application provides users an interface to run commands and schedule them to run periodically. Jobs can be arranged into groups and their access governed by CTL's acls.xml configuration.
- Reporting: The Reportcenter application provides a historical log of command and job executions. It also provides a query interface that supports RSS feeds. Additionally, it also supports adhoc reporting from either a HTTP-based api as well as BIRT integration.
- Dashboard and notification: The ops-scorecard application provides a service-oriented dashboard providing several trending graphs and event notification subscriptions.
- Resource definition: The Workbench application (along with project.xml) provide tools and interface to upload and view resource model definitions.
Figure: CTL Center feature areas
Administrative functions reside in a separate area of the interface and support the following:
- Clients: Register nodes and install CTL clients. Obtain heartbeat status
- Users: Define users and user groups and access control roles.
CTL Server
Services
Services are accessible using the REST model.
- Inventory
- CRUD operations for resource (nodes, resources, resource sets) and process model definitions
- Execution
- CRUD+Run,Kill operations for on-demand and scheduled process and command execution
- Reporting
- Event registration and querying of report messages
- Artifacts
- Upload and request packaged system artifacts (controllers, processes)
- Session
- Shared workflow session data
Queues
Several queues support the operation of the system:
- System
- Jobs governing internal processes such as reporting and notifications
- Administrative
- Jobs managing client management tasks (eg client installation, artifact distribution)
- User
- User-defined jobs to execute commands and processes on demand or on a scheduled basis
RDB
Domain model:
- Resources
- Nodes
- BusinessServices (rename)
- Processes
- Jobs (Executions)
- Reports
- Execution History
- Change History
- Notification subscriptions
- Node heartbeats
- Users and groups
- Profile and configuration
WebDAV
System artifacts:
- resources
- controllers
- processes
User artifacts:
- files and directories
- packages
CTL Client
CTL Server connector
Receives requests from the #CTL Server and also acts as an extension point to support future communication transports.
Dispatcher API
A java-based API used by CTL tasks, types, CTL Server and the process language.
Figure: Dispatcher client API layer
Module type
A new fundamental type "Module" is introduced to support all control types. It effectively replaces the Managed-Entity type (though Managed-Entity can subtype Module to maintain the large base of existing types).
The Module type will provide a basic set of primitives and resource model features that allow most use cases to use it directly, without the need for creating subtypes.
Commands
Module supports a number of low level commands useful for executing commands via the command dispatcher.
- Common opts: -output <> -timeout <> -nodeincludes <> -nodeexcludes <> -args <>
| command | description | usage |
|---|---|---|
| call | Call a defined command | call [common-opts] -type <> -name <> [-args <>] |
| exec | Execute a system command and/or script | exec [common-opts] -executable <> [-args <>] [-script <>] [-scriptfile] [-spawn] |
| relay | relay command to matching resources | relay [common-opts] -name <.*> -type <.*> [-errorcommand <>] [-threadcount <1>] |
| sequence | execute multiple commands | sequence [common-opts] -commands <> [-errorcommand <>] [-threadcount <1>] |
Properties
Supports a easily extensible set of instance properties.
Session data
Dispatcher performatives to support session data access
- string/file
- get/set/remove/list
- shared/local
- encrypted/cleartext
Data storage
- Each process gets its own collection
- Session elements can be stored as files or properties
Directory hierarchy
The CTL directory hierarchy (aka "depot") is simplified and made more coherent. This is done by consolidating the "module" and "entity" directory structures into one. The directory names are also revised to adhere to the rationalized set of terms:
- "modules" is used in place of "modules" (eg projects/projects/demo/modules)
- "resources" is used in place of "deployments" and refer to instances (eg projects/projects/demo/deployments)
File system structure
`-- projects/
`-- projName/
|-- modules/
| `-- moduleName/
| |-- bin/
| |-- commands/
| | |-- cmdA.xml
| | `-- cmdB.xml
| |-- commands.properties
| |-- lib/
| |-- templates/
| |-- type.properties
| `-- type.xml
`-- resources/
`-- resType/
`-- resName/
`-- var/
`-- resources.properties
Shell commands
Besides the existing ctl and ctl-exec shell commands two new ones are introduced:
- ctl-build
- processes definitions into usable artifacts (types, processes, resources, libraries)
- ctl-process
- executes a controltier process
The role of the ctl-project command may also change with the CTL Server's planned functionality to drive artifact management on the client hosts.
Model
The new architecture aims to rectify some of the inconsistencies and complexities encountered in the current 3.4 based system. It also introduces a new concept of a self contained process definition that will support a new programming model that offers an easier learning curve and direct control over defining multi step distributed processes.
Resources
The resource model is defined in two layers,
- a type system which describes the model schema. This defines type hierarchy, properties, defaults and commands. These types will continue to be defined by an updated type.xml format.
- instances of the types along with their interconnections, as well as, instance-specific properties that can be managed independently of the type's definition. (Conceptually similar to the Groovy Expando class)
Processes
The process definition will allow users to write a "dispatcher script" that can control multi-step processes comprised of ad hoc and defined commands.
Artifacts
Process artifacts
The sources process definition can be maintained in a directory structure and then "built" into a packaged archive that is then distributed to the ControlTier system. The source file structure assumes a directory named by the process. It contains the process definition file and possibly extra resource definitions, as well as, user specific files.
`-- processName/
|-- process.xml
|-- resources/
| |-- resources1.xml
| `-- resources2.xml
`-- files/
|-- aFile
`-- bFile
Process artifacts are published under this policy:
- Published to the CTL server
- Contains the node set criteria to which they should reside
- Admin confirms targeted nodes and approves distribution
- Updates the inventory about what artifacts are on which nodes
- Archive distributed to all nodes
- Shared
Security
Access control
- Users
- Roles
- Owners
- Admin
- Definition: CRUD set by Admin
- Operation: RunApproval,Run,StopApproval,Stop, set by owner or admin
| ||||||||||||||
