Apache Brooklyn: User Manual
Contents
Features
^Blueprinting
Composable blueprints
A YAML service spec can refer to other blueprints,
either in the catalog or by URL,
and can supply custom configuration.
Portable machines specs -- or location-specific identifiers
Define machine specs using portable constraints,
or, when you need to, use specific imageId, hardware profiles, and more
Policy-Based Management
Live metrics
Collect live metrics for use in policies,
either from metric stores or directly using REST, JMX, SSH, and more.
Management policies
Choose from built-in policies including auto-scaling, failover, and follow-the-sun, or create new policies to perform custom runtime management.
Use config keys to customize the policies to suit your systems, right in the YAML blueprint.
Dynamic reconfiguration
Reconfigure policies, suspend them, or add new ones on-the-fly
through the REST API.
Operations
Brooklyn console
Brooklyn runs with a GUI console giving easy access to the
management hierarchy, sensors, and activities.
High availability
Run standby nodes which can optionally automatically promote to master
in the event of master failure. Hot standby nodes can provide additional
read-only access to entity information.
State persistence
Blueprint, catalog, topology and sensor information can be
automatically persisted to any file system or object store to
stop Brooklyn and restart resuming where you left off.
REST API
The console is pure JS-REST, and all the data shown in the GUI is available through a straightforward REST/JSON API.
In many cases, the REST API is simply the GUI endpoint without the
leading #. For instance the data for
#/v1/applications/ is available at /v1/applications/.
And in all cases, Swagger doc is available in the product.
Groovy console
With the right permissions, Groovy scripts can be sent via
the GUI or via REST, allowing open-heart surgery on your systems.
(Use with care!)
Versioning
Blueprints in the catalog can be versioned on-the-fly.
Running entities are attached to the version against which
they were launched to preserve integrity, until manual
version updates are performed.
Deep task information
The console shows task flows in real-time,
including the `stdin` and `stdout` for shell commands,
making it simpler to debug those pesky failures.
Java
Discoverable configuration
Config keys, sensors, and effectors can be defined on the classes
such that they are automatically discoverable at runtime.
Type information, parameters, documentation, and default values
are returned through the REST API and shown in the GUI.
Type hierarchy
Use interfaces and mix-ins to share and inherit behavior in a strongly typed way.
Sensor feeds
Fluent builder-style API's are included for collecting sensor information
from REST endpoints, SSH commands, JMX connectors, and more.
Task libraries
Fluent builder-style task libraries are included for building activity
chains which run in parallel or sequentially,
executing SSH, REST, or arbitrary Java commands.
Task status, result, hierarchies, and errors are exposed through the REST API and in the GUI.
Downloads
^ {% include fields.md %} {% if site.brooklyn.is_snapshot %} **The downloads on this page have not been voted on and should be used at your own risk. The latest stable release can be accessed on the [main download page](/website/download/).** {% endif %} ## Download Version {{ site.brooklyn-version }} ## Examples Examples can be found in the main Brooklyn codebase, in the `/examples` directory. A good example to start with is the [Elastic Web Cluster](/guide/use/examples/webcluster.md). {% endcomment %} ## Maven If you use Maven, you can add Brooklyn with the following in your pom: {% highlight xml %}Brooklyn Concepts
^ This introduces brooklyn and describes how it simplifies the deployment and management of big applications. It is intended for people who are using brooklyn-supported application components (such as web/app servers, data stores) to be able to use brooklyn to easily start their application in multiple locations with off-the-shelf management policies. {% include list-children.html %}Entities
^ The central concept in a Brooklyn deployment is that of an ***entity***. An entity represents a resource under management, either *base* entities (individual machines or software processes) or logical collections of these entities. Fundamental to the processing model is the capability of entities to be the *parent* of other entities (the mechanism by which collections are formed), with every entity having a single parent entity, up to the privileged top-level ***application*** entity. Entities are code, so they can be extended, overridden, and modified. Entities can have events, operations, and processing logic associated with them, and it is through this mechanism that the active management is delivered. The main responsibilities of an entity are: - Provisioning the entity in the given location or locations - Holding configuration and state (attributes) for the entity - Reporting monitoring data (sensors) about the status of the entity - Exposing operations (effectors) that can be performed on the entity - Hosting management policies and tasks related to the entityApplication, Parent and Membership
^ All entities have a ***parent*** entity, which creates and manages it, with one important exception: *applications*. Application entities are the top-level entities created and managed externally, manually or programmatically. Applications are typically defined in Brooklyn as an ***application descriptor***. This is a Java class specifying the entities which make up the application, by extending the class ``AbstractApplication``, and specifying how these entities should be configured and managed. All entities, including applications, can be the parent of other entities. This means that the "child" is typically started, configured, and managed by the parent. For example, an application may be the parent of a web cluster; that cluster in turn is the parent of web server processes. In the management console, this is represented hierarchically in a tree view. A parallel concept is that of ***membership***: in addition to one fixed parent, an entity may be a ***member*** of zero or more ***groups*** (a "group" is a special kind of entity). Membership of a group can be used for whatever purpose is required; it is commonly used to manage a collection of entities together for one purpose. For example, a group could be used to indicate the targets for a load balancer, getting the address of each. These targets do not need to have the same parent (e.g. the members may have been deployed in different locations, where the 'parent' hierarchy mirrors the different locations). Another example would be deployment of a set of VMs where there is a "sidecar" logging process on each VM. The parent hierarchy could be a top-level cluster with a child entity per VM. Under each of these VM entities there could be two child entities for the two processes. There could then be a "group" entity whose members were all the sidecar logger entities.Configuration, Sensors and Effectors
^ ### Configuration All entities contain a map of config information. This can contain arbitrary values, typically keyed under static `ConfigKey` fields on the `Entity` sub-class. These values are inherited, so setting a configuration value at the application level will make it available in all entities underneath unless it is overridden. Configuration is propagated when an application "goes live" (i.e. it becomes "managed", either explicitly or when its `start()` method is invoked), so config values must be set before this occurs. Configuration values can be specified in a configuration file (`brooklyn.cfg`) to apply universally, and/or programmatically to a specific entity and its descendants by calling `.configure(KEY, VALUE)` in the entity spec when creating it. There is also an `entity.config().set(KEY, VALUE)` method. Additionally, many common configuration parameters are available as "flags" which can be supplied as Strings when constructing the entity, in the form `EntitySpec.create(MyEntity.class).configure("config1", "value1").configure("config2", "value2")`. Documentation of the flags available for individual entities can normally be found in the javadocs. The `@SetFromFlag` annotations on `ConfigKey` static field definitions in the entity's interface is the recommended mechanism for exposing configuration options. ### Sensors and Effectors ***Sensors*** (activity information and notifications) and ***effectors*** (operations that can be invoked on the entity) are defined by entities as static fields on the `Entity` subclass. Sensors can be updated by the entity or associated tasks – each update generates an event. An entity can subscribe to sensor events from itself or other entities. A common pattern is for an entity to subscribe to its children (or for a group to subscribe to its members), to aggregate the sensor data and to respond to those events. Effectors can be invoked by an entity's parent, and the invoker is able to track the execution of that effector. Effectors can also be invoked by other entities, but use this functionality with care to prevent too many managers! An entity consists of a Java interface (used when interacting with the entity) and a Java class. For resilience, it is recommended to store the entity's state in attributes (see `getAttribute(AttributeKey)`). If internal fields are used then the data will be lost on brooklyn restart, and may cause problems if the entity is to be moved to a different brooklyn management node.Lifecycle and ManagementContext
^ Under-the-covers, at heart of the brooklyn management plane is the ``ManagementContext``. This is started automatically when launching an application using the brooklyn CLI. For programmatic use, see ``BrooklynLauncher.newLauncher().launch()``. A Brooklyn deployment consists of many entities in a hierarchical tree, with the privileged *application* entity at the top level. An application entity (``Application`` class) is responsible for starting the deployment of all its child entities (i.e. the entire entity tree under its ownership). An ``Application``'s ``start()`` method begins provisioning the child entities of the application (and their entities, recursively). Provisioning of entities typically happens in parallel automatically, although this can be customized. This is implemented as ***tasks*** which are tracked by the management plane and is accessible in the web-based management console and REST API. Customized provisioning can be useful where two starting entities depend on each other. For example, it is often necessary to delay start of one entity until another entity reaches a certain state, and to supply run-time information about the latter to the former. When new entities are created, the entity is wired up to an application by giving it a parent. The entity is then explicitly "managed", which allows other entities to discover it. Typically a Brooklyn deployment has a single management context which records: * all entities under management that are reachable by the application(s) via the parent-child relationships, * the state associated with each entity, * subscribers (listeners) to sensor events arising from the entities, * active tasks (jobs) associated with the entity, * which Brooklyn management node is mastering (managing) each entity. In a multi-location deployment, management operates in all regions, with brooklyn entity instances being mastered in the relevant region.Dependent Configuration
^ Under the covers Brooklyn has a sophisticated sensor event and subscription model, but conveniences around this model make it very simple to express cross-entity dependencies. Consider the example where Tomcat instances need to know the URL of a database (or a set of URLs to connect to a Monterey processing fabric, or other entities) {% highlight java %} setConfiguration(UsesJava.JAVA_OPTIONS, ImmutableMap.of("mysql.url", attributeWhenReady(mysql, MySqlNode.MY_SQL_URL) )) {% endhighlight %} The ``attributeWhenReady(Entity, Sensor)`` call (a static method on the class ``DependentConfiguration``) causes the configuration value to be set when that given entity's attribute is ready. In the example, ``attributeWhenReady()`` causes the JVM system property ``mysql.url`` to be set to the value of the ``MySqlNode.MY_SQL_URL`` sensor from ``mysql`` when that value is ready. As soon as the database URL is announced by the MySql entity, the configuration value will be available to the Tomcat cluster. By default "ready" means being *set* (non-null) and, if appropriate, *non-empty* (for collections and strings) or *non-zero* (for numbers). Formally the interpretation of ready is that of "Groovy truth" defined by an ``asBoolean()`` method on the class and in the Groovy language extensions. You can customize "readiness" by supplying a ``Predicate`` (Google common) or ``Closure`` (Groovy) in a third parameter. This evaluates candidate values reported by the sensor until one is found to be ``true``. For example, passing ``{ it.size()>=3 }`` as the readiness argument would require at least three management plane URLs. More information on this can be found in the javadoc for ``DependentConfiguration``, along with a few other methods such as ``valueWhenAttributeReady`` which allow post-processing of an attribute value. Note that if the value of ``CONFIG_KEY`` passed to ``Entity.getConfig`` is a Closure or Task (such as returned by ``attributeWhenReady``), the first access of ``Entity.getConfig(CONFIG_KEY)`` will block until the task completes. Typically this does the right thing, blocking when necessary to generate the right start-up sequence without the developer having to think through the order, but it can take some getting used to. Be careful not to request config information until really necessary (or to use non-blocking "raw" mechanisms), and in complicated situations be ready to attend to circular dependencies. Trying to resolve a circular dependency leads to a deadlock in those activities. The presence of such deadlocks can be seen by viewing the web-console's 'activities' tab of the (hung) entities – it will show what the executing activities are waiting for. Automated detection/alerting of deadlocks is currently not supported in Brooklyn. The management console gives useful information for understanding what is happening and resolving the cycle.Location
^ Entities can be provisioned/started in the location of your choice. Brooklyn transparently uses [jclouds](http://www.jclouds.org) to support different cloud providers and to support BYON (Bring Your Own Nodes). The implementation of an entity (e.g. Tomcat) is agnostic about where it will be installed/started. When writing the application definition specify the location or list of possible locations (``Location`` instances) for hosting the entity. ``Location`` instances represent where they run and indicate how that location (resource or service) can be accessed. For example, a ``JBoss7Server`` will usually be running in an ``SshMachineLocation``, which contains the credentials and address for sshing to the machine. A cluster of such servers may be running in a ``MachineProvisioningLocation``, capable of creating new ``SshMachineLocation`` instances as required.Policies
^ Policies perform the active management enabled by Brooklyn. Entities can have zero or more ``Policy`` instances attached to them. Policies can subscribe to sensors from entities or run periodically, and when they run they can perform calculations, look up other values, and if deemed necessary invoke effectors or emit sensor values from the entity with which they are associated.Execution
^ All processing, whether an effector invocation or a policy cycle, are tracked as ***tasks***. This allows several important capabilities: * active and historic processing can be observed by operators * the invocation context is available in the thread, to check entitlement (permissions) and maintain a hierarchical causal chain even when operations are run in parallel Some executions create new entities, which can then have tasks associated with them, and the system will record, for example, that a start effector on the new entity is a task associated with that entity, with that task created by a task associated with a different entity. The execution of a typical overall start-up sequence is shown below (click image to magnify): [](brooklyn-flow-websequencediagrams.com.png) ## Integration One vital aspect of Brooklyn is its ability to communicate with the systems it starts. This is abstracted using a ***driver*** facility in Brooklyn, where a driver describes how a process or service can be installed and managed using a particular technology. For example, a ``TomcatServer`` may implement start and other effectors using a ``TomcatSshDriver`` which inherits from ``JavaSoftwareProcessSshDriver`` (for JVM and JMX start confguration), inheriting from ``AbstractSoftwareProcessSshDriver`` (for SSH scripting support). Particularly for sensors, some technologies are used so frequently that they are packaged as ***feeds*** which can discover their configuration (including from drivers). These include JMX and HTTP (see ``JmxFeed`` and ``HttpFeed``). Brooklyn comes with entity implementations for a growing number of commonly used systems, including various web application servers, databases and NoSQL data stores, and messaging systems.Stop/start/restart behaviour
^ Many entities expose `start`, `stop` and `restart` effectors. The semantics of these operations (and the parameters they take) depends on the type of entity. ## Top-level applications A top-level application is a grouping of other entities, pulling them together into the "application" of your choice. This could range from a single app-server, to an app that is a composite of a no-sql cluster (e.g. MongoDB sharded cluster, or Cassandra spread over multiple datacenters), a cluster of load-balanced app-servers, message brokers, etc. ### start(Collection <Location>) This will start the application in the given location(s). Each child-entity within the application will be started concurrently, passing the location(s) to each child. The start effector will be called automatically when the application is deployed through the catalog. Is is strongly recommended to not call start again. ### stop() Stop will terminate the application and all its child entities (including releasing all their resources). The application will also be unmanaged, **removing** it from Brooklyn. ### restart() This will invoke `restart()` on each child-entity concurrently (with the default values for the child-entity's restart effector parameters). Is is strongly recommended to not call this, unless the application has been explicitly written to implement restart. ## Software Process (e.g MySql, Tomcat, JBoss app-server, MongoDB) ### start(Collection <Location>) This will start the software process in the given location. If a machine location is passed in, then the software process is started there. If a cloud location is passed in, then a new VM will be created in that cloud - the software process will be **installed+launched** on that new VM. The start effector will have been called automatically when deploying an application through the catalog. In normal usage, do not invoke start again. If calling `start()` a second time, with no locations passed in (e.g. an empty list), then it will go through the start sequence on the existing location from the previous call. It will **install+customize+launch** the process. For some entities, this could be *dangerous*. The customize step might execute a database initialisation script, which could cause data to be overwritten (depending how the initialisation script was written). If calling `start()` a second time with additional locations, then these additional locations will be added to the set of locations. In normal usage it is not recommended. This could be desired behaviour if the entity had previously been entirely stopped (including its VM terminated) - but for a simple one-entity app then you might as well have deleted the entire app and created a new one. ### stop(boolean stopMachine) If `stopMachine==true`, this effector will stop the software process and then terminate the VM (if a VM had been created as part of `start()`). This behaviour is the inverse of the first `start()` effector call. When stopping the software process, it does not uninstall the software packages / files. If `stopMachine==false`, this effector will stop just the software process (leaving the VM and all configuration files / install artifacts in place). ### restart(boolean restartMachine, boolean restartChildren) This will restart the software process. If `restartMachine==true`, it will also terminate the VM and create a new VM. It will then install+customize+launch the software process on the new VM. It is equivalent of invoking `stop(true)` and then `start(Collections.EMPTY_LIST)`. If `restartMachine==false`, it will first attempt to stop the software process (which should be a no-op if the process is not running), and will then start the software process (without going through the **install+customize** steps). If `restartChildren==true`, then after restarting itself it will call `restart(restartMachine, restartChildren)` on each child-entity concurrently. ## Recommended operations The recommended operations to invoke to stop just the software process, and then to restart it are: * Select the software process entity in the tree (*not* the parent application, but the child of that application). * Invoke `stop(stopMachine=false)` * Invoke `restart(restartMachine=false, restartChildren=false)`Writing Blueprints
^ {% include list-children.html %}The Basic Structure
^ ## A First Blueprint The easiest way to write a blueprint is as a YAML file. This follows the OASIS CAMP plan specification, with some extensions described below. (A [YAML reference](yaml-reference.md) has more information, and if the YAML doesn't yet do what you want, it's easy to add new extensions using your favorite JVM language.) ### The Basic Structure Here's a very simple YAML blueprint plan, to explain the structure: {% highlight yaml %} {% read example_yaml/simple-appserver.yaml %} {% endhighlight %} * The `name` is just for the benefit of us humans. * The `location` specifies where this should be deployed. If you've [set up passwordless localhost SSH access](/guide/locations#localhost) you can use `localhost` as above, but if not, just wait ten seconds for the next example. * The `services` block takes a list of the typed services we want to deploy. This is the meat of the blueprint plan, as you'll see below. Finally, the clipboard in the top-right corner of the example plan box above (hover your cursor over the box) lets you easily copy-and-paste into the web-console: simply [download and launch](/guide/start/running.md) Brooklyn, open a new browser window (usually) at [http://127.0.0.1:8081/](http://127.0.0.1:8081/). Click on the tile "Blueprint Composer", then on the double-arrow located on the top right of the screen (to switch to the YAML mode), paste the copied YAML into the editor and press "Deploy". There are several other ways to deploy, including `curl` and via the command-line, and you can configure users, HTTPS, persistence, and more, as described [in the ops guide](/guide/ops). [](web-console-yaml.png) ### More Information Topics to explore next on the topic of YAML blueprints are: {% include list-children.html %} Plenty of examples of blueprints exist in the Brooklyn codebase, so another starting point is to [`git clone`](/website/developers/code) it and search for `*.yaml` files therein. Brooklyn lived as a Java framework for many years before we felt confident to make a declarative front-end, so you can do pretty much anything you want to by dropping to the JVM. For more information on Java: * start with a [Maven archetype](/guide/blueprints/java/archetype.md) * see all [Brooklyn Java guide](/guide/blueprints/java) topics * look at test cases in the [codebase](https://github.com/apache/brooklyn) You can also come talk to us, on IRC (#brooklyncentral on Freenode) or any of the usual [hailing frequencies](/website/community), as these documents are a work in progress.Entity Configuration
^ Within a blueprint or catalog item, entities can be configured. The rules for setting this configuration, including when composing and extending existing entities, is described in this section. ### Basic Configuration Within a YAML file, entity configuration should be supplied within a `brooklyn.config` map. It is also possible to supply configuration at the top-level of the entity. However, that approach is discouraged as it can sometimes be ambiguous (e.g. if the config key is called "name" or "type"), and also it does not work in all contexts such as for an enricher's configuration. A simple example is shown below: {% highlight yaml %} services: - type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer brooklyn.config: webapp.enabledProtocols: http http.port: 9080 wars.root: https://search.maven.org/remotecontent?filepath=org/apache/brooklyn/example/brooklyn-example-hello-world-webapp/0.12.0/brooklyn-example-hello-world-webapp-0.12.0.war # BROOKLYN_VERSION {% endhighlight %} If no config value is supplied, the default for that config key will be used. For example, `http.port` would default to 8080 if not explicitly supplied. Some config keys also have a short-form (e.g. `httpPort` instead of `http.port` would also work in the YAML example above). However, that approach is discouraged as it does not work in all contexts such as for inheriting configuration from a parent entity. ### Configuration in a Catalog Item When defining an entity in the catalog, it can include configuration values like any other blueprint (i.e. inside the `brooklyn.config` block). It can also explicitly declare config keys, using the `brooklyn.parameters` block. The example below illustrates the principle: {% highlight yaml %} {% read example_yaml/entity-config-catalog.yaml %} {% endhighlight yaml %} Once added to the catalog, it can be used with the simple blueprint below (substituting the location of your choice). Because no configuration has been overridden, this will use the default value for `custom.message`, and will use the given values for `launch.command` and `checkRunning.command`: {% highlight yaml %} {% read example_yaml/entity-config-app.yaml %} {% endhighlight yaml %} For details of how to write and add catalog items, see [Catalog](/guide/blueprints/catalog), and for a complete reference on the syntax of `brooklyn.parameters` see that section of the [YAML Reference](/guide/blueprints/yaml-reference.md). #### Config Key Constraints The config keys in the `brooklyn.parameters` can also have a list of constraints defined, for what values are valid. If more than one constraint is defined, then they must all be satisfied. The constraints can be any of: * `required`: deployment will fail if no value is supplied for this config key * `regex:Setting Locations
^ {% include fields.md %} Brooklyn supports a very wide range of target locations. With deep integration to [Apache jclouds](https://jclouds.apache.org), most well-known clouds and cloud platforms are supported. See the [Locations guide](/guide/locations/) for details and more examples. ### Cloud Example The following example is for Amazon EC2: {% highlight yaml %} {% read example_yaml/simple-appserver-with-location.yaml %} {% endhighlight %} (You'll need to replace the `identity` and `credential` with the "Access Key ID" and "Secret Access Key" for your account, as configured in the [AWS Console](https://console.aws.amazon.com/iam/home).) Other popular public clouds include `softlayer`, `google-compute-engine`, and `rackspace-cloudservers-us`. Private cloud systems including `openstack-nova` and `cloudstack` are also supported, although for these you'll supply an `endpoint: https://9.9.9.9:9999/v2.0/` (or `client/api/` in the case of CloudStack) instead of the `region`. ### "Bring Your Own Nodes" (BYON) Example You can also specify pre-existing servers to use -- "bring-your-own-nodes". The example below shows a pool of machines that will be used by the entities within the application. {% highlight yaml %} {% read example_yaml/simple-appserver-with-location-byon.yaml %} {% endhighlight %} ### Single Line and Multi Line Locations A simple location can be specified on a single line. Alternatively, it can be split to have one configuration option per line (recommended for all but the simplest locations). For example, the two examples below are equivalent: {% highlight yaml %} location: byon(name="my loc",hosts="1.2.3.4",user="bob",privateKeyFile="~/.ssh/bob_id_rsa") {% endhighlight %} {% highlight yaml %} location: byon: name: "my loc" hosts: - "1.2.3.4" user: "bob" privateKeyFile: "~/.ssh/bob_id_rsa" {% endhighlight %} ### Specific Locations for Specific Entities One can define specific locations on specific entities within the blueprint (instead of, or as well as, defining the location at the top-level of the blueprint). The example below will deploy Tomcat and JBoss App Server to different Bring Your Own Nodes locations: {% highlight yaml %} {% read example_yaml/simple-appserver-with-location-per-entity.yaml %} {% endhighlight %} The rules for precedence when defining a location for an entity are: * The location defined on that specific entity. * If no location is defined, then the first ancestor that defines an explicit location. * If still no location is defined, then the location defined at the top-level of the blueprint. This means, for example, that if you define an explicit location on a cluster then it will be used for all members of that cluster. ### Multiple Locations Some entities are written to expect a set of locations. For example, a `DynamicFabric` will create a member entity in each location that it is given. To supply multiple locations, simply use `locations` with a yaml list. In the example below, it will create a cluster of app-servers in each location. One location is used for each `DynamicCluster`; all app-servers inside that cluster will obtain a machine from that given location. {% highlight yaml %} {% read example_yaml/fabric-with-multiple-locations.yaml %} {% endhighlight %} The entity hierarchy at runtime will have a `DynamicFabric` with two children, each of type `DynamicCluster` (each running in different locations), each of which initially has three app-servers. For brevity, this example excludes the credentials for aws-ec2. These could either be specificed in-line or defined as named locations in the catalog (see below). ### Adding Locations to the Catalog The examples above have given all the location details within the application blueprint. It is also possible (and indeed preferred) to add the location definitions to the catalog so that they can be referenced by name in any blueprint. For more information see the [Operations: Catalog](/guide/blueprints/catalog/) section of the User Guide. ### Externalized Configuration For simplicity, the examples above have included the cloud credentials. For a production system, it is strongly recommended to use [Externalized Configuration](/guide/ops/externalized-configuration.md) to retrieve the credentials from a secure credentials store, such as [Vault](https://www.vaultproject.io). ### Use of provisioning.properties An entity that represents a "software process" can use the configuration option `provisioning.properties` to augment the location's configuration. For more information, see [Entity Configuration](/guide/blueprints/entity-configuration.html#entity-provisioningproperties-overriding-and-merging) details.Configuring VMs
^ Another simple blueprint will just create a VM which you can use, without any software installed upon it: {% highlight yaml %} {% read example_yaml/simple-vm.camp.yaml %} {% endhighlight %} *We've omitted the `location` section here and in many of the examples elsewhere; add the appropriate choice when you paste your YAML. Note that the `provisioning.properties` will be ignored if deploying to `localhost` or `byon` fixed-IP machines.* This will create a VM with the specified parameters in your choice of cloud. In the GUI (and in the REST API), the entity is called "VM", and the hostname and IP address(es) are reported as [sensors](/guide/concepts/configuration-sensor-effectors.md). There are many more `provisioning.properties` supported here, including: * a `user` to create (if not specified it creates the same username as `brooklyn` is running under) * a `password` for him or a `publicKeyFile` and `privateKeyFile` (defaulting to keys in `~/.ssh/id_rsa{.pub,}` and no password, so if you have keys set up you can immediately ssh in!) * `machineCreateAttempts` (for dodgy clouds, and they nearly all fail occasionally!) * and things like `imageId` and `userMetadata` and disk and networking options (e.g. `autoAssignFloatingIp` for private clouds) For more information, see [Operations: Locations](/guide/locations).Multiple Services and Dependency Injection
^ We've seen the configuration of machines and how to build up clusters. Now let's return to our app-server example and explore how more interesting services can be configured, composed, and combined. ### Service Configuration We'll begin by using more key-value pairs to configure the JBoss server to run a real app: {% highlight yaml %} {% read example_yaml/appserver-configured.yaml %} {% endhighlight %} (As before, you'll need to add the `location` info; `localhost` will work for these and subsequent examples.) When this is deployed, you can see management information in the Brooklyn Web Console, including a link to the deployed application (downloaded to the target machine from the `hello-world` URL), running on port 8080. **Tip**: If port 8080 might be in use, you can specify `8080+` to take the first available port >= 8080; the actual port will be reported as a sensor by Brooklyn. ### Multiple Services If you explored the `hello-world-sql` application we just deployed, you'll have noticed it tries to access a database. And it fails, because we have not set one up. Let's do that now: {% highlight yaml %} {% read example_yaml/appserver-w-db.yaml %} {% endhighlight %} Here there are a few things going on: * We've added a second service, which will be the database; you'll note the database has been configured to run a custom setup script * We've injected the URL of the second service into the appserver as a Java system property (so our app knows where to find the database) * We've used externalized config to keep secret information out of the blueprint; this is loaded at runtime from an [externalized config provider](../ops/externalized-config.md), such as a remote credentials store **Caution: Be careful if you write your YAML in an editor which attempts to put "smart-quotes" in. All quote characters must be plain ASCII, not fancy left-double-quotes and right-double-quotes!** There are as many ways to do dependency injection as there are developers, it sometimes seems; our aim in Brooklyn is not to say this has to be done one way, but to support the various mechanisms people might need, for whatever reasons. (We each have our opinions about what works well, of course; the one thing we do want to call out is that being able to dynamically update the injection is useful in a modern agile application -- so we are definitively **not** recommending this Java system property approach ... but it is an easy one to demo!) The way the dependency injection works is again by using the `$brooklyn:` DSL, this time referring to the `component("db")` (looked up by the `id` field on our DB component), and then to a sensor emitted by that component. All the database entities emit a `database.url` sensor when they are up and running; the `attributeWhenReady` DSL method will store a pointer to that sensor (a Java Future under the covers) in the Java system properties map which the JBoss entity reads at launch time, blocking if needed. This means that the deployment occurs in parallel, and if the database comes up first, there is no blocking; but if the JBoss entity completes its installation and downloading the WAR, it will wait for the database before it launches. At that point the URL is injected, first passing it through `formatString` to include the credentials for the database (which are defined in the database creation script). ### An Aside: Substitutability Don't like JBoss? Is there something about MariaDB? One of the modular principles we follow in Brooklyn is substitutability: in many cases, the config keys, sensors, and effectors are defined in superclasses and are portable across multiple implementations. Here's an example deploying the same application but with different flavors of the components: {% highlight yaml %} {% read example_yaml/appserver-w-db-other-flavor.yaml %} {% endhighlight %} By changing two lines we've switched from JBoss and MySQL to Tomcat and MariaDB. We've also brought in the `provisioning.properties` from the VM example earlier so our database has 8GB RAM. Any of those properties, including `imageId` and `user`, can be defined on a per-entity basis.Custom Entities
^ So far we've covered how to configure and compose entities. There's a large library of blueprints available, but there are also times when you'll want to write your own. For complex use cases, you can write JVM, but for many common situations, some of the highly-configurable blueprints make it easy to write in YAML, including `bash` and Chef. ### Vanilla Software using `bash` The following blueprint shows how a simple script can be embedded in the YAML (the `|` character is special YAML which makes it easier to insert multi-line text): {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat.yaml %} {% endhighlight %} This starts a simple `nc` listener on port 4321 which will respond `hello` to the first session which connects to it. Test it by running `telnet localhost 4321` or opening `http://localhost:4321` in a browser. Note that it only allows you connect once, and after that it fails. This is deliberate! We'll repair this later in this example. Until then however, in the *Applications* view you can click the server, go to the `Effectors` tab, and click `restart` to bring if back to life. This is just a simple script, but it shows how any script can be easily embedded here, including a script to download and run other artifacts. Many artifacts are already packaged such that they can be downloaded and launched with a simple script, and `VanillaSoftwareProcess` can also be used for them. #### Downloading Files We can specify a `download.url` which downloads an artifact (and automatically unpacking TAR, TGZ, and ZIP archives) before running `launch.command` relative to where that file is installed (or unpacked), with the default `launch.command` being `./start.sh`. So if we create a file `/tmp/netcat-server.tgz` containing just `start.sh` in the root which contains the line `echo hello | nc -l 4321`, we can instead write our example as: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-file.yaml %} {% endhighlight %} #### Determining Successful Launch The default method used to determine a successful launch of `VanillaSoftwareProcess` is to run a command over ssh to do a health check. The health check is done post-launch (repeating until it succeeds, before then reporting that the entity has started). The default command used to carry out this health check will determine if the pid, written to `$PID_FILE` is running. This is why we included in the entity's launch script the line `echo $! > $PID_FILE`. You'll observe this if you connect to one of the netcat services (e.g. via `telnet localhost 4321`): the `nc` process exits afterwards, causing Brooklyn to set the entity to an `ON_FIRE` state. (You can also test this with a `killall nc`). There are other options for determining health: you can set `checkRunning.command` and `stop.command` instead, as documented on the javadoc and config keys of the {% include java_link.html class_name="VanillaSoftwareProcess" package_path="org/apache/brooklyn/entity/software/base" project_subpath="software/base" %} class, and those scripts will be used instead of checking and stopping the process whose PID is in `$PID_FILE`. For example: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-more-commands.yaml %} {% endhighlight %} #### Periodic Health Check After start-up is complete, the health check described above is also run periodically, defaulting to every 5 seconds (configured with the config key `softwareProcess.serviceProcessIsRunningPollPeriod`). This ssh-based polling can be turned off by configuring `sshMonitoring.enabled: false`. However, if no alternative health-check is defined then failure of the process would never be detected by Brooklyn. See [Health Check Sensors](#health-check-sensors) for alternative ways of detecting failures. #### Port Inferencing If you're deploying to a cloud machine, a firewall might block the port 4321. We can tell Brooklyn to open this port explicitly by specifying `inboundPorts: [ 4321 ]`; however a more idiomatic way is to specify a config ending with `.port`, such as: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-port.yaml %} {% endhighlight %} The regex for ports to be opened can be configured using the config `inboundPorts.configRegex` (which has `.*\.port` as the default value). Config keys of type `org.apache.brooklyn.api.location.PortRange` (aka `port`) have special behaviour: when configuring, you can use range notation `8000-8100` or `8000+` to tell Brooklyn to find **one** port matching; this is useful when ports might be in use. In addition, any such config key will be opened, irrespective of whether it matches the `inboundPorts.configRegex`. To prevent any inferencing of ports to open, you can set the config `inboundPorts.autoInfer` to `false`. Furthermore, the port inferencing capability takes in account static `ConfigKey` fields that are defined on any Entity sub-class. So, `ConfigKey` fields that are based on `PortRanges` type will be also included as required open ports. Note that in the example above, `netcat.port` must be specified in a `brooklyn.config` block. This block can be used to hold any config (including for example the `launch.command`), but for convenience Brooklyn allows config keys declared on the underlying type to be specified up one level, alongside the type. However config keys which are *not* declared on the type *must* be declared in the `brooklyn.config` block. ### Passing custom variables Blueprint scripts can be parametrised through environment variables, making them reusable in different use-cases. Define the variables in the `env` block and then reference them using the standard bash notation: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-env.yaml %} {% endhighlight %} Non-string objects in the `env` map will be serialized to JSON before passing them to the script. #### Declaring New Config Keys We can define config keys to be presented to the user using the `brooklyn.parameters` block: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-port-parameter.yaml %} {% endhighlight %} The example above will allow a user to specify a message to send back and the port where netcat will listen. The metadata on these parameters is available at runtime in the UI and through the API, and is used when populating a catalog. The example also shows how these values can be passed as environment variables to the launch command. The `$brooklyn:config(...)` function returns the config value supplied or default. For the type `port`, an attribute sensor is also created to report the *actual* port used after port inference, and so the `$brooklyn:attributeWhenReady(...)` function is used. (If `$brooklyn:config("netcat.port")` had been used, `4321+` would be passed as `NETCAT_PORT`.) This gives us quite a bit more power in writing our blueprint: * Multiple instances of the server can be launched simultaneously on the same host, as the `4321+` syntax enables Brooklyn to assign them different ports * If this type is added to the catalog, a user can configure the message and the port; we'll show this in the next section ### Using the Catalog and Clustering The *Catalog* component allows you to add blueprints which you can refer to in other blueprints. In that tab, click *+* then *YAML*, and enter the following: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-catalog.bom %} {% endhighlight %} This is the same example as in the previous section, wrapped according to the catalog YAML requirements, with one new block added defining an enricher. An enricher creates a new sensor from other values; in this case it will create a `main.uri` sensor by populating a `printf`-style string `"http://%s:%s"` with the sensor values. With this added to the catalog, we can reference the type `netcat-example` when we deploy an application. Return to the *Home* or *Applications* tab, click *+*, and submit this YAML blueprint: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-reference.yaml %} {% endhighlight %} This extends the previous blueprint which we registered in the catalog, meaning that we don't need to include it each time. Here, we've elected to supply our own message, but we'll use the default port. More importantly, we can package it for others to consume -- or take items others have built. We can go further and use this to deploy a cluster, this time giving a custom port as well as a custom message: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-cluster.yaml %} {% endhighlight %} In either of the above examples, if you explore the tree in the *Applications* view and look at the *Summary* tab of any of the server instances, you'll now see the URL where netcat is running. But remember, netcat will stop after one run, so you'll only be able to use each link once before you have to restart it. You can also run `restart` on the cluster, and if you haven't yet experimented with `resize` on the cluster you might want to do that. ### Attaching Policies Besides detecting this failure, Brooklyn policies can be added to the YAML to take appropriate action. A simple recovery here might be just to restart the process automatically: {% highlight yaml %} {% read example_yaml/vanilla-bash-netcat-restarter.yaml %} {% endhighlight %} Autonomic management in Brooklyn often follows the principle that complex behaviours emerge from composing simple policies. The blueprint above uses one policy to trigger a failure sensor when the service is down, and another responds to such failures by restarting the service. This makes it easy to configure various aspects, such as to delay to see if the service itself recovers (which here we've set to 15 seconds) or to bail out on multiple failures within a time window (which again we are not doing). Running with this blueprint, you'll see that the service shows as on fire for 15s after a `telnet localhost 4321`, before the policy restarts it. ### Sensors and Effectors #### Effectors For an even more interesting way to test it, look at the blueprint defining [a netcat server and client](example_yaml/vanilla-bash-netcat-w-client.yaml). This uses `brooklyn.initializers` (see in the [YAML reference](yaml-reference.md)) to define an effector to `sayHiNetcat` on the `Simple Pinger` client, using `env` variables to inject the `netcat-server` location and `parameters` to pass in per-effector data: env: TARGET_HOSTNAME: $brooklyn:entity("netcat-server").attributeWhenReady("host.name") brooklyn.initializers: - type: org.apache.brooklyn.core.effector.ssh.SshCommandEffector brooklyn.config: name: sayHiNetcat description: Echo a small hello string to the netcat entity command: | echo $message | nc $TARGET_HOSTNAME 4321 parameters: message: description: The string to pass to netcat defaultValue: hi netcat #### Sensors This blueprint also uses initializers to define sensors on the `netcat-server` entity so that the `$message` we passed above gets logged and reported back: launch.command: | echo hello | nc -l 4321 >> server-input & echo $! > $PID_FILE brooklyn.initializers: - type: org.apache.brooklyn.core.sensor.ssh.SshCommandSensor brooklyn.config: name: output.last period: 1s command: tail -1 server-input More information on this and the sensors described below are in the [YAML Reference](yaml-reference.md). #### Container Sensor This blueprint uses initializers to define a `ContainerSensor` on a `BasicStartable` entity so that a random number is generated by executing a command on a container for this specific purpose, and destroyed afterwards. {% highlight yaml %} name: entity-with-container-sensor services: - type: 'org.apache.brooklyn.entity.stock.BasicStartable:1.1.0-SNAPSHOT' brooklyn.initializers: - type: org.apache.brooklyn.tasks.kubectl.ContainerSensor brooklyn.config: image: perl imagePullPolicy: never bashScript: | echo hello name: test-sensor period: 20s {% endhighlight %} `ContainerSensor` shared configuration details with `ContainerEffector`, so for more details about the `ContainerSensor` available configuration keys, see [here](effectors.md#ContainerEffector). #### Windows Command Sensor Like the blueprint above, the following example also uses `brooklyn.initializers` to define sensors on the entity, this time however it is a Windows VM and uses `WinRmCommandSensor`. - type: org.apache.brooklyn.entity.software.base.VanillaWindowsProcess brooklyn.config: launch.command: echo launching checkRunning.command: echo running brooklyn.initializers: - type: org.apache.brooklyn.core.sensor.windows.WinRmCommandSensor brooklyn.config: name: ip.config period: 60s command: hostname #### Health Check Sensors As mentioned [previously](#periodic-health-check), the default health check is to execute the check-running command over ssh every 5 seconds. This can be very CPU intensive when there are many entities. An alternative is to disable the ssh-polling (by setting `sshMonitoring.enabled: false`) and to configure a different health-check. See documentation on the [Entity's error status](/guide/ops/troubleshooting/overview.md#entitys-error-status) for how Brooklyn models an entity's health. In the snippet below, we'll define a new health-check sensor (via HTTP polling), and will automatically add this to the `service.notUp.indicators`. If that map is non-empty, then the entity's `service.isUp` will be set automatically to `false`: services: - type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess brooklyn.config: launch.command: | ... checkRunning.command: true sshMonitoring.enabled: false brooklyn.initializers: - type: org.apache.brooklyn.core.sensor.http.HttpRequestSensor brooklyn.config: name: http.healthy period: 5s suppressDuplicates: true jsonPath: "$" uri: $brooklyn:formatString: - "http://%s:8080/healthy" - $brooklyn:attributeWhenReady("host.name") brooklyn.enrichers: - type: org.apache.brooklyn.enricher.stock.UpdatingMap brooklyn.config: enricher.sourceSensor: $brooklyn:sensor("http.healthy") enricher.targetSensor: $brooklyn:sensor("service.notUp.indicators") enricher.updatingMap.computing: $brooklyn:object: type: "com.google.guava:com.google.common.base.Functions" factoryMethod.name: "forMap" factoryMethod.args: - true: null false: "false" - "no value" The `HttpRequestSensor` configures the entity to poll every 5 seconds on the given URI, taking the JSON result as the sensor value. The `UpdatingMap` enricher uses that sensor to populate an entry in the `service.notUp.indicators`. It transforms the `http.healthy` sensor value using the given function: if the HTTP poll returned `true`, then it is mapped to `null` (so is removed from the `service.noUp.indicators`); if the poll returned `false`, then `"false"` is added to the indicators map; otherwise `"no value"` is added to the indicators map. #### Summary These examples do relatively simple things, but they illustrate many of the building blocks used in real-world blueprints, and how they can often be easily described and combined in Brooklyn YAML blueprints.Uploading Script and Configuration Files
^ Blueprints often require that parameterized scripts and configuration files are available to be copied to the target VM. These must be URLs resolvable from the Brooklyn instance, or on the Brooklyn classpath. There are two types of file that can be uploaded: plain files and templated files. A plain file is uploaded unmodified. A templated file is interpreted as a [FreeMarker](http://freemarker.org) template. This supports a powerful set of substitutions. In brief, anything (unescaped) of the form `${name}` will be substituted, in this case looking up "name" for the value to use. ## Writing templates Templated files (be they configuration files or scripts) give a powerful way to inject dependent configuration when installing an entity (e.g. for customising the install, or for referencing the connection details of another entity). Available substitutions are: | Substitution | Effect | |---------------------------|--------------------------------------------------------------------| | `${config['key']}` | Equivalent to `entity.config().get(key)` | | `${attribute['key']}` | Equivalent to `entity.sensors().get(key)` | | `${mgmt['key']}` | Loads the value for `key` from the management context's properties | | `${entity.foo}` | FreeMarker calls `getFoo` on the entity | | `${driver.foo}` | FreeMarker calls `getFoo` on the entity's driver | | `${location.foo}` | FreeMarker calls `getFoo` on the entity's location | | `${javaSysProps.foo.bar}` | Loads the system property named `foo.bar` | Additional substitutions can be given per-entity by setting the `template.substitutions` key. For example, to include the address of an entity called db: brooklyn.config template.substitutions: databaseAddress: $brooklyn:entity("db").attributeWhenReady("host.address") The value can be referenced in a template with `${databaseAddress}`. FreeMarker evaluates all expressions between `${}` which may be inappropriate in certain kinds of files. To include the literal `${value}` in a script you might: * specify a [raw string literal](http://freemarker.org/docs/dgui_template_exp.html#dgui_template_exp_direct_string): `${r"${value}"}` * use the [noparse](http://freemarker.org/docs/ref_directive_noparse.html) directive: `<#noparse>${value}` * use FreeMarker's [alternative syntax](http://freemarker.org/docs/dgui_misc_alternativesyntax.html). A common pattern for templating Bash files is to set environment variables at the top of the script and to surround the rest of its contents with `noparse`. For example: GREETING=${config['greeting']} NAME=${config['name']} <#noparse> # The remainder of the script can be written as normal. echo "${GREETING}, ${NAME}!" ## Using templates in blueprints Files can be uploaded at several stages of an entity's lifecycle: | Config key | Copied before lifecycle phase | Templated | Relative to | |------------------------|-------------------------------|-----------|--------------| | `files.preinstall` | Pre-install | ✕ | `installDir` | | `files.install` | Install | ✕ | `installDir` | | `files.customize` | Pre-customize command | ✕ | `installDir` | | `files.runtime` | Pre-launch command | ✕ | `run.dir` | | `templates.preinstall` | Pre-install | ✓ | `installDir` | | `templates.install` | Install | ✓ | `installDir` | | `templates.customize` | Pre-customize command | ✓ | `installDir` | | `templates.runtime` | Pre-launch command | ✓ | `run.dir` | Each key accepts a map of values where a key indicates the source of a file and a value its destination on the instance. Files can be referenced as URLs. This includes support for: * `classpath://mypath/myfile.bat`, which looks for the given (fully qualified) resource on the Brooklyn classpath or inside the bundle, if using the OSGi version of Brooklyn with a catalog blueprint. * `file://`, which looks for the given file on the Brooklyn server, and * `http://`, which requires the file to be accessible from the Brooklyn instance. Destinations may be absolute or relative. Absolute paths need not exist beforehand, but Brooklyn's SSH user must have sufficient permission to create all parent directories and the file itself. Relative paths are copied as described in the table above. ### Example files.preinstall: # Reference a fixed resource classpath://com/acme/installAcme.ps1: C:\\acme\installAcme.ps1 # Inject the source from a config key $brooklyn:config("acme.conf"): C:\\acme\acme.conf ## Windows notes * When writing scripts for Windows ensure that each line ends with "\r\n", rather than just "\n". * The backslash character (\\) must be escaped in paths. For example: `C:\\install7zip.ps1`.Workflow
^ Apache Brooklyn has a powerful workflow engine and syntax for defining entities, effectors, sensors, and policies. For example, to define an effector `one-and-two` which invokes effector `one` then effector `two`, you can write: ```yaml brooklyn.initializers: - type: workflow-effector name: one-and-two steps: - invoke-effector one - invoke-effector two ``` This can be used within [most Apache Brooklyn resources](defining.md). The syntax supports [longhand, conditions, loops, error-handling](common.md), [variables](variables.md), a large set of [built-in step types](steps/), and the ability to [define custom step types](nested-workflow.md). You can also get started by looking at a variety [examples](examples/). Finally, you can run individual workflows in on an existing deployment using the "Run workflow" facility in the UI. This is a good way to build familiarity with workflows. Access this by selecting an entity in App Inspector and opening the top-right three-lines menu. To create a sandbox entity to experiment on, deploy a `services: [ { type:org.apache.brooklyn.entity.stock.BasicEntity } ]` or a `services: [ { type: server, location: localhost } ]`. If you prefer to read sequentially, simply follow the sub-menu options.Defining Workflow
^ Let's start by discussing _why_ workflow is introduced and where it can and should be used. The Apache Brooklyn Workflow is designed to make it easy to describe behaviour of entities, effectors, sensors, and policies in blueprints. It has the sophistication of a programming language including [conditions, loops, and error-handling](common.md) and [variables](variables.md), but more important are the [steps](steps/) which delegate to other systems, such as containers or SSH or HTTP endpoints. Complex programming logic will typically be done in another system, such as a container; but where effectors and sensors need to interact with Brooklyn, such as reading and setting sensors, or invoking effectors, it is often simplest to do that directly in the blueprint. Workflow in a blueprint has the benefit of being transparent to users and having the Brooklyn context easily available. It also has the advantage that Brooklyn will retry in certain situations. However it has the disadvantage that, although we've tried to make it as easy as we can for people to write, you are still writing code in YAML rather than in a first-class programming language. The following are places that workflow can be used. ### Effectors An effector can be defined using workflow and added to an entity as follows: ``` - type: some-entity brooklyn.initializers: - type: workflow-effector brooklyn.config: name: say-hi-and-publish-sensor steps: - log Hi - set-sensor boolean said_hi = true ``` This initializer will define the effector `say-hi-and-publish-sensor` which uses workflow to do just that. The config to define the effector is: * `name`: the name of the effector to define (required) * `parameters`: an optional map of parameters to advertise for the effector, keyed by the parameter name against the definition as a map including optionally `type`, `description`, and `defaultValue`; see [nested workflow](nested-workflow.md) for more details To define the workflow, this requires: * `steps`: to supply the list of steps defining the workflow (required) And the following optional common configuration keys are supported, with the same semantics as for individual steps as described under [Common Step Properties](common.md): * `condition`: an optional condition on the effector which if set and evaluating to false, prevents the effector workflow from running; this does _not_ support interpolated variables * `input`: a map of keys to values to make accessible in the workflow, in addition to effector `parameters` * `output`: defines the output of the workflow, often referring to workflow [variables](variables.md) or the output of the last step in the workflow * other [common step properties](common.md) such as `timeout`, `on-error`, and `next` * other [workflow settings properties](settings.md) such as `lock`, `retention`, `replayable`, and `idempotent` ### Sensors A sensor feed to use a workflow to compute a sensor value based on triggers and/or a schedule can be defined as follows: ``` - type: some-entity brooklyn.initializers: - type: workflow-sensor brooklyn.config: sensor: count-how-often-other_sensor-is-published triggers: - other_sensor steps: - let integer x = ${entity.sensor.x} + 1 ?? 0 - return ${x} ``` This initializer will add the sensor to the entity's type signature, then run the indicated workflow, periodically and/or on sensors, and set the return value as the value of the sensor. The config to define the sensor feed is: * `sensor`: (required) a string specifying the sensor name or map with keys `name` (required) being the sensor name and optionally `type` being a type to record for the sensor * `triggers`: a list of sensors which should trigger this when published, each entry declared either as the string name if on the local entity, or a map of `sensor` containing the name and `entity` containing the entity or entity ID where the sensor should be listened for, or if just a single local sensor, that sensor name supplied as a string * `period`: whether the feed should run periodically * `skip_initial_run`: by default sensors and policies will run when created (if any `condition` is met); this can be set `true` to prevent that, ensuring it is only run after the initial `period` or when one of the `triggers` fires In addition to defining the `sensor` name, at least one of `triggers` or `period` must be supplied. The `steps` must also be defined, as per `workflow-effector` above, and the same common configuration is supported. ### Policies A policy to to run a workflow based on triggers and/or a schedule can be defined as follows: ``` - type: some-entity brooklyn.policies: - type: workflow-policy brooklyn.config: name: invoke-effector-other_sensor-is-published triggers: - other_sensor steps: - invoke-effector some_effector ``` This initializer will add the policy to the entity's defined management adjuncts, then run the indicated workflow, periodically and/or on sensors. The config to define the policy feed is: * `name`: the name for the policy (recommended) * `triggers` and/or `period`: per `workflow-sensor` above (at least one required) The `steps` must also be defined, as per above, and the same common configuration is supported. ### Initializer A workflow can be made to run when an entity is created using a `workflow-initializer`. This can do any custom entity setup, including the above tasks using `add-policy` or `apply-initializer` steps, as follows: ``` - type: some-entity brooklyn.initializers: - type: workflow-initializer brooklyn.config: name: initializer-to-say-hi-then-add-effector-and-sensor steps: - log Hi this workflow initializer will run at entity creation - step: add-policy blueprint: type: workflow-policy brooklyn.config: name: invoke-effector-other_sensor-is-published triggers: - other_sensor steps: - invoke-effector say-hi-and-publish-sensor - step: apply-initializer blueprint: type: workflow-effector brooklyn.config: name: say-hi-and-publish-sensor steps: - log Hi - set-sensor boolean said_hi = true ``` The `workflow-initializer` takes the same config as `workflow-effector` with the exception of `parameters`. ### Workflow Entities New entities can be written to use workflow for their start and stop behavior by extending the type `workflow-entity`. This requires a `start` and `stop` configuration to be supplied defining the workflow for those steps. Optionally `restart` can be supplied, which if omitted will default to stopping then starting. ``` - type: workflow-entity brooklyn.config: start: steps: - log Starting up stop: steps: - log Stopping ``` The `workflow-entity` will automatically set the `service.isUp` and `service.state` sensors based on invocation of `start` and `stop` and the success of the workflow. It will also take into consideration the map sensors `service.problems` and `service.notUp.indicators`, where any entry in the former will cause `service.state` to show as "on-fire" if it is meant to be running, and any entry in the latter will cause `service.isUp` to become false (which will trigger an entry in `service.problems` if it is meant to be running). Thus liveness and health checks can, and often should, be added, such as in the following getting the `status_code` from the `main.uri`, and setting a `service.problems` if it is unavailable: ``` - type: workflow-entity brooklyn.config: start: steps: - ... # start the service, e.g. using container or http call - clear-sensor status_code - set-sensor main.uri = ... # get the URL from the previous; this will trigger sensor feed - step: wait ${entity.sensor.status_code} timeout: 5m stop: # omitted brooklyn.initializers: - type: workflow-sensor brooklyn.config: sensor: status_code period: 1m triggers: - main.uri steps: - step: http ${main.uri} on-error: - set-sensor service.problems['endpoint-live'] = ${error} - fail rethrow message Endpoint is not accessible - clear-sensor service.problems['endpoint-live'] - return ${status_code} ``` The workflow entity does not automatically start or stop children. If this is required, it should be part of the start/stop workflow, as follows: ``` - type: workflow-entity brooklyn.config: start: steps: - ... # start this # now start children - type: workflow target: children steps: - invoke-effector start stop: steps: # stop children first - type: workflow target: children steps: - invoke-effector stop - ... # then stop this ``` ### Workflow Software Process Entities Entities that run software or another machine-based process can also be defined, relying on Apache Brooklyn to provision a machine if required, then using workflow for the install, customize, launch, check-running, and stop phases. These entities will provision a machine if if given a cloud or machine provisioning location, and use the same latches and pre/post phase on-box commands as the ofther `SoftwareProcess` entities, but for the key phases it will take a `workflow` object as for `workflow-entity`. The `steps` will often run `ssh` and possibly `set-sensor`, and they can access the run dir and install dir using Freemarker `${entity.driver.installDir}` and `${entity.driver.runDir}`. The `workflow` object can also define inputs and outputs for use locally, and `on-error` handlers. The `checkRunning.workflow` should return `true` or `false` to indicate whether the software is running. Alternatively, by setting the entity config `usePidFile: true` and in the launch command writing the process id (PID) to `${entity.driver.pidFile}`, Brooklyn's automatic PID detection can be used. As an example: ``` - type: workflow-software-process brooklyn.config: install.workflow: steps: - ssh yum update - ssh yum install ... launch.workflow: steps: - let PORT = 4321 - 'ssh { echo hello | nc -l ${PORT} & } ; echo $! > /tmp/brooklyn-nc.pid' - 'set-sensor main.uri = http://localhost:${PORT}/' checkRunning.workflow: steps: - step: ssh ps -p `cat /tmp/brooklyn-nc.pid` timeout: 10s on-error: - return false - return true stop.workflow: steps: - ssh kill -9 `cat /tmp/brooklyn-nc.pid` ```Common Step Properties
^ ### Shorthand and Longhand Syntax Where possible, and in most examples so far, we've used the "shorthand" syntax for steps which uses a string template to make it easy to write workflow steps: ``` steps: - ssh echo today is `DATE` - http http://some-rest-api/ - container my/container command - set-sensor the-output = ssh says ${1.stdout}, http ${2.data}, container ${3.stdout} ``` This always starts with the type name, and the remainder is parsed according to a template defined by that step type. Internally, steps are maps with multiple inputs and properties, and the shorthand syntax makes it possible to set the most common ones. For `ssh` this is the command to run; for `container` an image; for `http` a URL; and for `set-xxx` an expression of the form `key=value`. It is always possible to write the longhand map syntax, and if using inputs or properties which aren't supported in the shorthand template (such as the `condition` property described below), the longhand map syntax is required: ``` steps: - type: ssh input: command: echo today is `DATE` condition: target: ${skip_date} not: { equals: true } - ... ``` Shorthand can be combined as part of a map by providing the step shorthand string in a key `step` (or `s` or `shorthand`). The type must be included as the first word, and the `type` key must not be used. ``` steps: - step: ssh echo today is `DATE` condition: target: ${scratch.skip_date} not: { equals: true } ``` Care should be taken when using `:` in a step with shorthand. YAML will parse it as a map if it is not quoted in YAML. However at runtime, if the step looks like it came from an accidental colon causing a map, it will be reverted to a string with the colon re-introduced, so you can write steps with shorthand `- log Your name is: ${name}`. All steps support a number of common properties, described below. ### Explicit IDs and Name Steps can define an explicit ID for use with `next`, for correlation in the UI, and to be able to reference the output or input from a specific step using the [workflow expression syntax](variables.md). They can also include a `name` used in the UI. ``` steps: - type: no-op next: skipping-ssh-date condition: target: ${scratch.skip_date} equals: true - step: ssh echo today is `date` name: Doing SSH next: end - id: skipping-ssh-date name: Not doing SSH step: log skipping ssh date command ``` ### Conditions The previous example shows the use of conditions, as mentioned as one of the properties common to all steps. This makes use of the recent [Predicate DSL](../yaml-reference.md#predicate-dsl) conditions framework. It is normally necessary to supply a `target`, unless one of the entity-specific target keys (e.g. `sensor` or `config`) is used. The target and arguments here can use the [workflow expression syntax](variables.md). The condition is evaluated when the step is about to run, and if the condition is not satisfied, the workflow moves to the following step in the sequence, or ends if that was the last step. Apart from `name` and `id` above, if a step's `condition` is unmet, the other properties set on a step are ignored. The `if` step can be used for simple conditions, as shown in the next section. ### Jumping with "Next" or "Goto" The common property `next` allows overriding the workflow sequencing, indicating that a different step should be gone to next. This does not apply if a step's condition is not satisfied, as noted at the end of the previous section. The value of the `next` property should be the ID of the step to go to or one of the following reserved words: * `start`: return to the start of the workflow * `end`: exit the workflow (or if in a block where this doesn't make sense, such as `retry`, go to the last executed step) * `exit`: if in an error handler, exit that error handler The `goto` step type is equivalent to the `no-op` step with `next` set, as a simpler idiom for controlling workflow flow. While `goto` is "considered harmful" in many programming environments, for declarative workflow it is fairly common, because it can simplify what might otherwise involve multiple nested workflows. That said, the confusion that `goto` can cause should be kept in mind, and its availability not abused: in particular where a task can be better done in a proper high-level programming language, consider putting that program into a container and calling it from your workflow. Thus the above workflow can be written more concisely as: ``` steps: - if ${scratch.skip_date} then goto skipping-ssh-date - step: ssh echo today is `date` next: end - id: skipping-ssh-date name: Not doing SSH step: log skipping ssh date command ``` ### Input and Output Most steps take input parameters and return output. Many step-specific input parameters can be set in the shorthand, but not all. All input parameters can be specified in an `input` block. It is also possible to customize the output from a step using an `output` block. For example: ``` - step: let target = aws condition: target: location tag: aws next: picked-target - step: let target = azure condition: target: location tag: azure next: picked-target - input: location_name: ${entity.location.name} step: log Unrecognized cloud ${location_name}, using default output: cloud: default next: end - id: picked-target step: log Picked target ${target} output: cloud: ${target} ``` The above will return an output map containing a key `cloud` and a value of either `azure`, `aws`, or `default`. In addition, a custom `input` variable is passed to the third step. (This is not the simplest way to write this logic, but it illustrates the concepts.) This example also shows the expression syntax. More on inputs, outputs, variables, and expressions is covered [here](variables.md). ### Timeout Any step and/or an entire workflow can define a `timeout:Variables and Expressions
^ Steps can take input and return output, as do the containing workflows in most cases. In workflow, Apache Brooklyn supports an interpolation expression syntax to access these values. You can also use the Brooklyn DSL, but in most cases the interpolation syntax is more convenient. For example, you can write: ``` - log Starting workflow ${workflow.name} - let integer x = 1 - id: log step: log The value for x is now ${x} - step: let x = ${x} + 1 next: log condition: target: ${x} less-than: 3 ``` This will output a series of log messages such as: ``` Starting workflow: my-workflow The value for x is now 1 The value for x is now 2 The value for x is now 3 ``` ### Workflow Variables The above illustration showed how `let x =Workflow Steps
^ Apache Brooklyn workflow supports a range of step types covering the most common activities commonly done as part of application management. These are divided into the following groups: * [Workflow Control](#workflow-control): use local variables and control flow, with step types such as `let`, `return`, and `retry` * [External Actions](#external-actions): interact with external systems using step types such as `container`, `ssh`, `winrm`, and `http` * [Application Models](#application-models): work with the models stored in Brooklyn, doing things such as `invoke-effector`, `set-sensor`, `deploy-application`, and `add-entity` * [General Purpose](#general-purpose): miscellaneous step types such as `log` and `sleep` Custom step types can be written and added to the catalog, either written as workflow using these primitives (including doing virtually anything in a container) or by implementing a Java type, both [as described here](../nested-workflow.md). An index of all out-of-the-box step types is included at the [end of this section](#index-of-step-types). {% jsonball steps from yaml file steps.yaml %} {% assign step_summaries = "" | split: ", " %} {% for x in steps %} {% assign step_summaries = step_summaries | concat: x.steps %} ## {{ x.section_name }} {{ x.section_intro }}
{% for step in x.steps %}
* [`{{ step.name }}`](#{{ step.name }})
{% endfor %}
{% for step in x.steps %}
### `{{ step.name }}`
{{ step.summary }}
{% if step.shorthand %}
**Shorthand**: {{ step.shorthand }}
{% endif %}
{% if step.input %}
**Input parameters**:
{{ step.input }}
{% endif %}
{% if step.output %}
**Output return value**:
{{ step.output }}
{% endif %}
{% endfor %}
{% endfor %}
## Index of Step Types
{% assign step_summaries = step_summaries | sort: "name" %}
{% for x in step_summaries %}
* [{% if x.shorthand %}`{{ x.name }}`{% else %}`{{ x.name }}`{% endif %}](#{{ x.name }})
{% endfor %}
Nested and Custom Workflow
^ The `type: workflow` can be used to create new step types and save them to the Brooklyn type registry. New step types can be used in the same way as the [standard step types](steps/). This allows common workflow logic to be saved and re-used as steps in other workflow. The usual properties of steps -- inputs, outputs, error handling -- are all supported, and it supports defining parameters and even custom shorthand templates. This step type can also be used directly within a workflow to run a "nested workflow", for instance to structure code and to isolate a sequence of steps or to apply `on-error` behavior to a group of steps. Nested and custom workflows are not permitted to access data from their containing workflow; instead, they accept an `input` block like other steps. This type permits all the [common step properties](common.md) and all the [workflow settings properties](settings.md), plus a few others, `target`, `concurrency`, `parameters`, and `shorthand` as described below. ### Basic Usage in a Workflow When used in a workflow, the nested workflow should be defined as a list in the `steps` key. An `input` section can also be defined to pass data from the outer workflow to the nested workflow. For example: ``` - log This is about to run a nested workflow - let x=1 - let y=2 - type: workflow input: x: ${x} steps: - log This is a nested workflow, able to see x=${x} from input but not y from the output workflow. on-error: # error handler which runs if the nested workflow fails (i.e. if any step therein fails and does not correct it) ``` ### Loops and Parallelization The `workflow` type can also be used to run a sequence of steps on multiple targets. If given a `target` value, Brooklyn will run the workflow against that target or targets, as follows: * If the target is a managed entity, e.g. `$brooklyn:entity("some-child")`, the nested workflow will run in the scope of that entity. It will be visible in the UI under that entity, and references to sensors and effectors will be aginst that entity. * If the target is any value which resolves to a list, it will be run against every entry in the list, with the variable expression `${target}` available in the sub-workflow to refer to the relevant entry * If the target is `children` or `members` it will run against each entity in the relevant list * If the target is of the form `M..N` for integers `M` and `N` it will run for all integers in that range, inclusive (so the string `1..4` is equivalent to the list `[1,2,3,4]`) The scratch variables `target` and `target_index` are available referring to to the specific target and its 0-indexed position. These names can be overridden with the `target_var_name` and `target_index_var_name` keys. Where a list is supplied, the result of the step is the list collecting the output of each sub-workflow. If a `condition` is supplied when a list is being used, that `condition` is applied to each entry in the list. The `workflow` step itself will always appear to run, but might have 0 nested workflows, and in that case will return the empty list. If the condition is intended to block the list from being evaluated, and should be evaluated in the containing workflow's scope, consider using a `subworkflow` with the outer condition containing the `workflow`. An example of this is included below, using `service.isUp` with `children`. ### Associated Step Types The `foreach` type is a simplified variant of `workflow` when recursing over a list, taking the same arguments as `workflow` but supporting a shorthand syntax where the `target_var_name` and `target` can be specified as in `foreach VAR_NAME in TARGET`. A single step can be supplied as `do STEP` at the end; either that or `steps` is required (but not both), and that step or those steps are run for all items in the target list. The `subworkflow` type is another simplified variant that does not allow any target (so no recursing over a list and no concurrency), and which -- unlike the other workflow step types -- shares variables with the calling workflow. All variables from the immediate containing workflow can be accessed and updated in a `subworkflow`, and new variables defined in the `subworkflow` are subsequently available in the outer workflow. The `steps` are required/ A `name` is frequently given for readability, as are `condition` or `on-error` entries to apply a condition or error-handler to the block of steps. If no step type is defined, but `steps` are defined, this `subworkflow` step type is assumed. The `if` type acts like `subworkflow` but allows specifying a condition in shorthand. #### Example ``` - foreach x in 1..3 do return ${x} ``` The above loop will return `[1,2,3]` and proceed to the next step. ``` # foreach, nested workflow - step: foreach x in 1..3 steps: - let x = ${x} + 1 - return ${x} ``` The above loop will return `[2,3,4]` and proceed to the next step. It is the same as the previous except it is supplying multiple steps to run on each iteration. Note that the assignment to `${x}` does not side-effect the loop, and nor is it available to the outer workflow, when using `foreach` or `workflow`. Use `${output}` to access the result, and see the `reducing` section below. ``` # subworkflow - let x = 1 - steps: - let x = ${x} + 1 - return ${x} - let x = Not run, but if it were it could access ${x} ``` The above implicit lightweight subworkflow _does_ run in the parent workflow's context, so it will access and update the same instance of `${x}`, and it will `return` from the workflow, not running the next step. ``` # if - let x = 1 - if ${x} == 1 then let x = ${x} + 1 - return ${x} ``` The above `if` step can access and update outer variables, like `subworkflow`, providing a convenient shorthand for simple conditions and steps. It is functionally identical to this: ``` - let x = 1 - condition: target: ${x} equals: 1 steps: - let x = ${x} + 1 - return ${x} ``` ### Reducing Each nested workflow runs in its own scope and does not share workflow variables with the parent, apart from values specified as `input`, or with other iterations of a loop. Where it is desired to share variables across iterations, the key `reducing` can be supplied, giving a map of variable names to be shared and their initial values. When `reducing`, the indicated variables are available as scratch workflow variables in the calling workflow. This cannot be used with `subworkflow`, as all variables are considered reduced in a `subworkflow`, and it cannot be used with `concurrency` as that risks race conditions in updating the reduced variables. (You can use `inputs` with both to provide new local variables.) #### Example ``` - step: foreach x in 1..3 do let sum = ${sum} + ${x} reducing: sum: 0 ``` The above loop will return `6`. ### Concurrency By default nested workflows with list targets run sequentially over the entries, but this can be varied by setting `concurrency`. The following values are allowed: * a number to indicate the maximum number of simultaneous executions (with `1` being the default, for no concurrency) * the string `unlimited` to allow all to run in parallel * a negative number to indicate all but a certain number * a percentage to indicate a percentage of the targets * the string `min(...)` or `max(...)`, where `...` is a comma separated list of valid values This concisely allows complicated -- but important in the real world -- logic such as `max(1, min(50%, -10))` to express running concurrently over up to half if more than twenty, otherwise all but 10, and always allowing 1. This might be used for example to upgrade a cluster in situ, leaving the larger of 10 instances or half the cluster alone, if possible. If the concurrency expression evaluates to 0, or to a negative number whose absolute value is larger than the number of values, the step will fail before executing, to ensure that if e.g. "-10" is specified when there are fewer than 10 items in the target list, the workflow does not run. (Use "max(1, -10)" to allow it to run 1 at a time if there are 10 or fewer.) Note: Concurrency cannot be specified when `reducing`. #### Example This example invokes an effector on all children which are `service.isUp`, running in batches of up to 5 but not more than a third of the children at once: ``` - type: workflow target: children concurrency: max(1, min(33%, 5)) condition: sensor: service.isUp equals: true steps: - invoke-effector effector-on-children ``` As noted above, the `condition` here is evaluated on each child. To have it evaluated on the parent context instead, you can write: ``` - condition: sensor: service.isUp equals: true steps: - type: workflow target: children concurrency: max(1, min(33%, 5)) steps: - invoke-effector effector-on-children ``` ### Defining Custom Workflow Steps This type can be used to define new step types and add them as new types in the type registry. The definition must specify the `steps`, and may in addition specify: * `parameters`: a map of parameters accepted by the workflow, with the key the parameter name, and the value map possibly empty or providing optional `type` (default `string`), `defaultValue` (default none), `required` (default `false`), `description` (default none), and/or `constraints` * `shorthand`: a template, as described below * `output`: an output map or value to be returned by the step, evaluated in the context of the nested workflow When this type is used to define a new workflow step, the newly defined step does _not_ allow the `steps` or any of the parameters listed above to be overridden. Instead it accepts the parameters defined in the `parameters` key of the definition. It also accepts the standard step keys such as `input`, `timeout` on `on-error`. A user of the defined step type can also supply `output` which, as per other steps, is evaluated in the context of the outer workflow, with visibility of the output from the current step. When supplying a workflow in contexts where a `workflow` is already expected, such as in a config key that takes a `workflow` (a Java `CustomWorkflowStep`), it is not necessary to specify the `type: workflow`, and additionally, if the only things being set is `steps`, those steps can be provided as a list without the `steps` keyword. Internally a _list_ will coerce to a `workflow` by interpreting the list as the steps. #### Shorthand Template Syntax A custom workflow step can define a `shorthand` template which permits a user to use the workflow step as a string rather than a map, even with parameters. The shorthand template syntax consists of a sequence of the following tokens: * `${VAR}` - to set VAR, which should be of the regex [A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*, with dot separation used to set nested maps * `${VAR...}` - as `${VAR}` but allowing it to match multiple words * `"LITERAL"` - to expect the user to supply the exact token `LITERAL`; this should include spaces if spaces are required * `[Workflow Settings
^ Wherever workflows are defined, such as in [effectors, sensors and policies](defining.md) and in [nested workflow](nested-workflow.md), there are a number of properties which can be defined. The most common of these, `input`, `output`, and `parameters` are described in the sections above. Some of the common properties permitted on [steps](common.md) also apply to workflow definitions, including `condition`, `timeout`, and `on-error`. This rest of this section describes the remaining properties for more advanced use cases including mutex locking and resilient workflows with replay points. ## Locks and Mutual Exclusion Behavior In some cases, it is important to ensure that the same workflow does not run concurrently with itself, or more generally to assign a mutual exclusion "mutex" lock to make sure that at most one executing instance from a group can run at any point in time. This can be done in Apache Brooklyn by specifying `lock: LOCK-NAME` on the workflow. The lock is scoped to the entity, and means that if a workflow instance running at the entity enters this block, it acquires that "lock", and no other workflow instance at the entity can enter that block until the first one exits the block and releases the lock. Workflow instances at the entity that seek to `lock` the same `LOCK-NAME` will block until the lock becomes available. For example to ensure that `start` and `stop` do not run simultaneously, we could write: ``` brooklyn.initializers: - type: workflow-effector name: start lock: start-stop steps: - ... - type: workflow-effector name: stop lock: start-stop steps: - ... ``` If `stop` is run while `start` is still running, or a second `start` is run, they will not run until the first `start` completes and releases the lock. An operator with appropriate access permissions could also manually cancel the `start`. Details of why the effector is blocked are shown in the UI and available via the API, as part of the workflow data. To set a lock shared across multiple entities, the `lock` can be set as map of the form `{ name:Workflow Examples
^ The following examples are available: {% include list-children.html %}Ansible and BASH over SSH for a Web Server
^ The following code defines an entity which uses Ansible and BASH on a CentOS server to configure and run a web server (Apache `httpd`) running a simple HTML site. {% highlight yaml %} {% readj example-ansible-and-bash.yaml %} {% endhighlight %}Custom Step to Retrieve a Git Repository
^ This BOM file can be added to the catalog (download and run `br catalog addOAuth web request workflow
^ The following code defines an entity with a workflow which makes web requests, automatically refreshing with an OAuth token if the error message indicates that it should, and using backoff/retry strategies. The blueprint assumes that a Google App requiring OAuth is set up. This is easy done at Google, or the code should be straightforward to adapt for any other OAuth-based site. The blueprint expects the following three values from an [externalized config provider](/guide/ops/externalized-configuration.md) called `google-oauth`: * `google_client_id` - the client ID for the Google App (created when the App is created there) * `google_client_secret` - the client secret for the Google App (created when the App is created there) * `google_refresh_token` - a refersh token acquired for the app for a logged in user; because this interaction is intended to be automated, it expects to be configured with a valid refresh token (rather than redirect a user to a webpage); this is not a token for accessing the API directly, but for acquiring a token to do so, and can be retrieved by interacting with the OAuth API ahead of time or by inspecting the traffic for a UI-based log-in The blueprint is as follows: {% highlight yaml %} {% readj oauth.yaml %} {% endhighlight %}AWS CloudFormation Stack Discovery
^ The `update-children` step makes it straightforward to keep an Apache Brooklyn model in synch with external resources, from a cloud, GitHub or Jira tickets, or any data source you choose. The Brooklyn blueprint can then be used to attach management logic, including for example automatically deploying branched resources into ephemeral test environments. This example shows how CloudFormation stacks in AWS can be synchronized. Firstly, we define our type to represent discovered stack and be able to refresh `on_update`: {% highlight yaml %} {% readj aws-cfn-type.bom %} {% endhighlight %} This should be added to the catalog. We can then deploy our Brooklyn application to discover and monitor stacks: {% highlight yaml %} {% readj aws-discoverer.yaml %} {% endhighlight %} Create and delete stacks, and see them update in Brooklyn. Then consider: * Modify the `ssh aws` step in the "discoverer" to filter based on your preferred tags. * Use the `transform ... | merge` operator to combine lists from different regions. * Add other policies to check for drift on stacks and show failures in AMP if there is drift. * Create a similar workflow to monitor pull requests using the `gh` CLI; then create, update, delete, and track ephemeral test deployments based on thoseCatalog
^ Apache Brooklyn provides a **catalog**, which is a persisted collection of versioned blueprints and other resources. A set of blueprints is loaded from the `default.catalog.bom` in the Brooklyn folder by default and additional ones can be added through the web console or CLI. Blueprints in the catalog can be deployed directly, via the Brooklyn CLI or the web console, or referenced in other blueprints using their `id`. {% include list-children.html %}Catalog Items YAML Syntax
^ ### Catalog Items YAML Syntax An item or items to be added to the catalog is defined by a YAML file, specifying the catalog metadata for the items and the actual blueprint or resource definition. #### General YAML Schema Catalog items can be defined using the general structure below: ~~~ yaml brooklyn.catalog:Templates and Quick Launch
^ A `template` item type is a full application or service. It consists of one or more entities together with policies, configuration, parameters and locations. It is also composable: it can be used as an entity in another entity or application template. The "Home" screen in the UI includes a "Quick Launch" bar which by default shows all items in the catalog with `itemType: template`, making it easy for users to deploy things without any composition. When an item is selected, it can be simply configured and immediately deployed, or it can be edited in a simple in-line YAML editor or it can be opened in the composer. If the template is a valid plan without a location, the simple configuration view is used. This prompts a user to select a name, location, and values for the config parameters. The user can then easily deploy it, or if they wish open it in one of the other two modes. The simple YAML editor will edit the YAML that _references_ the template, whereas the composer will actually load the YAML _of_ the item, with any values set in the simple view. If the template declares a location -- or if the template is not valid YAML -- then the simple editor is opened immediately, with the template loaded exactly as it is defined. Comments can be included to facilitate editing. It is also possible to curate which items appear in "Quick Launch", and which locations. If the tag `catalog_quick_launch` is set on _any_ template in the catalog, then _only_ templates with that tag will be displayed. (If nothing has such a tag, then all templates are displayed.) Of course other tags can be added as usual. Similarly with locations, if any locations have that tag, then only locations with that tag will be available in the deploy dropdown. If there is only one location, it is pre-selected. So by tagging just one location, it can be ensured to be automatically selected. Additionally if any locations have that tag, the "Create Location" option is suppressed, as it is assumed the Quick Launch is being used in a curated environment where that option is not meant for end-users. (This can be changed through config options in the code, for users who wish to produce their own UI.) For example, the following catalog BOM will add a template for a single server to the catalog, ensuring it appears in Quick Launch (with the tag), and triggering the simple YAML editor when opened (because it has a location defined): ~~~ yaml brooklyn.catalog: items: - id: 1-server-template itemType: template name: "Template 1: Server" tags: [ catalog_quick_launch ] description: | Sample YAML to provision a server in a cloud with illustrative VM properties item: name: Server (Brooklyn Example) # this basic example shows how Brooklyn can provision a single raw VM # in the cloud or location of your choice services: - type: server name: My VM # location can be e.g. `softlayer` or `jclouds:openstack-nova:https://9.9.9.9:9999/v2.0/`, # or `localhost` or `byon:(hosts="10.9.1.1,10.9.1.2,produser2@10.9.2.{10,11,20-29}")` location: jclouds:aws-ec2: # edit these to use your credential (or delete if credentials specified in brooklyn.properties) identity:Versioning
^ Brooklyn supports multiple versions of a type to be installed and used at the same time. Versions are a first-class concept and are often prominently displayed in the UI. In order to do this, Brooklyn requires that the `id:version` string be unique across the catalog: it is normally an error to add a type if a type with the same `id:version` is present. The exceptions to this are if the definition is identical, or if the `version` is noted as a `SNAPSHOT`. In extraordinary circumstances it may be appropriate to delete a given `id:version` definition and then add the new one, but this is discouraged and the usual practice is to: * Use a `-SNAPSHOT` qualifer suffix on your version when developing * Increase the version number when making a change to a non-SNAPSHOT type When adding to the catalog, if no version is supplied, Brooklyn will typically use `0.0.0-SNAPSHOT`, and some clients will automatically create and increment the version number for the catalog item. When deploying a blueprint, if a version number is not specified Brooklyn will typically use the highest ordered version (see "Ordering" below) in the catalog for the referenced type, and will thereafter lock the use of that version in that blueprint. (An exception is where types are co-bundled or an explicit search path is defined; in the context of evaluating one type, Brooklyn may prefer versions in the same bundle or on the search path.) #### Versioning Syntax Version numbers in Brooklyn are recommended to follow the following syntax: ~~~Catalog Management
^ ### Catalog Management The Catalog module (from the tile on the homepage, or the module switch on the top right) in the web console will show all versions of catalog items, and allow you to add new items. #### Adding to the Catalog There are three ways of adding items to the catalog from the UI. 1. **From the Catalog Module**: Click on the "Upload to catalog" button located on the top right corner then follow the instructions. Alternatively, you can directly drag and drop items on the screen. This will accept `*.bom`, `*.jar` and `*.zip` files. 2. **From the Blueprint Composer module**: If you have composed a blueprint on the fly in the Blueprint Composer, you can save it into the catalog by clicking on the "Add to catalog button" located on the bottom right on the screen. A modal will then be displayed which will ask you for the required metadata regarding this new item. 3. **From the Blueprint Importer module**: If you already have catalog items, you can paste them into the importer editor then click on the "Import button". This allows you to import any types of items, including locations. In addition to the GUI, items can be added to the catalog via the REST API with a `POST` of the YAML file to `/v1/catalog` endpoint. To do this using `curl`: ~~~ bash curl -u admin:password http://127.0.0.1:8081/v1/catalog --data-binary @/path/to/riak.catalog.bom ~~~ Or using the CLI: ~~~ bash br catalog add /path/to/riak.catalog.bom ~~~ #### Deleting from the Catalog On the UI, if you are viewing a bundle page, a "Delete" button can be used to delete it from the catalog. Using the REST API, you can delete a versioned item from the catalog using the corresponding endpoint. For example, to delete the item with id `datastore` and version `1.0` with `curl`: ~~~ bash curl -u admin:password -X DELETE http://127.0.0.1:8081/v1/catalog/applications/datastore/1.0 ~~~ **Note:** Catalog items should not be deleted if there are running apps which were created using the same item. During rebinding the catalog item is used to reconstruct the entity. If you have running apps which were created using the item you wish to delete, you should instead deprecate the catalog item. Deprecated catalog items will not appear in the add application wizard, or in the catalog list but will still be available to Brooklyn for rebinding. The option to display deprecated catalog items in the catalog list will be added in a future release. Deprecation applies to a specific version of a catalog item, so the full id including the version number is passed to the REST API as follows: ~~~ bash curl -u admin:password -X POST http://127.0.0.1:8081/v1/catalog/entities/MySQL:1.0/deprecated/true ~~~Bundling
^ ### Bundling Catalog Resources It is possible to upload catalog items and associated resources as a single bundle to Brooklyn. This is useful when you have a blueprint that needs to reference external scripts, icons, config files or other resources, or when you have multiple blueprints that you want to keep in sync. Brooklyn will persist any uploaded bundles so that they are available after a restart or on HA failover. The bundle must be a ZIP file including a `catalog.bom` in the root. (The `br` CLI will create a ZIP from a local folder, for convenience.) The `catalog.bom` must declare a `bundle` identifier and a `version`, following Brooklyn's [versioning](versioning.md) rules. Brooklyn will keep track of that bundle, allowing items to be added and removed as a group, and associated resources to be versioned and included alongside them. With SNAPSHOT-version bundles, it allows replacement of multiple related items at the same time, and in advanced cases it allows setting up dependent bundles (specified as `brooklyn.libraries` or, for people familiar with OSGi, the `Required-bundle` manifest header) which will be searched if a blueprint in one bundle references resources from another bundle. Resources in the bundle can be referenced from the `catalog.bom` by using the `classpath:` URL protocol, as in `classpath://path/to/script.sh`. This can also be used to load resources in explicitly declared dependent bundles, using `classpath:bundle:version:/path/to/file.sh`; if omitted, typically the bundle of the entity resolving the link will be searched along with its OSGi dependencies and `brooklyn.library` dependencies. (This means if you define an entity type in a bundle, resources in that bundle will be found automatically, but if your bundle defines a composite blueprint referencing other types as children, those children will _not_ automatically search your bundle and a reference will be needed.) Optionally, an icon can be provided for the bundle using `bundleIconUrl`, which accepts `http:`, `https:`, and `classpath:` protocols, e.g. `classpath://brooklyn/icons/server.svg` or `https://brooklyn.apache.org/style/img/bridge.png` ### Example In this example, we will create a simple `my-server` catalog item, bundled with a simple script. The script will be run when launching the server. First, create a folder called bundleFolder, then add a file called myfile.sh to it. The contents of myfile.sh should be as follows: ~~~ bash echo Hello, World! ~~~ Now create a file in bundleFolder called `catalog.bom` with the following contents: ~~~ yaml brooklyn.catalog: bundle: MyServerBundle bundleIconUrl: classpath://brooklyn/icons/server.svg version: 1.0.0 items: - id: my-server item: type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess brooklyn.config: files.runtime: classpath://myfile.sh: files/myfile.sh launch.command: | chmod +x ./files/myfile.sh ./files/myfile.sh checkRunning.command: echo "Running" ~~~ The `bundle: MyServerBundle` line specifies the OSGI bundle name for this bundle. Any resources included in this bundle will be accessible on the classpath, but will be scoped to this bundle. This prevents an issue where multiple bundles include the same resource. To create the bundle, simply use the br command as follows. This will create a zip and send it to Brooklyn. Please note you can also specify a zip file (either on the file system or hosted remotely): ~~~ bash br catalog add bundleFolder ~~~ This will have added our bundle to the catalog. We can now deploy an instance of our server as follows. Please note that in this example we deploy to localhost. If you have not setup your machine to use localhost please see the instructions [here](../../locations#localhost-setup) or use a non localhost location. ~~~ yaml location: localhost services: - type: my-server ~~~ We can now see the result of running that script. In the UI find the activities for this application. The start activity has a sub task called launch (you will have to click through multiple activities called start/launch. Looking at the stdout of the launch task you should see: ~~~ bash Hello, World! ~~~ Alternatively you can view the script directly if you run the following against localhost. Please note that brooklyn-username and the id of your app will be different. ~~~ bash cat /tmp/brooklyn-username/apps/nl9djqbq2i/entities/VanillaSoftwareProcess_g52gahfxnt/files/myfile.sh ~~~ It should look like this: ~~~ bash echo Hello, World! ~~~ Now modify `myfile.sh` to contain a different message, change the version number in `catalog.bom` to `1.1.0`, and use the br command to send the bundle to the server. If you now deploy a new instance of the server using the same YAML as above, you should be able to confirm that the new script has been run (either by looking at the stdout of the launch task, or looking at the script itself) At this point, it is also possible to deploy the original `Hello, World!` version by explicitly stating the version number in the YAML: ~~~ yaml location: localhost services: - type: my-server:1.0.0 ~~~ To demonstrate the scoping, you can create another bundle with the following `catalog.bom`. Note the bundle name and entity id have been changed, but it still references a script with the same name. ~~~ yaml brooklyn.catalog: bundle: DifferentServerBundle version: 1.0.0 item: id: different-server type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess brooklyn.config: files.runtime: classpath://myfile.sh: files/myfile.sh launch.command: | chmod +x ./files/myfile.sh ./files/myfile.sh checkRunning.command: echo "Running" ~~~ Now create a new `myfile.sh` script with a different message, and use the br command to send the bundle to Brooklyn. Now deploy a blueprint which deploys all three servers. Each of the three deployments will utilise the script that was included with their bundle. ~~~ yaml location: localhost services: - type: my-server:1.0.0 - type: my-server:1.1.0 - type: different-server ~~~ **Note**: All three entities copy a file from `classpath://myfile.sh`, but as they are in different bundles, the scripts copied to the server will be different.Brooklyn Server Command Line Arguments
^ ### Brooklyn Server Command Line Arguments The command line arguments when launching `brooklyn` include several commands for working with the catalog. * `--catalogAddClusters, Specs, and Composition
^ What if you want multiple machines? One way is just to repeat the `- type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess` block, but there's another way which will keep your powder [DRY](http://en.wikipedia.org/wiki/Don't_repeat_yourself): {% highlight yaml %} {% read example_yaml/cluster-vm.yaml %} {% endhighlight %} Here we've composed the previous blueprint introducing some new important concepts, the `DynamicCluster` the `$brooklyn` DSL, and the "entity-spec". Let's unpack these. ## Dynamic Cluster The `DynamicCluster` creates a set of homogeneous instances. At design-time, you specify an initial size and the specification for the entity it should create. At runtime you can restart and stop these instances as a group (on the `DynamicCluster`) or refer to them individually. You can resize the cluster, attach enrichers which aggregate sensors across the cluster, and attach policies which, for example, replace failed members or resize the cluster dynamically. The specification is defined in the `dynamiccluster.memberspec` key. As you can see it looks very much like the previous blueprint, with one extra line. Entries in the blueprint which start with `$brooklyn:` refer to the Brooklyn DSL and allow a small amount of logic to be embedded (if there's a lot of logic, it's recommended to write a blueprint YAML plugin or write the blueprint itself as a plugin, in Java or a JVM-supported language). In this case we want to indicate that the parameter to `dynamiccluster.memberspec` is an entity specification (`EntitySpec` in the underlying type system); the `entitySpec` DSL command will do this for us. The example above thus gives us 5 VMs identical to the one we created in the previous section. ### Configuration The following configuration keys can be specified for dynamic cluster: | Config Key | Default | Description | |-------------------------------------------|-------------|-----------------------------------------------------------------| | dynamiccluster.restartMode | | How this cluster should handle restarts; by default it is disallowed, but this key can specify a different mode. Modes supported by dynamic cluster are 'off', 'sequential', or 'parallel'. However subclasses can define their own modes or may ignore this. | | dynamiccluster.quarantineFailedEntities | true | If true, will quarantine entities that fail to start; if false, will get rid of them (i.e. delete them) | | dynamiccluster.quarantineFilter | | Quarantine the failed nodes that pass this filter (given the exception thrown by the node). Default is those that did not fail with NoMachinesAvailableException (Config ignored if quarantineFailedEntities is false) | | cluster.initial.quorumSize | -1 | Initial cluster quorum size - number of initial nodes that must have been successfully started to report success (if < 0, then use value of INITIAL_SIZE) | | dynamiccluster.memberspec | | Entity spec for creating new cluster members | | dynamiccluster.firstmemberspec | | Entity spec for creating the first member of the cluster (if unset, will use the member spec for all) | | dynamiccluster.removalstrategy | | strategy for deciding what to remove when down-sizing | | dynamiccluster.customChildFlags | | Additional flags to be passed to children when they are being created | | dynamiccluster.zone.enable | false | Whether to use availability zones, or just deploy everything into the generic location | | dynamiccluster.zone.failureDetector | | Zone failure detector | | dynamiccluster.zone.placementStrategy | BalancingNodePlacementStrategy | Node placement strategy | | dynamiccluster.availabilityZones | | availability zones to use (if non-null, overrides other configuration) | | dynamiccluster.numAvailabilityZones | | number of availability zones to use (will attempt to auto-discover this number) | | cluster.member.id | | The unique ID number (sequential) of a member of a cluster | | cluster.initial.size | 1 | Initial cluster size | | start.timeout | | Time to wait (after members' start() effectors return) for SERVICE_UP before failing (default is not to wait) | | cluster.max.size | 2147483647 | Size after which it will throw InsufficientCapacityException | | dynamiccluster.maxConcurrentChildCommands | 0 | *Beta* The maximum number of effector invocations that will be made on children at once (e.g. start, stop, restart). Any value null or less than or equal to zero means invocations are unbounded | | UP_QUORUM_CHECK | QuorumChecks.atLeastOne() | Up check, applied by default to members, requiring at least one present and up | | RUNNING_QUORUM_CHECK | QuorumChecks.all() | Problems check from children actual states (lifecycle), applied by default to members and children, not checking upness, but requiring by default that none are on-fire | ### Effectors Dynamic cluster has a set of effectors which allow its members to be manipulated, these are detailed below. | Effector Name | Parameters | Description | |---------------|-------------|-----------------------------------------------------------------| | replaceMember | memberId | Replaces a specific member of the cluster given its ID | | resize | desiredSize | Resizes the cluster to a `desiredSize` | | resizeByDelta | delta | Resizes the cluster by a `delta` | Note that resizing of a cluster is limited by `cluster.max.size` and 0. When increasing the size of a cluster to larger than the `cluster.max.size`, if there is any headroom between the cluster and `cluster.max.size`, the resize call will resize the cluster to `cluster.max.size`. Any calls to increase the size of the cluster when it is already at `cluster.max.size` will result in an `InsufficientCapacityException`. Note that the new size of the cluster is returned by the resize effector calls. ### Sensors A set of sensors are defined for dynamic cluster to feed back information on its status. These are: | Sensor Name | Description | |-----------------------------------|-----------------------------------------------------------------| | group.members | Members of the group | | dynamiccluster.entityQuarantined | Entity failed to start, and has been quarantined | | dynamiccluster.quarantineGroup | Group of quarantined entities that failed to start | | dynamiccluster.subLocations | Locations for each availability zone to use | | dynamiccluster.failedSubLocations | Sub locations that seem to have failed | | cluster.one_and_all.members.up | True if the cluster is running, there is at least one member, and all members are service.isUp | ### Policies Dynamic cluster has a set of policies which can auto-replace and resize the members as well as determine primary nodes and other higher level actions. These policies are detailed on the [clusters and policies](clusters-and-policies.md) page.Enrichers
^ Enrichers provide advanced manipulation of an entity's sensor values. See below for documentation of the stock enrichers available in Apache Brooklyn. #### Transformer [`org.apache.brooklyn.enricher.stock.Transformer`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/enricher/stock/Transformer.html) Takes a source sensor and modifies it in some way before publishing the result in a new sensor. See below an example using `$brooklyn:formatString`. {% highlight yaml %} {% read example_yaml/enricher-transformer.yaml %} {% endhighlight %} #### Propagator [`org.apache.brooklyn.enricher.stock.Propagator`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/enricher/stock/Propagator.html) Use propagator to duplicate one sensor as another, giving the supplied sensor mapping. The other use of Propagator is where you specify a producer (using `$brooklyn:entity(...)` as below) from which to take sensors; in that mode you can specify `propagate` as a list of sensors whose names are unchanged, instead of (or in addition to) this map. {% highlight yaml %} {% read example_yaml/enricher-propagator.yaml %} {% endhighlight %} #### Custom Aggregating [`org.apache.brooklyn.enricher.stock.Aggregator`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/enricher/stock/Aggregator.html) Aggregates multiple sensor values (usually across a tier, esp. a cluster) and performs a supplied aggregation method to them to return an aggregate figure, e.g. sum, mean, median, etc. {% highlight yaml %} {% read example_yaml/enricher-aggregator.yaml %} {% endhighlight %} There are a number of additional configuration keys available for the Aggregators: | Configuration Key | Default | Description | |-----------------------------------|---------|---------------------------------------------------------------------| | enricher.transformation.untyped | list | Specifies a transformation, as a function from a collection to the value, or as a string matching a pre-defined named transformation, such as 'average' (for numbers), 'sum' (for numbers), 'isQuorate' (to compute a quorum), 'first' (the first value, or null if empty), or 'list' (the default, putting any collection of items into a list) | | quorum.check.type | | The requirement to be considered quorate -- possible values: 'all', 'allAndAtLeastOne', 'atLeastOne', 'atLeastOneUnlessEmpty', 'alwaysHealthy'", "allAndAtLeastOne" | | quorum.total.size | 1 | The total size to consider when determining if quorate | #### Joiner [`org.apache.brooklyn.enricher.stock.Joiner`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/enricher/stock/Joiner.html) Joins a sensor whose output is a list into a single item joined by a separator. {% highlight yaml %} {% read example_yaml/enricher-joiner.yaml %} {% endhighlight %} There are a number of additional configuration keys available for the joiner: | Configuration Key | Default | Description | |-----------------------------------|---------|---------------------------------------------------------------------| | enricher.joiner.separator | , | Separator string to insert between each argument | | enricher.joiner.keyValueSeparator | = | Separator string to insert between each key-value pair | | enricher.joiner.joinMapEntries | false | Whether to add map entries as key-value pairs or just use the value | | enricher.joiner.quote | true | Whether to bash-escape each parameter and wrap in double-quotes | | enricher.joiner.minimum | 0 | Minimum number of elements to join; if fewer than this, sets null | | enricher.joiner.maximum | null | Maximum number of elements to join (null means all elements taken) | #### Delta Enricher [`org.apache.brooklyn.policy.enricher.DeltaEnricher`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/policy/enricher/DeltaEnricher.html) Converts an absolute sensor into a delta sensor (i.e. the difference between the current and previous value) #### Time-weighted Delta [`org.apache.brooklyn.enricher.stock.YamlTimeWeightedDeltaEnricher`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/enricher/stock/YamlTimeWeightedDeltaEnricher.html) Converts absolute sensor values into a difference over time. The `enricher.delta.period` indicates the measurement interval. {% highlight yaml %} {% read example_yaml/enricher-time-weighted-delta.yaml %} {% endhighlight %} #### Rolling Mean [`org.apache.brooklyn.policy.enricher.RollingMeanEnricher`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/policy/enricher/RollingMeanEnricher.html) Transforms a sensor into a rolling average based on a fixed window size. This is useful for smoothing sample type metrics, such as latency or CPU time #### Rolling Time-window Mean [`org.apache.brooklyn.policy.enricher.RollingTimeWindowMeanEnricher`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/policy/enricher/RollingTimeWindowMeanEnricher.html) Transforms a sensor's data into a rolling average based on a time window. This time window can be specified with the config key `confidenceRequired` - Minimum confidence level (ie period covered) required to publish a rolling average (default `8d`). #### Http Latency Detector [`org.apache.brooklyn.policy.enricher.RollingTimeWindowMeanEnricher.HttpLatencyDetector`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/policy/enricher/HttpLatencyDetector.html) An Enricher which computes latency in accessing a URL, normally by periodically polling that URL. This is then published in the sensors `web.request.latency.last` and `web.request.latency.windowed`. There are a number of additional configuration keys available for the Http Latency Detector: | Configuration Key | Default | Description | |-----------------------------------|---------|----------------------------------------------------------------------| | latencyDetector.url | | The URL to compute the latency of | | latencyDetector.urlSensor | | A sensor containing the URL to compute the latency of | | latencyDetector.urlPostProcessing | | Function applied to the urlSensor value, to determine the URL to use | | latencyDetector.rollup | | The window size (in duration) over which to compute | | latencyDetector.requireServiceUp | false | Require the service is up | | latencyDetector.period | 1s | The period of polling | #### Combiner [`org.apache.brooklyn.enricher.stock.Combiner`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/enricher/stock/Combiner.html) Can be used to combine the values of sensors. This enricher should be instantiated using `Enrichers.builder().combining(..)`. This enricher is only available in Java blueprints and cannot be used in YAML. #### Note On Enricher Producers If an entity needs an enricher whose source sensor (`enricher.sourceSensor`) belongs to another entity, then the enricher configuration must include an `enricher.producer` key referring to the other entity. For example, if we consider the Transfomer from above, suppose that `enricher.sourceSensor: $brooklyn:sensor("urls.tcp.list")` is actually a sensor on a different entity called `load.balancer`. In this case, we would need to supply an `enricher.producer` value. {% highlight yaml %} {% read example_yaml/enricher-transformer.yaml %} {% endhighlight %} It is important to note that the value supplied to `enricher.producer` must be immediately resolvable. While it would be valid DSL syntax to write: {% highlight yaml %} enricher.producer: brooklyn:entity($brooklyn:attributeWhenReady("load.balancer.entity")) {% endhighlight %} (assuming the `load.balancer.entity` sensor returns a Brooklyn entity), this will not function properly because `enricher.producer` will unsuccessfully attempt to get the supplied entity immediately.Policies
^ Policies perform the active management enabled by Brooklyn. They can subscribe to entity sensors and be triggered by them (or they can run periodically, or be triggered by external systems). Policies can add subscriptions to sensors on any entity. Normally a policy will subscribe to sensors on either its associated entity, that entity's children and/or to the members of a "group" entity. Common uses of a policy include the following: * perform calculations, * look up other values, * invoke effectors (management policies) or, * cause the entity associated with the policy to emit sensor values (enricher policies). Entities can have zero or more `Policy` instances attached to them. {% include list-children.html %}Off-the-Shelf Policies
^ Policies are highly reusable as their inputs, thresholds and targets are customizable. Config key details for each policy can be found in the Catalog in the Brooklyn UI. ### HA/DR and Scaling Policies #### AutoScaler Policy - org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy Increases or decreases the size of a Resizable entity based on an aggregate sensor value, the current size of the entity, and customized high/low watermarks. An AutoScaler policy can take any sensor as a metric, have its watermarks tuned live, and target any resizable entity - be it an application server managing how many instances it handles, or a tier managing global capacity. e.g. if the average request per second across a cluster of Tomcat servers goes over the high watermark, it will resize the cluster to bring the average back to within the watermarks. {% highlight yaml %} brooklyn.policies: - type: org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy brooklyn.config: metric: webapp.reqs.perSec.perNode metricUpperBound: 3 metricLowerBound: 1 resizeUpStabilizationDelay: 2s resizeDownStabilizationDelay: 1m maxPoolSize: 3 {% endhighlight %} #### ServiceRestarter Policy - org.apache.brooklyn.policy.ha.ServiceRestarter Attaches to a SoftwareProcess or to anything Startable which emits `ha.entityFailed` on failure (or other configurable sensor), and invokes `restart` on that failure. If there is a subsequent failure within a configurable time interval or if the restart fails, this gives up and emits `ha.entityFailed.restart` for other policies to act upon or for manual intervention. {% highlight yaml %} brooklyn.policies: - type: org.apache.brooklyn.policy.ha.ServiceRestarter brooklyn.config: failOnRecurringFailuresInThisDuration: 5m {% endhighlight %} Typically this is used in conjunction with the ServiceFailureDetector enricher to emit the trigger sensor. The [introduction to policies](/guide/start/policies.md) shows a worked example of these working together. #### ServiceReplacer Policy - org.apache.brooklyn.policy.ha.ServiceReplacer The ServiceReplacer attaches to a DynamicCluster and replaces a failed member in response to `ha.entityFailed` (or other configurable sensor) as typically emitted by the ServiceFailureDetector enricher. The [introduction to policies](/guide/start/policies.md) shows a worked example of this policy in use. #### ServiceFailureDetector Enricher - org.apache.brooklyn.policy.ha.ServiceFailureDetector The ServiceFailureDetector enricher detects problems and fires an `ha.entityFailed` (or other configurable sensor) for use by ServiceRestarter and ServiceReplacer. The [introduction to policies](/guide/start/policies.md) shows a worked example of this in use. #### SshMachineFailureDetector Policy - org.apache.brooklyn.policy.ha.SshMachineFailureDetector The SshMachineFailureDetector is an HA policy for monitoring an SshMachine, emitting an event if the connection is lost/restored. #### ConnectionFailureDetector Policy - org.apache.brooklyn.policy.ha.ConnectionFailureDetector The ConnectionFailureDetector is an HA policy for monitoring an HTTP connection, emitting an event if the connection is lost/restored. ### Primary Election / Failover Policies {% include_relative _elect-primary-policies.md %} ### Optimization Policies #### PeriodicEffector Policy - org.apache.brooklyn.policy.action.PeriodicEffectorPolicy The `PeriodicEffectorPolicy` calls an effector with a set of arguments at a specified time and date. The policy monitors the sensor configured by `start.sensor` and will only start when this is set to `true`. The default sensor checked is `service.isUp`, so that the policy will not execute the effector until the entity is started. The following example calls a `resize` effector to resize a cluster up to 10 members at 8am and then down to 1 member at 6pm. - type: org.apache.brooklyn.policy.action.PeriodicEffectorPolicy brooklyn.config: effector: resize args: desiredSize: 10 period: 1 day time: 08:00:00 - type: org.apache.brooklyn.policy.action.PeriodicEffectorPolicy brooklyn.config: effector: resize args: desiredSize: 1 period: 1 day time: 18:00:00 #### ScheduledEffector Policy - org.apache.brooklyn.policy.action.ScheduledEffectorPolicy The `ScheduledEffectorPolicy` calls an effector at a specific time. The policy monitors the sensor configured by `start.sensor` and will only execute the effector at the specified time if this is set to `true`. There are two modes of operation, one based solely on policy configuration where the effector will execute at the time set using the `time` key or after the duration set using the `wait` key, or by monitoring sensors. The policy monitors the `scheduler.invoke.now` sensor and will execute the effector immediately when its value changes to `true`. When the `scheduler.invoke.at` sensor changes, it will set a time in the future when the effector should be executed. The following example calls a `backup` effector every night at midnight. - type: org.apache.brooklyn.policy.action.ScheduledEffectorPolicy brooklyn.config: effector: backup time: 00:00:00 #### FollowTheSun Policy - org.apache.brooklyn.policy.followthesun.FollowTheSunPolicy The FollowTheSunPolicy is for moving work around to follow the demand. The work can be any Movable entity. This currently available in yaml blueprints. #### LoadBalancing Policy - org.apache.brooklyn.policy.loadbalancing.LoadBalancingPolicy The LoadBalancingPolicy is attached to a pool of "containers", each of which can host one or more migratable "items". The policy monitors the workrates of the items and effects migrations in an attempt to ensure that the containers are all sufficiently utilized without any of them being overloaded. ### Lifecycle and User Management Policies #### StopAfterDuration Policy - org.apache.brooklyn.policy.action.StopAfterDurationPolicy The StopAfterDurationPolicy can be used to limit the lifetime of an entity. After a configure time period expires the entity will be stopped. #### ConditionalSuspend Policy - org.apache.brooklyn.policy.ha.ConditionalSuspendPolicy The ConditionalSuspendPolicy will suspend and resume a target policy based on configured suspend and resume sensors. #### CreateUser Policy - org.apache.brooklyn.policy.jclouds.os.CreateUserPolicy The CreateUserPolicy Attaches to an Entity and monitors for the addition of a location to that entity, the policy then adds a new user to the VM with a randomly generated password, with the SSH connection details set on the entity as the createuser.vm.user.credentials sensor. #### AdvertiseWinRMLogin Policy - org.apache.brooklyn.location.winrm.WinRmMachineLocation This is similar to the CreateUserPolicy. It will monitor the addition of WinRmMachineLocation to an entity and then create a sensor advertising the administrative user's credentials. ### Synchronization Policies #### nginx-multi-upstream-sync Policy *nginx-multi-upstream-sync* policy is designed to be used in combination with `nginx-multi` type that allows to achieve blue-green deployment. Here is an example: 1. Deploy cluster of apps version 1 with the following blueprint: ```yaml name: My Application services: - type: nginx-multi:1.1.0-SNAPSHOT id: my-nginx-multi - type: org.apache.brooklyn.entity.webapp.DynamicWebAppCluster id: my-app-cluster-v1 name: App Cluster v1 brooklyn.config: latch.launch: $brooklyn:component("my-nginx-multi").attributeWhenReady("service.isUp") dynamiccluster.quarantineFailedEntities: false dynamiccluster.memberspec: '$brooklyn:entitySpec': type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer brooklyn.config: wars.root: https://repo1.maven.org/maven2/org/apache/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.8.0-incubating/brooklyn-example-hello-world-sql-webapp-0.8.0-incubating.war brooklyn.policies: - type: nginx-multi-upstream-sync brooklyn.config: group: $brooklyn:component("my-app-cluster-v1") sensorsToTrack: - service.isUp nginxNode: $brooklyn:component("my-nginx-multi") groupName: v1 ``` 2. Navigate to NGINX effectors and render routing for `v1.myapp.com` pointing to v1 app group. `v1.myapp.com` is an endpoint to test cluster of apps version 1. * logicalNme `v1` * hostName `v1.myapp.com` * groupName `v1` 3. Find IP address of the NGINX in sensors and map it to `v1.myapp.com` in `/etc/hosts` and load `v1.myapp.com` in the browser on the machine where `/etc/hosts` is modified to test version 1, e.g. `11.22.33.44 v1.myapp.com`. 4. Render routing for `myapp.com` pointing to v1 app group. `myapp.com` in this example is a production endpoint. * logicalNme `production` * hostName `myapp.com` * groupName `v1` 5. Map IP address of the NGINX to myapp.com in `/etc/hosts` and load `myapp.com` in the browser on the machine where `/etc/hosts` is modified to verify that production endpoint loads version 1. 6. Deploy cluster of apps version 2 alongside version 1 in the same application: ```yaml services: - type: org.apache.brooklyn.entity.webapp.DynamicWebAppCluster id: my-app-cluster-v2 name: App Cluster v2 brooklyn.config: latch.launch: $brooklyn:component("my-nginx-multi").attributeWhenReady("service.isUp") dynamiccluster.quarantineFailedEntities: false dynamiccluster.memberspec: '$brooklyn:entitySpec': type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer brooklyn.config: wars.root: https://repo1.maven.org/maven2/org/apache/brooklyn/example/brooklyn-example-hello-world-sql-webapp/1.0.0/brooklyn-example-hello-world-sql-webapp-1.0.0.war brooklyn.policies: - type: nginx-multi-upstream-sync brooklyn.config: group: $brooklyn:component("my-app-cluster-v2") sensorsToTrack: - service.isUp nginxNode: $brooklyn:component("my-nginx-multi") groupName: v2 ``` ***Hint:*** add as a child to a deployed application. 7. Render routing for `v2.myapp.com` pointing to v2 app group. `v2.myapp.com` is an endpoint to test cluster of apps version 2. * logicalNme `v2` * hostName `v2.myapp.com` * groupName `v2` 8. Map IP address of the NGINX to `v2.myapp.com` in `/etc/hosts` and load `v2.myapp.com` in the browser on the machine where `/etc/hosts` is modified to test version 2. 9. Render routing for `myapp.com` pointing to v2 app group to switch the app version at production endpoint. * logicalNme `production` * hostName `myapp.com` * groupName `v2` 10. Refresh myapp.com in the browser on the machine where `/etc/hosts` is modified to verify that production endpoint loads version 2. 11. Try to resize cluster of app version 2 and see routing configuration updated in activities of nginx-multi node. ***Note***, `/etc/hosts` is used to simplify demonstration of the policy in a blue-green deployment. Writing a Policy ---------------- ### Your First Policy Policies perform the active management enabled by Brooklyn. Each policy instance is associated with an entity, and at runtime it will typically subscribe to sensors on that entity or children, performing some computation and optionally actions when a subscribed sensor event occurs. This action might be invoking an effector or emitting a new sensor, depending the desired behavior is. Writing a policy is straightforward. Simply extend [``AbstractPolicy``](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/policy/AbstractPolicy.html), overriding the [``setEntity``](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/objs/AbstractEntityAdjunct.html#setEntity-org.apache.brooklyn.api.entity.EntityLocal-) method to supply any subscriptions desired: {% highlight java %} @Override public void setEntity(EntityLocal entity) { super.setEntity(entity) subscribe(entity, TARGET_SENSOR, this) } {% endhighlight %} and supply the computation and/or activity desired whenever that event occurs: {% highlight java %} @Override public void onEvent(SensorEventWriting a Policy
^ ### Your First Policy Policies perform the active management enabled by Brooklyn. Each policy instance is associated with an entity, and at runtime it will typically subscribe to sensors on that entity or children, performing some computation and optionally actions when a subscribed sensor event occurs. This action might be invoking an effector or emitting a new sensor, depending the desired behavior is. Writing a policy is straightforward. Simply extend [``AbstractPolicy``](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/policy/AbstractPolicy.html), overriding the [``setEntity``](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/objs/AbstractEntityAdjunct.html#setEntity-org.apache.brooklyn.api.entity.EntityLocal-) method to supply any subscriptions desired: ```java @Override public void setEntity(EntityLocal entity) { super.setEntity(entity) subscribe(entity, TARGET_SENSOR, this) } ``` and supply the computation and/or activity desired whenever that event occurs: ```java @Override public void onEvent(SensorEventEffectors
^ Effectors perform an operation of some kind, carried out by a Brooklyn Entity. They can be manually invoked or triggered by a [Policy](/guide/blueprints/policies/). Common uses of an effector include the following: * Perform a command on a remote machine. * Collect data and publish them to sensors. Entities have default effectors, the lifecycle management effectors like `start`, `stop`, `restart`, and clearly more `Effectors` can be attached to them. Off-the-Shelf Effectors ---------------------- Effectors are highly reusable as their inputs, thresholds and targets are customizable. ### SshCommandEffector An `Effector` to invoke a command on a node accessible via SSH. It enables execution of a `command` in a specific `execution director` (executionDir) by using a custom `shell environment` (shellEnv). By default, the specified command will be executed on the entity where the effector is attached or on all *children* or all *members* (if it is a group) by configuring `executionTarget`. There are a number of additional configuration keys available for the `SshCommandEffector`: | Configuration Key | Default | Description | |-----------------------------------|---------|--------------------------------------------------------------------------------------| | command | | command to be executed on the execution target | | executionDir | | possible values: 'GET', 'HEAD', 'POST', 'PUT', 'PATCH', 'DELETE', 'OPTIONS', 'TRACE' | | shellEnv | | custom shell environment where the command is executed | | executionTarget | ENTITY | possible values: 'MEMBERS', 'CHILDREN' | Here is a simple example of an `SshCommandEffector` definition: {% highlight yaml %} brooklyn.initializers: - type: org.apache.brooklyn.core.effector.ssh.SshCommandEffector brooklyn.config: name: sayHiNetcat description: Echo a small hello string to the netcat entity command: | echo $message | nc $TARGET_HOSTNAME 4321 parameters: message: description: The string to pass to netcat defaultValue: hi netcat {% endhighlight %} See [`here`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/effector/ssh/SshCommandEffector.html) for more details. ### ContainerEffector This defines an effector to invoke a command or (a list of commands) on a container node accessible via `kubectl`. This that the `kubectl` CLI is available on the host where Apache Brooklyn is running and configured to access the desired Kubernetes cluster and the associated images. Supported Kubernetes environments to run containers include EKS, GKE, AKS, and (locally) Minikube and Docker Desktop. This effector is defined in the blueprint to be added to the entity using Apache Brooklyn initializers. It enables execution of a `command` in a specific container managed by a Kubernetes cluster. _Under the covers_ the commands and other configurations are used to generate a Kubernetes job that will execute in its own namespace. Regardless of the job execution result (success or failure) the namespace is deleted at the end, unless configured otherwise. There are a number of configuration keys available for the `ContainerEffector`: | Configuration Key | Default | Description | |-----------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | image | | Docker image name, the container will be created from. (mandatory) | | imagePullPolicy | `Always` | Possible values: `IfNotPresent`, `Always`, `Never`. Same values from the Kubernetes official documentation, the only advantage is that Apache Brooklyn is case insensitive. So, for example 'NEVER` and `never` are accepted as well. | | jobIdentifier | | An identifier to use to identify the jobs and containers in Kubernetes, with salt added (defaults to Brooklyn entity ID) | | keepContainerForDebug | `false` | When set to true, the namespace" and associated resources and services are not destroyed after execution, thus allowing access to the container for in-dept debugging. | | bashScript | | A bash script to run (convenience for command `bash` `-c` and args as supplied here; a list, multiline string, or single line string are all accepted | | command | | The command (and args) to execute on the container. | | args | | If the container is declared with an `ENTRYPOINT`, you might want to provide only arguments for the default command configured by the container. | | timeout | `5m` | How much should Kubernetes wait before considering a job to be failed and mark the container as failed as well. Defaults to 5m. | | workingDir | | The directory where the commands should be executed, can be a directory in the container or on a volume attached to it. | | volumeMounts | | Configuration to mount a volume into a container.(Same syntax as Kubernetes.) | | volumes | | List of directories with data that is accessible across multiple containers. These directories must exists and be configured in the Kubernetes cluster. | Environment variables using the `shell.env` Apache Brooklyn property are passed on to the container. The following example shows a sample blueprint of configuring the `ContainerEffector` for a simple `BasicStartable` entity with a list of simple commands being run on a container based on the [Perl](https://hub.docker.com/_/perl) image. Notice the last `echo $hello` command in the list; this prints the value of the `hello` environment variable configured using `shell.env` Apache Brooklyn configuration key. {% highlight yaml %} name: container-effector services: - type: 'org.apache.brooklyn.entity.stock.BasicStartable:1.1.0-SNAPSHOT' brooklyn.initializers: - type: org.apache.brooklyn.tasks.kubectl.ContainerEffector brooklyn.config: name: container-effector description: Very simple container effector shell.env: hello: world-amp image: perl imagePullPolicy: IfNotPresent bashScript: > HELLO=$(ls -la) echo $HELLO date echo $hello brooklyn.initializers: - type: org.apache.brooklyn.tasks.kubectl.ContainerEffector brooklyn.config: name: run-spark-job image: my-spark-container {% endhighlight %} The following example shows a sample blueprint of configuring the `ContainerEffector` for a simple `BasicStartable` entity with a simple command passes as arguments to the [Perl](https://hub.docker.com/_/perl) image. {% highlight yaml %} name: container-effector services: - type: 'org.apache.brooklyn.entity.stock.BasicStartable:1.1.0-SNAPSHOT' brooklyn.initializers: - type: org.apache.brooklyn.tasks.kubectl.ContainerEffector brooklyn.config: name: container-effector description: Very simple container effector shell.env: hello: world-amp image: perl imagePullPolicy: IfNotPresent args: - echo - hello {% endhighlight %} **Note:** Not all Kubernetes configuration properties are supported at the moment. **Note:** Job template properties `completions`, `parallelism` and `backoffLimit` have been enforced to 1 in an attempt to prevent Kubernetes to attempt more than one job run. In case of failure, by default, Kubernetes tries to run the same job 6 times, thus creating six pods. **Note:** For trying this effector locally, we recommend using downloading [Minikube](https://minikube.sigs.k8s.io) or install it on your local using package manager. **Note:** If you want to use your own image you can try customizing an existing one. We recommend you keep the image small to keep things quick. For example, the image described by the following Docker file is only 73MB in size(based on the minimal [Alpine](https://hub.docker.com/_/alpine)) and can be used to execute terraform commands. {% highlight yaml %} FROM alpine:latest RUN apk update && apk add --no-cache wget terraform unzip CMD ["/bin/sh"] {% endhighlight %} The same Apache Brooklyn configuration can be used to declare a `ContainerSensor` for an entity, as shown in the following blueprint. The smaller image is very suitable for a container sensor, since sensors are evaluated periodically. {% highlight yaml %} name: entity-with-container-sensor services: - type: 'org.apache.brooklyn.entity.stock.BasicStartable:1.1.0-SNAPSHOT' brooklyn.initializers: - type: org.apache.brooklyn.tasks.kubectl.ContainerSensor brooklyn.config: image: perl imagePullPolicy: never args: - echo - hello name: test-sensor period: 20s {% endhighlight %} ### HttpCommandEffector An `Effector` to invoke HTTP endpoints. It allows the user to specify the URI, the HTTP verb, credentials for authentication and HTTP headers. There are a number of additional configuration keys available for the `HttpCommandEffector`: | Configuration Key | Default | Description | |-----------------------------------|------------------|---------------------------------------------------------------------------------------------------------------| | uri | | URI of the endpoint | | httpVerb | | possible values: 'GET', 'HEAD', 'POST', 'PUT', 'PATCH', 'DELETE', 'OPTIONS', 'TRACE' | | httpUsername | | user name for the authentication | | httpPassword | | password for the authentication | | headers | application/json | It explicitly supports `application/x-www-form-urlencoded` | | httpPayload | | The body of the HTTP request | | jsonPath | | A jsonPath expression to extract values from a JSON object | | jsonPathAndSensors | | A map where keys are jsonPath expressions and values the name of the sensor where to publish extracted values | When the header `HttpHeaders.CONTENT_TYPE` is equals to *application/x-www-form-urlencoded* and the `httpPayload` is a `map`, the payload is transformed into a single string using `URLEncoded`. {% highlight yaml %} brooklyn.initializers: - type: org.apache.brooklyn.core.effector.http.HttpCommandEffector brooklyn.config: name: request-access-token description: Request an access token for the Azure API uri: $brooklyn:formatString: - "https://login.windows.net/%s/oauth2/token" - $brooklyn:config("tenant.id") httpVerb: POST httpPayload: resource: https://management.core.windows.net/ client_id: $brooklyn:config("application.id") grant_type: client_credentials client_secret: $brooklyn:config("application.secret") jsonPathAndSensors: $.access_token: access.token headers: Content-Type: "application/x-www-form-urlencoded" {% endhighlight %} See [`here`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/effector/http/HttpCommandEffector.html) for more details. ### AddChildrenEffector An `Effector` to add a child blueprint to an entity. {% highlight yaml %} brooklyn.initializers: - type: org.apache.brooklyn.core.effector.AddChildrenEffector brooklyn.config: name: add_tomcat blueprint_yaml: | name: sample description: Tomcat sample JSP and servlet application. origin: http://www.oracle.com/nCAMP/Hand services: - type: io.camp.mock:AppServer name: Hello WAR wars: /: hello.war controller.spec: port: 80 brooklyn.catalog: name: catalog-name type: io.camp.mock.MyApplication version: 0.9 libraries: - name: org.apache.brooklyn.test.resources.osgi.brooklyn-test-osgi-entities version: 0.1.0 url: classpath:/brooklyn/osgi/brooklyn-test-osgi-entities.jar auto_start: true {% endhighlight %} One of the config keys `BLUEPRINT_YAML` (containing a YAML blueprint (map or string)) or `BLUEPRINT_TYPE` (containing a string referring to a catalog type) should be supplied, but not both. See [`here`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/effector/AddChildrenEffector.html) for more details. Writing an Effector ------------------- ### Your First Effector Effectors generally perform actions on entities. Each effector instance is associated with an entity, and at runtime it will typically exectute an operation, collect the result and, potentially, publish it as sensor on that entity, performing some computation. Writing an effector is straightforward. Simply extend [`AddEffector`](https://brooklyn.apache.org/v/latest/misc/javadoc/org/apache/brooklyn/core/effector/AddEffector.html), providing an implementation for `newEffectorBuilder` and adding a constructor that consumes the builder or override an existing effector. {% highlight java %} public MyEffector(ConfigBag params) { super(newEffectorBuilder(params).build()); } public static EffectorBuilderClusters and Policies
^ Now let's bring the concept of the "cluster" back in. We could wrap our appserver in the same `DynamicCluster` we used earlier, although then we'd need to define and configure the load balancer. But another blueprint, the `ControlledDynamicWebAppCluster`, does this for us. It takes the same `dynamiccluster.memberspec`, so we can build a fully functional elastic 3-tier deployment of our `hello-world-sql` application as follows: {% highlight yaml %} {% read example_yaml/appserver-clustered-w-db.yaml %} {% endhighlight %} This sets up Nginx as the controller by default, but that can be configured using the `controllerSpec` key. This uses the same [externalized config](/guide/ops/externalized-configuration.md) as in other examples to hide the password. JBoss is actually the default appserver in the `ControlledDynamicWebAppCluster`, so because `brooklyn.config` keys in Brooklyn are inherited by default, the same blueprint can be expressed more concisely as: {% highlight yaml %} {% read example_yaml/appserver-clustered-w-db-concise.yaml %} {% endhighlight %} The other nicety supplied by the `ControlledDynamicWebAppCluster` blueprint is that it aggregates sensors from the appserver, so we have access to things like `webapp.reqs.perSec.windowed.perNode`. These are convenient for plugging in to policies! We can set up our blueprint to do autoscaling based on requests per second (keeping it in the range 10..100, with a maximum of 5 appserver nodes) as follows: {% highlight yaml %} {% read example_yaml/appserver-w-policy.yaml %} {% endhighlight %} Use your favorite load-generation tool (`jmeter` is one good example) to send a huge volume of requests against the server and see the policies kick in to resize it.Java Entities
^ Java blueprints are powerful, but also rather more difficult to write than YAML. Advanced Java skills are required. {% include list-children.html %} The main uses of Java-based blueprints are: * Integration with a service's API (e.g. for an on-line DNS service). This could take advantage of existing Java-based clients, or of Java's flexibility to chain together multiple calls. * Complex management logic, for example when the best practices for adding/removing nodes from a cluster is fiddly and has many conditionals. * Where the developer has a strong preference for Java. Anything that can be done in YAML can be done in the Java API. Once the blueprint is added to the catalog, the use of Java will be entirely hidden from users of that blueprint. The Apache Brooklyn community is striving to make YAML-based blueprints as simple as possible - if you come across a use-case that is hard to do in YAML then please let the community know.Creating from a Maven Archetype
^ ### Maven Archetype Brooklyn includes a maven archetype, which can be used to create the project structure for developing a new Java entity, and generating the OSGi bundle for it. #### Generating the Project The archetype can be used interactively, by running: {% highlight bash %} $ mvn archetype:generate {% endhighlight %} The user will be prompted for the archetype to use (i.e. group "org.apache.brooklyn" and artifact "brooklyn-archetype-quickstart"), as well as options for the project to be created. Alternatively, all options can be supplied at the command line. For example, if creating a project named "autobrick" for "com.acme": {% highlight bash %} $ BROOKLYN_VERSION={{ site.brooklyn-version }} $ mvn archetype:generate \ -DarchetypeGroupId=org.apache.brooklyn \ -DarchetypeArtifactId=brooklyn-archetype-quickstart \ -DarchetypeVersion=${BROOKLYN_VERSION} \ -DgroupId=com.acme \ -DartifactId=autobrick \ -Dversion=0.1.0-SNAPSHOT \ -DpackageName=com.acme.autobrick \ -DinteractiveMode=false {% endhighlight %} This will create a directory with the artifact name (e.g. "autobrick" in the example above). Note that if run from a directory containing a pom, it will also modify that pom to add this as a module! The project will contain an example Java entity. You can test this using the supplied unit tests, and also replace it with your own code. The `README.md` file within the project gives further guidance. #### Building To build, run the commands: {% highlight bash %} $ cd autobrick $ mvn clean install {% endhighlight %} #### Adding to the Catalog The build will produce an OSGi bundle in `target/autobrick-0.1.0-SNAPSHOT.jar`, suitable for use in the [Brooklyn catalog](/guide/blueprints/catalog) (using `brooklyn.libraries`). To use this in your Brooklyn catalog you will first have to copy the target jar to a suitable location. For developing/testing purposes storing on the local filesystem is fine. For production use, we recommend uploading to a remote maven repository or similar. Once your jar is in a suitable location the next step is to add a new catalog item to Brooklyn. The project comes with a `catalog.bom` file, located in `src/main/resources`. Modify this file by adding a 'brooklyn.libraries' statement to the bom pointing to the jar. For example: {% highlight yaml %} brooklyn.catalog: brooklyn.libraries: - file:///path/to/jar/autobrick-0.1.0-SNAPSHOT.jar version: "0.1.0-SNAPSHOT" itemType: entity items: - id: com.acme.autobrick.MySample item: type: com.acme.autobrick.MySample {% endhighlight %} The command below will use the CLI to add this to the catalog of a running Brooklyn instance: {% highlight bash %} br catalog add catalog.bom {% endhighlight %} After running that command, the OSGi bundle will have been added to the OSGi container, and the entity will have been added to your catalog. It can then be used in the same way as regular Brooklyn entities. For example, you can use the blueprint: {% highlight yaml %} services: - type: com.acme.autobrick.MySample {% endhighlight %} ### Testing Entities The project comes with unit tests that demonstrate how to test entities, both within Java and also using YAML-based blueprints. A strongly recommended way is to write a YAML test blueprint using the test framework, and making this available to anyone who will use your entity. This will allow users to easily run the test blueprint in their own environment (simply by deploying it to their own Brooklyn server) to confirm that the entity is working as expected. An example is contained within the project at `src/test/resources/sample-test.yaml`.Defining and Deploying
^ ## Intro This walkthrough will set up a simple entity, add it to the catalog, and provision it. For illustration purposes, we will write an integration with [Github Gist](https://gist.github.com/), with an effector to create new gists. ## Project Setup Follow the instructions to create a new Java project using the [archetype](archetype.html), and import it into your [favorite IDE](/guide/dev/env/ide/). This example assumes you used the groupId `com.acme` and artifact id `autobrick`. First ensure you can build this project at the command line, using `mvn clean install`. ## Java Entity Classes For this particular example, we will use a third party Gist library, so will need to add that as a dependency. Add the following to your `pom.xml` inside the `Handling Bundle Dependencies
^ Some Java blueprints will require third party libraries. These need to be made available to the Apache Brooklyn runtime. There are a number of ways this can be achieved. ### Classic Mode: Dropins Folder In Brooklyn classic mode (i.e. when not using Karaf), jars can be added to `./lib/dropins/`. After restarting Brooklyn, these will be available on the classpath. In Brooklyn classic mode, there is an embedded OSGi container. This is used for installing libraries referenced in catalog items. ### OSGi Bundles #### Introduction to OSGi Bundles An [OSGi bundle](https://en.wikipedia.org/wiki/OSGi#Bundles) is a jar file with additional metadata in its manifest file. The `MANIFEST.MF` file contains the symbolic name and version of the bundle, along with details of its dependencies and of the packages it exports (which are thus visible to other bundles). The [maven-bundle-plugin](http://felix.apache.org/documentation/subprojects/apache-felix-maven-bundle-plugin-bnd.html) is a convenient way of building OSGi bundles. #### OSGi Bundles Declared in Catalog Items Within a [catalog item](/guide/blueprints/catalog), a list of URLs can be supplied under `brooklyn.libraries`. Each URL should point to an OSGi bundle. This list should include the OSGi bundle that has the Java code for your blueprint, and also the OSGi bundles that it depends on (including all transitive dependencies). It is vital that these jars are built correctly as OSGi bundles, and that all transitive dependencies are included. The bundles will be added to Karaf in the order given, so a bundle's dependencies should be listed before the bundle(s) that depend on them. In the [GistGenerator example](/guide/blueprints/java/defining-and-deploying.md), the [catalog.bom file](/guide/blueprints/java/gist_generator/gist_generator.bom) included the URL of the dependency `org.eclipse.egit.github.core`. It also (before that line) included its transitive dependency, which is a specific version of `gson`. For Java blueprint developers, this is often the most convenient way to share a blueprint. Similarly for those wishing to use a new blueprint, this is often the simplest mechanism: the dependencies are fully described in the catalog item, which makes it convenient for deploying to Apache Brooklyn instances where there is not direct access to Karaf or the file system. #### Adding Bundles and Features Directly to Karaf Bundles and features can be added manually, directly to Karaf. However, note this only affects the single Karaf instance. If running in HA mode or if provisioning a new instance of Apache Brooklyn, the bundles will also need to be added to these Karaf instances. ##### Karaf Console Login to the [Karaf console](https://karaf.apache.org/manual/latest/#_shell_console_basics) using `./bin/client`, and add the bundles and features as desired. Examples of some useful commands are shown below: {% highlight bash %} karaf@amp> bundle:install -s http://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles.egit.github.core/2.1.5_1/org.apache.servicemix.bundles.egit.github.core-2.1.5_1.jar Bundle ID: 316 karaf@amp> bundle:list -t 0 -s | grep github 318 | Active | 80 | 2.1.5.1 | org.apache.servicemix.bundles.egit.github.core karaf@amp> bundle:headers org.apache.servicemix.bundles.egit.github.core ... karaf@amp> bundle:uninstall org.apache.servicemix.bundles.egit.github.core {% endhighlight %} ##### Karaf Deploy Folder Karaf support [hot deployment](https://karaf.apache.org/manual/latest/#_deployers). There are a set of deployers, such as feature and KAR deployers, that handle deployment of artifacts added to the `deploy` folder. Note that the Karaf console can give finer control (including for uninstall). ### Karaf KAR files [Karaf KAR](https://karaf.apache.org/manual/latest/kar) is an archive format (Karaf ARchive). A KAR is a jar file (so a zip file), which contains a set of feature descriptors and bundle jar files. This can be a useful way to bundle a more complex Java blueprint (along with its dependencies), to make it easier for others to install. A KAR file can be built using the [maven plugin org.apache.karaf.tooling:features-maven-plugin](https://karaf.apache.org/manual/latest/#_maven). ### Karaf Features A [karaf feature.xml](https://karaf.apache.org/manual/latest/#_create_a_features_xml_karaf_feature_archetype) defines a set of bundles that make up a feature. Once a feature is defined, one can add it to a Karaf instance: either directly (e.g. using the [Karaf console](https://karaf.apache.org/manual/latest/#_shell_console_basics)), or by referencing it in another feature.xml file. ### Embedded Dependencies An OSGi bundle can [embed jar dependencies](https://felix.apache.org/documentation/subprojects/apache-felix-maven-bundle-plugin-bnd.html#_embedding_dependencies) within it. This allows dependencies to be kept private within a bundle, and easily shipped with that bundle. To keep these private, it is vital that the OSGi bundle does not import or export the packages contained within those embedded jars, and does not rely on any of those packages in the public signatures of any packages that are exported or imported. ### Converting Non-OSGi Dependencies to Bundles If a dependencies is not available as an OSGi bundle (and you don't want to just [embed the jar](#embedded-dependencies)), there are a few options for getting an equivalent OSGi bundle: * Use a ServiceMix re-packaged jar, if available. ServiceMix have re-packed many common dependencies as OSGi bundles, and published them on [Maven Central](https://search.maven.org). * Use the `wrap:` prefix. The [PAX URL Wrap protocol](https://ops4j1.jira.com/wiki/display/paxurl/Wrap+Protocol) is an OSGi URL handler that can process your legacy jar at runtime and transform it into an OSGi bundle. This can be used when declaring a dependency in your feature.xml, and when using the Karaf console's `bundle:install`. Note that it is not yet supported in Brooklyn's `brooklyn.libraries` catalog items. When using `wrap:` include the `Bundle-SymbolicName` and `Bundle-Version` headers as parameters. (e.g. `wrap:mvn:javax.xml.ws/jaxws-api/2.3.0$Bundle-Symbolic-Name=javax.xml.ws.api&Bundle-Version=2.3.0`) * Re-package the bundle yourself, offline, to produce a valid OSGi bundle.Topology, Dependencies, and Management Policies
^ Applications written in YAML can similarly be written in Java. However, the YAML approach is recommended. ## Define your Application Blueprint The example below creates a three tier web service, composed of an Nginx load-balancer, a cluster of Tomcat app-servers, and a MySQL database. It is similar to the [YAML policies example](/guide/start/policies.md), but also includes the MySQL database to demonstrate the use of dependent configuration. {% highlight java %} {% read java_app/ExampleWebApp.java %} {% endhighlight %} To describe each part of this: * The application extends `AbstractApplication`. * It implements `init()`, to add its child entities. The `init` method is called only once, when instantiating the entity instance. * The `addChild` method takes an `EntitySpec`. This describes the entity to be created, defining its type and its configuration. * The `brooklyn.example.db.url` is a system property that will be passed to each `TomcatServer` instance. Its value is the database's URL (discussed below). * The policies and enrichers provide in-life management of the application, to restart failed instances and to replace those components that repeatedly fail. * The `NginxController` is the load-balancer and reverse-proxy: by default, it round-robins to the ip:port of each member of the cluster configured as the `SERVER_POOL`. ## Dependent Configuration Often a component of an application will depend on another component, where the dependency information is only available at runtime (e.g. it requires the IP of a dynamically provisioned component). For example, the app-servers in the example above require the database URL to be injected. The "DependentConfiguration" methods returns a future (or a "promise" in the language of some other programming languages): when the value is needed, the caller will block to wait for the future to resolve. It will block only "at the last moment" when the value is needed (e.g. after the VMs have been provisioned and the software is installed, thus optimising the provisioning time). It will automatically monitor the given entity's sensor, and generate the value when the sensor is populated. The `attributeWhenReady` is used to generate a configuration value that depends on the dynamic sensor value of another entity - in the example above, it will not be available until that `MySqlNode.DATASTORE_URL` sensor is populated. At that point, the JDBC URL will be constructed (as defined in the `formatString` method, which also returns a future).Common Classes and Entities
^ ### Entity Class Hierarchy By convention in Brooklyn the following words have a particular meaning: * *Group* - a homogeneous grouping of entities (which need not all be managed by the same parent entity) * *Cluster* - a homogeneous collection of entities (all managed by the "cluster" entity) * *Fabric* - a multi-location collection of entities, with one per location; often used with a cluster per location * *Application* - a top-level entity, which can have one or more child entities. The following constructs are often used for Java entities: * *entity spec* defines an entity to be created; used to define a child entity, or often to define the type of entity in a cluster. * *traits* (mixins) providing certain capabilities, such as *Resizable* and *Startable*. * *Resizable* entities can re-sized dynamically, to increase/decrease the number of child entities. For example, scaling up or down a cluster. It could similarly be used to vertically scale a VM, or to resize a disk. * *Startable* indicates the effector to be executed on initial deployment (`start()`) and on tear down (`stop()`). ### Configuration Configuration keys are typically defined as static named fields on the Entity interface. These define the configuration values that can be passed to the entity during construction. For example: {% highlight java %} public static final ConfigKeyFeeds
^ ### Feeds `Feed`s within Apache Brooklyn are used to populate an entity's sensors. There are a variety of feed types, which commonly poll to retrieve the raw metrics of the entity (for example polling an HTTP management API, or over JMX). #### Persistence There are two ways to associate a feed with an entity. The first way is (within the entity) to call `feeds().addFeed(...)`. This persists the feed: the feed will be automatically added to the entity when the Brooklyn server restarts. It is important that all configuration of the feed is persistable (e.g. not using any in-line anonymous inner classes to define functions). The feed builders can be passed a `uniqueTag(...)`, which will be used to ensure that on rebind there will not be multiple copied of the feed (e.g. if `rebind()` had already re-created the feed). The second way is to just pass to the feed's builder the entity. When using this mechanism, the feed will be wired up to the entity but it will not be persisted. In this case, it is important that the entity's `rebind()` method recreates the feed. #### Types of Feed ##### HTTP Feed An `HttpFeed` polls over HTTP(S). An example is shown below: {% highlight java %} private HttpFeed feed; @Override protected void connectSensors() { super.connectSensors(); feed = feeds().addFeed(HttpFeed.builder() .period(200) .baseUri(String.format("http://%s:%s/management/subsystem/web/connector/http/read-resource", host, port)) .baseUriVars(ImmutableMap.of("include-runtime","true")) .poll(new HttpPollConfig(SERVICE_UP) .onSuccess(HttpValueFunctions.responseCodeEquals(200)) .onError(Functions.constant(false))) .poll(new HttpPollConfig(REQUEST_COUNT) .onSuccess(HttpValueFunctions.jsonContents("requestCount", Integer.class))) .build()); } @Override protected void disconnectSensors() { super.disconnectSensors(); if (feed != null) feed.stop(); } {% endhighlight %} ##### SSH Feed An SSH feed executes a command over ssh periodically. An example is shown below: {% highlight java %} private AbstractCommandFeed feed; @Override protected void connectSensors() { super.connectSensors(); feed = feeds.addFeed(SshFeed.builder() .machine(mySshMachineLachine) .poll(new CommandPollConfig(SERVICE_UP) .command("rabbitmqctl -q status") .onSuccess(new Function() { public Boolean apply(SshPollValue input) { return (input.getExitStatus() == 0); }})) .build()); } @Override protected void disconnectSensors() { super.disconnectSensors(); if (feed != null) feed.stop(); } {% endhighlight %} ##### WinRm CMD Feed A WinRM feed executes a Windows command over WinRM periodically. An example is shown below: {% highlight java %} private AbstractCommandFeed feed; //@Override protected void connectSensors() { super.connectSensors(); feed = feeds.addFeed(CmdFeed.builder() .entity(entity) .machine(machine) .poll(new CommandPollConfigWriting an Entity
^ ## Ways to write an entity There are several ways to write a new entity: * For Unix/Linux, write YAML blueprints, for example using a **`VanillaSoftwareProcess`** and configuring it with your scripts. * For Windows, write YAML blueprints using **`VanillaWindowsProcess`** and configure the PowerShell scripts. * For composite entities, use YAML to compose exiting types of entities (potentially overwriting parts of their configuration), and wire them together. * Use **[Chef recipes](/guide/blueprints/chef)**. * Use **[Salt formulas](/guide/blueprints/salt)**. * Use **[Ansible playbooks](/guide/blueprints/ansible)**. * Write pure-java, extending existing base-classes. For example, the `GistGenerator` [example](defining-and-deploying.md). These can use utilities such as `HttpTool` and `BashCommands`. * Write pure-Java blueprints that extend `SoftwareProcess`. However, the YAML approach is strongly recommended over this approach. * Write pure-Java blueprints that compose together existing entities, for example to manage a cluster. Often this is possible in YAML and that approach is strongly recommended. However, sometimes the management logic may be so complex that it is easier to use Java. The rest of this section covers writing an entity in pure-java (or other JVM languages). ## Things To Know All entities have an interface and an implementation. The methods on the interface are its effectors; the interface also defines its sensors. Entities are created through the management context (rather than calling the constructor directly). This returns a proxy for the entity rather than the real instance, which is important in a distributed management plane. All entity implementations inherit from `AbstractEntity`, often through one of the following: * **`SoftwareProcessImpl`**: if it's a software process * **`VanillaJavaAppImpl`**: if it's a plain-old-java app * **`JavaWebAppSoftwareProcessImpl`**: if it's a JVM-based web-app * **`DynamicClusterImpl`**, **`DynamicGroupImpl`** or **`AbstractGroupImpl`**: if it's a collection of other entities Software-based processes tend to use *drivers* to install and launch the remote processes onto *locations* which support that driver type. For example, `AbstractSoftwareProcessSshDriver` is a common driver superclass, targetting `SshMachineLocation` (a machine to which Brooklyn can ssh). The various `SoftwareProcess` entities above (and some of the exemplars listed at the end of this page) have their own dedicated drivers. Finally, there are a collection of *traits*, such as `Resizable`, in the package ``brooklyn.entity.trait``. These provide common sensors and effectors on entities, supplied as interfaces. Choose one (or more) as appropriate. ## Key Steps *NOTE: Consider instead writing a YAML blueprint for your entity.* So to get started: 1. Create your entity interface, extending the appropriate selection from above, to define the effectors and sensors. 2. Include an annotation like `@ImplementedBy(YourEntityImpl.class)` on your interface, where `YourEntityImpl` will be the class name for your entity implementation. 3. Create your entity class, implementing your entity interface and extending the classes for your chosen entity super-types. Naming convention is a suffix "Impl" for the entity class, but this is not essential. 4. Create a driver interface, again extending as appropriate (e.g. `SoftwareProcessDriver`). The naming convention is to have a suffix "Driver". 5. Create the driver class, implementing your driver interface, and again extending as appropriate. Naming convention is to have a suffix "SshDriver" for an ssh-based implementation. The correct driver implementation is found using this naming convention, or via custom namings provided by the `BasicEntityDriverFactory`. 6. Wire the `public Class getDriverInterface()` method in the entity implementation, to specify your driver interface. 7. Provide the implementation of missing lifecycle methods in your driver class (details below) 8. Connect the sensors from your entity (e.g. overriding `connectSensors()` of `SoftwareProcessImpl`).. See the sensor feeds, such as `HttpFeed` and `JmxFeed`. Any JVM language can be used to write an entity. However use of pure Java is encouraged for entities in core brooklyn. ## Helpful References A few handy pointers will help make it easy to build your own entities. Check out some of the exemplar existing entities (note, some of the other entities use deprecated utilities and a deprecated class hierarchy; it is suggested to avoid these, looking at the ones below instead): * `JBoss7Server` * `MySqlNode` You might also find the following helpful: * **[Entity Design Tips](/guide/dev/tips#EntityDesign)** * The **[User Guide](/guide)** * The **[Mailing List](https://mail-archives.apache.org/mod_mbox/brooklyn-dev/)**Custom Entity Development
^ This section details how to create new custom application components or groups as brooklyn entities. The Entity Lifecycle -------------------- - Importance of serialization, ref to How mananagement works - Parents and Membership (groups) What to Extend -- Implementation Classes ---------------------------------------- - entity implementation class hierarchy - `SoftwareProcess` as the main starting point for base entities (corresponding to software processes), and subclasses such as `VanillaJavaApp` - `DynamicCluster` (multiple instances of the same entity in a location) and `DynamicFabric` (clusters in multiple location) for automatically creating many instances, supplied with an `EntityFactory` (e.g. `BaseEntityFactory`) in the `factory` flag - `AbstractGroup` for collecting entities which are parented elsewhere in the hierachy - `AbstractEntity` if nothing else fits - traits (mixins, otherwise known as interfaces with statics) to define available config keys, sensors, and effectors; and conveniences e.g. ``StartableMethods.{start,stop}`` is useful for entities which implement ``Startable`` - the ``Entities`` class provides some generic convenience methods; worth looking at it for any work you do A common lifecycle pattern is that the ``start`` effector (see more on effectors below) is invoked, often delegating either to a driver (for software processes) or children entities (for clusters etc). Configuration ------------- - AttributeSensorAndConfigKey fields can be automatically converted for ``SoftwareProcess``. This is done in ``preStart()``. This must be done manually if required for other entities, often with ``ConfigToAttributes.apply(this)``. - Setting ports is a special challenge, and one which the ``AttributeSensorAndConfigKey`` is particularly helpful for, cf ``PortAttributeSensorAndConfigKey`` (a subclass), causing ports automatically get assigned from a range and compared with the target ``PortSupplied`` location. Syntax is as described in the PortRange interface. For example, "8080-8099,8800+" will try port 8080, try sequentially through 8099, then try from 8800 until all ports are exhausted. This is particularly useful on a contended machine (localhost!). Like ordinary configuration, the config is done by the user, and the actual port used is reported back as a sensor on the entity. - Validation of config values can be applied by supplying a ``Predicate`` to the ``constraint`` of a ConfigKey builder. Constraints are tested after an entity is initialised and before an entity managed. Useful predicates include: - ``StringPredicates.isNonBlank``: require that a String key is neither null nor empty. - ``ResourcePredicates.urlExists``: require that a URL that is loadable by Brooklyn. Use this to confirm that necessary resources are available to the entity. - ``Predicates.in``: require one of a fixed set of values. - ``Predicates.containsPattern``: require that a value match a regular expression pattern. An important caveat is that only constraints on config keys that are on an entity's type hierarchy can be tested automatically. Brooklyn has no knowledge of the true type of other keys until they are retrieved with a ``config().get(key)``. Implementing Sensors -------------------- - e.g. HTTP, JMX Sensors at base entities are often retrieved by feeds which poll the entity's corresponding instance in the real world. The ``SoftwareProcess`` provides a good example; by subclassing it and overriding the ``connectSensors()`` method you could wire some example sensors using the following: {% highlight java %} public void connectSensors() { super.connectSensors() httpFeed = HttpFeed.builder() .entity(this) .period(200) .baseUri(mgmtUrl) .poll(new HttpPollConfigService State
^ Any entity can use the standard "service-up" and "service-state" sensors to inform other entities and the GUI about its status. In normal operation, entities should publish at least one "service not-up indicator", using the `ServiceNotUpLogic.updateNotUpIndicator` method. Each such indicator should have a unique name or input sensor. `Attributes.SERVICE_UP` will then be updated automatically when there are no not-up indicators. When there are transient problems that can be detected, to trigger `ON_FIRE` status entity code can similarly set `ServiceProblemsLogic.updateProblemsIndicator` with a unique namespace, and subsequently clear it when the problem goes away. These problems are reflected at runtime in the `SERVICE_PROBLEMS` sensor, allowing multiple problems to be tracked independently. When an entity is changing the expected state, e.g. starting or stopping, the expected state can be set using `ServiceStateLogic.setExpectedState`; this expected lifecycle state is considered together with `SERVICE_UP` and `SERVICE_PROBLEMS` to compute the actual state. By default the logic in `ComputeServiceState` is applied. For common entities, good out-of-the-box logic is applied, as follows: * For `SoftwareProcess` entities, lifecycle service state is updated by the framework and a service not-up indicator is linked to the driver `isRunning()` check. * For common parents, including `AbstractApplication` and `AbstractGroup` subclasses (including clusters, fabrics, etc), the default enrichers analyse children and members to set a not-up indicator (requiring at least one child or member who is up) and a problem indicator (if any children or members are on-fire). In some cases other quorum checks are preferable; this can be set e.g. by overriding the `UP_QUORUM_CHECK` or the `RUNNING_QUORUM_CHECK`, as follows: public static final ConfigKeyEntitlements
^ Brooklyn supports a plug-in system for defining "entitlements" -- essentially permissions. Any entitlement scheme can be implemented by supplying a class which implements one method on one class: public interface EntitlementManager { publicWindows Blueprints
^ Brooklyn can deploy to Windows servers using WinRM to run commands. These deployments can be expressed in pure YAML, and utilise PowerShell to install and manage the software process. This approach is similar to the use of SSH for UNIX-like servers. About WinRM ----------- WinRM - or *Windows Remote Management* to give its full title - is a system administration service provided in all recent Windows Server operating systems. It allows remote access to system information (provided via WMI) and the ability to execute commands. For more information refer to [Microsoft's MSDN article on Windows Remote Management](https://msdn.microsoft.com/en-us/library/aa384426(v=vs.85).aspx). WinRM is available by default in Windows Server, but is not enabled by default. Brooklyn will, in most cases, be able to switch on WinRM support, but this is dependent on your cloud provider supporting a user metadata service with script execution (see [below](/guide/blueprints/winrm/troubleshoot.md#user-metadata-service-requirement)) Locations for Windows --------------------- You must define a new location in Brooklyn for Windows deployments. Windows deployments require a different VM image ID to Linux, as well as some other special configuration, so you must have separate Brooklyn locations for Windows and Linux deployments. In particular, you will most likely want to set these properties on your location: * `imageId` or `imageNameRegex` - select your preferred Windows Server image from your cloud provider. * `hardwareId` or `minRam`/`minCores` - since Windows machines generally require more powerful servers, ensure you get a machine with the required specification. * `useJcloudsSshInit` - this must be set to `false`. Without this setting, jclouds will attempt to connect to the new VMs using SSH, which will fail on Windows Server. * `templateOptions` - you may also wish to request a larger disk size. This setting is cloud specific; on AWS, you can request a 100GB disk by setting this property to `{mapNewVolumeToDeviceName: ["/dev/sda1", 100, true]}`. In your YAML blueprint: location: jclouds:aws-ec2: region: us-west-2 identity: AKA_YOUR_ACCESS_KEY_ID credential:WinRM4j Client
^ ## WinRM4j parameters Per the `ConfigKey` entries in [org.apache.brooklyn.location.winrm.WinRmMachineLocation](https://github.com/apache/brooklyn-server/blob/master/software/winrm/src/main/java/org/apache/brooklyn/location/winrm/WinRmMachineLocation.java), the parameters available for WinRM are: * hostIf no port is specified then it defaults to a port depending on the `winrm.useHttps` flag. * winrm.useHttps
Default value: `false` * retriesOfNetworkFailures
Default value: `4` * winrm.useNtlm
Default value: `true` * winrm.computerName
Default value: `null` * user
Default value: `null` * password
Default value: `null` * waitWindowsToStart
Default value: `null` If there are location config keys prefixed with `brooklyn.winrm.config.` prefix will be removed and it will be used to instantiate a `org.apache.brooklyn.util.core.internal.winrm.WiRmTool` implementation. ## WinRM Connectivity Diagnostics If you are experiencing problems with a Windows blueprint, with an error about failing to connect (or about an authorization conduit), try the following quick list: 1. Apache Brooklyn is using correct username and password 1. Apache Brooklyn can reach the IP of the provisioned machine. WinRM port 5985 or 5986 is also reachable from Apache Brooklyn. 1. Check whether `WinRmMachineLocation#getDefaultUserMetadataString(ConfigurationSupportInternal)` is applied on the VM. This script should be passed to the cloud and executed in order to configure WinRM according to Apache Brooklyn requirements for authentication. So far Windows startup script are known to be supported on AWS EC2 and VCloud Director. If your cloud doesn't use this script then tune WinRM parameters accordingly. 1. Check whether you use WinRM over HTTP or over HTTPS. If you are using WinRM over HTTP then make sure WinRM service on target VM has `AllowUnencrypted = true` (see below). If the quick list above doesn't help then follow the steps below. To speed up diagnosing the problem if you don't already have a static machine to target, we advise to trigger a deployment with the JcloudsLocation flag `destroyOnFailure: false` so you can check status of the provisioned machine or try later different WinRM parameters with a Apache Brooklyn [BYON Location](/guide/locations/index.html#byon). After you determine what is the username and the password you can proceed with next steps. *(Notice that for cloud providers which use Auto Generated password will not be logged. For these cases use Java Debug to retrieve ot or provision a VM manually with the same parameters when using Apache Brooklyn to provision a jclouds location.)* The first step is to find what is the WinRM service configuration on the target host. 1. If you have RDP access or KVM like access to the VM then check the WinRM service status with the command bellow. `winrm get winrm/config/service` If you are using HTTP you should have AllowUnencrypted to false. Encryption is supported only over HTTPS. Sample output: MaxConcurrentOperations = 4294967295 MaxConcurrentOperationsPerUser = 1500 EnumerationTimeoutms = 240000 MaxConnections = 300 MaxPacketRetrievalTimeSeconds = 120 AllowUnencrypted = true Auth Basic = false Kerberos = true Negotiate = true Certificate = false CredSSP = true CbtHardeningLevel = Relaxed DefaultPorts HTTP = 5985 HTTPS = 5986 IPv4Filter = * IPv6Filter = * EnableCompatibilityHttpListener = false EnableCompatibilityHttpsListener = false CertificateThumbprint AllowRemoteAccess = true Use an Apache Brooklyn BYON blueprint to try easily other connection options. location: byon: hosts: - winrm: 10.0.0.1 user: Administrator password: pa55w0rd osFamily: windows services: - type: org.apache.brooklyn.entity.software.base.VanillaWindowsProcess brooklyn.config: checkRunning.command: echo checkRunning install.command: echo installCommand 1. Check IP is reachable from Apache Brooklyn instance Check whether `telnet 10.0.0.1 5985` makes successfully a socket. 1. Check that WinRM works, before delving deep in to the client: `Test-WSMan TARGET` and/or `winrs -r:10.0.2.15 -unencrypted -u:Administrator -p:pa55w0rd ipconfig`; many of the tips below will fix underlying WinRM problems, not just Winrm4j. *If the cloud provider doesn't support passing `sysprep-specialize-script-cmd` it may be necessary to modify the source Windows VM image to enable WinRM.* 1. Check your username and password. Notice in Windows passwords are case sensitive. Here is how it looks log from a wrong password: INFO: Authorization loop detected on Conduit "{http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd}WinRmPort.http-conduit" on URL "http://192.168.122.1:5985/wsman" with realm "null" Oct 21, 2016 10:43:11 AM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging WARNING: Interceptor for {http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd}WinRmService#{http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd}Create has thrown exception, unwinding now org.apache.cxf.interceptor.Fault: Could not send Message. at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:64) 1. Try `./User` instead of `User`. 1. Check whether you need to specify a different domain: this is possible from `brooklyn.winrm.config.winrm.computerName` location config. 1. Ensure all Windows machines consider the other side a "trusted host". On a private subnet, it may be appropriate to run: `Set-Item wsman:\localhost\client\trustedhosts *` whereas in other environments you will need to specify the list of machines. 1. Restart WinRM on both machines (some changes need a restart to take effect): `Restart-Service WinRM` 1. If you want to configure Windows target host with HTTPS then check the article [Configuring WINRM for HTTPS](https://support.microsoft.com/en-us/kb/2019527) In some cases the problems may be outwith the client, and it might be useful to look at [Troubleshooting](troubleshoot.md).
WinRM4j Tips and Tricks
^ ### PowerShell PowerShell commands can be supplied using config options such as `launch.powershell.command`. This is an alternative to supplying a standard batch command using config such as `launch.command`. For a given phase, only one of the commands (PowerShell or Batch) should be supplied. ### Execution Phases Bear in mind that the best practices for other entities (e.g. using [VanillaSoftwareProcess](/guide/blueprints/custom-entities.md#vanilla-software-using-bash)) apply for WinRM as well. Blueprint authors are strongly encouraged to provide an implementation for install, launch, stop and checkRunning. These are vital for the generic effectors such as stopping and restarting the process. ### Getting the Right Exit Codes WinRM (or at least the chosen WinRM client!) can return a zero exit code even on error in certain circumstances. It is therefore advisable to follow the guidelines below. *For a given command, write the PowerShell or Batch script as a separate multi-command file. Upload this (e.g. by including it in the `files.preinstall` configuration). For the configuration of the given command, execute the file.* When you have a command inside the PowerShell script which want to report its non zero exiting, please consider adding a check for its exit code after it. Example: & "C:\install.exe" If ($lastexitcode -ne 0) { exit $lastexitcode } For PowerShell files, consider including $ErrorActionPreference = "Stop" at the start of the file. `$ErrorActionPreference` Determines how Windows PowerShell responds to a non-terminating error (an error that does not stop the cmdlet processing) at the command line or in a script, cmdlet, or provider, such as the errors generated by the Write-Error cmdlet. https://technet.microsoft.com/en-us/library/hh847796.aspx See [Incorrect Exit Codes](limitations.md#incorrect-exit-codes) under Known Limitations. ### Executing Scripts From Batch Commands In a batch command, you can execute a batch file or PowerShell file. For example: install.command: powershell -NonInteractive -NoProfile -Command "C:\\install7zip.ps1" Or alternatively: install.command: C:\\install7zip.bat ### Executing Scripts From PowerShell In a PowerShell command, you can execute a batch file or PowerShell file. There are many ways to do this (see official PowerShell docs). For example: install.powershell.command: "& C:\\install7zip.ps1" Or alternatively: install.powershell.command: "& C:\\install7zip.bat" Note the quotes around the command. This is because the "&" has special meaning in a YAML value. ### Parameterised Scripts Calling parameterised Batch and PowerShell scripts is done in the normal Windows way - see offical Microsoft docs. For example: install.command: "c:\\myscript.bat myarg1 myarg2" Or as a PowerShell example: install.powershell.command: "& c:\\myscript.ps1 -key1 myarg1 -key2 myarg2" It is also possible to construct the script parameters by referencing attributes of this or other entities using the standard `attributeWhenReady` mechanism. For example: install.command: $brooklyn:formatString("c:\\myscript.bat %s", component("db").attributeWhenReady("datastore.url")) ### PowerShell - Using Start-Process When you are invoking a command from a PowerShell script with `Start-Process` cmdlet, please use the `-Wait` and the `-PassThru` arguments. Example `Start-Process C:\mycommand -Wait -PassThru` Using `-Wait` guarantees that the script process and its children and thus the WinRM session won't be terminated until it is finished. `-PassThru` Returns a process object for each process that the cmdlet started. By default, this cmdlet does not generate any output. See https://technet.microsoft.com/en-us/library/hh849848.aspx ### Rebooting Where a reboot is required as part of the entity setup, this can be configured using config like `pre.install.reboot.required` and `install.reboot.required`. If required, the installation commands can be split between the pre-install, install and post-install phases in order to do a reboot at the appropriate point of the VM setup. We strongly recommend to **write blueprints in a way that they do NOT restart automatically Windows** and use one of the `pre.install.reboot.required` or `install.reboot.required` parameters to perform restart, as script and Brooklyn connectivity behaviour is difficult to continue across reboots! ### Install Location Blueprint authors are encouraged to explicitly specify the full path for file uploads, and for paths in their PowerShell scripts (e.g. for installation, configuration files, log files, etc). ### How and Why to re-authenticate within a PowerShell script Some installation scripts require the use of security-related operations. In some environments, these fail by default when executed over WinRM, even though the script may succeed when run locally (e.g. by using RDP to connect to the machine and running the script manually). There may be no clear indication from Windows why it failed (e.g. for MSSQL install, the only clue is a security exception in the installation log). When a script is run over WinRM, the credentials under which the script are run are marked as 'remote' credentials, which are prohibited from running certain security-related operations. The solution is to obtain a new set of credentials within the script and use those credentials to required commands. The WinRM client uses Negotiate+NTLM to authenticate against the machine. This mechanism applies certain restrictions to executing commands on the Windows host. For this reason you should enable CredSSP on the Windows host which grants all privileges available to the user. https://technet.microsoft.com/en-us/library/hh849719.aspx#sectionSection4 To use `Invoke-Command -Authentication CredSSP` the Windows Machine has to have: - Up and running WinRM over HTTP. The custom-enable-credssp.ps1 script enables WinRM over HTTP because `Invoke-Command` use WinRM over HTTP by default. Invoke-Command can be used with -UseSSL option but this will lead to modifying PowerShell scripts. With always enabling WinRM over HTTP on the host, blueprint's PowerShell scripts remain consistent and not depend on the WinRM HTTPS/HTTP environments. We hope future versions of winrm4j will support CredSSP out of the box and wrapping commands in Invoke-Command will not be needed. - Added trusted host entries which will use Invoke-Command - Allowed CredSSP All the above requirements are enabled in Apache Brooklyn through [brooklyn-server/software/base/src/main/resources/org/apache/brooklyn/software/base/custom-enable-credssp.ps1](https://github.com/apache/brooklyn-server/blob/master/software/base/src/main/resources/org/apache/brooklyn/software/base/custom-enable-credssp.ps1) script which enables executing commands with CredSSP in the general case. The script works for most of the Windows images out there version 2008 and later. Check Microsoft Documentation for more information on how to [negotiate authenticate mechanisms](https://msdn.microsoft.com/en-us/library/windows/desktop/aa378748(v=vs.85).aspx) Re-authentication also requires that the password credentials are passed in plain text within the script. Please be aware that it is normal for script files - and therefore the plaintext password - to be saved to the VM's disk. The scripts are also accessible via the Brooklyn web-console's activity view. Access to the latter can be controlled via [Entitlements](/guide/blueprints/java/entitlements.md). As an example (taken from MSSQL install), the command below works when run locally, but fails over WinRM: ( $driveLetter + "setup.exe") /ConfigurationFile=C:\ConfigurationFile.ini The code below can be used instead (note this example uses Freemarker templating): & winrm set winrm/config/service/auth '@{CredSSP="true"}' & winrm set winrm/config/client/auth '@{CredSSP="true"}' # $pass = '${attribute['windows.password']}' $secpasswd = ConvertTo-SecureString $pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($($env:COMPUTERNAME + "\${location.user}"), $secpasswd) # $exitCode = Invoke-Command -ComputerName $env:COMPUTERNAME -Credential $mycreds -ScriptBlock { param($driveLetter) $process = Start-Process ( $driveLetter + "setup.exe") -ArgumentList "/ConfigurationFile=C:\ConfigurationFile.ini" -RedirectStandardOutput "C:\sqlout.txt" -RedirectStandardError "C:\sqlerr.txt" -Wait -PassThru -NoNewWindow $process.ExitCode } -Authentication CredSSP -ArgumentList $driveLetter # exit $exitCode In this example, the `${...}` format is FreeMarker templating. Those sections will be substituted before the script is uploaded for execution. To explain this example in more detail: * `${attribute['windows.password']}` is substituted for the entity's attribute "windows.password". This (clear-text) password is sent as part of the script. Assuming that HTTPS and NTLM is used, the script will be encrypted while in-flight. * The `${location.user}` gets (from the entity's machine location) the username, substituting this text for the actual username. In many cases, this will be "Administrator". However, on some clouds a different username (with admin privileges) will be used. * The username and password are used to create a new credential object (having first converted the password to a secure string). * Credential Security Service Provider (CredSSP) is used for authentication, to pass the explicit credentials when using `Invoke-Command`. To avoid the inclusion of clear-text passwords in the scripts, we recommend the use of a secure keystore such as Vault, which can be used by the script to retrieve sensitive data such as credentials. ### Windows AMIs on AWS Windows AMIs in AWS change regularly (to include the latest Windows updates). If using the community AMI, it is recommended to use an AMI name regex, rather than an image id, so that the latest AMI is always picked up. If an image id is used, it may fail as Amazon will delete their old Windows AMIs. If using an image regex, it is recommended to include the image owner in case someone else uploads a similarly named AMI. For example: location: jclouds:aws-ec2:us-west-2 imageNameRegex = Windows_Server-2012-R2_RTM-English-64Bit-Base-.* imageOwner = 801119661308 ... ## stdout and stderr in a PowerShell script When calling an executable in a PowerShell script, the stdout and stderr will usually be output to the console. This is captured by Brooklyn, and shown in the activities view under the specific tasks. An alternative is to redirect stdout and stderr to a file on the VM, which can be helpful if one expects sys admins to log into the VM. However, be warned that this would hide the stdout/stderr from Brooklyn's activities view. For example, instead of running the following: D:\setup.exe /ConfigurationFile=C:\ConfigurationFile.ini The redirect can be achieved by using the `Start-Process` scriptlet: Start-Process D:\setup.exe -ArgumentList "/ConfigurationFile=C:\ConfigurationFile.ini" -RedirectStandardOutput "C:\sqlout.txt" -RedirectStandardError "C:\sqlerr.txt" -PassThru -Wait The `-ArgumentList` is simply the arguments that are to be passed to the executable, `-RedirectStandardOutput` and `RedirectStandardError` take file locations for the output (if the file already exists, it will be overwritten). The `-PassThru` argument indicates that PowerShell should write to the file *in addition* to the console, rather than *instead* of the console. The `-Wait` argument will cause the scriptlet to block until the process is complete. Further details can be found on the [Start-Process documentation page](https://technet.microsoft.com/en-us/library/hh849848.aspx) on the Microsoft TechNet site.WinRM4j Limitations
^ ### Apache Brooklyn limitations on using WinRM over HTTP and HTTPS HTTPS is supported and recommended for use with WinRM via Apache Brooklyn in any wide-area scenario. Note however that there is no straightforward way to configure specific certificates to trust, and thus Apache Brooklyn will accept any certificate used in a HTTPS WinRM connection; traffic will be encrypted but other means will be necessary to protect against MITM attacks. WinRM 2.0 supports encryption mechanisms on top of HTTP. However those are not supported in Apache Brooklyn. For production usage also see the [Microsoft Guidelines](https://msdn.microsoft.com/en-us/library/ee309366(v=vs.85).aspx). ### Incorrect Exit Codes Some limitations with WinRM (or at least the chosen WinRM Client!) are listed below: ##### Single-line PowerShell files When a PowerShell file contains just a single command, the execution of that file over WinRM returns exit code 0 even if the command fails! This is the case for even simple examples like `exit 1` or `thisFileDoesNotExist.exe`. A workaround is to add an initial command, for example: Write-Host dummy line for workaround exit 1 ##### Direct Configuration of PowerShell commands If a command is directly configured with PowerShell that includes `exit`, the return code over WinRM is not respected. For example, the command below will receive an exit code of 0. launch.powershell.command: | echo first exit 1 ##### Direct Configuration of Batch commands If a command is directly configured with a batch exit, the return code over WinRM is not respected. For example, the command below will receive an exit code of 0. launch.command: exit /B 1 ##### Non-zero Exit Code Returned as One In some configurations, scripts can report any non-zero exit code as `1`. It may be possible to workaround this where the exit code is needeed by using e.g. `install.command: powershell -command "C:\\installmssql.ps1"` instead of `install.powershell.command: "C:\\installmssql.ps1"` If this is problematic, please consider submitting a patch to `VanillaWindowsProcess`! ### PowerShell "Preparing modules for first use" The first command executed over WinRM has been observed to include stderr saying "Preparing modules for first use", such as that below: < CLIXMLWinRM4j Troubleshooting
^ Note: in addition to the Windows-specific points here, much of the [operations troubleshooting guide](/guide/ops/troubleshooting) is applicable for Windows blueprints. ### WinRM Basics If you can't get WinRM to work at all, see the notes on [the Winrm4j client](client.md) which includes detailed troubleshooting for basic connectivity. ### User metadata service requirement WinRM requires activation and configuration before it will work in a standard Windows Server deployment. To automate this, Brooklyn will place a setup script in the user metadata blob. Services such as Amazon EC2's `Ec2ConfigService` will automatically load and execute this script. If your chosen cloud provider does not support `Ec2ConfigService` or a similar package, or if your cloud provider does not support user metadata, then you must pre-configure a Windows image with the required WinRM setup and make Brooklyn use this image. If the configuration options `userMetadata` or `userMetadataString` are used on the location, then this will override the default setup script. This allows one to supply a custom setup script. However, if userMetadata contains something else then the setup will not be done and the VM may not not be accessible remotely over WinRM. ### Credentials and privileges requiring special configuration When a script is run over WinRM over HTTP, the credentials under which the script are run are marked as 'remote' credentials, which are prohibited from running certain security-related operations. This may prevent certain operations. The installer from Microsoft SQL Server is known to fail in this case, for example. For a workaround, please refer to [How and Why to re-authenticate withing a PowerShell script](tips.md#how-and-why-to-re-authenticate-within-a-powershell-script) above. In some cases where security-related operation are to be executed, it may require the use of `CredSSP` to obtain the correct Administrator privileges: you may otherwise get an access denied error. See the sub-section [How and Why to re-authenticate within a powershell script](tips.md#how-and-why-to-re-authenticate-within-a-powershell-script) for more details. ### WebServiceException: Could not send Message We detected a `WebServiceException` and different `SocketException` during deployment of long-lasting Application Blueprint against VcloudDirector. Launching the blueprint below was giving constantly this type of error on launch step. services: type: org.apache.brooklyn.entity.software.base.VanillaWindowsProcess brooklyn.config: pre.install.command: echo preInstallCommand install.command: echo installCommand > C:\\install.txt post.install.command: echo postInstallCommand customize.command: echo customizeCommand pre.launch.command: echo preLaunchCommand launch.powershell.command: | Start-Sleep -s 400 Write-Host Test Completed post.launch.command: echo postLaunchCommand checkRunning.command: echo checkRunningCommand stop.command: echo stopCommand With series of tests we concluded that on the Vcloud Director environment we were using a restart was happening about 2 minutes after the VM is provisioned. Logging in the host and search for System event of type 1074 in Windows Event Viewer, we found two 1074 events where the second one was The process C:\Windows\system32\winlogon.exe (W2K12-STD) has initiated the restart of computer WIN-XXXX on behalf of user NT AUTHORITY\SYSTEM for the following reason: Operating System: Upgrade (Planned) Reason Code: 0x80020003 Shutdown Type: restart Comment: Normally on other clouds only one restart event is registered and the first time WinRM connection is made the Windows VM is ready for use. For this particular case when you want this second restart to finish we made `waitWindowsToStart` location parameter which basically adds additional check assuring the Windows VM provisioning is done. For example when using `waitWindowsToStart: 5m` location parameter, Apache Brooklyn will wait 5 minutes to see if a disconnect occurs. If it does, then it will again wait 5m for the machine to come back up. The default behaviour in Apache Brooklyn is to consider provisioning done on the first successful WinRM connection, without waiting for restart. To determine whether you should use this parameter you should carefully inspect how the image you choose to provision is behaving. If the description above matches your case and you are getting **connection failure message in the middle of the installation process** for your blueprints, a restart probably occurred and you should try this parameter. Before using this parameter we advice to check whether this is really your case. ### AMIs not found If using the imageId of a Windows community AMI, you may find that the AMI is deleted after a few weeks. ### VM Provisioning Times Out In some environments, provisioning of Windows VMs can take a very long time to return a usable VM. If the image is old, it may install many security updates (and reboot several times) before it is usable. On a VMware vCloud Director environment, the guest customizations can cause the VM to reboot (sometimes several times) before the VM is usable. This could cause the WinRM connection attempts to timeout. The location configuration option `waitForWinRmAvailable` defaults to `30m` (i.e. 30 minutes). This can be increased if required. Incorrectly prepared Windows templates can cause the deployment to time-out expecting an interaction by the user. You can verify if this is the case by RDPing to the in-progress deployment. It is recommended that any new Windows template be tested with a manually deployment to verify that it can be used for unattended installations and it doesn't wait and/or require an input by the user. See [Windows template settings for an Unattended Installation](limitations.md#windows-template-settings-for-an-unattended-installation) under Known Limitations below. ### Windows log files Details of the commands executed, and their results, can be found in the Brooklyn log and in the Brooklyn web-console's activity view. There will also be log files on the Windows Server. System errors in Windows are usually reported in the Windows Event Log - see [https://technet.microsoft.com/en-us/library/cc766042.aspx](https://technet.microsoft.com/en-us/library/cc766042.aspx) for more information. Additional logs may be created by some Windows programs. For example, MSSQL creates a log in `%programfiles%\Microsoft SQL Server\130\Setup Bootstrap\Log\` - for more information see [https://msdn.microsoft.com/en-us/library/ms143702.aspx](https://msdn.microsoft.com/en-us/library/ms143702.aspx). ### WinRM Commands Fail on Java Version 8u161 As described in bug [BROOKLYN-592](https://issues.apache.org/jira/browse/BROOKLYN-592), WinRM commands in an entity fail for certain versions of Java 8 (from 8u161, fixed in 8u192). This is caused by the Java bug [JDK-8196491](https://bugs.openjdk.java.net/browse/JDK-8196491). The error within Brooklyn will look like: ``` org.apache.brooklyn.util.core.internal.winrm.WinRmException: (Administrator@52.87.226.190:5985) : failed to execute command: SOAPFaultException: Marshalling Error: Entity References are not allowed in SOAP documents at org.apache.brooklyn.util.core.internal.winrm.winrm4j.Winrm4jTool.propagate(Winrm4jTool.java:257) at org.apache.brooklyn.util.core.internal.winrm.winrm4j.Winrm4jTool.exec(Winrm4jTool.java:214) at org.apache.brooklyn.util.core.internal.winrm.winrm4j.Winrm4jTool.executeCommand(Winrm4jTool.java:117) ... Caused by: java.lang.UnsupportedOperationException: Entity References are not allowed in SOAP documents at com.sun.xml.internal.messaging.saaj.soap.SOAPDocumentImpl.createEntityReference(SOAPDocumentImpl.java:148) ... ``` The workaround is to downgrade Java to 8u151 or similar, or upgrade to 8u192 or later.Testing YAML Blueprints
^ Brooklyn provides a selection of test entities which can be used to validate Blueprints via YAML. The basic building block is a TargetableTestComponent, which is used to resolve a target. There are two different groups of entities that inherit from TargetableTestComponent. The first is structural, which effects how the tests are run, for example by affecting the order they are run in. The second group is validation, which is used to confirm the application is deployed as intended, for example by checking some sensor value. Structural test entities include: - `TestCase` - starts child entities sequentially. - `ParallelTestCase` - starts child entities in parallel. - `LoopOverGroupMembersTestCase` - creates a TargetableTestComponent for each member of a group. - `InfrastructureDeploymentTestCase` - will create the specified Infrastructure and then deploy the target entity specifications there. Validation test entities include: - `TestSensor` - perform assertion on a specified sensor. - `TestEffector` - perform assertion on response to effector call. - `TestHttpCall` - perform assertion on response to specified HTTP GET Request. - `TestSshCommand` - test assertions on the result of an ssh command on the same machine as the target entity. - `TestWinrmCommand` - test assertions on the result of a WinRM command on the same machine as the target entity. - `TestEndpointReachable` - assert that a TCP endpoint is reachable. The endpoint can be in a number of different formats: a string in the form of `ip:port` or URI format; or a `com.google.common.net.HostAndPort` instance; or a `java.net.URI` instance; or a `java.net.URL` instance. TargetableTestComponents can be chained together, with the target being inherited by the components children. For example, a ParallelTestCase could be created that has a TestHttpCall as a child. As long as the TestHttpCall itself does not have a target, it will use the target of its parent, ParallelTestCase. Using this technique, we can build up complex test scenarios. The following sections provide details on each test entity along with examples of their use. {% include list-children.html %}Blueprint Test Entities
^ {% include fields.md %} ## Structural Test Entities ### TestCase The `TestCase` entity acts as a container for a list of child entities which are started *sequentially*. {% highlight yaml %} {% read example_yaml/entities/testcase-entity.yaml %} {% endhighlight %} This can be used to enforce a strict ordering, for example ensuring a sensor has a certain value before attempting to invoke an effector. Timeouts on child entities should be set relative to the completion of the preceding entity. The `ParallelTestCase` entity can be added as a child to run a subset of entities in parallel as a single step. #### Parameters - `on.error.spec` - Spec of entity to instantiate (and start, if startable) if the test-case fails - `on.finally.spec` - Spec of entity to instantiate (and start, if startable) after a test-case either passes or fails - `continueOnFailure` - Whether to continue executing subsequent children if an earlier child fails ### ParallelTestCase The `ParallelTestCase` entity acts as a container for a list of child entities which are started in *parallel*. {% highlight yaml %} {% read example_yaml/entities/paralleltestcase-entity.yaml %} {% endhighlight %} This can be used to run a subset of entities in parallel as a single step when nested under a `TestCase` entity. Timeouts on child entities should be set relative to the start of the `ParallelTestCase`. ### LoopOverGroupMembersTestCase The `LoopOverGroupMembersTestCase` entity is configured with a target group and a test specification. For each member of the targeted group, the test case will create a TargetableTestComponent entity from the supplied test specification and set the components target to be the group member. {% highlight yaml %} {% read example_yaml/entities/loopovergroupmembers-entity.yaml %} {% endhighlight %} #### Parameters - `target` - group who's members are to be tested, specified via DSL. For example, `$brooklyn:entity("tomcat")`. See also the `targetId` parameter. - `targetId` - alternative to the `target` parameter which wraps the DSL component lookup requiring only the `id` be supplied. For example, `tomcat`. Please note, this must point to a group. - `test.spec` - The TargetableTestComponent to create for each child. ### InfrastructureDeploymentTestCase The `InfrastructureDeploymentTestCase` will first create and deploy an infrastructure from the `infrastructure.deployment.spec` config. It will then retrieve a deployment location by getting the value of the infrastructures `infrastructure.deployment.location.sensor` sensor. It will then create and deploy all entities from the `infrastructure.deployment.spec` config to the deployment location. {% highlight yaml %} {% read example_yaml/entities/infrastructuredeploymenttestcase-entity.yaml %} {% endhighlight %} #### Parameters - `infrastructure.deployment.spec` - the infrastructure to be deployed. - `infrastructure.deployment.entity.specs` - the entities to be deployed to the infrastructure - `infrastructure.deployment.location.sensor` - the name of the sensor on the infrastructure to retrieve the deployment location ## Validation Test Entities ### TestSensor The `TestSensor` entity performs an assertion on a specified sensors value. {% highlight yaml %} {% read example_yaml/entities/testsensor-entity.yaml %} {% endhighlight %} #### Parameters - `target` - entity whose sensor will be tested, specified via DSL. For example, `$brooklyn:entity("tomcat")`. See also the `targetId` parameter. - `targetId` - alternative to the `target` parameter which wraps the DSL component lookup requiring only the `id` be supplied. For example, `tomcat`. - `sensor` - sensor to evaluate. For example `service.isUp`. - `timeout` - duration to wait on assertion to return a result. For example `10s`, `10m`, etc - `assert` - assertion to perform on the specified sensor value. See section on assertions below.
Tip: If the
### TestEffector
The `TestEffector` entity invokes the specified effector on a target entity. If the result of the effector is a String, it will then perform assertions on the result.
{% highlight yaml %}
{% read example_yaml/entities/testeffector-entity.yaml %}
{% endhighlight %}
#### Parameters
- `target` - entity whose effector will be invoked, specified via DSL. For example, `$brooklyn:entity("tomcat")`. See also the `targetId` parameter.
- `targetId` - alternative to the `target` parameter which wraps the DSL component lookup requiring only the `id` be supplied. For example, `tomcat`.
- `timeout` - duration to wait on the effector task to complete. For example `10s`, `10m`, etc
- `effector` - effector to invoke, for example `deploy`.
- `params` - parameters to pass to the effector, these will depend on the entity and effector being tested. The example above shows the `url` and `targetName` parameters being passed to Tomcats `deploy` effector.
- `assert` - assertion to perform on the returned result. See section on assertions below.
TestSensor is wrapped within a TestCase,
ParallelTestCase or LoopOverGroupMembersTestCase that set the target,
you don't need to specify the target, unless you want to test a different entity.
Tip: If the
### TestHttpCall
The `TestHttpCall` entity performs a HTTP GET on the specified URL and performs an assertion on the response.
{% highlight yaml %}
{% read example_yaml/entities/testhttpcall-entity.yaml %}
{% endhighlight %}
#### Parameters
- `url` - URL to perform GET request on, this can use DSL for example `$brooklyn:entity("tomcat").attributeWhenReady("webapp.url")`.
- `timeout` - duration to wait on a HTTP response. For example `10s`, `10m`, etc
- `applyAssertionTo` - The filed to apply the assertion to. For example `status`, `body`
- `assert` - assertion to perform on the response. See section on assertions below.
TestEffector is wrapped within a TestCase,
ParallelTestCase or LoopOverGroupMembersTestCase that set the target,
you don't need to specify the target, unless you want to test a different entity.
Tip: If the
### TestSshCommand
The TestSshCommand runs a command on the host of the target entity.
The script is expected not to run indefinitely, but to return a result (process exit code), along with its
standard out and error streams, which can then be tested using assertions.
If no assertions are explicitly configured, the default is to assert a non-zero exit code.
Either a bash command may be provided in the YAML, or a URL for a script which will be executed.
{% highlight yaml %}
{% read example_yaml/entities/testsshcommand-entity.yaml %}
{% endhighlight %}
#### Parameters
- `command` - The shell command to execute. (This and `downloadUrl` are mutually exclusive.)
- `downloadUrl` - URL for a script to download and execute. (This and `command` are mutually exclusive.)
- `shell.env` - Map of environment variables to be set.
- `scriptDir` - if `downloadUrl` is used. The directory on the target host where downloaded scripts should be copied to.
- `runDir` - the working directory where the command or script will be executed on the target host.
- `assertStatus` - Assertions on the exit code of the command or script. See section on assertions below.
- `assertOut` - Assertions on the standard output of the command as a String.
- `assertErr` - Assertions on the standard error of the command as a String.
TestHttpCall is wrapped within a TestCase,
ParallelTestCase or LoopOverGroupMembersTestCase that set the target,
you don't need to specify the target, unless you want to test a different entity.
Tip: If the
## Assertions
The following conditions are provided by those test entities above that include assertions
- `isNull` - asserts that the actual value is `null`.
- `notNull` - asserts that the actual value is NOT `null`.
- `isEqualTo` - asserts that the actual value equals an expected value.
- `equalTo` - a synonym for `isEqualTo`
- `equals` - a synonym for `isEqualTo`
- `notEqual` - asserts that the actual value does not equal the expected value.
- `matches` - asserts that the actual value matches a [regex pattern](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html?is-external=true), for example `".*hello.*"`.
Note that regular expressions follow the Java defaults, e.g. for [line terminators](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#lt).
One can use `"(?m)"` for multi-line mode (so that `^` and `$` also match just after/before line terminators,
rather than just the start/end of the entire input sequence). The dotall mode, set with `"(?s)"`, can also
be useful (so that `.` matches any character, including line terminators).
- `containsMatch` - asserts that the value contains a string that matches a regex. It follow the behaviour of
the [Matcher.find() method](http://docs.oracle.com/javase/7/docs/api/java/util/regex/Matcher.html#find()).
This can be useful for simplifying matching in multi-line input.
- `contains` - asserts that the actual value contains the supplied value
- `isEmpty` - asserts that the actual value is an empty string
- `notEmpty` - asserts that the actual value is a non empty string
- `hasTruthValue` - asserts that the actual value has the expected interpretation as a boolean
- `greaterThan` - asserts that the actual value is greater than the expected value according to Java's
[Comparable](https://docs.oracle.com/javase/8/docs/api/java/lang/Comparable.html) interface. Actual and
expected must be instances of the same type and implement `Comparable`.
- `lessThan` - asserts that the actual value is less than the expected value according to Java's
[Comparable](https://docs.oracle.com/javase/8/docs/api/java/lang/Comparable.html) interface. Actual and
expected must be instances of the same type and implement `Comparable`.
Assertions may be provided as a simple map:
assert:
contains: 2 users
matches: .*[\d]* days.*
If there is the need to make multiple assertions with the same key, the assertions can be specified
as a list of such maps:
assert:
- contains: 2 users
- contains: 2 days
TestSshCommand is wrapped within a TestCase,
ParallelTestCase or LoopOverGroupMembersTestCase that set the target,
you don't need to specify the target, unless you want to test a different entity.
Example Blueprint Tests
^ {% include fields.md %} ## Introduction This section describes some simple tests based on the [Getting Started](/guide/start/blueprints.md#launching-from-a-blueprint) example blueprint: {% highlight yaml %} {% read /guide/start/_my-web-cluster.yaml %} {% endhighlight %} The following sections contain yaml snippets that be appended to the list of services in the blueprint above, a complete blueprint is also provided [below](#full-example). Note that unless otherwise specified the following tests are executed in parallel with the deployment of the application services, all `timeout` values are set accordingly. ### Sensor Test Demonstrates the following sensor assertion: - asserts that the `webappcluster` entity `service.isUp` sensor is `true` within 10 minutes of blueprint being deployed. {% highlight yaml %} {% read example_yaml/testcases/sensor-test-snippet.yaml %} {% endhighlight %} ### Effector Test (via TestCase entity) This `TestEffector` example demonstrates the use of the `TestCase` and `TestSensor` entities to ensure the service has started before invoking an effector using the `TestEffector` entity. - `TestCase` entity starts its children sequentially - asserts that the `webappcluster` entity `service.isUp` sensor is `true` within 10 minutes of the parent `TestCase` entity starting. Blocks start of the next child until it obtains a result (or times out). - `deploy` effector invoked to deploy WAR to a `newcontext` with a 5 minute timeout to allow completion of the deploy task. - asserts `/newcontext` URL returns a HTTP status code 200 within 5 minutes of the effector being invoked (Note that this timeout is relative to the preceding test entity as they are being sequentially run as children of a `TestCase` entity). {% highlight yaml %} {% read example_yaml/testcases/effector-test-snippet.yaml %} {% endhighlight %} ### HTTP Call Tests Demonstrates the following HTTP Call assertions against the specified `url`, which in these examples are being built from the `webappcluster` entities `host.address` and `proxy.http.port` sensors (the tester having flexibility in how the test URL is to be constructed): - asserts the response status code is 200 within 10 minutes of the blueprint being deployed. - asserts the response body matches the regex `(?s).*Br[o]{2}klyn Deployed.*` within 10 minutes of the blueprint being deployed. Note the presence of the `(?s)` dotall flag to test a multiline response. {% highlight yaml %} {% read example_yaml/testcases/http-test-snippet.yaml %} {% endhighlight %} ### Full Example A sample blueprint containing all the tests described above is available [here](./example_yaml/testcases/getting-started-test-example.yaml). This blueprint will deploy the [Getting Started](/guide/start/blueprints.html#launching-from-a-blueprint) application and run all of the test entities, which if successful should appear in the web console as follows. [](images/getting-started-blueprint-test-large.png)Ansible in YAML Blueprints
^ This guide describes how Brooklyn entities can be created using the Ansible infrastructure management tool ([ansible.com](http://ansible.com)). At present Brooklyn provides basic support for Ansible, operating in a 'masterless' mode. Comments on this support and suggestions for further development are welcome. This guide assumes you are familiar with the basics of [creating YAML blueprints](/guide/blueprints). {% include list-children.html %}About Ansible
^ ## What you need to know about Ansible [Ansible](http://docs.ansible.com/ansible/) is a deployment tool designed to work in an agent-less manner, normally performing its operations on a node over SSH from some central administrating node. Brooklyn can deploy software via Ansible on one of its managed nodes, by first installing Ansible on the node itself and then using Ansible to deploy the required software. A 'Playbook' in Ansible is a specification of the configuration and deployment of a system. Playbooks are expressed in YAML format, and contain a number of 'plays', which are in turn lists of tasks to carry out to achieve the desired configuration on the system. 'Roles' are pre-written modular collections of tasks, and can be included in playbooks. Ansible comes with built-in support for many software systems, and has a community repository of roles exists at [https://galaxy.ansible.com](https://galaxy.ansible.com). ### How Brooklyn interacts with Ansible Brooklyn provides a Ansible entity type. An entity of this type can be specified in a blueprint in order to provision the node through Ansible. The Ansible entity will download Ansible and install it on the node. The entity type supports the configuration of Ansible playbooks to download, or write inline in the blueprint, for simple playbooks. Configuration values for the playbooks can be supplied in the blueprint. Brooklyn will deploy the software specified in the playbook when the entity starts. In addition, an effector is provided on the entity that supports general purpose Ansible instructions.Creating Blueprints with Ansible
^ To write a blueprint to use Ansible with Brooklyn it will help to have a degree of familiarity with Ansible itself. In the sections below, when the Brooklyn configuration is described, the underlying Ansible operation is also noted briefly, for clarity for readers who know Ansible. To manage a node with Ansible, create a blueprint containing a service of type `org.apache.brooklyn.entity.cm.ansible.AnsibleEntity` and define and minimum the `playbook` value, and one or other of `playbook.url` or `playbook.yaml`. You must also define the `service.name` that will be tested in order to determine if the entity is running successfully. For example: name: myweb location: ... services: - type: org.apache.brooklyn.entity.cm.ansible.AnsibleEntity id: apache name: apache service.name: apache2 playbook: apache-playbook playbook.url: http://myhost/projectX/apache-playbook.yaml This example specifies that Brooklyn should use Ansible to download the playbook from the repository on "myhost". The playbook contains the instructions to install the Apache web server. To start the entity, Brooklyn will use Ansible's "ansible-playbook" command to run the playbook, which will bring up the web server. ### Lifecycle of AnsibleEntity The start effector applies the playbook and verifies that it has started the software correctly by checking the service defined as `service.name` is running. This can be customized, see `ansible.service.start` configuration below. The stop effector will stop the service `service.name`. Again, this can be customized, with `ansible.service.stop`. The restart effector will apply stop and then start. ### Configuration of AnsibleEntity The `playbook` configuration key names the top level list of states that will be applied using Ansible. This configuration key is mandatory. The playbook must be defined using one or other (both together are not permitted) of `playbook.yaml` or `playbook.url`. The former allows the playbook content to be defined inline within the blueprint, using the normal YAML format of an Ansible playbook. The latter obtains the playbook from an external URL. The `ansible.service.start` configuration key allows the blueprint author to override the command used by default to verify that the service `service.name` is running (or to start it, if the playbook did not specify it should run by default). The default value is: sudo ansible localhost -c local -m service -a "name=Chef in YAML Blueprints
^ This guide describes how Brooklyn entities can be easily created from Chef cookbooks. As of this writing (May 2014) some of the integration points are under active development, and comments are welcome. A plan for the full integration is online [here](https://docs.google.com/a/cloudsoftcorp.com/document/d/18ZwzmncbJgJeQjnSvMapTWg6N526cvGMz5jaqdkxMf8). This guide assumes you are familiar with the basics of [creating YAML blueprints](/guide/blueprints). {% include list-children.html %}About Chef
^ ## What you need to know about Chef Chef works in two different modes, *server* and *solo*. *Server* is where the Chef client talks to a central server to retrieve information about its roles, policies and cookbooks (where a cookbook defines how to install and configure a particular piece of software). With *solo*, the client works in isolation, therefore its configuration and cookbooks must be supplied by another means. Chef *client* is the Chef agent. This is a Ruby application which is installed on each and every managed host. When invoked in server mode, it will contact the Chef server to check for updates to cookbooks and policy; it then "runs" the recipes in its run lists, to converge the machine to a known state. In solo mode, it reads the locally-maintained cookbooks and policies. The client may be run as a daemon that checks the server regularly, or it could merely be run manually when required. The *policy* is a set of rules on the Chef server. A client starts with a set of *attributes*, which could be as simple as its name and a recipe runlist, or which may involve a more complex set of attributes about how it is to be configured. The client then augments this with auto-detected metadata - a tool called `ohai` is run that collects detailed information about the host. Next, the policy on the server modifies these attributes - overriding some, setting defaults for others - to produce a final set of attributes. It is these which are the input to the recipes. Finally, the attributes are uploaded to the server where they are stored as metadata for the node, where they can be inspected and modified by the system operator. Also of interest is `knife`, which is the workstation toolkit for Chef. Typically this would be installed on the operation engineer's workstation, where it would be used to interact with the Chef server and clients. Of particular interest to us is the *bootstrap* operation, which is used for setting up new Chef clients - given a virtual machine, it will install the Chef client on it, configure it with enough information to find the Chef server and performs its first run, and then kicks off the Chef client for the first time. There is often a preconception about how a Chef client is bootstrapped; mistakenly, there is the belief that the `knife` tool configures the Chef server with information about the client, and the client finds out about itself from the server. This is not the case - the bootstrap operation does not involve `knife` talking to the server. Instead, `knife` packages up all of the required information and sends it to the client - the client will then introduce itself to the server, passing on its configuration. This diagram summarises the interaction between Brooklyn, the new node, and the various Chef tools. Note that there is no interaction between the Apache Brooklyn Server and the Chef Server. [](chef-call-flow.png) ### How Brooklyn interacts with Chef Brooklyn understands both the *server* and *solo* modes of operation. Server mode utilises the `knife` toolkit, and therefore `knife` must be installed onto the Apache Brooklyn server and configured appropriately. Solo mode does not have any special requirements; when running in solo mode, Brooklyn will install and configure the Chef client over SSH, just like it does most other kinds of entities.Creating Blueprints from Chef
^ In a nutshell, a new Chef-based entity can be defined as a service by specifying `chef:cookbook_name` as the `service_type`, along with a collection of optional configuration. An illustrative example is below: {% highlight yaml %} {% read example_yaml/mysql-chef-1.yaml %} {% endhighlight %} *This works without any installation: try it now, copying-and-pasting to the Brooklyn console. (Don't forget to add your preferred `location: some-cloud` to the spec.)* Notice, if you target `google-compute-engine` location, you may need to specify `bind_address: 0.0.0.0` for the `mysql` cookbook, as described [here](https://github.com/sous-chefs/mysql/blob/main/documentation/resource_mysql_service.md). We'll now walk through the important constituent parts, and then proceed to describing things which can be done to simplify the deployment. ### Cookbook Primary Name The first thing to note is the type definition: - type: chef:mysql This indicates that the Chef entity should be used (`org.apache.brooklyn.entity.chef.ChefEntity`) to interpret and pass the configuration, and that it should be parameterised with a `brooklyn.chef.cookbook.primary.name` of `mysql`. This is the cookbook namespace used by default for determining what to install and run. ### Importing Cookbooks Next we specify which cookbooks are required and where they can be pulled from: cookbook_urls: mysql: https://github.com/opscode-cookbooks/mysql/archive/v4.0.12.tar.gz openssl: https://github.com/opscode-cookbooks/openssl/archive/v1.1.0.tar.gz build-essential: https://github.com/opscode-cookbooks/build-essential/archive/v1.4.4.tar.gz Here, specific versions are being downloaded from the canonical github repository. Any URL can be used, so long as it is resolvable on either the target machine or the Brooklyn server; this includes `file:` and `classpath:` URLs. The archive can be ZIP or TAR or TGZ. The structure of the archive must be that a single folder is off the root, and in that folder contains the usual Chef recipe and auxiliary files. For example, the archive might contain `mysql-master/recipes/server.rb`. Archives such as those above from github match this format. The name of that folder does not matter, as often they contain version information. When deployed, these will be renamed to match the short name (the key in the `cookbooks_url` map, for instance `mysql` or `openssl`). If Chef server is configured (see below), this section can be omitted. ### Launch Run List and Attributes The next part is to specify the Chef run list and attributes to store when launching the entity: launch_run_list: - mysql::server launch_attributes: mysql: server_root_password: p4ssw0rd server_repl_password: p4ssw0rd server_debian_password: p4ssw0rd For the `launch_run_list`, you can use either the YAML `- recipe` syntax or the JSON `[ "recipe" ]` syntax. The `launch_attributes` key takes a map which will be stored against the `node` object in Chef. Thus in this example, the parameter `node['mysql']['server_root_password']` required by the mysql blueprint is set as specified. You can of course set many other attributes in this manner, in addition to those that are required! ### Simple Monitoring The final section determines how Brooklyn confirms that the service is up. Sophisticated solutions may install monitoring agents as part of the `launch_run_list`, with Brooklyn configured to read monitoring information to confirm the launch was successful. However for convenience, two common mechanisms are available out of the box: #service_name: mysqld pid_file: /var/run/mysqld/mysqld.pid If `service_name` is supplied, Brooklyn will check the return code of the `status` command run against that service, ensuring it is 0. (Note that this is not universally reliable, although it is the same mechanism which Chef typically uses to test status when determining whether to start a service. Some services, e.g. postgres, will return 0 even if the service is not running.) If a `pid_file` is supplied, Brooklyn will check whether a process with the PID specified in that file is running. This has been selected for mysql because it appears to be more portable: the service name varies among OS's: it is `mysqld` on CentOS but `mysql` on Ubuntu!Writing Chef for Blueprints
^ ## Making it Simpler The example we've just seen shows how existing Chef cookbooks can be used as the basis for entities. If you're *writing* the Chef recipes, there are a few simple techniques we've established with the Chef community which make blueprints literally as simple as: - type: chef:mysql brooklyn.config: mysql_password: p4ssw0rd pid_file: /var/run/mysqld/mysqld.pid ### Some Basic Conventions * **A `start` recipe**: The first step is to provide a `start` recipe in `recipes/start.rb`; if no `launch_run_list` is supplied, this is what will be invoked to launch the entity. It can be as simple as a one-line file: include_recipe 'mysql::server' * **Using `brooklyn.config`**: All the `brooklyn.config` is passed to Chef as node attributes in the `node['brooklyn']['config']` namespace. Thus if the required attributes in the mysql recipe are set to take a value set in `node['brooklyn']['config']['mysql_password']`, you can dispense with the `launch_attributes` section. ## Using Chef Server The examples so far have not required Chef Server, so they will work without any external Chef dependencies (besides the built-in install from `https://www.opscode.com/chef/install.sh` and the explicitly referenced cookbooks). If you use Chef Server, however, you'll want your managed nodes to be integrated with it. This is easy to set up, with a few options: If you have `knife` set up in your shell environment, the Brooklyn Chef support will use it by default. If the recipes are installed in your Chef server, you can go ahead and remove the `cookbooks_url` section! Use of `solo` or `knife` can be forced by setting the `chef_mode` flag (`brooklyn.chef.mode` config key) to either of those values. (It defaults to `autodetect`, which will use `knife` if it is on the path and satisfies sanity checks). If you want to specify a different configuration, there are a number of config keys you can use: * `brooklyn.chef.knife.executableFile`: this should be point to the knife binary to use * `brooklyn.chef.knife.configFile`: this should point to the knife configuration to use * `brooklyn.chef.knife.setupCommands`: an optional set of commands to run prior to invoking knife, for example to run `rvm` to get the right ruby version on the Brooklyn server If you're interested in seeing the Chef REST API be supported directly (without knife), please let us know. We'd like to see this too, and we'll help you along the way! ## Tips and Tricks To help you on your way writing Chef blueprints, here are a handful of pointers particularly useful in this context: * Configuration keys can be inherited from the top-level and accessed using `$brooklyn:entity('id').config('key_name')`. An example of this is shown in the `mysql-chef.yaml` sample recipe contained in the Brooklyn code base and [here](example_yaml/mysql-chef-2.yaml) for convenience. Here, `p4ssw0rd` is specified only once and then used for all the attributes required by the stock mysql cookbook. * Github tarball downloads! You'll have noticed these in the example already, but they are so useful we thought we'd call them out again. Except when you're developing, we recommend using specific tagged versions rather than master. * The usual machine `provisioning.properties` are supported with Chef blueprints, so you can set things like `minRam` and `osFamily` * To see more configuration options, and understand the ones presented here in more detail, see the javadoc or the code for the class `ChefConfig` in the Brooklyn code base.Advanced Chef Integration
^ ### Adding Sensors and Effectors Custom sensors and effectors can be added using an `entity.initializer` section in the YAML blueprint. One common pattern is to have sensors which extract information from Ohai. Another common pattern is to install a monitoring agent as part of the run list, configured to talk to a monitoring store, and then to add a sensor feed which reads data from that store. On the effector side, you can add SSH-based effectors in the usual way. You can also describe additional chef converge targets following the pattern set down in `ChefLifecycleEffectorTasks`, making use of conveniences in `ChefSoloTasks` and `ChefServerTasks`, or provide effectors which invoke network API's of the systems under management (for example to supply the common `executeScript` effector as on the standard `MySqlNode`). ### Next Steps: Simpifying sensors and effectors, transferring files, and configuring ports The Brooklyn-Chef integration is work in progress, with a few open issues we'd still like to add. Much of the thinking for this is set forth in the [Google document](https://docs.google.com/a/cloudsoftcorp.com/document/d/18ZwzmncbJgJeQjnSvMapTWg6N526cvGMz5jaqdkxMf8) indicated earlier. If you'd like to work with us to implement these, please let us know. ## Reference A general schema for the supported YAML is below: {% highlight yaml %} - type: chef:cookbook_name brooklyn.config: cookbook_urls: cookbook_name: url://for/cookbook.tgz dependency1: url://for/dependency1.tgz launch_run_list: [ "cookbook_name::start" ] launch_attributes: # map of arguments to set in the chef node service_name: cookbook_service pid_file: /var/run/cookbook.pid {% endhighlight %} If you are interested in exploring the Java code for creating blueprints, start with the `TypedToyMySqlEntiyChef` class, which essentially does what this tutorial has shown; and then move on to the `DynamicToyMySqlEntiyChef` which starts to look at more sophisticated constructs. (Familiarity with BASH and basic Java blueprints may be useful at that stage.)Salt in YAML Blueprints
^ This guide describes how Brooklyn entities can be created using the [Salt](https://docs.saltproject.io/) infrastructure management tool. At present Brooklyn provides basic support for Salt, operating in a 'masterless' mode. Comments on this support and suggestions for further development are welcome. This guide assumes you are familiar with the basics of [creating YAML blueprints](/guide/blueprints). {% include list-children.html %}About Salt
^ ## What you need to know about Salt Salt is designed to work in either what it calls a 'master/minion' or a 'masterless' topology. In the former, the master server acts as a managing controller for any number of client nodes, called 'minions'. A salt daemon running on the minion connects back to the master server for its operation, and manages the software on the minion according to a specification of 'states' defined in Salt configuration files. In the latter, there is no master, and the salt daemon on the minion operates based on the Salt files on the minion node. This is the mode currently supported by Brooklyn. A 'State' in Salt is a specification of a configuration of some aspect of a system, such as what packages are installed, what system services are running, what files exist, etc. Such states are described in "SLS" files (for SaLt State file). These files are typically written as templates using the "Jinja" templating engine. The actual SLS files for the minion are then created by processing the templates using configuration data provided via Salt's "Pillar" system. Salt comes with built-in support for many software systems, and has a repository of pre-written Salt states known as 'formulas' on GitHub, at [https://github.com/saltstack-formulas](https://github.com/saltstack-formulas). ### How Brooklyn interacts with Salt Brooklyn provides a Salt entity type. An entity of this type can be specified in a blueprint in order to provision the node through Salt. The Salt entity will download Salt and install it on the node. The entity type supports the configuration of Salt formulas and Pillar data to download, and the configuration of what Salt states to apply. These are managed using Salt in 'masterless' mode using the 'salt-call' functionality of Salt. The Salt 'highstate' (the collection of states applied to the system) is exposed as a sensor on the entity. An effector is provided on the entity that supports general purpose Salt instructions.Creating Blueprints with Salt
^ To write a blueprint to use Salt with Brooklyn it will help to have a degree of familiarity with Salt itself. In the sections below, when the Brooklyn configuration is described, the underlying Salt operation is also noted briefly, for clarity for readers who know Salt. To manage a node with Salt, create a blueprint containing a service of type `org.apache.brooklyn.entity.cm.salt.SaltEntity` and define the `formulas` and `start_states` For example: name: Salt Example setting up Apache httpd location: my-cloud services: - id: httpd-from-salt type: org.apache.brooklyn.entity.cm.salt.SaltEntity formulas: - https://github.com/saltstack-formulas/apache-formula/archive/master.tar.gz start_states: - apache This example specifies that Brooklyn should use Salt to download the `apache-formula` from the Saltstack repository on Github. The apache formula contains the Apache web server with a simple "it worked" style index page. To start the entity, Brooklyn will use Salt to apply the `apache` state, which will bring up the web server. A typical usage of the Salt entity might be to include a formula from the Saltstack repository, such as `apache` above, and another formula created by the blueprint author, with additional states, such as web site content for the apache server. ### Start States The `start_states` configuration key defines the top level list of states that will be applied using Salt. These values are added to the Salt `top.sls` file and applied using `state.apply`. This configuration key is mandatory. ### Stop States The `stop_states` configuration key is used to specify states that should be applied when the 'stop' effector is invoked on the entity. For example, the Saltstack `mysql` [formula](https://github.com/saltstack-formulas/mysql-formula) supplies a state `mysql.disabled` that will shut down the database server. If the Saltstack formula does not supply a suitable stop state, the blueprint author can create a suitable state and include it in an additional formula to be supplied in the `formulas` section. The `stop_states` configuration key is optional; if it is not provided, Brooklyn assumes that each state `S` in the `start_states` will have a matching `S.stop` state. If any `S` does not have such a state, the stop effector will fail stopping processes. Note that on a machine created for this entity, Brooklyn's default behaviour may be to proceed to destroy the VM, so stop states are not always needed unless there is a cleaner shutdown process or you are using long-running servers. ### Restart States For completeness, Brooklyn also provides a `restart_states` configuration key. These states are applied by the restart effector, and blueprint authors may choose to provide custom states to implement restart if that is applicable for their application. This key is again optional. If not supplied, Brooklyn will go through each of the states `S` in `start_states` looking for a matching `S.restart` state defined in the formulas. If all exist, these will be applied by the restart effector. If none exist, Brooklyn will invoke the `stop` and then `start` effectors -- so `restart` states are not required for Brooklyn to use Salt. (If some but not all have matching `restart` states, Brooklyn will fail the restart, on the assumption that the configuration is incomplete.) ### Formulas The `formulas` key provides the URLs for archives containing the Salt formulas that defined the required states. These archives are downloaded to the `/srv/formula` directory on the minion and added to the state filesystem roots configuration in Salt's minion config, so that their states are available for a `state.apply`. ### Pillar Configuration A typical Salt deployment will include both states (provided via Salt formulas) and configuration data, provided through Salt's "Pillar" component. Brooklyn provides configuration keys for the Salt entity to specify where to get the Pillar configuration data. For example: name: Salt Example setting up MySQL with a Pillar location: my-cloud services: - id: mysql-from-salt-with-my-pillar type: org.apache.brooklyn.entity.cm.salt.SaltEntity formulas: - https://github.com/saltstack-formulas/mysql-formula/archive/master.tar.gz - http://myhost:8080/my-mysql-formula.tar.gz start_states: - mysql stop_states: - mysql.disabled pillars: - mysql pillarUrls: - http://myhost:8080/my-mysql-pillar.tar.gz This blueprint contains the MySQL database, and includes a formula available from `myhost` which includes the schema information for the DB. The MySQL formula from Saltstack has extensive configurability through Salt Pillars. In the blueprint above, Brooklyn is instructed to apply the pillars defined in the `pillars` configuration key. (This will add these values to the Salt Pillars `top.sls` file.) The pillar data must be downloaded; for this, the `pillarUrls` key provides the address of an archive containing the Pillar data. The contents of the archive will be extracted and put in the `/srv/pillar` directory on the minion, in order to be available to Salt when applying the pillar. For example, the archive above can simply have the structure pillar/ | +- mysql/ | +- init.sls The init.sls contains the pillar configuration values, such as # Manage databases database: - orders schema: orders: load: True source: salt://mysql/files/orders.schema Meanwhile the `my-mysql-formula.tar.gz` formula archive contains the schema: my-mysql-formula/ | +- mysql/ | +- files/ | +- orders.schema Note that the blueprint above defines an `id` for the Salt entity. This id, if provided, is set as the minion id in the Salt configuration file. This is useful particularly for Pillar configuration, as, if there are more than one Salt managed nodes in the application, they can share a common Pillar file, with appropriate subdivisions of pillar data based on minion id. ### Highstate Sensors The Salt entity exposes the Salt "highstate" on the node via Brooklyn sensors. Firstly a single sensor `salt.states` contains a list of all the top level Salt state ID declarations in the highstate. For example, for the mysql case above, this might look like: ["mysql_additional_config", "mysql_config", "mysql_db_0", "mysql_db_0_load", "mysql_db_0_schema", "mysql_debconf", "mysql_debconf_utils", "mysql_python", "mysql_user_frank_localhost", "mysql_user_frank_localhost_0", "mysql_user_nopassuser_localhost", "mysqld"] Then, for each ID and each Salt state function in that ID, a Brooklyn sensor is created, containing a map of the data from the highstate. For example, the `salt.state.mysqld.service.running` sensor would have a value like: {"name":"mysql", "enable":true, "watch":[{"pkg":"mysqld"}, {"file":"mysql_config"}], "order":10005} ### saltCall Effector The Salt entity includes a general purpose Salt effector, `saltCall`, which permits execution of Salt commands via `salt-call --local`. It contains a single parameter, `spec`, which specifies the command to invoke. For example, invoking the effector with a `spec` value of `network.interfaces --out=yaml` would return a YAML formatted map of the network interfaces on the minion.YAML Blueprint Advanced Example
^ By this point you should be familiar with the fundamental concepts behind both Apache Brooklyn and YAML blueprints. This section of the documentation is intended to show a complete, advanced example of a YAML blueprint. The intention is that this example is used to learn the more in-depth concepts, and also to serve as a reference when writing your own blueprints. This page will first explain what the example application is and how to run it, then it will spotlight interesting features. Please note, there is now a much more up-to-date ELK blueprint that can be found [here](https://github.com/brooklyncentral/brooklyn-elk/). We've using an older version of this in the tutorial as it highlights some key Brooklyn concepts. ### ELK Stack Example This example demonstrates the deployment of an ELK Stack (Elasticsearch, Logstash and Kibana), using the provided blueprint to deploy, install, run and manage all three. Briefly, the component parts are: * Elasticsearch: A clustered search engine * Logstash: Collects, parses and stores logs. For our example it will store logs in Elasticsearch * Kibana: A web front end to Elasticsearch We also deploy a simple webserver whose logs will be collected. * Tomcat 8: Web server whose logs will be stored in Elasticsearch by Logstash. For more about the ELK stack, please see the documentation [here](https://www.elastic.co/webinars/introduction-elk-stack). #### The Blueprints ----------- There are four blueprints that make up this application. Each of them is used to add one or more catalog items to Brooklyn. You can find them below: * [Elasticsearch](example_yaml/brooklyn-elasticsearch-catalog.bom) * [Logstash](example_yaml/brooklyn-logstash-catalog.bom) * [Kibana](example_yaml/brooklyn-kibana-catalog.bom) * [ELK](example_yaml/brooklyn-elk-catalog.bom) #### Running the example First, add all four blueprints to the Brooklyn Catalog. This can be done by going to the "Catalog" component, then clicking the button "Upload to catalog" and selecting the above blueprints (or drag-and-drop them). Once this is done, search for an entity called "ELK Stack". You will be able to deploy it by clicking on the "Deploy" button from the details page. Note that you can also perform this operation from the quick launch panel on the homepage. Alternatively use the `br` Brooklyn command line tool and add the files with `br catalog add`. #### Exploring the example After the application has been deployed, you can ensure it is working as expected by checking the following: * There is a Kibana sensor called `main.uri`, the value of which points to the Kibana front end. You can explore this front end, and observe the logs stored in Elasticsearch. Many Brooklyn applications have a `main.uri` set to point you in the right direction. * You can also use the Elasticsearch REST API to explore further. The Elasticsearch Cluster entity has a `urls.http.list` sensor. Using a host:port from that list you will be able to access the REST API. The following URL will give you the state of the cluster `http://Blueprinting Tips
^ ## YAML Recommended The recommended way to write a blueprint is as a YAML file. This is true both for building an application out of existing blueprints, and for building new integrations. The use of Java is reserved for those use-cases where the provisioning or configuration logic is very complicated. ## Be Familiar with Brooklyn Be familiar with the stock entities available in Brooklyn. For example, prove you understand the concepts by making a deployment of a cluster of Tomcat servers before you begin writing your own blueprints. ## Ask For Help Ask for help early. The community is keen to help, and also keen to find out what people find hard and how people use the product. Such feedback is invaluable for improving future versions. ## Faster Dev-Test Writing a blueprint is most efficient and simple when testing is fast, and when testing is done incrementally as features of the blueprint are written. The slowest stages of deploying a blueprint are usually VM provisioning and downloading/installing of artifacts (e.g. RPMs, zip files, etc). Options for speeding up provisioning include those below. #### Deploying to Bring Your Own Nodes (BYON) A [BYON location](/guide/locations/index.html#byon) can be defined, which avoids the time required to provision VMs. This is fast, but has the downside that artifacts installed during a previous run can interfere with subsequent runs. These VMs should mirror the target environment as much as possible. #### Deploying to the "localhost" location This is fast and simple, but has some obvious downsides: * Artifacts are installed directly on your desktop/server. * The artifacts installed during previous runs can interfere with subsequent runs. * Some entities require `sudo` rights, which must be granted to the user running Brooklyn. #### Deploying to Containers Containers can give a fast, convenient way to test blueprints, to run apps, or to failover temporarily. #### Local Repository of Install Artifacts To avoid re-downloading install artifacts on every run, these can be saved to `~/.brooklyn/repository/`. The file structure is a sub-directory with the entity's simple name, then a sub-directory with the version number, and then the files to be downloaded. For example, `~/.brooklyn/repository/TomcatServer/7.0.56/apache-tomcat-7.0.56.tar.gz`. If possible, synchronise this directory with your test VMs. #### Re-install on BYON If using BYON or localhost, the install artifacts will by default be installed to a directory like `/tmp/brooklyn-myname/installs/`. If install completed successfully, then the install stage will be subsequently skipped (a marker file being used to indicate completion). To re-test the install phase, delete the install directory (e.g. delete `/tmp/brooklyn-myname/installs/TomcatServer_7.0.56/`). Where installation used something like `apt-get install` or `yum install`, then re-testing the install phase will require uninstalling these artifacts manually. ## Monitoring and Managing Applications Think about what it really means for an application to be running. The out-of-the-box default for a software process is the lowest common denominator: that the process is still running. Consider checking that the app responds over HTTP etc. If you have control of the app code, then consider adding an explicit health check URL that does more than basic connectivity tests. For example, can it reach the database, message broker, and other components that it will need for different operations. ## Writing Composite Blueprints Write everything in discrete chunks that can be composed into larger pieces. Do not write a single mega-blueprint. For example, ensure each component is added to the catalog independently, along with a blueprint for the composite app. Experiment with lots of small blueprints to test independent areas before combining them into the real thing. ## Writing Entity Tests Use the [test framework](/guide/blueprints/test) to write test cases. This will make automated (regression) testing easier, and will allow others to easily confirm that the entity works in their environment. If using Maven/Gradle then use the [Brooklyn Maven plugin](https://github.com/brooklyncentral/brooklyn-maven-plugin) to test blueprints at build time. ## Custom Entity Development If writing a custom integration, the following recommendations may be useful: * Always be comfortable installing and running the process yourself before attempting to automate it. * For the software to be installed, use its Installation and Admin guides to ensure best practices are being followed. Use blogs and advice from experts, when available. * Where there is a choice of installation approaches, use the approach that is most appropriate for production use-cases (even if this is harder to test on locahost). For example, prefer the use of RPMs versus unpacking zip files, and prefer the use of services versus invoking a `bin/start` script. * Ensure every step is scriptable (e.g. manual install does not involve using a text editor to modify configuration files, or clicking on things in a web-console). * Write scripts (or Chef recipes, or Puppet manifests, etc), and test these by executing manually. Only once these work in isolation, add them to the entity blueprint. * Externalise the configuration where appropriate. For example, if there is a configuration file then include a config key for the URL of the configuration file to use. Consider using FreeMarker templating for such configuration files. * Focus on a single OS distro/version first, and clearly document these assumptions. * Breakdown the integration into separate components, where possible (and thus develop/test them separately). For example, if deploying a MongoDB cluster then first focus on single-node MongoDB, and then make that configurable and composable for a cluster. * Where appropriate, share the new entity with the Brooklyn community so that it can be reviewed, tested and improved by others in the community! ## Cloud Portability You get a lot of support out-of-the-box for deploying blueprints to different clouds. The points below may also be of help: * Test (and regression test) on each target cloud. * Be aware that images on different clouds can be very different. For example, two CentOS 6.6 VMs might have different pre-installed libraries, different default iptables or SE Linux settings, different repos, different sudo configuration, etc. * Different clouds handle private and public IPs differently. One must be careful about which address to advertise to for use by other entities. * VMs on some clouds may not have a well-configured hostname (e.g. `ping $(hostname)` can fail). * VMs in different clouds have a different number of NICs. This is important when choosing whether to listen on 0.0.0.0 or on a specific NIC. ## Investigating Errors ALWAYS keep logs when there is an error. See the [Troubleshooting](/guide/ops/troubleshooting) guide for more information.Locations
^ Locations are the environments to which Brooklyn deploys applications. Most commonly these are cloud services such as AWS, Azure, and GCE. Brooklyn also supports deploying to a pre-provisioned network or to localhost (primarily useful for testing blueprints). See also: * The [Locations yaml guide](/guide/blueprints/setting-locations.md) * Use within an entity of the configuration option [provisioning.properties](/guide/blueprints/entity-configuration.md#entity-provisioningproperties-overriding-and-merging) * How to add location definitions to the [Catalog](/guide/blueprints/catalog); and * How to use [Externalized Configuration](/guide/ops/externalized-configuration.md). The requirements for how a provisioned machine should behave will depend on the entites subsequently deployed there. Below are a set of common assumptions, made by many entity implementations, which could cause subsequent errors if they do not hold. These relate to the machine's configuration, rather than additional networking or security that a given Cloud might offer. Also see the [Troubleshooting](/guide/ops/troubleshooting) docs. {% child_content %} ## Remote Access ### SSH or WinRM Access Many entities require ssh'ing (or using WinRM for Windows), to install and configure the software. An example of disabling all ssh'ing is shown below: {% read _provisioned-machine-requirements.camp.md %} ### Parsing SSH stdout: No Extra Lines For entities that execute ssh commands, these sometimes parse the resulting stdout. It is strongly recommended that VMs are configured so that no additional stdout is written when executing remote ssh (or WinRM) commands. Such stdout risks interfering with the response parsing in some blueprints. For example, if configuring the VM to write out "Last login" information, this should be done for only "interactive" shells (see [Stackoverflow](http://stackoverflow.com/a/415444/1393883) for more details). ### Passwordless Sudo Does passwordless sudo work? Try executing: sudo whoami See [Passwordless Sudo](#passwordless-sudo). ## Advertised Addresses ### Hostname Resolves Locally Does the hostname known at the box resolve at the box? Try executing: ping $(hostname) if not, consider setting `generate.hostname: true` in the location config, for jclouds-based locations. ### IP Resolves Locally For the IP address advertised in Brooklyn using the sensor `host.addresses.private` (or `host.subnet.address`), can the machine reach that IP? Get the sensor value, and then try executing: ping ${PRIVATE_IP} Is there a public IP (advertised using the sensor `host.addresses.public`, or `host.address`), and can the machine reach it? Get the sensor value, and then try executing: ping ${PUBLIC_IP} ## Networking ### Public Internet Access Can the machine reach the public internet, and does DNS resolve? Try executing: ping www.example.org ### Machine's Hostname in DNS Is the machine hostname well-known? If ones does a DNS lookup, e.g. from the Brooklyn server, does it resolve and does it return the expected IP (e.g. the same IP as the `host.addresses.public` sensor)? Try using the hostname that the machine reports when you execute `hostname`. Many blueprints do not require this, instead using IP addresses directly. Some blueprints may include registration with an appropriate DNS server. Some clouds do this automatically. ### Reachability When provisioning two machines, can these two machines reach each other on the expected IP(s) and hostname(s)? Try using `ping` from one machine to another using the public or subnet ip or hostname. However, note that `ping` requires access over ICMP, which may be disabled. Alternatively, try connecting to a specific TCP port using `telnetReference Guide
^ {% include list-children.html %}Production Installation
^ {% include fields.md %} To install Apache Brooklyn on a production server: 1. [Set up the prerequisites](#prerequisites) 1. [Download Apache Brooklyn](#download) 1. [Configuring brooklyn.cfg](#configuring-properties) 1. [Configuring Karaf Security](#configuring-karaf-security) 1. [Configuring default.catalog.bom](#configuring-catalog) 1. [Test the installation](#confirm) This guide covers the basics. You may also wish to configure: * [Logging](/guide/ops/logging/index.md) * [Persistence](persistence) * [High availability](high-availability) ### Set up the Prerequisites Check that the server meets the [requirements](requirements.md). Then configure the server as follows: * install Java JRE or JDK (version 8 or later) * enable "Java Cryptography Extension" (already enabled out of the box of OpenJDK installs) * install an [SSH key](/guide/locations#ssh-keys), if not available * if the "localhost" location will be used, enable [passwordless ssh login](/guide/locations#ssh-keys) * create a `~/.brooklyn` directory on the host with `$ mkdir ~/.brooklyn` * check your `iptables` or other firewall service, making sure that incoming connections on port 8443 is not blocked * check that the [Linux kernel entropy](/guide/ops/troubleshooting/increase-entropy.md) is sufficient * check that the [ulimit values](/guide/ops/troubleshooting/increase-system-resource-limits.md) are sufficiently high * ensure external libraries are up-to-date, including `nss` for SSL. * ensure the time is continually accurate, ideally by running a service like the [ntp daemon](http://www.ntp.org/). ### Download Apache Brooklyn Download Brooklyn and obtain a binary build as described on [the download page](/website/download/). {% if brooklyn_version contains 'SNAPSHOT' %} Expand the `tar.gz` archive (note: as this is a -SNAPSHOT version, your filename will be slightly different): {% else %} Expand the `tar.gz` archive: {% endif %} {% if brooklyn_version contains 'SNAPSHOT' %} {% highlight bash %} % tar -zxf apache-brooklyn-dist-{{ site.brooklyn-stable-version }}-timestamp-dist.tar.gz {% endhighlight %} {% else %} {% highlight bash %} % tar -zxf apache-brooklyn-{{ site.brooklyn-stable-version }}-dist.tar.gz {% endhighlight %} {% endif %} This will create a `apache-brooklyn-{{ site.brooklyn-stable-version }}` folder. Let's setup some paths for easy commands. {% highlight bash %} % cd apache-brooklyn-{{ site.brooklyn-stable-version }} % BROOKLYN_DIR="$(pwd)" % export PATH=$PATH:$BROOKLYN_DIR/bin/ {% endhighlight %} ### Configuring brooklyn.cfg Set up `brooklyn.cfg` as described [here](configuration/brooklyn_cfg.md): * Configure the users who should have access * Turn on HTTPS * Supply credentials for any pre-defined clouds ### Configuring Karaf Security Out of the box, Apache Brooklyn includes the default Karaf security configuration. This configuration is used to manage connections to the ssh port of Karaf (which is available to localhost connections only). It is recommended that you update the credentials as detailed in the [Karaf Security](https://karaf.apache.org/manual/latest/security#_users_groups_roles_and_passwords) page. ### Configuring the Catalog By default Brooklyn loads the catalog of available application components and services from `default.catalog.bom` on the classpath. The initial catalog is in `conf/brooklyn/` in the dist. If you have a preferred catalog, simply replace that file. [More information on the catalog is available here.](/guide/blueprints/catalog) ### Confirm Installation Launch Brooklyn in a disconnected session so it will remain running after you have logged out: {% highlight bash %} % nohup bin/brooklyn launch > /dev/null 2&>1 & {% endhighlight %} Apache Brooklyn should now be running on port 8081 (or other port if so specified). To install on a different port edit config in `etc/org.ops4j.pax.web.cfg`.Starting, Stopping and Monitoring
^ **NOTE:** This document is for information on starting an Apache Brooklyn Server. For information on using the Brooklyn Client CLI to access an already running Brooklyn Server, refer to [Client CLI Reference](cli). ## Packages for RHEL/CentOS and Ubuntu If you are using the `.rpm` or `.deb` package of Apache Brooklyn, then Brooklyn will integrate with your OS service management. Commands such as `service brooklyn start` will work as expected, and Brooklyn's PID file will be stored in the normal location for your OS, such as `/var/run/brooklyn.pid`. ## Platform-independent distributions The platform-independent distributions are packaged in `.tar.gz` and `.zip` files. ### Starting To launch Brooklyn, from the directory where Brooklyn is unpacked, run: {% highlight bash %} % bin/start {% endhighlight %} With no configuration, this will launch the Brooklyn web console and REST API on [`http://localhost:8081/`](http://localhost:8081/), listening on all network interfaces. No credentials are required by default. It is strongly recommended to [configure security](configuration). See the [Server CLI Reference](server-cli-reference.md) for more information about the Brooklyn server process. ### Stopping To stop Brooklyn, from the directory where Brooklyn is unpacked, run: For example: {% highlight bash %} % bin/stop {% endhighlight bash %} ## Monitoring For `.tar.gz` and `.zip` distributions of Brooklyn, the Brooklyn startup script will create a file name `pid_java` at the root of the Brooklyn directory, which contains the PID of the last Brooklyn process to be started. You can examine this file to discover the PID, and then test that the process is still running. `.rpm` and `.deb` distributions of Brooklyn will use the normal mechanism that your OS uses, such as writing to `/var/run/brooklyn.pid`. This should lead to a fairly straightforward integration with many monitoring tools - the monitoring tool can discover the expected PID, and can execute the start or stop commands shown above as necessary. For example, here is a fragment of a `monitrc` file as used by [Monit](https://mmonit.com/monit/), for a Brooklyn `.tar.gz` distribution unpacked and installed at `/opt/apache-brooklyn`: {% highlight text %} check process apachebrooklyn with pidfile /opt/apache-brooklyn/pid_java start program = "/bin/bash -c '/opt/apache-brooklyn/bin/brooklyn launch --persist auto & disown'" with timeout 10 seconds stop program = "/bin/bash -c 'kill $( cat /opt/apache-brooklyn/pid_java )'" {% endhighlight %} In addition to monitoring the Brooklyn process itself, you will almost certainly want to monitor resource usage of Brooklyn. In particular, please see the [Requirements](requirements.md#disk-space) section for a discussion on Brooklyn's disk space requirements.Server CLI Reference
^ **NOTE:** This document is for information on starting a Brooklyn Server. For information on using the Brooklyn Client CLI to access an already running Brooklyn Server, refer to [Client CLI Reference](cli). ## Launch command To launch Brooklyn, from the directory where Brooklyn is unpacked, run: {% highlight bash %} % nohup bin/brooklyn launch > /dev/null 2>&1 & {% endhighlight %} With no configuration, this will launch the Brooklyn web console and REST API on [`http://localhost:8081/`](http://localhost:8081/), listening on all network interfaces. No credentials are required by default. For a production system, or if Apache Brooklyn is publicly reachable, it is strongly recommended to [configure security](configuration/brooklyn_cfg.md). By default, Brooklyn will write log messages at the INFO level or above to `brooklyn.info.log` and messgages at the DEBUG level or above to `brooklyn.debug.log`. Redirecting the output to `/dev/null` prevents the default console output being written to `nohup.out`. You may wish to [add Brooklyn to your path](#path-setup); assuming you've done this, to get information the supported CLI options at any time, just run `brooklyn help`: {% highlight bash %} % bin/brooklyn help usage: brooklyn [(-q | --quiet)] [(-v | --verbose)]Client CLI Reference
^ **NOTE:** These documents are for using the Brooklyn Client CLI tool to access a running Brooklyn Server. For information on starting on a Brooklyn Server, refer to [Server CLI Reference](/guide/ops/server-cli-reference.md). ## Obtaining the CLI tool A selection of distributions of the CLI tool, `br`, are available to download from the download site {% if site.brooklyn-version contains 'SNAPSHOT' %} [here](https://repository.apache.org/service/local/artifact/maven/redirect?r=snapshots&g=org.apache.brooklyn&a=brooklyn-client-cli&v={{site.brooklyn-version}}&c=bin&e=zip). {% else %} here: * [Windows](https://www.apache.org/dyn/closer.lua/brooklyn/apache-brooklyn-{{site.brooklyn-version}}-client-cli-windows.zip) * [Linux](https://www.apache.org/dyn/closer.lua/brooklyn/apache-brooklyn-{{site.brooklyn-version}}-client-cli-linux.tar.gz) * [OSX](https://www.apache.org/dyn/closer.lua/brooklyn/apache-brooklyn-{{site.brooklyn-version}}-client-cli-macosx.tar.gz) {% endif %} Alternatively the CLI tool is available as an executable binary for many more platforms in the Apache Brooklyn distribution, under `bin/brooklyn-client-cli/`, with each build in its own subdirectory: * Mac: `darwin.amd64/` * Windows 32-bit: `windows.386/` * Windows 64-bit: `windows.amd64/` * Linux 32-bit: `linux.386/` * Linux 64-bit: `linux.amd64/` The binary is completely self-contained so you can either copy it to your `bin/` directory or add the appropriate directory above to your path: PATH=$PATH:$HOME/apache-brooklyn/bin/brooklyn-client-cli/linux.amd64/ ## Documentation {% include list-children.html %}CLI Reference Guide
^ ## Usage {% highlight text %} NAME: br - A Brooklyn command line client application USAGE: br [global options] command [command options] [arguments...] {% endhighlight %} ## List of Commands Commands whose description begins with a `*` character are particularly experimental and likely to change in upcoming releases. {% highlight text %} COMMANDS: Commands whose description begins with a "*" character are particularly experimental and likely to change in upcoming releases. If not otherwise specified, "SCOPE" below means application or entity scope. If an entity scope is not specified, the application entity is used as a default. access Show access control activity Show the activity for an application / entity add-catalog (deprecated, use catalog add) Add a new catalog item from the supplied YAML (a file or HTTP URL) add-children * Add a child or children to this entity from the supplied YAML application Show the status and location of running applications catalog Catalog operations config Show the config for an application or entity delete * Delete (expunge) a brooklyn application deploy Deploy a new application from the given YAML (read from file or URL, or stdin) destroy-policy Destroy a policy effector Show the effectors for an application or entity entity Show the entities of an application or entity env Show the ENV stream for a given activity invoke Invoke an effector of an application and entity locations * List the available locations login Login to brooklyn policy Show the policies for an application or entity rename Rename an application or entity restart Invoke restart effector on an application and entity sensor Show values of all sensors or named sensor for an application or entity set Set config for an entity spec Get the YAML spec used to create the entity, if available start Invoke start effector on an application and entity start-policy Start or resume a policy stderr Show the STDERR stream for a given activity stdin Show the STDIN stream for a given activity stdout Show the STDOUT stream for a given activity stop Invoke stop effector on an application and entity stop-policy Suspends a policy tree * Show the tree of all applications version Display the version of the connected Brooklyn help GLOBAL OPTIONS: --skipSslChecks Skip verification of server's certificate chain and hostname (for use with self-signed certs) --help, -h show help --version, -v print the version {% endhighlight %} ## Scopes Many commands require a "scope" expression to indicate the target on which they operate. Where this is required the usage statements below will use the shorthand nomenclature of `CLI Usage Guide
^ This document provides a brief overview of using the most common Brooklyn CLI commands, by using the CLI to deploy an application then examine various aspects of it. The YAML blueprint for the application that will be deployed is shown at the end of this document. **NOTE:** In the sample output, some additional line-wrapping has been used to aid readabilty. Additionally, the vertical bar character is omitted from table output for readability. For scripting purposes it can be useful in conjunction with a shell pipeline like `some_command | cut -f 1 -d '|' | xargs -L1 some_other_command`. ## Login First, login to the running Brooklyn server. This example assumes that the Brooklyn server is running on `localhost` configured to use basic authorization; change the URL and credentials as necessary. {% highlight text %} $ br login http://localhost:8081 admin Enter Password: * Connected to Brooklyn version 0.9.0-SNAPSHOT at http://localhost:8081 {% endhighlight %} The version of the connected Brooklyn server may be viewed with the `version` command: {% highlight text %} $ br version 0.9.0-SNAPSHOT {% endhighlight %} ## Applications Deploy the application; on success the Id of the new application is displayed: {% highlight text %} $ br deploy webapp-policy.yaml Id: lmOcZbsT Name: WebCluster Status: In progress {% endhighlight %} The `application` command can be used to list a summary of all the running applications. After all of the entities have been started, the application status changes to `RUNNING`: {% highlight text %} $ br application Id Name Status Location YeEQHwgW AppCluster RUNNING CNTBOtjI lmOcZbsT WebCluster RUNNING CNTBOtjI {% endhighlight %} Further details of an application can be seen by using the ApplicationID or Name as a parameter for the `application` command: {% highlight text %} $ br application WebCluster Id: lmOcZbsT Name: WebCluster Status: RUNNING ServiceUp: true Type: org.apache.brooklyn.entity.stock.BasicApplication CatalogItemId: null LocationId: CNTBOtjI LocationName: FixedListMachineProvisioningLocation:CNTB LocationSpec: byon LocationType: org.apache.brooklyn.location.byon.FixedListMachineProvisioningLocation {% endhighlight %} The configuration details of an application can be seen with the `config` command: {% highlight text %} $ br application WebCluster config Key Value camp.template.id TYWVroRz brooklyn.wrapper_app true {% endhighlight %} ## Entities The entities of an application can be viewed with the `entity` command: {% highlight text %} $ br app WebCluster entity Id Name Type xOcMooka WebApp org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster thHnLFkP WebDB org.apache.brooklyn.entity.database.mysql.MySqlNode {% endhighlight %} It is common for an entity to have child entities; these can be listed by providing an entity-scope for the `entity` command: {% highlight text %} $ br app WebCluster entity WebApp entity Id Name Type e5pWAiHf Cluster of TomcatServer org.apache.brooklyn.entity.webapp.DynamicWebAppCluster CZ8QUVgX NginxController:CZ8Q org.apache.brooklyn.entity.proxy.nginx.NginxController {% endhighlight %} or by using `-c` (or `--children`) flag with the `entity` command: {% highlight text %} $ br app WebCluster entity -c e5pWAiHf Id Name Type x0P2LRxZ quarantine org.apache.brooklyn.entity.group.QuarantineGroup QK6QjmrW TomcatServer:QK6Q org.apache.brooklyn.entity.webapp.tomcat.TomcatServer {% endhighlight %} As for applications, the configuration details of an entity can be seen with the `config` command: {% highlight text %} $ br app WebCluster entity thHnLFkP config Key Value install.unique_label MySqlNode_5.6.26 brooklyn.wrapper_app true datastore.creation.script.template.url https://bit.ly/brooklyn-visitors-creation-script camp.template.id dnw3GqN0 camp.plan.id db onbox.base.dir /home/vagrant/brooklyn-managed-processes onbox.base.dir.resolved true {% endhighlight %} The value of a single configuration item can be displayed by using the configuration key as a parameter for the `config` command: {% highlight text %} $ br app WebCluster entity thHnLFkP config datastore.creation.script.template.url https://bit.ly/brooklyn-visitors-creation-script {% endhighlight %} The value of a configuration item can be changed by using the `set` command: {% highlight text %} $ br app WebCluster entity thHnLFkP config datastore.creation.script.template.url set \"https://bit.ly/new-script\" {% endhighlight %} ## Sensors The sensors associated with an application or entity can be listed with the `sensor` command: {% highlight text %} $ br app WebCluster entity CZ8QUVgX sensor Name Value download.addon.urls: {"stickymodule":"https://bitbucket.org/nginx-goodies/n ginx-sticky-module-ng/get/${addonversion}.tar.gz","pcr e":"ftp://ftp.csx.cam.ac.uk/pub/software/programming/p cre/pcre-${addonversion}.tar.gz"} download.url: http://nginx.org/download/nginx-${version}.tar.gz expandedinstall.dir: /home/vagrant/brooklyn-managed-processes/installs/Ngi nxController_1.8.0/nginx-1.8.0 host.address: 192.168.52.102 host.name: 192.168.52.102 host.sshAddress: vagrant@192.168.52.102:22 host.subnet.address: 192.168.52.102 host.subnet.hostname: 192.168.52.102 http.port: 8000 install.dir: /home/vagrant/brooklyn-managed-processes/installs/Ngin xController_1.8.0 log.location: /home/vagrant/brooklyn-managed-processes/apps/FoEXXwJ2 /entities/NginxController_CZ8QUVgX/console main.uri: http://192.168.52.102:8000/ member.sensor.hostandport: member.sensor.hostname: {"typeToken":null,"type":"java.lang.String","name":"ho st.subnet.hostname","description":"Host name as known internally in the subnet where it is running (if diffe rent to host.name)","persistence":"REQUIRED"} member.sensor.portNumber: {"typeToken":null,"type":"java.lang.Integer","name":"h ttp.port","description":"HTTP port","persistence":"RE QUIRED","configKey":{"name":"http.port","typeToken":nu ll,"type":"org.apache.brooklyn.api.location.PortRange" ,"description":"HTTP port","defaultValue":{"ranges":[{ "port":8080},{"start":18080,"end":65535,"delta":1}]}," reconfigurable":false,"inheritance":null,"constraint": "ALWAYS_TRUE"}} nginx.log.access: /home/vagrant/brooklyn-managed-processes/apps/FoEXXwJ2 /entities/NginxController_CZ8QUVgX/logs/access.log nginx.log.error: /home/vagrant/brooklyn-managed-processes/apps/FoEXXwJ2 /entities/NginxController_CZ8QUVgX/logs/error.log nginx.pid.file: /home/vagrant/brooklyn-managed-processes/apps/FoEXXwJ2 /entities/NginxController_CZ8QUVgX/pid.txt nginx.url.answers.nicely: true proxy.domainName: proxy.http.port: 8000 proxy.https.port: 8443 proxy.protocol: http proxy.serverpool.targets: {"TomcatServerImpl{id=QK6QjmrW}":"192.168.52.103:8080"} run.dir: /home/vagrant/brooklyn-managed-processes/apps/FoEXXwJ2 /entities/NginxController_CZ8QUVgX service.isUp: true service.notUp.diagnostics: {} service.notUp.indicators: {} service.problems: {} service.process.isRunning: true service.state: RUNNING service.state.expected: running @ 1449314377781 / Sat Dec 05 11:19:37 GMT 2015 softwareprocess.pid.file: softwareservice.provisioningLocation: {"type":"org.apache.brooklyn.api.location.Location","i d":"zhYBc6xt"} webapp.url: http://192.168.52.102:8000/ {% endhighlight %} Details for an individual sensor can be shown by providing the Sensor Name as a parameter to the `sensor` command: {% highlight text %} $ br app WebCluster entity CZ8QUVgX sensor service.state.expected running @ 1449314377781 / Sat Dec 05 11:19:37 GMT 2015 {% endhighlight %} ## Effectors The effectors for an application or entity can be listed with the `effector` command: {% highlight text %} $ br app WebCluster effector Name Description Parameters restart Restart the process/service represented by an entity start Start the process/service represented by an entity locations stop Stop the process/service represented by an entity {% endhighlight %} {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q effector Name Description Parameters deploy Deploys an archive ... getCurrentConfiguration Gets the current ... populateServiceNotUpDiagnostics Populates the attribute ... reload Forces reload of ... restart Restart the process/service ... restartChildren,restartMachine start Start the process/service ... locations stop Stop the process/service ... stopProcessMode,stopMachineMode update Updates the entities ... {% endhighlight %} Details of an individual effector can be viewed by using the name as a parameter for the `effector` command: {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q effector update Name: update Description: Updates the entities configuration, and then forces reload of that configuration Parameters: {% endhighlight %} An effector can be invoked by using the `invoke` command with an effector-scope: {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q effector update invoke {% endhighlight %} Parameters can also be passed to the effector: {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q effector restart invoke -P restartChildren=true {% endhighlight %} If a parameter value is complex or spans multiple lines, it may be provided in a file and used like this: {% highlight text %} $ br app WebCluster effector start invoke -P locations=@data.txt {% endhighlight %} Shortcut commands are available for the 3 standard effectors of `start`, `restart` and `stop`. These commands can be used directly with an app-scope or entity-scope: {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q restart $ br app WebCluster stop {% endhighlight %} ## Policies The policies associated with an application or entity can be listed with the `policy` command: {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q policy Id Name State VcZ0cfeO Controller targets tracker RUNNING {% endhighlight %} Details of an individual policy may be viewed by using the PolicyID as a parameter to the `policy` command: {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q policy VcZ0cfeO Name Value Description group DynamicWebAppClusterImpl{id=TpbkaK4D} group notifyOnDuplicates false Whether to notify listeners when a sensor is published with the same value as last time sensorsToTrack [Sensor: host.subnet.hostname Sensors of members to be monitored (java.lang.String), Sensor: http.port (implicitly adds service-up (java.lang.Integer)] to this list, but that behaviour may be deleted in a subsequent release!) {% endhighlight %} ## Activities The activities for an application or entity may be listed with the `activity` command: {% highlight text %} $ br app WebCluster activity Id Task Submitted Status Streams Wb6GV5rt start Sat Dec 19 11:08:01 GMT 2015 Completed q2MbyyTo invoking start[locations] on 2 nodes Sat Dec 19 11:08:01 GMT 2015 Completed {% endhighlight %} {% highlight text %} $ br app WebCluster entity NginxController:CZ8Q activity Id Task Submitted Status Streams GVh0pyKG start Sun Dec 20 19:18:06 GMT 2015 Completed WJm908rA provisioning (FixedListMachineProvisi... Sun Dec 20 19:18:06 GMT 2015 Completed L0cKFBrW pre-start Sun Dec 20 19:18:06 GMT 2015 Completed D0Ab2esP ssh: initializing on-box base dir ./b... Sun Dec 20 19:18:06 GMT 2015 Completed env,stderr,stdin,stdout tumLAdo4 start (processes) Sun Dec 20 19:18:06 GMT 2015 Completed YbF2czKM copy-pre-install-resources Sun Dec 20 19:18:06 GMT 2015 Completed o3YdqxsQ pre-install Sun Dec 20 19:18:06 GMT 2015 Completed TtGw4qMZ pre-install-command Sun Dec 20 19:18:06 GMT 2015 Completed duPvOSDB setup Sun Dec 20 19:18:06 GMT 2015 Completed WLtkbhgW copy-install-resources Sun Dec 20 19:18:06 GMT 2015 Completed ZQtrImnl install Sun Dec 20 19:18:06 GMT 2015 Completed hzi49YD6 ssh: setting up sudo Sun Dec 20 19:18:06 GMT 2015 Completed env,stderr,stdin,stdout eEUHcpfi ssh: Getting machine details for: Ssh... Sun Dec 20 19:18:07 GMT 2015 Completed env,stderr,stdin,stdout juTe2qLG ssh: installing NginxControllerImpl{i... Sun Dec 20 19:18:08 GMT 2015 Completed env,stderr,stdin,stdout hXqwEZJl post-install-command Sun Dec 20 19:18:08 GMT 2015 Completed vZliYwBI customize Sun Dec 20 19:18:08 GMT 2015 Completed O4Wwb0bP ssh: customizing NginxControllerImpl{... Sun Dec 20 19:18:08 GMT 2015 Completed env,stderr,stdin,stdout sDwMSkE2 copy-runtime-resources Sun Dec 20 19:18:08 GMT 2015 Completed yDYkdkS8 ssh: create run directory Sun Dec 20 19:18:08 GMT 2015 Completed env,stderr,stdin,stdout W7dI8r1c pre-launch-command Sun Dec 20 19:18:08 GMT 2015 Completed OeZKwM5z launch Sun Dec 20 19:18:08 GMT 2015 Completed y50Gne5E scheduled:nginx.url.answers.nicely @ ... Sun Dec 20 19:18:08 GMT 2015 Scheduler, ARTninGE scheduled:service.process.isRunning @... Sun Dec 20 19:18:08 GMT 2015 Scheduler, tvZoNUTN ssh: launching NginxControllerImpl{id... Sun Dec 20 19:18:08 GMT 2015 Completed env,stderr,stdin,stdout YASrjA4w post-launch-command Sun Dec 20 19:18:09 GMT 2015 Completed jgLYv8pE post-launch Sun Dec 20 19:18:09 GMT 2015 Completed UN9OcWLS post-start Sun Dec 20 19:18:09 GMT 2015 Completed nmiv97He reload Sun Dec 20 19:18:09 GMT 2015 Completed FJfPbNtp ssh: restarting NginxControllerImpl{i... Sun Dec 20 19:18:10 GMT 2015 Completed env,stderr,stdin,stdout Xm1tjvKf update Sun Dec 20 19:18:40 GMT 2015 Completed Row67vfa reload Sun Dec 20 19:18:40 GMT 2015 Completed r8QZXlxJ ssh: restarting NginxControllerImpl{i... Sun Dec 20 19:18:40 GMT 2015 Completed env,stderr,stdin,stdout {% endhighlight %} The detail for an individual activity can be viewed by providing the ActivityID as a parameter to the `activity` command (an app-scope or entity-scope is not not needed for viewing the details of an activity): {% highlight text %} $ br activity tvZoNUTN Id: tvZoNUTN DisplayName: ssh: launching NginxControllerImpl{id=OxPUBk1p} Description: EntityId: OxPUBk1p EntityDisplayName: NginxController:OxPU Submitted: Sun Dec 20 19:18:08 GMT 2015 Started: Sun Dec 20 19:18:08 GMT 2015 Ended: Sun Dec 20 19:18:09 GMT 2015 CurrentStatus: Completed IsError: false IsCancelled: false SubmittedByTask: OeZKwM5z Streams: stdin: 1133, stdout: 162, stderr: 0, env 0 DetailedStatus: "Completed after 1.05s Result: 0" {% endhighlight %} The activity command output shows whether any streams were associated with it. The streams and environment for an activity can be viewed with the commands `stdin`, `stdout`, `stderr` and `env`: {% highlight text %} $ br activity tvZoNUTN stdin export RUN_DIR="/home/vagrant/brooklyn-managed-processes/apps/V5GQCpIT/entities/NginxController_OxPUBk1p" mkdir -p $RUN_DIR cd $RUN_DIR cd /home/vagrant/brooklyn-managed-processes/apps/V5GQCpIT/entities/NginxController_OxPUBk1p { which "./sbin/nginx" || { EXIT_CODE=$? && ( echo "The required executable \"./sbin/nginx\" does not exist" | tee /dev/stderr ) && exit $EXIT_CODE ; } ; } nohup ./sbin/nginx -p /home/vagrant/brooklyn-managed-processes/apps/V5GQCpIT/entities/NginxController_OxPUBk1p/ -c conf/server.conf > /home/vagrant/brooklyn-managed-processes/apps/V5GQCpIT/entities/NginxController_OxPUBk1p/console 2>&1 & for i in {1..10} do test -f /home/vagrant/brooklyn-managed-processes/apps/V5GQCpIT/entities/NginxController_OxPUBk1p/logs/nginx.pid && ps -p `cat /home/vagrant/brooklyn-managed-processes/apps/V5GQCpIT/entities/NginxController_OxPUBk1p/logs/nginx.pid` && exit sleep 1 done echo "No explicit error launching nginx but couldn't find process by pid; continuing but may subsequently fail" cat /home/vagrant/brooklyn-managed-processes/apps/V5GQCpIT/entities/NginxController_OxPUBk1p/console | tee /dev/stderr {% endhighlight %} {% highlight text %} $ br activity tvZoNUTN stdout ./sbin/nginx PID TTY TIME CMD 6178 ? 00:00:00 nginx Executed /tmp/brooklyn-20151220-191808796-CaiI-launching_NginxControllerImpl_.sh, result 0 {% endhighlight %} The child activities of an activity may be listed by providing an activity-scope for the `activity` command: {% highlight text %} $ br activity OeZKwM5z Id: OeZKwM5z DisplayName: launch Description: EntityId: OxPUBk1p EntityDisplayName: NginxController:OxPU Submitted: Sun Dec 20 19:18:08 GMT 2015 Started: Sun Dec 20 19:18:08 GMT 2015 Ended: Sun Dec 20 19:18:09 GMT 2015 CurrentStatus: Completed IsError: false IsCancelled: false SubmittedByTask: tumLAdo4 Streams: DetailedStatus: "Completed after 1.06s No return value (null)" $ br activity OeZKwM5z activity Id Task Submitted Status Streams tvZoNUTN ssh: launching NginxControllerImpl{id... Sun Dec 20 19:18:08 GMT 2015 Completed env,stderr,stdin,stdout {% endhighlight %} or by using the `-c` (or `--children`) flag with the `activity` command: {% highlight text %} $ br activity -c OeZKwM5z Id Task Submitted Status Streams tvZoNUTN ssh: launching NginxControllerImpl{id... Sun Dec 20 19:18:08 GMT 2015 Completed env,stderr,stdin,stdout {% endhighlight %} ## YAML Blueprint This is the YAML blueprint used for this document, based on the [web cluster](../../blueprints/clusters-and-policies.html) examples. {% highlight text %} name: WebCluster location: byon: user: vagrant password: vagrant hosts: - 192.168.52.101 - 192.168.52.102 - 192.168.52.103 - 192.168.52.104 - 192.168.52.105 services: - type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster name: WebApp brooklyn.config: wars.root: https://search.maven.org/remotecontent?filepath=org/apache/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.12.0/brooklyn-example-hello-world-sql-webapp-0.12.0.war # BROOKLYN_VERSION java.sysprops: brooklyn.example.db.url: > $brooklyn:formatString("jdbc:%s%s?user=%s&password=%s", component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", $brooklyn:external("brooklyn-demo-sample", "hidden-brooklyn-password")) brooklyn.policies: - type: org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy brooklyn.config: metric: webapp.reqs.perSec.windowed.perNode metricLowerBound: 2 metricUpperBound: 10 minPoolSize: 1 maxPoolSize: 2 resizeUpStabilizationDelay: 1m resizeDownStabilizationDelay: 5m - type: org.apache.brooklyn.entity.database.mysql.MySqlNode id: db name: WebDB brooklyn.config: creation.script.password: $brooklyn:external("brooklyn-demo-sample", "hidden-brooklyn-password") creationScriptUrl: https://bit.ly/brooklyn-visitors-creation-script {% endhighlight %}GUI Guide
^ {% include list-children.html %}Launching
^ This guide will walk you through connecting to the Brooklyn Server Graphical User Interface and performing various tasks. For an explanation of common Brooklyn Concepts see the [Brooklyn Concepts Quickstart](/guide/start/concept-quickstart.md) or see the full guide in the [Brooklyn Concepts](/guide/concepts) chapter of the [User Guide](/guide). This guide assumes that you are using Linux or Mac OS X and that Brooklyn Server will be running on your local system. ## Launch Apache Brooklyn If you haven't already done so, you will need to start Brooklyn Server using the commands shown below. It is not necessary at this time, but depending on what you are going to do, you may wish to set up some other configuration options first, * [Security](/guide/ops/configuration/brooklyn_cfg.md) * [Persistence](/guide/ops/persistence) Now start Brooklyn with the following command: {% highlight bash %} $ cd apache-brooklyn-{{ site.brooklyn.version }} $ bin/brooklyn launch {% endhighlight %} Please refer to the [Server CLI Reference](/guide/ops/server-cli-reference.md) for details of other possible command line options. Brooklyn will output the address of the management interface: ``` INFO Starting Brooklyn web-console with no security options (defaulting to no authentication), on bind addressDeploying Blueprints
^ {% include fields.md %} ## Launching from a Blueprint When you first access the web console on [http://127.0.0.1:8081](http://127.0.0.1:8081) you will be greeted by the GUI homepage comprised of: - the quick launch panel - links to UI modules We'll start by deploying an application via a YAML blueprint consisting of the following layers. - MySQL DB - Dynamic web application cluster - Nginx load balancer - Tomcat app server cluster [](images/add-blueprint-large.png) Go the "Blueprint Composer" and switch to the YAML editor view (by clicking on the double horizontal arrows icon on the top right corner) and copy the blueprint below into the editor. But *before* you submit it, modify the YAML to specify the location where the application will be deployed. {% highlight yaml %} {% read _my-web-cluster.yaml %} {% endhighlight %} Replace the `location:` element with values for your chosen target environment, for example to use SoftLayer rather than AWS (updating with your own credentials): {% highlight yaml %} location: jclouds:softlayer: identity: ABCDEFGHIJKLMNOPQRST credential: s3cr3tsq1rr3ls3cr3tsq1rr3ls3cr3tsq1rr3l {% endhighlight %} With the modified YAML, click on the "Deploy" button. Brooklyn will begin deploying your application and redirect you to the "Inspector". In this screen, you will see your application as "Starting". [](images/app-deploying-large.png) Depending on your choice of location it may take some time for the application nodes to start, the next page describes how you can monitor the progress of the application deployment and verify its successful deployment. ## Launching from the Catalog Instead of pasting the YAML blueprint each time, it can be added to the Brooklyn Catalog where it will be accessible from the "Quick Launch" panel or "Catalog" and "Blueprint Composer" UI modules. [](images/app-quicklaunch-large.png) ## Next So far we have touched on Brooklyn's ability to *deploy* an application blueprint to a cloud provider. The next section will show how to **[Monitor and Manage Applications](managing.md)**.Monitoring and Managing Applications
^ From the Home page, go the "Inspector" module. We can explore the management hierarchy of the application, which will show us the entities it is composed of. Starting from the application use the arrows to expand out the list of entities. * My Web Cluster (A `BasicApplication`) * My DB (A `MySqlNode`) * My Web (A `ControlledDynamicWebAppCluster`) * Cluster of TomcatServer (A `DynamicWebAppCluster`) * quarantine (A `QuarantineGroup`) * TomcatServer (A `TomcatServer`) * NginxController (An `NginxController`) Clicking on the "My Web Cluster" entity will show the "Summary" tab, giving a very high level of what that component is doing. Click on each of the child components in turn for more detail on that component. Note that the cluster of web servers includes a "quarantine group", to which members of the cluster that fail will be added. These are excluded from the load-balancer's targets. [](images/my-web-large.png) ## Activities The "Activities" tab allows us to drill down into the tasks each entity is currently executing or has recently completed. It is possible to drill down through all child tasks, and view the commands issued, along with any errors or warnings that occurred. For example clicking on the NginxController in the left hand tree and opening its Activity tab you can observe the 'start' task is 'Finished' and 'Successful', symbolised by a green circle. **Note**: You may observe different tasks depending on how far your deployment has progressed). [](images/my-db-activities-step1-large.png) Clicking on the 'start' task you can discover more details on the actions being carried out by that task (a task may consist of additional subtasks). [](images/my-db-activities-step2-large.png) Continuing to drill down into the tasks, you will eventually reach the some where you can investigate the ssh command executed on the target node including the current stdin, stdout and stderr output. [](images/my-db-activities-step3-large.png) ## Sensors Now click on the "Sensors" tab: these data feeds drive the real-time picture of the application. As you navigate in the tree at the left, you can see more targeted statistics coming in in real-time. Explore the sensors and the tree to find the URL where the _NginxController_ for the webapp we just deployed is running. This can be found in '**My Web Cluster** -> **My Web** -> **NginxController** -> **_main.uri_**'. Quickly return to the **‘Brooklyn JS REST client’** web browser tab showing the "Sensors" and observe the '**My Web Cluster** -> **My Web** -> **Cluster of TomcatServer** -> **_webapp.reqs.perSec.last_**' sensor value increase. ## Stopping the Application To stop an application, select the application in the tree view (the top/root entity), click on the Effectors tab, and invoke the "Stop" effector. This will cleanly shutdown all components in the application and return any cloud machines that were being used. [](images/my-web-cluster-stop-confirm-large.png) ## Next Brooklyn's real power is in using **[Policies](policies.md)** to automatically *manage* applications.Using Policies
^ ## Exploring and Testing Policies To see an example of policy based management, please deploy the following blueprint (changing the location details as for the example shown earlier): {% highlight yaml %} {% read _my-web-cluster2.yaml %} {% endhighlight %} The app server cluster has an `AutoScalerPolicy`, and the loadbalancer has a `targets` policy. Use the Applications tab in the web console to drill down into the Policies section of the ControlledDynamicWebAppCluster. You will see that the `AutoScalerPolicy` is running. This policy automatically scales the cluster up or down to be the right size for the cluster's current load. One server is the minimum size allowed by the policy. The loadbalancer's `targets` policy ensures that the loadbalancer is updated as the cluster size changes. Sitting idle, this cluster will only contain one server, but you can use a tool like [jmeter](http://jmeter.apache.org/) pointed at the nginx endpoint to create load on the cluster. Download a jmeter test plan [here](https://github.com/apache/brooklyn-library/blob/master/examples/simple-web-cluster/resources/jmeter-test-plan.jmx). As load is added, Apache Brooklyn requests a new cloud machine, creates a new app server, and adds it to the cluster. As load is removed, servers are removed from the cluster, and the infrastructure is handed back to the cloud. ## Under the Covers The `AutoScalerPolicy` here is configured to respond to the sensor reporting requests per second per node, invoking the default `resize` effector. By clicking on the policy, you can configure it to respond to a much lower threshhold or set long stabilization delays (the period before it scales out or back). An even simpler test is to manually suspend the policy, by clicking "Suspend" in the policies list. You can then switch to the "Effectors" tab and manually trigger a `resize`. On resize, new nodes are created and configured, and in this case a policy on the nginx node reconfigures nginx whenever the set of active targets changes. ## Next This guide has given a quick overview of using the Apache Brooklyn GUI to deploy, monitor and manage applications. The GUI also allows you to perform various Advanced management tasks and to explore and use the REST API (from the Script tab). Please take some time now to become more familiar with the GUI. Then continue to read through the [Operations Guide](/guide/ops).REST API
^ Apache Brooklyn exposes a powerful REST API, allowing it to be scripted from bash or integrated with other systems. For many commands, the REST call follows the same structure as the web console URL scheme, but with the `#` at the start of the path removed; for instance the catalog item `cluster` in the web console is displayed at: http://localhost:8081/#v1/catalog/entities/cluster:1.2.0-SNAPSHOT And in the REST API it is accessed at: http://localhost:8081/v1/catalog/entities/cluster:1.2.0-SNAPSHOT A full reference for the REST API is automatically generated by the server at runtime. It can be found in the Brooklyn web console by clicking `REST API` under the menu icon in the top right corner. This opens a Swagger UI that allows users to see the full reference for the API capabilities, as well as execute the API commands with a user-friendly interface. Here we include some of the most common REST examples and other advice for working with the REST API. ### Tooling Tips For command-line access, we recommend `curl`, with tips below. For navigating in a browser we recommend getting a plugin for working with REST; these are available for most browsers and make it easier to authenticate, set headers, and see JSON responses. For manipulating JSON responses on the command-line, the library `jq` from [stedolan's github](https://stedolan.github.io/jq/) is very useful, and available in most package repositories, including `port` and `brew` on Mac. ### Common REST Examples Here are some useful snippets: * List applications curl http://localhost:8081/v1/applications * Deploy an application from `__FILE__` curl http://localhost:8081/v1/applications --data-binary @__FILE__ * Get details of a task with ID `__ID__` (where the `id` is returned by the above, optionally piped to `jq .id`) curl http://localhost:8081/v1/activities/__ID__ * Get the value of sensor `service.state` on entity `e1` in application `app1` (note you can use either the entity's ID or its name) curl http://localhost:8081/v1/applications/app1/entities/e1/sensors/service.state * Get all sensor values (using the pseudo-sensor `current-state`) curl http://localhost:8081/v1/applications/app1/entities/e1/sensors/service.state * Invoke an effector `eff` on `e1`, with argument `arg1` equal to `hi` (note if no arguments, you must specify `-d ""`; for multiple args, just use multiple `-d` entries, or a JSON file with `--data-binary @...`) curl http://localhost:8081/v1/applications/app1/entities/e1/effectors/eff -d arg1=hi * Add an item to the catalog from `__FILE__` curl http://localhost:8081/v1/catalog --data-binary @__FILE__ ### Curl Cheat Sheet * if authentication is required, use `--user username:password` * to see detailed output, including headers, code, and errors, use `-v` * where the request is not a simple HTTP GET, use `-X POST` or `-X DELETE` * to pass key-value data to a post, use `-d key=value` * where you are posting from a file `__FILE__`, use `--data-binary @__FILE__` (implies a POST) or `-T __FILE__ -X POST` * to add a header, use `-H "key: value"`, for example `-H "Brooklyn-Allow-Non-Master-Access: true"` * to specify that a specific content-type is being uploaded, use `-H "Content-Type: application/json"` (or `application/yaml`) * to specify the content-type required for the result, use `-H "Accept: application/json"` (or `application/yaml`, or for sensor values, `text/plain`)Brooklyn Configuration and Options
^ Apache Brooklyn contains a number of configuration options managed across several files. Historically Brooklyn has been configured through a brooklyn.properties file, this changed to a [brooklyn.cfg](brooklyn_cfg.md) file when the Karaf release became the default in Brooklyn 0.12.0. The configurations for [persistence](/guide/ops/persistence) and [high availability](/guide/ops/high-availability) are described elsewhere in this manual. Configuration of Apache Brooklyn when running under Karaf is largely done through standard Karaf mechanisms. The Karaf "Configuration Admin" subsystem is used to manage configuration values loaded at first boot from the `.cfg` files in the `etc` directory of the distribution. In the Karaf command line these can then be viewed and manipulated by the `config:` commands, see the [Karaf documentation](https://karaf.apache.org/manual/latest/) for full details. ## Configuring Brooklyn Properties To configure the Brooklyn runtime create an `etc/brooklyn.cfg` file. If you have previously used `brooklyn.properties` it follows the same file format. Values can be viewed and managed dynamically via the OSGI configuration admin commands in Karaf, e.g. `config:property-set`. The global `~/.brooklyn/brooklyn.properties` is still supported and has higher priority for duplicate keys, but its values can't be manipulated with the Karaf commands, so its use is discouraged. You can use the standard `~/.brooklyn/brooklyn.properties` file to configure Brooklyn. Alternatively create `etc/brooklyn.cfg` inside the distribution folder (same file format). The keys in the former override those in the latter. Web console related configuration is done through the corresponding Karaf mechanisms: * The port is set in `etc/org.ops4j.pax.web.cfg`, key `org.osgi.service.http.port`. * For authentication the JAAS realm "webconsole" is used; by default it will use any SecurityProvider implementations configured in Brooklyn falling back to auto generating the password. To configure a custom JAAS realm see the `jetty.xml` file in `brooklyn-server/karaf/jetty-config/src/main/resources` and override it by creating a custom one in `etc` folder. Point the "webconsole" login service to the JAAS realm you would like to use. * For other Jetty related configuration consult the Karaf and pax-web docs. ### Memory Usage The amount of memory required by Apache Brooklyn process depends on the usage - for example the number of entities/VMs under management. For a standard Brooklyn deployment, the defaults are to start with 256m, and to grow to 2g of memory. These numbers can be overridden by setting the `JAVA_MAX_MEM` and `JAVA_MAX_PERM_MEM` in the `bin/setenv` script: export JAVA_MAX_MEM="2G" Apache Brooklyn stores a task history in-memory using [soft references](http://docs.oracle.com/javase/7/docs/api/java/lang/ref/SoftReference.html). This means that, once the task history is large, Brooklyn will continually use the maximum allocated memory. It will only expunge tasks from memory when this space is required for other objects within the Brooklyn process. ### Authentication and Security There are two areas of authentication used in Apache Brooklyn, these are as follows: * Karaf authentication Apache Brooklyn uses [Apache Karaf](https://karaf.apache.org) as a core platform, this has user level security and groups which can be configured as detailed [here](https://karaf.apache.org/manual/latest/security#_users_groups_roles_and_passwords). * Apache Brooklyn authentication Users and passwords for Brooklyn can be configured in the brooklyn.cfg as detailed [here](brooklyn_cfg#authentication). ### HTTPS Configuration See [HTTPS Configuration](https.md) for general information on configuring HTTPS. ## Catalog in OSGi OSGi bundles may include a `catalog.bom` file at root, defining the elements to be added to the Apache Brooklyn catalog when the bundle is installed. These bundles may be installed using the API or as part of the initial catalog. The initial catalog is loaded from `etc/default.catalog.bom` which typically references files in `catalog/*`, as described in the section on [installation](/guide/ops/production-installation.md). For example, the `catalog.bom` file for Brooklyn's Webapp bundle looks like (abbreviated): brooklyn.catalog: version: ... items: - id: org.apache.brooklyn.entity.webapp.nodejs.NodeJsWebAppService itemType: entity item: type: org.apache.brooklyn.entity.webapp.nodejs.NodeJsWebAppService name: Node.JS Application ... - id: resilient-bash-web-cluster-template itemType: template name: "Template: Resilient Load-Balanced Bash Web Cluster with Sensors" description: | Sample YAML to provision a cluster of the bash/python web server nodes, with sensors configured, and a load balancer pointing at them, and resilience policies for node replacement and scaling item: name: Resilient Load-Balanced Bash Web Cluster (Brooklyn Example) In the above YAML the first item declares that the bundle provides an entity whose type is `org.apache.brooklyn.entity.webapp.nodejs.NodeJsWebAppService`, and whose name is 'Node.JS Application'. The second item declares that the bundle provides a template application, with id `resilient-bash-web-cluster-template`, and includes a description for what this is. Note: the `type` argument can only refer to unqualified class names where the bundle is in scope, such as contained in the same bundle as one containing the `catalog.bom` (in some cases), or a bundle specified in the `org.apache.brooklyn.classloader.fallback.bundles` list supplied at startup; to reference a type in another bundle, the syntax `osgi-bundle:org.package.Class` is supported in most places. ### Configuring applications in the Catalog When running some particular deployment of Brooklyn it may not be desirable for the sample applications to appear in the catalog (for clarity, "application" here in the sense of an item with `itemType: template`). For example, if you have developed some bundle with your own application and added it to Karaf then you might want only your own application to appear in the catalog. Brooklyn contains a mechanism to allow you to configure what bundles will add their applications to the catalog. The Karaf configuration file `/etc/org.apache.brooklyn.core.catalog.bomscanner.cfg` contains two properties, one `whitelist` and the other `blacklist`, that bundles must satisfy for their applications to be added to the catalog. Each property value is a comma-separated list of regular expressions. The symbolic id of the bundle must match one of the regular expressions on the whitelist, and not match any expression on the blacklist, if its applications are to be added to the bundle. The default values of these properties are to admit all bundles, and forbid none. ### Configuring custom bundle resolvers, type-plan transformers, and other bundles As described throughout this user guide, Apache Brooklyn by default uses the CAMP YAML format to define types, including entities, and supports the `catalog.bom` format defined elsewhere and ZIP bundles containing `catalog.bom` or OSGi metadata information. It is possible to extend this, and supply additional item type definition formats and bundle resolution strategies. This is done by defining OSGi services in an OSGi bundle blueprint, implementing `BrooklynTypePlanTransformer` and/or `BrooklynCatalogBundleResolver`. This can be used to add support for any type of plan or bundle format, such as Kubernetes Helm charts, TOSCA YAML topology definitions, or TOSCA CSAR bundles. These services, or any additional bundles to install, can be specified in any of several ways: * As part of Karaf startup, by specifying it in `etc/startup.properties` or as a boot feature/bundle * Adding it to the `/etc/default.catalog.bom` * Putting it in the OSGi `/deploy` folder (before or after startup) * Manually after startup through the API or CLI (e.g. via `br catalog add`) and subsequently restored through rebind *Note*: If the initial catalog `/etc/default.catalog.bom` requires those bundles to be installed, you must use the first option, otherwise, because OSGi startup can be non-deterministic, the bundles might not be installed when the initial catalog is loaded. In addition, you must specify that the services from those bundles are required prior to starting the initial catalog (and before rebind). This can be done with the following setting in `brooklyn.cfg`: brooklyn.osgi.dependencies.services.filters=brooklyn.cfg
^ {% include fields.md %} The file `brooklyn.cfg` is read when Apache Brooklyn starts in order to load any server configuration values. It can be found in the Brooklyn configuration folder. You can check [here](/guide/ops/paths.md) for the location of your Brooklyn configuration folder ## Quick Setup The most common properties set in this file are for access control. Without this, Brooklyn's web-console and REST API will require no authentication. The simplest way to specify users and passwords is shown below (but see the [Authentication](#authentication) section for how to avoid storing passwords in plain text): {% highlight properties %} brooklyn.webconsole.security.users=admin,bob brooklyn.webconsole.security.user.admin.password=AdminPassw0rd brooklyn.webconsole.security.user.bob.password=BobPassw0rd {% endhighlight %} In many cases, it is preferable instead to use an external credentials store such as LDAP. Information on configuring these is [below](#authentication). If coming over a network it is highly recommended additionally to use `https`. This can be configured with: {% highlight properties %} brooklyn.webconsole.security.https.required=true {% endhighlight %} More information, including setting up a certificate, is described [further below](#https-configuration). ## Camp YAML Expressions Values in `brooklyn.cfg` can use the Camp YAML syntax. Any value starting `$brooklyn:` is parsed as a Camp YAML expression. This allows [externalized configuration](/guide/ops/externalized-configuration.md) to be used from `brooklyn.cfg`. For example: {% highlight properties %} brooklyn.location.jclouds.aws-ec2.identity=$brooklyn:external("vault", "aws-identity") brooklyn.location.jclouds.aws-ec2.credential=$brooklyn:external("vault", "aws-credential") {% endhighlight %} If for some reason one requires a literal value that really does start with `$brooklyn:` (i.e. for the value to not be parsed), then this can be achieved by using the syntax below. This example returns the property value `$brooklyn:myexample`: {% highlight properties %} example.property=$brooklyn:literal("$brooklyn:myexample") {% endhighlight %} ## Java Arbitrary data can be set in the `brooklyn.cfg`. This can be accessed in java using `ManagementContext.getConfig(KEY)`. ## Authentication **Security Providers** are the mechanism by which different authentication authorities are plugged in to Brooklyn. These can be configured by specifying `brooklyn.webconsole.security.provider` equal to the name of a class implementing `SecurityProvider`. An implementation of this could point to Spring, LDAP, OpenID or another identity management system. The default implementation, `ExplicitUsersSecurityProvider`, reads from a list of users and passwords which should be specified as configuration parameters e.g. in `brooklyn.cfg`. This configuration could look like: {% highlight properties %} brooklyn.webconsole.security.users=admin brooklyn.webconsole.security.user.admin.salt=OHDf brooklyn.webconsole.security.user.admin.sha256=91e16f94509fa8e3dd21c43d69cadfd7da6e7384051b18f168390fe378bb36f9 {% endhighlight %} The `users` line should contain a comma-separated list. The special value `*` is accepted to permit all users. To generate this, there is a script shipped with Brooklyn: {% highlight bash %} ./bin/generate-password.sh --user admin Enter password: Re-enter password: Please add the following to your etc/brooklyn.cfg: brooklyn.webconsole.security.users=admin brooklyn.webconsole.security.user.admin.salt=OHDf brooklyn.webconsole.security.user.admin.sha256=91e16f94509fa8e3dd21c43d69cadfd7da6e7384051b18f168390fe378bb36f9 {% endhighlight %} Alternatively, in dev/test environments where a lower level of security is required, the syntax `brooklyn.webconsole.security.user.HTTPS Configuration
^ ## Getting the Certificate To enable HTTPS web access, you will need a server certificate in a java keystore. To create a self-signed certificate, for testing and non-production use, you can use the tool `keytool` from your Java distribution. (A self-signed certificate will cause a warning to be displayed by a browser when viewing the page. The various browsers each have ways to import the certificate as a trusted one, for test purposes.) The following command creates a self-signed certificate and adds it to a keystore, `keystore.jks`: {% highlight bash %} % keytool -genkey -keyalg RSA -alias brooklyn \ -keystoreCORS Configuration
^ To enable / configure [cross-origin resource sharing (CORS)](https://en.wikipedia.org/wiki/Cross-origin_resource_sharing). The following file must be added to [`org.apache.brooklyn.rest.filter.cors.cfg`](/guide/ops/paths.md) {% highlight properties %} # Enables experimental support for Cross Origin Resource Sharing (CORS) filtering in Apache Brooklyn REST API. cors.enabled=true # @see CrossOriginResourceSharingFilter#setAllowOrigins(ListPersistence
^ By default Brooklyn persists its state to storage so that a server can be restarted without loss or so a high availability standby server can take over. Brooklyn can persist its state to one of two places: the file system, or to an [object store](https://en.wikipedia.org/wiki/Object_storage) of your choice. # Configuration To configure persistence, edit the file `org.apache.brooklyn.osgilauncher.cfg` in the `etc` directory of your Brooklyn instance. The following options are available: `persistMode` - This is the mode in which persistence is running, in and is set to `AUTO` by default. The possible values are: * `AUTO` - will rebind to any existing state, or start up fresh if no state; * `DISABLED` - will not read or persist any state; * `REBIND` - will rebind to the existing state, or fail if no state available; * `CLEAN` - will start up fresh (removing any existing state) `persistenceDir` - This is the directory or object store container name to which Apache Brooklyn reads and writes its persistence data. The default location depends on your installation method. Checkout [this page](/guide/ops/paths.md) for more information. `persistenceLocation` - This is the location for an object store to read and write persisted state. `persistPeriod` - This is an interval period which can be set to reduce the frequency with which persistence is carried out, for example `1s`. # File-based Persistence Apache Brooklyn starts with file-based persistence by default, saving data in the [persisted state folder](/guide/ops/paths.md). For the rest of this document we will refer to this location as `%persistence-home%`. If there is already data at `%persistence-home%/data`, then a backup of the directory will be made. This will have a name like `%persistence-home%/backups/%date%-%time%-jvyX7Wis-promotion-igFH`. This means backups of the data directory will be automatically created each time Brooklyn is restarted (or if a standby Brooklyn instances takes over as master). The state is written to the given path. The file structure under that path is: * `./catalog/` * `./enrichers/` * `./entities/` * `./feeds/` * `./locations/` * `./nodes/` * `./plane/` * `./policies/` In each of those directories, an XML file will be created per item - for example a file per entity in `./entities/`. This file will capture all of the state - for example, an entity's: id; display name; type; config; attributes; tags; relationships to locations, child entities, group membership, policies and enrichers; and dynamically added effectors and sensors. # Object Store Persistence Apache Brooklyn can persist its state to any Object Store API supported by [Apache jclouds](https://jclouds.apache.org/) including [S3](https://aws.amazon.com/s3), [Swift](http://docs.openstack.org/developer/swift) and [Azure](https://azure.microsoft.com/services/storage/). This gives access to any compatible Object Store product or cloud provider including AWS-S3, SoftLayer, Rackspace, HP and Microsoft Azure. For a complete list of supported providers, see [jclouds](http://jclouds.apache.org/reference/providers/#blobstore). To configure the Object Store, add the credentials to `brooklyn.cfg` by creating a new location such as: {% highlight properties %} brooklyn.location.named.aws-s3-eu-west-1=aws-s3:eu-west-1 brooklyn.location.named.aws-s3-eu-west-1.identity=ABCDEFGHIJKLMNOPQRSTU brooklyn.location.named.aws-s3-eu-west-1.credential=abcdefghijklmnopqrstuvwxyz1234567890ab/c {% endhighlight %} or: {% highlight properties %} brooklyn.location.named.softlayer-swift-ams01=jclouds:openstack-swift:https://ams01.objectstorage.softlayer.net/auth/v1.0 brooklyn.location.named.softlayer-swift-ams01.identity=ABCDEFGHIJKLM:myname brooklyn.location.named.softlayer-swift-ams01.credential=abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12 brooklyn.location.named.softlayer-swift-ams01.jclouds.keystone.credential-type=tempAuthCredentials {% endhighlight %} Then edit the `persistenceLocation` to point at this object store: `softlayer-swift-ams01`. # Rebinding to State When Brooklyn starts up pointing at existing state, it will recreate the entities, locations and policies based on that persisted state. Once all have been created, Brooklyn will "manage" the entities. This will bind to the underlying entities under management to update the each entity's sensors (e.g. to poll over HTTP or JMX). This new state will be reported in the web-console and can also trigger any registered policies. ## Handling Rebind Failures It is possible to confuse Apache Brooklyn such that it is unable to rebind to previously persisted state after a restart or when running from a different instance. Detailed steps to troubleshoot and correct these situations can be found [here](/guide/ops/troubleshooting/fails-to-start.md). # Writing Persistable Code The most common problem on rebind is that custom entity code has not been written in a way that can be persisted and/or rebound. The rule of thumb when implementing new entities, locations, policies and enrichers is that all state must be persistable. All state must be stored as config or as attributes, and must be serializable. For making backwards compatibility simpler, the persisted state should be clean. Below are tips and best practices for when implementing an entity in Java (or any other JVM language). How to store entity state: * Config keys and values are persisted. * Store an entity's runtime state as attributes. * Don't store state in arbitrary fields - the field will not be persisted (this is a design decision, because Brooklyn cannot intercept the field being written to, so cannot know when to persist). * Don't just modify the retrieved attribute value (e.g. `getAttribute(MY_LIST).add("a")` is bad). The value may not be persisted unless setAttribute() is called. * For special cases, it is possible to call `entity.requestPerist()` which will trigger asynchronous persistence of the entity. * Overriding (and customizing) of `getRebindSupport()` is discouraged - this will change in a future version. How to store policy/enricher/location state: * Store values as config keys where applicable. * Unfortunately these (currently) do not have attributes. Normally the state of a policy or enricher is transient - on rebind it starts afresh, for example with monitoring the performance or health metrics rather than relying on the persisted values. * For special cases, you can annotate a field with `@SetFromFlag` for it be persisted. When you call `requestPersist()` then values of these fields will be scheduled to be persisted. *Warning: the `@SetFromFlag` functionality may change in future versions.* Persistable state: * Ensure values can be serialized. This (currently) uses xstream, which means it does not need to implement `Serializable`. * Always use static (or top-level) classes. Otherwise it will try to also persist the outer instance! * Any reference to an entity or location will be automatically swapped out for marker, and re-injected with the new entity/location instance on rebind. The same applies for policies, enrichers, feeds, catalog items and `ManagementContext`. Behaviour on rebind: * By extending `SoftwareProcess`, entities get a lot of the rebind logic for free. For example, the default `rebind()` method will call `connectSensors()`. See [`SoftwareProcess` Lifecycle](/guide/blueprints/java/entities.md) for more details. * If necessary, implement rebind. The `entity.rebind()` is called automatically by the Brooklyn framework on rebind, after configuring the entity's config/attributes but before the entity is managed. Note that `init()` will not be called on rebind. * Feeds will be persisted if and only if `entity.addFeed(...)` was called. Otherwise the feed needs to be re-registered on rebind. *Warning: this behaviour may change in future version.* * All functions/predicates used with persisted feeds must themselves be persistable - use of anonymous inner classes is strongly discouraged. * Subscriptions (e.g. from calls to `subscribe(...)` for sensor events) are not persisted. They must be re-registered on rebind. *Warning: this behaviour may change in future version.* Below are tips to make backwards-compatibility easier for persisted state: * Never use anonymous inner classes - even in static contexts. The auto-generated class names are brittle, making backwards compatibility harder. * Always use sensible field names (and use `transient` whenever you don't want it persisted). The field names are part of the persisted state. * Consider using Value Objects for persisted values. This can give clearer separation of responsibilities in your code, and clearer control of what fields are being persisted. * Consider writing transformers to handle backwards-incompatible code changes. Brooklyn supports applying transformations to the persisted state, which can be done as part of an upgrade process. # Persisted State Backup ### File system backup When using the file system it is important to ensure it is backed up regularly. One could use `rsync` to regularly backup the contents to another server. It is also recommended to periodically create a complete archive of the state. A simple mechanism is to run a CRON job periodically (e.g. every 30 minutes) that creates an archive of the persistence directory, and uploads that to a backup facility (e.g. to S3). Optionally, to avoid excessive load on the Brooklyn server, the archive-generation could be done on another "data" server. This could get a copy of the data via an `rsync` job. An example script to be invoked by CRON is shown below: DATE=`date "+%Y%m%d.%H%M.%S"` BACKUP_FILENAME=/path/to/archives/back-${DATE}.tar.gz DATA_DIR=/path/to/base/dir/data tar --exclude '*/backups/*' -czvf $BACKUP_FILENAME $DATA_DIR # For AWS CLI installation see https://aws.amazon.com/cli aws s3 cp $BACKUP_FILENAME s3://mybackupbucket rm $BACKUP_FILENAME ### Object store backup Object Stores will normally handle replication. However, many such object stores do not handle versioning (i.e. to allow access to an old version, if an object has been incorrectly changed or deleted). The state can be downloaded periodically from the object store, archived and backed up. An example script to be invoked by CRON is shown below: DATE=`date "+%Y%m%d.%H%M.%S"` BACKUP_FILENAME=/path/to/archives/back-${DATE}.zip HOSTNAME=localhost USERNAME=admin PASSWORD=pa55wOrd curl -v -u "${USERNAME}:${PASSWORD}" \ https://${HOSTNAME}:8443/v1/server/ha/persist/export \ > ${BACKUP_FILENAME} # For AWS CLI installation see https://aws.amazon.com/cli aws s3 cp $BACKUP_FILENAME s3://mybackupbucket rm $BACKUP_FILENAME # PERSISTED STATE IMPORT/EXPORT Rest API offers capability to `export` the current persistence state to an archive, which then can be restored by `import`. * `/ha/persist/export`: (GET) Captures the current state of persistence and saves it as an archive * `/ha/persist/import`: (POST) Takes a previously exported persistence state archive and restores it into the current persistence, by merging the states, adding the catalog bundles, types, locations and deployed applications under the current management plane. `Export`/`Import` functionality is also available from the Brooklyn UI, using the respective options in the `About` page.High Availability
^ Brooklyn will automatically run in HA mode if multiple Brooklyn instances are started pointing at the same persistence store. One Brooklyn node (e.g. the first one started) is elected as HA master: all *write operations* against Brooklyn entities, such as creating an application or invoking an effector, should be directed to the master. Once one node is running as `MASTER`, other nodes start in either `STANDBY` or `HOT_STANDBY` mode: * In `STANDBY` mode, a Brooklyn instance will monitor the master and will be a candidate to become `MASTER` should the master fail. Standby nodes do *not* attempt to rebind until they are elected master, so the state of existing entities is not available at the standby node. However a standby server consumes very little resource until it is promoted. * In `HOT_STANDBY` mode, a Brooklyn instance will read and make available the live state of entities. Thus a hot-standby node is available as a read-only copy. As with the standby node, if a hot-standby node detects that the master fails, it will be a candidate for promotion to master. * In `HOT_BACKUP` mode, a Brooklyn instance will read and make available the live state of entities, as a read-only copy. However this node is not able to become master, so it can safely be used to test compatibility across different versions. To explicitly specify what HA mode a node should be in, the following options are available for the config option `highAvailabilityMode` in [`org.apache.brooklyn.osgilauncher.cfg`](/guide/ops/paths.md): * `DISABLED`: management node works in isolation; it will not cooperate with any other standby/master nodes in management plane * `AUTO`: will look for other management nodes, and will allocate itself as standby or master based on other nodes' states * `MASTER`: will startup as master; if there is already a master then fails immediately * `STANDBY`: will start up as lukewarm standby; if there is not already a master then fails immediately * `HOT_STANDBY`: will start up as hot standby; if there is not already a master then fails immediately * `HOT_BACKUP`: will start up as hot backup; this can be done even if there is not already a master; this node will not be a master The REST API offers live detection and control of the HA mode, including setting priority to control which nodes will be promoted on master failure: * `/server/ha/state`: Returns the HA state of a management node (GET), or changes the state (POST) * `/server/ha/states`: Returns the HA states and detail for all nodes in a management plane * `/server/ha/priority`: Returns the HA node priority for MASTER failover (GET), or sets that priority (POST) The High Availability management plane can also be controlled via the Brooklyn UI. More information is available in [`Configuraing HA - an example`](/guide/ops/high-availability/high-availability-supplemental.md) Note that when making a POST request to a non-master server it is necessary to pass a `Brooklyn-Allow-Non-Master-Access: true` header. For example, the following cURL command could be used to change the state of a `STANDBY` node on `localhost:8082` to `HOT_STANDBY`: curl -v -X POST -d mode=HOT_STANDBY -H "Brooklyn-Allow-Non-Master-Access: true" http://localhost:8082/v1/server/ha/state When running a single server, you can disable HA mode. You can recover from a failure by restarting the process or launching a replacement machine, pointing at the same persisted state. A single server running in HA mode will have the following differences in behaviour: * If you run Brooklyn and then kill it (e.g. `kill -9` or turn off the VM), when you start Brooklyn again it will wait to confirm the previous server is really dead. It waits for 30 seconds after the old server's last heartbeat, by default. * The HA status shows all previous runs of the Brooklyn server (it gets a new node-id each time it restarts). This list will get longer and longer if you keep restarting Brooklyn, while pointing at the same persisted state, until you clear out terminated instances from the list (via the UI or the REST API). * The logging at startup can be quite different (e.g. in HA mode, "Brooklyn initialisation (part two) complete" can mean that the server has finished becoming the 'standby'. Care should be taken if searching or parsing the logs.Configuring HA - an example
^ This supplements the [High Availability](/guide/ops/high-availability) documentation and provides an example of how to configure a pair of Apache Brooklyn servers to run in master-standby mode with a shared NFS datastore ### Prerequisites - Two VMs (or physical machines) have been provisioned - NFS or another suitable file system has been configured and is available to both VMs* - An NFS folder has been mounted on both VMs at `/mnt/brooklyn-persistence` and both machines can write to the folder \* Brooklyn can be configured to use either an object store such as S3, or a shared NFS mount. The recommended option is to use an object store as described in the [Object Store Persistence](/guide/ops/persistence#object-store-persistence) documentation. For simplicity, a shared NFS folder is assumed in this example ### Launching To start, download and install the latest Apache Brooklyn release on both VMs following the instructions in [Running Apache Brooklyn](/guide/start/running.md) On the first VM, which will be the master node, set the following configuration options in [`org.apache.brooklyn.osgilauncher.cfg`](/guide/ops/paths.md): - highAvailabilityMode: MASTER - persistMode: AUTO - persistenceDir: /mnt/brooklyn-persistence Then launch Brooklyn with: {% highlight bash %} $ bin/start {% endhighlight %} If you are using RPMs/deb to install, please see the [Running Apache Brooklyn](/guide/start/running.md) documentation for the appropriate launch commands Once Brooklyn has launched, on the second VM, set the following configuration options in [`org.apache.brooklyn.osgilauncher.cfg`](../paths.html): - highAvailabilityMode: AUTO - persistMode: AUTO - persistenceDir: /mnt/brooklyn-persistence Then launch the standby Brooklyn with: {% highlight bash %} $ bin/start {% endhighlight %} ### Failover When running as a HA standby node, each standby Brooklyn server (in this case there is only one standby) will check the shared persisted state every one second to determine the state of the HA master. If no heartbeat has been recorded for 30 seconds, then an election will be performed and one of the standby nodes will be promoted to master. At this point all requests should be directed to the new master node. If the master is terminated gracefully, the secondary will be immediately promoted to mater. Otherwise, the secondary will be promoted after heartbeats are missed for a given length of time. This defaults to 30 seconds, and is configured in `brooklyn.cfg` using `brooklyn.ha.heartbeatTimeout` In the event that tasks - such as the provisioning of a new entity - are running when a failover occurs, the new master will display the current state of the entity, but will not resume its provisioning or re-run any partially completed tasks. In this case it may be necessary to remove the entity and reprovision it. In the case of a failover whilst executing a task called by an effector, it may be possible to simple call the effector again ### High Availability Management On top of the [`API`](/guide/ops/high-availability/index.md), High Availability can be explicitly controlled from the Brooklyn UI, which allows for the server to change its priority and request to promote itself to master. This can be achieved via the `HA Status` table in the `About` page, which displays information about nodes in the current management plane. The control menu is opened by selecting the `Manage` option on the current server entry in the table. The following menu allows to change the priority value, as well as the status of the node. - If a node is `MASTER`, it can demote itself by changing to another state. In such case, a new master is selected from available standby servers, basing on their priority. - If a node is `STANDBY`, or `HOT_STANDBY`, it can promote itself by changing to `MASTER` state. It is recommended for this server to have the highest priority amongst all available servers. Additionally, terminated servers can be removed from the persistence with `Remove` option (visible upon hover over the terminated node in the `HA Status` Table). All terminated servers can be removed at once with `Remove terminated nodes` option. These operations are only available to the master node. ### Client Configuration It is the responsibility of the client to connect to the master Brooklyn server. This can be accomplished in a variety of ways: * ###Reverse Proxy To allow the client application to automatically fail over in the event of a master server becoming unavailable, or the promotion of a new master, a reverse proxy can be configured to route traffic depending on the response returned by `https://Externalized Configuration
^ Sometimes it is useful that configuration in a blueprint, or in Brooklyn itself, is not given explicitly, but is instead replaced with a reference to some other storage system. For example, it is undesirable for a blueprint to contain a plain-text password for a production system, especially if (as we often recommend) the blueprints are kept in the developer's source code control system. To handle this problem, Apache Brooklyn supports externalized configuration. This allows a blueprint to refer to a piece of information that is stored elsewhere. `brooklyn.cfg` defines the external suppliers of configuration information. At runtime, when Brooklyn finds a reference to externalized configuration in a blueprint, it consults `brooklyn.cfg` for information about the supplier, and then requests that the supplier return the information required by the blueprint. Take, as a simple example, a web app which connects to a database. In development, the developer is running a local instance of PostgreSQL with a simple username and password. But in production, an enterprise-grade cluster of PostgreSQL is used, and a dedicated service is used to provide passwords. The same blueprint can be used to service both groups of users, with `brooklyn.cfg` changing the behaviour depending on the deployment environment. Here is the blueprint: {% read external/_externalised-configuration-blueprint.camp.md %} You can see that when we are building up the JDBC URL, we are using the `external` function. This takes two parameters: the first is the name of the configuration supplier, the second is the name of a key that is stored by the configuration supplier. In this case we are using two different suppliers: `servers` to store the location of the server, and `credentials` which is a security-optimized supplier of secrets. Developers would add lines like this to the `brooklyn.cfg` file on their workstation: {% highlight properties %} brooklyn.external.servers=org.apache.brooklyn.core.config.external.InPlaceExternalConfigSupplier brooklyn.external.servers.postgresql=127.0.0.1 brooklyn.external.credentials=org.apache.brooklyn.core.config.external.InPlaceExternalConfigSupplier brooklyn.external.credentials.postgresql-user=admin brooklyn.external.credentials.postgresql-password=admin {% endhighlight %} In this case, all of the required information is included in-line in the local `brooklyn.cfg`. Whereas in production, `brooklyn.cfg` might look like this: {% highlight properties %} brooklyn.external.servers=org.apache.brooklyn.core.config.external.PropertiesFileExternalConfigSupplier brooklyn.external.servers.propertiesUrl=https://ops.example.com/servers.properties brooklyn.external.credentials=org.apache.brooklyn.core.config.external.vault.VaultAppIdExternalConfigSupplier brooklyn.external.credentials.endpoint=https://vault.example.com brooklyn.external.credentials.path=secret/enterprise-postgres brooklyn.external.credentials.appId=MyApp {% endhighlight %} In this case, the list of servers is stored in a properties file located on an Operations Department web server, and the credentials are stored in an instance of [Vault](https://www.vaultproject.io/). More information on these providers is below. For demo purposes, there is a pre-defined external provider called `brooklyn-demo-sample` which defines `hidden-brooklyn-password` as `br00k11n`. This is used in some of the sample blueprints, referencing `$brooklyn:external("brooklyn-demo-sample", "hidden-brooklyn-password")`. The value used here can be overridden with the following in your `brooklyn.cfg`: {% highlight properties %} brooklyn.external.brooklyn-demo-sample=org.apache.brooklyn.core.config.external.InPlaceExternalConfigSupplier brooklyn.external.brooklyn-demo-sample.hidden-brooklyn-password=new_password {% endhighlight %} ## Defining Suppliers External configuration suppliers are defined in `brooklyn.cfg`. The minimal definition is of the form: brooklyn.external.*supplierName* = *className* This defines a supplier named *supplierName*. Brooklyn will attempt to instantiate *className*; it is this class which will provide the behaviour of how to retrieve data from the supplier. Brooklyn includes a number of supplier implementations; see below for more details. Suppliers may require additional configuration options. These are given as additional properties in `brooklyn.cfg`: {% highlight properties %} brooklyn.external.supplierName = className brooklyn.external.supplierName.firstConfig = value brooklyn.external.supplierName.secondConfig = value {% endhighlight %} ## Referring to External Configuration in Blueprints Externalized configuration adds a new function to the Brooklyn blueprint language DSL, `$brooklyn:external`. This function takes two parameters: 1. supplier 2. key When resolving the external reference, Brooklyn will first identify the *supplier* of the information, then it will give the supplier the *key*. The returned value will be substituted into the blueprint. {% read external/_externalised-configuration-twoways.camp.md %} ## Referring to External Configuration in brooklyn.cfg The same blueprint language DSL can be used from `brooklyn.cfg`. For example: {% highlight properties %} brooklyn.location.jclouds.aws-ec2.identity=$brooklyn:external("mysupplier", "aws-identity") brooklyn.location.jclouds.aws-ec2.credential=$brooklyn:external("mysupplier", "aws-credential") {% endhighlight %} ## Referring to External Configuration in Catalog Items {% read external/_externalised-configuration-catalog.camp.md %} ## Suppliers available with Brooklyn Brooklyn ships with a number of external configuration suppliers ready to use. ### In-place **InPlaceExternalConfigSupplier** embeds the configuration keys and values as properties inside `brooklyn.cfg`. For example: {% highlight properties %} brooklyn.external.servers=org.apache.brooklyn.core.config.external.InPlaceExternalConfigSupplier brooklyn.external.servers.postgresql=127.0.0.1 {% endhighlight %} Then, a blueprint which referred to `$brooklyn:external("servers", "postgresql")` would receive the value `127.0.0.1`. ### Properties file **PropertiesFileExternalConfigSupplier** loads a properties file from a URL, and uses the keys and values in this file to respond to configuration lookups. Given this configuration: {% highlight properties %} brooklyn.external.servers=org.apache.brooklyn.core.config.external.PropertiesFileExternalConfigSupplier brooklyn.external.servers.propertiesUrl=https://ops.example.com/servers.properties {% endhighlight %} This would cause the supplier to download the given URL. Assuming that the file contained this entry: {% highlight properties %} postgresql=127.0.0.1 {% endhighlight %} Then, a blueprint which referred to `$brooklyn:external("servers", "postgresql")` would receive the value `127.0.0.1`. ### Vault [Vault](https://www.vaultproject.io) is a server-based tool for managing secrets. Brooklyn provides suppliers that are able to query the Vault REST API for configuration values. The different suppliers implement alternative authentication options that Vault provides. Brooklyn supports the "Key/Value" Secrets Engine, with API version 1 or version 2. See [KV Secrets Engine](https://www.vaultproject.io/docs/secrets/kv) in the Vault documentation to help you decide which one to use. If you are using version 1, here are the properties you can set: {% highlight properties %} brooklyn.external.supplierName.endpoint=Logging
^ Brooklyn uses the SLF4J logging facade, which allows use of many popular frameworks including `logback`, `java.util.logging` and `log4j`. Most loggers follow the ``package.ClassName`` naming standard, apart from some high-level ones which simplify the package and classname. For handy reference, the most commonly used logging categories are: * `org.apache.brooklyn.rest.security.LoginLogging`: login and logouts * `org.apache.brooklyn.REST`: REST handling details * `org.apache.brooklyn.SSH`: SSH i/o details * `org.apache.brooklyn.WinRm`: WinRM i/o details * `org.apache.brooklyn.core.mgmt.internal.BrooklynGarbageCollector`: Java memory GC details The default logging is to write INFO+ messages to `brooklyn.info.log`, and DEBUG+ to `brooklyn.debug.log`. Each is a rolling log file, where the past 10 files will be kept. INFO level, and above, messages will be logged to the karaf console. Use the `log:` commands in the karaf client, e.g. `log:tail`, to read these messages. The convention for log levels is as follows: * `ERROR` and above: exceptional situations which indicate that something has unexpectedly failed or some other problem has occurred which the user is expected to attend to * `WARN`: exceptional situations which the user may which to know about but which do not necessarily indicate failure or require a response. * `INFO`: a synopsis of activity, but which should not generate large volumes of events nor overwhelm a human observer. * `DEBUG`: detail of activity which might merit closer inspection under certain circumstances. * `TRACE` and lower: detail of activity which is not normally of interest, but which might merit closer inspection under certain circumstances including sensitive information (e.g. secrets) that should not be exposed in higher lover levels. A configuration example for TRACE level is present in the log configuration file, but is commented because of security concerns. ## Using Logback through OSGi Pax Logging In the OSGi based Apache Brooklyn logging is configured from ops4j pax logging. See: [https://ops4j1.jira.com/wiki/display/paxlogging/Configuration](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration) ## Standard Configuration A `org.ops4j.pax.logging.cfg` file is included in the `etc/` directly of the Brooklyn distro; this is read by `brooklyn` at launch time. Changes to the logging configuration, such as new appenders or different log levels, can be made directly in this file. Karaf logging is highly configurable. For example enable the sift appender to log to separate log files for each bundle as described here: [Advanced configuration](https://karaf.apache.org/manual/latest/#_advanced_configuration) By default, file logging uses the following pattern, in the PAX logging file: ```properties log4j2.pattern = %d{ISO8601}Z %X{task.id}-%X{entity.ids} %-5.5p %3X{bundle.id} %c{1.} [%.16t] %m%n ``` This uses UTC time, and includes any context thread (`%t`), as well as the bundle ID, task ID, and entity IDs. This makes the log easy to search, either from within AMP or using standard tools. As log messages are written for all tasks when they are created, these IDs make it possible trace back through the creation stack and identify the originating task or REST API call. The `bundle.id`, `task.id` and `entity.ids` are set as logging context variables, accessed using `%X`. The context variable `username` is also available if in the context of a REST API request, and can be included by inserting `%X{username}` into the pattern above. ## Advanced Configuration The default `logback.xml` file references a collection of other log configuration files included in the Brooklyn jars. It is necessary to understand the source structure in the [logback-includes]({{ site.brooklyn.url.git }}/logging/logback-includes) project. For example, to change the debug log inclusions, create a folder `brooklyn` under `conf` and create a file `logback-debug.xml` based on the [brooklyn/logback-debug.xml]({{ site.brooklyn.url.git }}/logging/logback-includes/src/main/resources/brooklyn/logback-debug.xml) from that project. A full explanation of logging in karaf is available [here](https://karaf.apache.org/manual/latest/#_log). ## Karaf Log commands Logging commands are available through the karaf console. These let you interact with the logs and dynamically change logging configuration in a running application. Some useful log: commands are: log:display mylogger -p "%d - %c - %m%n" - Show the log entries for a specific logger with a different pattern. log:get/set - Show / set the currently configured log levels log:tail - As display but will show continuously log:exception-display - Display the last exception ## Log File Backup *This sub-section is a work in progress; feedback from the community is extremely welcome.* The default rolling log files can be backed up periodically, e.g. using a CRON job. Note however that the rolling log file naming scheme will rename the historic zipped log files such that `brooklyn.debug-1.log.zip` is the most recent zipped log file. When the current `brooklyn.debug.log` is to be zipped, the previous zip file will be renamed `brooklyn.debug-2.log.zip`. This renaming of files can make RSYNC or backups tricky. An option is to covert/move the file to a name that includes the last-modified timestamp. For example (on mac): ```shell LOG_FILE=brooklyn.debug-1.log.zip TIMESTAMP=`stat -f '%Um' $LOG_FILE` mv $LOG_FILE /path/to/archive/brooklyn.debug-$TIMESTAMP.log.zip ``` ## Logging Aggregators Integration with systems like Logstash and Splunk is possible using standard log4j configuration. Log4j can be configured to write to syslog using the SyslogAppender which can then [feed its logs to Logstash](http://www.logstash.net/docs/1.4.2/inputs/syslog). ## For More Information The following resources may be useful when configuring logging: * The [logback-includes]({{ site.brooklyn.url.git }}/logging/logback-includes) project * [Brooklyn Developer Guide](/guide/dev/tips/logging/) logging tips * The [Logback Project](http://logback.qos.ch/) home page * [OPS4J Pax Logging](https://ops4j1.jira.com/wiki/display/paxlogging/Configuration) * [Elasticsearch Best Practices](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/best_practices.html) * [Elasticsearch Memory Usage](https://www.elastic.co/blog/significantly-decrease-your-elasticsearch-heap-memory-usage) * [OpenSearch Index Management](https://opensearch.org/docs/im-plugin/ism/index/) and [policies](https://opensearch.org/docs/im-plugin/ism/policies/) {% child_content %}Requirements
^ ## Server Specification The size of server required by Brooklyn depends on the amount of activity. This includes: * the number of entities/VMs being managed * the number of VMs being deployed concurrently * the amount of management and monitoring required per entity For dev/test or when there are only a handful of VMs being managed, a small VM is sufficient. For example, an AWS m3.medium with one vCPU, 3.75GiB RAM and 4GB disk. For larger production uses, a more appropriate machine spec would be two or more cores, at least 8GB RAM and 100GB disk. The disk is just for logs, a small amount of persisted state, and any binaries for custom blueprints/integrations. ### Disk Space There are three main consumers of disk space: * **Static files**: these are the Apache Brooklyn distribution with its own dependencies, OSGi bundles for custom blueprints and integrations installed to the `deploy/` directory, plus `data/` directory which is generated on first launch. Note that Brooklyn requires that Java is installed which you may have to consider when calculating disk space requirements. * **Persisted state**: when using [Persistence](persistence) -- which is a prerequisite for [High Availability](high-availability) -- Brooklyn will save data to a store location. Items in the persisted state include metadata about the Brooklyn servers, catalog items, and metadata about all running applications and entities. * **Log files**: Brooklyn writes info and debug log files. By default, these are written to the local filesystem. This can be reconfigured to set the destination and to increase or decrease the detail in the logs. See the [Logging](/guide/ops/logging/index.md) section for more details. The Apache Brooklyn distribution itself, when unpacked, consumes approximately 75MB of disk space. This includes everything needed to run Brooklyn except for a Java VM. The space consumed by additional binaries for custom blueprints and integrations is application-specific. Persisted state, excluding catalog data, is relatively small, starting at approximately 300KB for a clean, idle Brooklyn server. Deploying blueprints will add to this - how much depends exactly on the entities involved and is therefore application specific, but as a guideline, a 3-node Riak cluster adds approximately 500KB to the persistence store. Log data can be a large consumer of disk space. By default Brooklyn generates two logfiles, one which logs notable information only, and another which logs at a debug level. Each logfile rotates when it hits a size of 100MB; a maximum of 10 log files are retained for each type. The two logging streams combined, therefore, can consume up to 2GB of disk space. In the default configuration of Brooklyn's `.tar.gz` and `.zip` distributions, logs are saved to the Brooklyn installation directory. You will most likely want to [reconfigure Brooklyn's logging](/guide/ops/logging/index.md) to save logs to a location elsewhere. In the `.rpm` and `.deb` packaging, logging files will be located under `/var/log`. You can further reconfiguring the logging detail level and log rotation according to your organisation's policy. ## OS Requirements Brooklyn is tested against CentOS (6 or later), RHEL (6 or later), Ubuntu (14.04 or later), OS X, and Windows. ## Software Requirements Brooklyn requires Java 8 (JRE or JDK) or later. OpenJDK is recommended. Brooklyn has also been tested on the Oracle JVM and IBM J9. ## Configuration Requirements ### Ports The ports used by Brooklyn are: * 8443 for HTTPS, to expose the web-console and REST API. * 8081 for HTTP, to expose the web-console and REST API. Whether to use HTTPS rather than HTTP is configurable using the CLI option `--https`; the port to use is configurable using the CLI option `--portSystem Paths Summary
^ Based on the installation method you choose, the paths to the installed components of Apache Brooklyn will be different. The following table will help you to easily locate these:| Installation method | Brooklyn Home | Brooklyn Logs | Brooklyn Configuration | Brooklyn Persisted state |
|---|---|---|---|---|
| RPM Package | /opt/booklyn (symlink to /opt/booklyn-<version>) |
/var/log/booklyn (symlink to /opt/booklyn-<version>/data/log) |
/etc/booklyn |
/var/lib/booklyn |
| DEB Package | /opt/booklyn (symlink to /opt/booklyn-<version>) |
/var/log/booklyn (symlink to /opt/booklyn-<version>/data/log) |
/etc/booklyn |
/var/lib/booklyn |
| Vagrant | /opt/booklyn (symlink to /opt/booklyn-<version>) |
/var/log/booklyn (symlink to /opt/booklyn-<version>/data/log) |
/etc/booklyn |
/var/lib/booklyn |
| Tarball Zip Package | /path/of/untar/archive |
/path/of/untar/archive/data/log |
/path/of/untar/archive/etc |
~/.brooklyn/brooklyn-persisted-state |
Upgrades
^ This section provides all necessary information to upgrade Apache Brooklyn for both the RPM/DEB and Tarball packages. ## Backwards Compatibility Apache Brooklyn version 0.12.0 onward runs primarily inside a Karaf container. When upgrading from 0.11.0 or below, this update changes the mechanisms for launching Brooklyn. This will impact any custom scripting around the launching of Brooklyn, and the supplying of command line arguments. Use of the `lib/dropins` and `lib/patch` folders will no longer work (because Karaf does not support that kind of classloading). Instead, code must be built and installed as [OSGi bundles](https://en.wikipedia.org/wiki/OSGi#Bundles). {% include_relative _server.md %} {% include_relative _blueprints.md %} {% include_relative _systems-under-mgmt.md %}Security Guidelines
^ ## Brooklyn Server ### Web-console and REST API Users are strongly encouraged to use HTTPS, rather than HTTP. The use of LDAP is encouraged, rather than basic auth. Configuration of "entitlements" is encouraged, to lock down access to the REST API for different users. ### Brooklyn user Users are strongly discouraged from running Brooklyn as root. For production use-cases (i.e. where Brooklyn will never deploy to "localhost"), the user under which Brooklyn is running should not have `sudo` rights. ### Persisted State Use of an object store is recommended (e.g. using S3 compliant or Swift API) - thus making use of the security features offered by the chosen object store. File-based persistence is also supported. Permissions of the files will automatically be 600 (i.e. read-write only by the owner). Care should be taken for permissions of the relevant mount points, disks and directories. ## Credential Storage For credential storage, users are strongly encouraged to consider using the "externalised configuration" feature. This allows credentials to be retrieved from a store managed by you, rather than being stored within YAML blueprints or `brooklyn.cfg`. A secure credential store is strongly recommended, such as use of [HashiCorp's Vault](https://www.vaultproject.io) - see `org.apache.brooklyn.core.config.external.vault.VaultExternalConfigSupplier`. ## Infrastructure Access ### Cloud Credentials and Access Users are strongly encouraged to create separate cloud credentials for Brooklyn's API access. Users are also encouraged to (where possible) configure the cloud provider for only minimal API access (e.g. using AWS IAM). ### VM Image Credentials Users are strongly discouraged from using hard-coded passwords within VM images. Most cloud providers/APIs provide a mechanism to instead set an auto-generated password or to create an entry in `~/.ssh/authorized_keys` (prior to the VM being returned by the cloud provider). If a hard-coded credential is used, then Brooklyn can be configured with this "loginUser" and "loginUser.password" (or "loginUser.privateKeyData"), and can change the password and disable root login. ### VM Users It is strongly discouraged to use the root user on VMs being created or managed by Brooklyn. SSH-ing on the VM should be done on rare cases such as initial Apache Brooklyn setup, Apache Brooklyn upgrade and other important maintenance occasions. ### SSH keys Users are strongly encouraged to use SSH keys for VM access, rather than passwords. This SSH key could be a file on the Brooklyn server. However, a better solution is to use the "externalised configuration" to return the "privateKeyData". This better supports upgrading of credentials. ## Install Artifact Downloads When Brooklyn executes scripts on remote VMs to install software, it often requires downloading the install artifacts. For example, this could be from an RPM repository or to retrieve `.zip` installers. By default, the RPM repositories will be whatever the VM image is configured with. For artifacts to be downloaded directly, these often default to the public site (or mirror) for that software product. Where users have a private RPM repository, it is strongly encouraged to ensure the VMs are configured to point at this. For other artifacts, users should consider hosting these artifacts in their own web-server and configuring Brooklyn to use this. See the documentation for `org.apache.brooklyn.core.entity.drivers.downloads.DownloadProducerFromProperties`. ## Controlling Sensitive Information Often it is necessary to include sensitive information such as credentials within blueprints and locations. Apache Brooklyn provides a secure mechanism for users to protect the integrity of this information, as follows: * Values should be set in a secure external store such as an HSM, Vault, AWS Secrets Manager, or properties file * The blueprint should provide a reference to these values in config keys using [externalized configuration](/guide/ops/externalized-configuration.md) * Config keys where these values are referenced should comply with the naming scheme below, where one of the "sensitive-named tokens" is contained in the key name Externalized configuration allows blueprints to be written such that they do not contain any secure information, and with the steps above Apache Brooklyn further guarantees that these values of config keys are not stored on disk as part of the catalog nor as part of an active deployment, they are not written to the log, and they are not displayed in the UI. Because the values are needed for blueprint execution, there are still ways that users with appropriate permissions can discover these values, such as by using them in scripts or using low-level API calls, so due care should be taken to secure the user entitlements, the Brooklyn server, and the systems under management. ### Sensitive-Named Fields Potentially sensitive information is identified heuristically, by default according to a naming scheme whereby config keys and environment variables are deemed potentially sensitive if they contain any of the following "sensitive-named tokens" (case insensitive): - `password` - `passwd` - `credential` - `secret` - `private` - `access.cert` - `access.key` This list can be customized by setting `brooklyn.security.sensitive.fields.tokens` in `etc/brooklyn.cfg` to a list of strings, e.g. instead of the list above, to treat keys containing "hidden" or "pass" as potentially sensitive, set: ``` brooklyn.security.sensitive.fields.tokens=[hidden,pass] ``` Brooklyn will suppress the values of potentially sensitive information in many places, as described below, but to ensure the values are treated as secure it is necessary to follow the naming scheme _and_ supply the values via externalized configuration. ### Preventing Plaintext Values for Sensitive Named Fields Apache Brooklyn can be configured to disallow plaintext values for potentially sensitive config keys with the following `/etc/brooklyn.cfg` property: ``` brooklyn.security.sensitive.fields.plaintext.blocked=true ``` With this set, Apache Brooklyn will prevent deployment of blueprints that provide plaintext values in these places, forcing users to follow security best practice. This will apply to potentially sensitive values embedded in a blueprint being deployed or in a blueprint from the catalog referenced by a blueprint being deployed. This will also block some additions to the catalog where secrets are set as plaintext config values (including types from the Composer, except in some cases where it is explicitly marked as a "template"). This does not apply to default values specified for parameters or to values supplied via Java, as it is expected in these contexts that users are less likely to accidentally supply sensitive values in plaintext. All functions and complex objects, including mechanisms such as `$brooklyn:literal("value")` (to escape at design-time and evaluate as `value` at runtime), are permitted as values. Sensitive field blocking can optionally be further restricted to exclude selected DSL values and complex objects where the string representation (unresolved `toString`) contains selected tokens or phrases, by using the `brooklyn.security.sensitive.fields.ext.blocked.phrases` configuration property. For example to prevent the usage of the `literal` DSL function anywhere in a supplied expression, the following setting can be used: ``` brooklyn.security.sensitive.fields.ext.blocked.phrases = [ "$brooklyn:literal" ] ``` ### Scripts, Sensors, and other Blueprint Execution Considerations When blueprints are executing, they will by design have access to the sensitive values, so authors should be careful to limit their usage and maintain the security around these values. In particular: * The sensitive-name scheme should be followed for all parameters which might contain the sensitive value, and these should refer to sensitive-named configuration properties which refer to an external provider * When needed for a script, sensitive value should be should be passed as environment variables following the sensitive-name scheme, taking their values by referring to sensitive configuration properties, and the values should not be output by the script * Sensitive values should not be used in template files, sensors, tags, task result, or any other places where the value might be returned in the UI or the logs If these steps are not followed, the security afforded by the externalized configuration might be compromised. ### Logging Log messages which may contain sensitive information are normally logged at TRACE level. Logging should configured such that TRACE is excluded or appropriately secured to prevent sensitive values from being compromised. A commented sample configuration for enabling TRACE logging is available in the `org.ops4j.pax.logging.cfg` logging configuration file. With this configuration enabled, all TRACE log entries are written to the `brooklyn.trace.log` file. Blueprint source code and some activity may be logged at DEBUG level or higher, so, as per above, secrets should not be included in plain text in blueprints unless the Apache Brooklyn environment and its logs are appropriately secured. If it is desired to suppress information that is logged at DEBUG or higher level, which should not ordinarily be needed but may be desired on occasion, this can be done by setting filter(s) and/or appender(s) on the appropriate logging category in `org.ops4j.pax.logging.cfg`. Some of the categories (or individual sub-categories of these) which may be relevant for exclusion or higher security are: * `org.apache.brooklyn.core.typereg`: resolution of bundles and registration of types * `org.apache.brooklyn.rest.resources`: log REST activity, including blueprints deployed * `org.apache.brooklyn.camp.brooklyn.spi.creation`: creation of entities from CAMP * `org.apache.brooklyn.camp.brooklyn.spi.dsl`: resolution of DSL expressions ### Sanitizing Outputs For security against accidental exposure, plaintext values for sensitive named fields are masked in many places in the logs and UI, typically replaced with a string indicating it is suppressed and supplying a MD5 hash prefix (the first 4 bytes / 8 hex chars). This is so that if you know the value, you can confirm it with high confidence, but attackers don't have enough information to uniquely crack typical-length passwords. For example, in the logs and in the UI, sensitive information might be displayed as: password:Troubleshooting
^ {% include list-children.html %}Brooklyn Fails to Start
^ If Apache Brooklyn does not start, or starts with errors, the problem is usually easy to resolve. The first place to look is the [logs](/guide/ops/logging/index.md): `grep` for the first `ERROR`, and sometimes look backwards for the first `WARN` message. There are a handful of common causes. ## Memory If there is not enough memory available either on the system or for the software, it will have problems. This may manifest itself as the process being killed, e.g. if the OS does not have enough memory (and there will usually be a message in the system log, e.g. `/var/log/syslog`); or some modules failing to load with an `OutOfMemoryException` in the log. If either of these occurs, you can assign additional memory if available on your system by editing the files in `bin/`, such as `JAVA_MAX_MEM` in `setenv` (or `setenv.bat` on Windows), or by running Apache Brooklyn on a system with more memory. ## Rebind Errors It is possible to get the persistent state into an incompatible state, where Apache Brooklyn cannot load its previous state. In this case it fails fast so as not to corrupt the state further. In addition, a backup of the persistent state will be written to the `backups/` folder in the persistent state directory. The log files contain detailed information about what is unable to be loaded and why; some causes include: * A type that is deployed is no longer available, e.g. because a `SNAPSHOT` bundle was installed, say with a type `X`, the type `X` is used in an active deployment, and then the bundle was either uninstalled or a new version installed at the same version (for `SNAPSHOT` or forced) that did not contain the type in use (`X`) * A deployment did not correctly clean up and leaked resources; this will happen only with Java entities or adjuncts that are incorrectly unmanaged * A dependency is unavailable, possibly because it was added via the `dropins/` folder or is not installed in the Brooklyn instance being started There are some good practices which can help avoid these errors: * Avoid the use of `SNAPSHOT` bundles in production (and do not `force` install bundles) * If `SNAPSHOT` bundles are updated in an incompatible way in a dev environment (eg blueprint name change), take care to remove pre-existing incompatible deployments * When upgrading or restarting Brooklyn, it is recommended to start a second instance as hot-standby first: this will flag the issue that there is an existing deployment which cannot be re-read on a clean start, and it can be removed from the primary Brooklyn If a rebind problem does occur, all is not lost. There are several ways that recovery can be achieved: * Delete the incompatible persisted state item files indicated in the logs (or simply delete all the persisted state in a dev environment) * Restore to a previous backup state (automatically written to the `backups/` folder with a datestamp) * Tell Brooklyn to ignore a certain number of rebind errors with settings in `brooklyn.cfg`: * `rebind.failureMode.danglingRefs.minRequiredHealthy`: takes `QuorumCheck` syntax, consisting of points on a line, e.g. `[[0,0],[10,5],[20,14]]` to allow up to 1 failure for every 2 items up to 10 items (5 needed when 10 items are persisted, per the second point), then subsequently 1 failure for every additional 10 items deployed (14 needed when 20 items are persisted, per the third point) * `rebind.failureMode.rebind`: either `FAIL_FAST`, `FAIL_AT_END`, or `CONTINUE`, for how to treat serious rebind problems (default `FAIL_AT_END`) * Further options available as per the JavaDoc on `RebindManagerImpl` config keys * When Brooklyn is stopped, remove the persisted state; then restart in a pristine environment, install any missing bundles, then import the offending persistent state via the UI (About) or REST API; alternatively in some cases it may be possible to add additional/missing bundles via the `dropins/` folder of Karaf or using the `karaf` console (`bundle:install -s ...`) * If the broken persisted state is critical, it is possible to edit them: they are simply an XML model of the items using a lot of unique identifiers designed so that references can be easily found using `grep` * Finally, if all else fails, open a support ticket: there are a number of other advanced techniques available, such as specifying that types should be automatically renamed or migrated by new bundles ([see the Persistence section here](/guide/ops/upgrades)). It may also be useful to review the sections on [Persistence](/guide/ops/persistence) and [HA](/guide/ops/high-availability).Other Resources
^ Further documentation specific to this version of Brooklyn includes: {% for item in page.menu %} * [{{ item.title_in_menu }}]({{ item.url }}) {% endfor %} Also see the [other versions](/website/meta/versions.md) or [general documentation](/website/documentation).Release Notes
^ ## Version {{ site.brooklyn-version }} {% if SNAPSHOT %} **You are viewing a SNAPSHOT release (master branch), so this list is in progress!** {% endif %} Thanks go to our community for their improvements, feedback and guidance, and to Brooklyn's commercial users for funding much of this development. ### New Features ### Backwards Compatibility Changes since 1.1.0: For changes in prior versions, please refer to the release notes for [1.1.0]({{ site.path.v | relative_url }}/1.1.0/misc/release-notes.html).Known Issues
^ ## Unable to Provision certain types of Debian VMs *Symptom*: Brooklyn fails to provision Debian VMs (e.g. in aws-ec2). *Cause*: `sudo` is not available on path, causing Brooklyn to fail to confirm that the VM is ssh'able. *Workaround*: Choose an image that does have sudo (see [wiki.debian.org/Cloud/AmazonEC2Image](http://wiki.debian.org/Cloud/AmazonEC2Image)). *Fix*: is [Pull #600](https://github.com/brooklyncentral/brooklyn/pull/600); you may also want to run with `brooklyn.location.jclouds.aws-ec2.user=root` if subsequent commands give permission errors. *Versions Affected*: 0.5.0-M2 ### Unable to Provision Ubuntu 8 VMs *Symptom: Brooklyn fails to provision Ubuntu 8 VMs (e.g. in aws-ec2) with the following error 'Cannot insert the iptables rule for port 22. Error: sudo: illegal option `-n''. *Cause: Ubuntu 8 is too old; the sudo command doesn't support the -n setting. *Workaround: Choose Ubuntu 10 or higher. *Versions Affected*: 0.5.0-M2Maven Build
^ ## The Basics The full build requires the following software to be installed: * Maven (v3.5.4+) * Java (v1.8) * Go (v1.6+) [if building the CLI client] * rpm tools (latest) [if building the dist packages for those platforms] * deb tools (latest) [if building the dist packages for those platforms] * docker (latest) [if building the dist package for this platform] With these in place, you should be able to build everything with a: {% highlight bash %} % mvn clean install {% endhighlight %} By default, only tarball and zip packages for `brooklyn-dist` will be built. You can enable each dist artifact with the following arguments: - for CLI client: `-Dcli` (requires `Go`) - for RPM package: `-Drpm` (requires `rpm tools`) - for DEB package: `-Ddeb` (requires `deb tools`) - for docker image: `-Ddocker` (requires `docker`) Alternatively, you can build everything by using the `release` profile: {% highlight bash %} mvn clean install -Prelease {% endhighlight %} Other tips: * Add ``-DskipTests`` to skip tests (builds much faster, but not as safe) * You may need more JVM memory, e.g. at the command-line (or in `.profile`): ``export MAVEN_OPTS="-Xmx1024m -Xms512m"`` * Run ``-PIntegration`` to run integration tests, or ``-PLive`` to run live tests ([tests described here](/guide/dev/code/tests.md)) * If building the `rpm` package, you can install rpm tools with: `brew install rpm` for Mac OS, `apt-get install rpm` for Ubuntu, `yum install rpm` for Centos/RHEL. On Mac OS you may also need to set `%_tmppath /tmp` in `~/.rpmmacros`. * If building the `deb` package, you can install deb tools with: `brew install dpkg` for Mac OS, `apt-get install deb` for Ubuntu, `yum install deb` for Centos/RHEL. * If you're looking at the maven internals, note that many of the settings are inherited from parent projects (see for instance `brooklyn-server/parent/pom.xml`) * For tips on building within various IDEs, look [here](ide). ## When the RAT Bites We use RAT to ensure that all files are compliant to Apache standards. Most of the time you shouldn't see it or need to know about it, but if it detects a violation, you'll get a message such as: [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.10:check (default) on project brooklyn-parent: Too many files with unapproved license: 1 See RAT report in: /Users/alex/Data/cloudsoft/dev/gits/brooklyn/target/rat.txt -> [Help 1] If there's a problem, see the file `rat.txt` in the `target` directory of the failed project. (Maven will show you this link in its output.) Often the problem is one of the following: * You've added a file which requires the license header but doesn't have it **Resolution:** Simply copy the header from another file * You've got some temporary files which RAT things should have headers **Resolution:** Move the files away, add headers, or turn off RAT (see below) * The project structure has changed and you have stale files (e.g. in a `target` directory) **Resolution:** Remove the stale files, e.g. with `git clean -df` (and if needed a `find . -name target -prune -exec rm -rf {} \;` to delete folders named `target`) To disable RAT checking on a build, set `rat.ignoreErrors`, e.g. `mvn -Drat.ignoreErrors=true clean install`. (But note you will need RAT to pass in order for a PR to be accepted!) If there is a good reason that a file, pattern, or directory should be permanently ignored, that is easy to add inside the root `pom.xml`. ## Other Handy Hints * The **mvnf** script ([get the gist here](https://gist.github.com/2241800)) simplifies building selected projects, so if you just change something in ``software-webapp`` and then want to re-run the examples you can do: ``examples/simple-web-cluster% mvnf ../../{software/webapp,usage/all}`` ## Appendix: Sample Output A healthy build will look something like the following, including a few warnings (which we have looked into and understand to be benign and hard to get rid of them, although we'd love to if anyone can help!): {% highlight bash %} % mvn clean install ... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Brooklyn Root 1.0.0-SNAPSHOT: [INFO] [INFO] Brooklyn Server Root ............................... SUCCESS [ 0.567 s] [INFO] Brooklyn Parent Project ............................ SUCCESS [ 1.552 s] [INFO] Brooklyn Test Support Utilities .................... SUCCESS [ 2.719 s] [INFO] Brooklyn Logback Includable Configuration .......... SUCCESS [ 0.355 s] [INFO] Brooklyn Common Utilities .......................... SUCCESS [ 7.237 s] [INFO] Brooklyn API ....................................... SUCCESS [ 1.229 s] [INFO] CAMP Server Parent Project ......................... SUCCESS [ 0.109 s] [INFO] CAMP Base .......................................... SUCCESS [ 0.893 s] [INFO] Brooklyn Test Support .............................. SUCCESS [ 0.897 s] [INFO] Brooklyn REST Swagger Apidoc Utilities ............. SUCCESS [ 0.733 s] [INFO] Brooklyn Logback Configuration ..................... SUCCESS [ 0.299 s] [INFO] CAMP Server ........................................ SUCCESS [ 1.385 s] [INFO] Brooklyn Felix Runtime ............................. SUCCESS [ 0.534 s] [INFO] Brooklyn Groovy Utilities .......................... SUCCESS [ 0.500 s] [INFO] Brooklyn Core ...................................... SUCCESS [ 31.521 s] [INFO] Brooklyn Policies .................................. SUCCESS [ 3.556 s] [INFO] Brooklyn WinRM Software Entities ................... SUCCESS [ 1.778 s] [INFO] Brooklyn Secure JMXMP Agent ........................ SUCCESS [ 1.108 s] [INFO] Brooklyn JMX RMI Agent ............................. SUCCESS [ 0.334 s] [INFO] Brooklyn Jclouds Location Targets .................. SUCCESS [ 5.202 s] [INFO] Brooklyn Software Base ............................. SUCCESS [ 6.690 s] [INFO] Brooklyn CAMP ...................................... SUCCESS [ 4.282 s] [INFO] Brooklyn Launcher Common ........................... SUCCESS [ 1.719 s] [INFO] Brooklyn REST API .................................. SUCCESS [ 3.866 s] [INFO] Brooklyn REST Resources ............................ SUCCESS [ 4.475 s] [INFO] Brooklyn REST Server ............................... SUCCESS [ 1.523 s] [INFO] Brooklyn Launcher .................................. SUCCESS [ 2.765 s] [INFO] Brooklyn Container Location Targets ................ SUCCESS [ 2.413 s] [INFO] Brooklyn Command Line Interface .................... SUCCESS [ 2.101 s] [INFO] Brooklyn Test Framework ............................ SUCCESS [ 2.537 s] [INFO] Brooklyn OSGi init ................................. SUCCESS [ 1.517 s] [INFO] Brooklyn OSGi start ................................ SUCCESS [ 1.497 s] [INFO] Brooklyn Karaf ..................................... SUCCESS [ 0.037 s] [INFO] Jetty config fragment .............................. SUCCESS [ 1.381 s] [INFO] Apache Http Component extension .................... SUCCESS [ 0.369 s] [INFO] Brooklyn Karaf Features ............................ SUCCESS [ 0.867 s] [INFO] Brooklyn Karaf Shell Commands ...................... SUCCESS [ 2.625 s] [INFO] Brooklyn UI :: Parent .............................. SUCCESS [ 25.412 s] [INFO] Brooklyn UI :: Modularity Server (parent) .......... SUCCESS [ 0.138 s] [INFO] Brooklyn UI :: Modularity Server :: UI Module API .. SUCCESS [ 1.085 s] [INFO] Brooklyn UI :: Modularity Server :: UI Module Registry SUCCESS [ 0.802 s] [INFO] Brooklyn UI :: Modularity Server :: UI Proxy ....... SUCCESS [ 0.602 s] [INFO] Brooklyn UI :: Modularity Server :: UI Metadata Registry SUCCESS [ 0.595 s] [INFO] Brooklyn UI :: Modularity Server :: External UI Modules Registration Hooks SUCCESS [ 1.134 s] [INFO] Brooklyn UI :: Modularity Server :: Features ....... SUCCESS [ 2.050 s] [INFO] Brooklyn UI :: Modules (parent) .................... SUCCESS [ 9.488 s] [INFO] Brooklyn UI :: Modules - UI Utils .................. SUCCESS [ 7.689 s] [INFO] Brooklyn UI :: Modules - Home ...................... SUCCESS [ 34.523 s] [INFO] Brooklyn UI :: Modules - App inspector ............. SUCCESS [ 37.624 s] [INFO] Brooklyn UI :: Modules - Blueprint composer ........ SUCCESS [ 39.765 s] [INFO] Brooklyn UI :: Modules - Blueprint importer ........ SUCCESS [ 31.316 s] [INFO] Brooklyn UI :: Modules - Catalog ................... SUCCESS [ 32.420 s] [INFO] Brooklyn UI :: Modules - Location manager .......... SUCCESS [ 30.421 s] [INFO] Brooklyn UI :: Modules - REST API Docs ............. SUCCESS [ 29.679 s] [INFO] Brooklyn UI :: Modules - Groovy console ............ SUCCESS [ 27.595 s] [INFO] Brooklyn UI :: Modules - Logout .................... SUCCESS [ 25.890 s] [INFO] Brooklyn UI :: Modules - Features .................. SUCCESS [ 1.720 s] [INFO] Brooklyn UI :: Features ............................ SUCCESS [ 0.160 s] [INFO] Brooklyn Library Root .............................. SUCCESS [ 0.318 s] [INFO] Brooklyn CM Chef ................................... SUCCESS [ 3.157 s] [INFO] Brooklyn CM SaltStack .............................. SUCCESS [ 1.531 s] [INFO] Brooklyn CM Ansible ................................ SUCCESS [ 1.414 s] [INFO] Brooklyn CM Integration Root ....................... SUCCESS [ 0.172 s] [INFO] Brooklyn Network Software Entities ................. SUCCESS [ 1.458 s] [INFO] Brooklyn OSGi Software Entities .................... SUCCESS [ 1.105 s] [INFO] Brooklyn Database Software Entities ................ SUCCESS [ 2.084 s] [INFO] Brooklyn Web App Software Entities ................. SUCCESS [ 2.996 s] [INFO] Brooklyn Messaging Software Entities ............... SUCCESS [ 3.046 s] [INFO] Brooklyn NoSQL Data Store Software Entities ........ SUCCESS [ 4.885 s] [INFO] Brooklyn Monitoring Software Entities .............. SUCCESS [ 1.048 s] [INFO] Brooklyn Web App Software Entities ................. SUCCESS [ 0.272 s] [INFO] Brooklyn QA ........................................ SUCCESS [ 3.819 s] [INFO] Brooklyn Examples Aggregator Project ............... SUCCESS [ 0.111 s] [INFO] Brooklyn Examples Aggregator Project - Webapps ..... SUCCESS [ 0.158 s] [INFO] hello-world-webapp Maven Webapp .................... SUCCESS [ 0.526 s] [INFO] hello-world-sql-webapp Maven Webapp ................ SUCCESS [ 0.591 s] [INFO] Brooklyn Simple Web Cluster Example ................ SUCCESS [ 1.919 s] [INFO] Brooklyn Library Karaf integration ................. SUCCESS [ 0.087 s] [INFO] Brooklyn Library Catalog ........................... SUCCESS [ 0.316 s] [INFO] Brooklyn Library Karaf Features .................... SUCCESS [ 0.238 s] [INFO] Brooklyn Downstream Project Parent ................. SUCCESS [ 0.082 s] [INFO] Brooklyn Dist Root ................................. SUCCESS [ 0.489 s] [INFO] Brooklyn All Things ................................ SUCCESS [ 1.868 s] [INFO] Brooklyn Distribution .............................. SUCCESS [ 7.541 s] [INFO] Brooklyn Karaf Distribution Parent ................. SUCCESS [ 0.064 s] [INFO] Brooklyn Karaf Server Configuration ................ SUCCESS [ 0.446 s] [INFO] Brooklyn Dist Karaf Features ....................... SUCCESS [ 0.152 s] [INFO] Brooklyn Karaf Distribution ........................ SUCCESS [ 10.222 s] [INFO] Brooklyn Karaf pax-exam itest ...................... SUCCESS [ 2.030 s] [INFO] Brooklyn Vagrant Getting Started Environment ....... SUCCESS [ 0.190 s] [INFO] Brooklyn Quick-Start Project Archetype ............. SUCCESS [ 0.723 s] [INFO] Brooklyn Shared Package Files ...................... SUCCESS [ 0.328 s] [INFO] Brooklyn Root ...................................... SUCCESS [ 0.439 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 08:21 min [INFO] Finished at: 2019-04-08T15:52:28+01:00 [INFO] ------------------------------------------------------------------------ {% endhighlight %}IDE Setup
^ Gone are the days when IDE integration always just works... Maven and Eclipse fight, neither quite gets along perfectly with Groovy, git branch switches (sooo nice) can be slow, etc etc. But with a bit of a dance the IDE can still be your friend, making it much easier to run tests and debug. As a general tip, don't always trust the IDE to build correctly; if you hit a snag, do a command-line ``mvn clean install`` (optionally with ``-DskipTests`` and/or ``-Dno-go-client``) then refresh the project. See instructions below for specific IDEs. ## Eclipse The default Eclipse downloads already include all of the plugins needed for working with the Brooklyn project. Optionally you can install the Groovy and TestNG plugins, but they are not required for building the project. You can install these using Help -> Install New Software, or from the Eclipse Marketplace: {% read eclipse.include.md %} As of this writing, Eclipse 4.5 and Eclipse 4.4 are commonly used, and the codebase can be imported (Import -> Existing Maven Projects) and successfully built and run inside an IDE. However there are quirks, and mileage may vary. Disable ``Build Automatically`` from the ``Project`` menu if the IDE is slow to respond. If you encounter issues, the following hints may be helpful: * If m2e reports import problems, it is usually okay (even good) to mark all to "Resolve All Later". The build-helper connector is useful if you're prompted for it, but do *not* install the Tycho OSGi configurator (this causes show-stopping IAE's, and we don't need Eclipse to make bundles anyway). You can manually mark as permanently ignored certain errors; this updates the pom.xml (and should be current). * A quick command-line build (`mvn clean install -DskipTests -Dno-go-client`) followed by a workspace refresh can be useful to re-populate files which need to be copied to `target/` * m2e likes to put `excluding="**"` on `resources` directories; if you're seeing funny missing files (things like not resolving jclouds/aws-ec2 locations or missing WARs), try building clean install from the command-line then doing Maven -> Update Project (clean uses a maven-replacer-plugin to fix `.classpath`s). Alternatively you can go through and remove these manually in Eclipse (Build Path -> Configure) or the filesystem, or use the following command to remove these rogue blocks in the generated `.classpath` files: {% highlight bash %} % find . -name .classpath -exec sed -i.bak 's/[ ]*..cluding="[\*\/]*\(\.java\)*"//g' {} \; {% endhighlight %} * You may need to ensure ``src/main/{java,resources}`` is created in each project dir, if (older versions) complain about missing directories, and the same for ``src/test/{java,resources}`` *if* there are tests (``src/test`` exists): {% highlight bash %} find . \( -path "*/src/main" -or -path "*/src/test" \) -exec echo {} \; -exec mkdir -p {}/{java,resources} \; {% endhighlight %} If the pain starts to be too much, come find us on IRC #brooklyncentral or [elsewhere](/website/community) and we can hopefully share our pearls. (And if you have a tip we haven't mentioned please let us know that too!) ## IntelliJ IDEA To develop or debug Brooklyn in IntelliJ, you will need to ensure that the Groovy and TestNG plugins are installed via the IntelliJ IDEA | Preferences | Plugins menu. Once installed, you can open Brooklyn from the root folder, (e.g. ``~/myfiles/brooklyn``) which will automatically open the subprojects. Brooklyn has informally standardized on arranging `import` statements as per Eclipse's default configuration. IntelliJ's default configuration is different, which can result in unwanted "noise" in commits where imports are shuffled backward and forward between the two types - PRs which do this will likely fail the review. To avoid this, reconfigure IntelliJ to organize imports similar to Eclipse. See [this StackOverflow answer](http://stackoverflow.com/a/17194980/68898) for a suitable configuration. ## Netbeans Tips from Netbeans users wanted! ## Debugging Tips To debug Brooklyn, you have 2 solutions: 1. **Launch the REST server + each UI module manually** Create a launch configuration which launches the ``BrooklynJavascriptGuiLauncher`` class. This will launch only the REST API. If you need the UI on top of it, you can create launch configuration for each of the UI module by calling `npm run start`. See `README.md` files on each UI modules for more information about this. 2. **Launch the fully built karaf distribution, attached to a Java debugger** First, you need to build the entire project. Then, create a launch configuration that will execute `brooklyn-dist/karaf/apache-brooklyn/target/assembly/bin/karaf` and have the following Java Options: `JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,address=127.0.0.1:8888,server=y,suspend=y`. Create a second launch configuration to start the remote debugger on port `8888`. When you start the first launch configuration, it will wait for the Java remote debugger to start. At that point, start the second launch configuration, the first will then resume as usual. This will give you a fully built Brooklyn including REST server and UI.Code Structure
^ Brooklyn is split into the following subprojects: * **brooklyn-server**: * **api**: the pure-Java interfaces for interacting with the system * **camp**: the components for a server which speaks with the CAMP REST API and understands the CAMP YAML plan language * **core**: the base class implementations for entities and applications, entity traits, locations, policies, sensor and effector support, tasks, and more * **karaf**: OSGi support * **launcher**: for launching brooklyn, either using a main method or invoked from the CLI project * **locations**: specific location integrations * **jclouds**: integration with many cloud APIs and providers via Apache jclouds * **logging**: how we enable configurable logging * **logback-includes**: Various helpful logback XML files that can be included; does not contain logback.xml * **logback-xml**: Contains a logback.xml that references the include files in brooklyn-logback-includes * **parent**: a meta-project parent to collect dependencies and other maven configuration for re-use * **policy**: collection of useful policies for automating entity activity * **rest**: supporting the REST API * **rest-api**: The API classes for the Brooklyn REST api * **rest-client**: A client Java implementation for using the Brooklyn REST API * **rest-server**: The server-side implementation of the Brooklyn REST API * **server-cli**: implementation of the Brooklyn *server* command line interface; not to be confused with the client CLI * **software**: support frameworks for creating entities which mainly launch software processes on machines * **base**: software process lifecycle abstract classes and drivers (e.g. SSH) * **winrm**: support for connecting to Windows machines * **test-framework**: provides Brooklyn entities for building YAML tests for other entities * **test-support**: provides Brooklyn-specific support for Java TestNG tests, used by nearly all projects in scope ``test``, building on `utils/test-support` * **utils**: projects with lower level utilities * **common**: Utility classes and methods developed for Brooklyn but not dependent on Brooklyn * **groovy**: Groovy extensions and utility classes and methods developed for Brooklyn but not dependent on Brooklyn * **jmx/jmxmp-ssl-agent**: An agent implementation that can be attached to a Java process, to give expose secure JMXMP * **jmx/jmxrmi-agent**: An agent implementation that can be attached to a Java process, to give expose JMX-RMI without requiring all high-number ports to be open * **rest-swagger**: Swagger REST API utility classes and methods developed for Brooklyn but not dependent on Brooklyn * **test-support**: Test utility classes and methods developed for Brooklyn but not dependent on Brooklyn * **brooklyn-ui**: Javascript web-app for the brooklyn management web console (builds a WAR) * **brooklyn-library**: a library of useful blueprints * **examples**: some canonical examples * **qa**: longevity and stress tests * **sandbox**: experimental items * **software**: blueprints for software processes * **webapp**: web servers (JBoss, Tomcat), load-balancers (Nginx), and DNS (Geoscaling) * **database**: relational databases (SQL) * **nosql**: datastores other than RDBMS/SQL (often better in distributed environments) * **messaging**: messaging systems, including Qpid, Apache MQ, RabbitMQ * **monitoring**: monitoring tools, including Monit * **osgi**: OSGi servers * **brooklyn-docs**: the markdown source code for this documentation * **brooklyn-dist**: projects for packaging Brooklyn and making it easier to consume * **all**: maven project to supply a shaded JAR (containing all dependencies) for convenience * **archetypes**: A maven archetype for easily generating the structure of new downstream projects * **dist**: builds brooklyn as a downloadable .zip and .tar.gz * **scripts**: various scripts useful for building, updating, etc. (see comments in the scripts)Tests
^ We have the following tests groups: * normal (i.e. no group) -- should run quickly, not need internet, and not side effect the machine (apart from a few /tmp files) * Integration -- deploys locally, may read and write from internet, takes longer. If you change an entity, rerun the relevant integration test to make sure all is well! * Live -- deploys remotely, may provision machines (but should clean up, getting rid of them in a try block) * Live-sanity -- a sub-set of "Live" that can be run regularly; a trade-off of optimal code coverage for the time/cost of those tests. * WIP -- short for "work in progress", this will disable the test from being run by the normal brooklyn maven profiles, while leaving the test enabled so that one can work on it in IDEs or run the selected test(s) from the command line. * Acceptance -- this (currently little-used) group is for very long running tests, such as soak tests To run these from the command line, use something like the following: * normal: `mvn clean install` * integration: `mvn clean verify -PEssentials,Locations,Entities,Integration -Dmaven.test.failure.ignore=true --fail-never` * Live: `mvn clean verify -PEntities,Locations,Entities,Live -Dmaven.test.failure.ignore=true --fail-never` * Live-sanity: `mvn clean verify -PEntities,Locations,Entities,Live-sanity -Dmaven.test.failure.ignore=true --fail-never` To run a single test, use something like the following: * run a single test class: `mvn -Dtest=org.apache.brooklyn.enricher.stock.EnrichersTest -DfailIfNoTests=false test` * run a single test method: `mvn -Dtest=org.apache.brooklyn.enricher.stock.EnrichersTest#testAdding -DfailIfNoTests=false test`License Considerations
^ The Apache Software Foundation, quite rightly, place a high standard on code provenance and license compliance. The Apache license is flexible and compatible with many other types of license, meaning there is generally little problem with incorporating other open source works into Brooklyn (with GPL being the notable exception). However diligence is required to ensure that the project is legally sound, and third parties are rightfully credited where appropriate. This page is an interpretation of the [Apache Legal Previously Asked Questions](http://www.apache.org/legal/resolved.html) page as it specifically applies to the Brooklyn project, such as how we organise our code and the releases that we make. However this page is not authoritative; if there is any conflict between this page and the Previously Asked Questions or other Apache Legal authority, they will take precedence over this page. If you have any doubt, please ask on the Brooklyn mailing list, and/or the Apache Legal mailing list. What code licenses can we bundle? --------------------------------- Apache Legal maintains the ["Category A" list](http://www.apache.org/legal/resolved.html#category-a), which is a list of licenses that are compatible with the Apache License; that is, code under these licenses can be imported into Brooklyn without violating Brooklyn's Apache License nor the code's original license (subject to correctly modifying the `LICENSE` and/or `NOTICE` files; see below). Apache Legal also maintain the ["Category X" list](http://www.apache.org/legal/resolved.html#category-x). Code licensed under a Category X license **cannot** be imported into Brooklyn without violating either Brooklyn's Apache license or the code's original license. There is also a ["Category B" list](http://www.apache.org/legal/resolved.html#category-b), which are licenses that are compatible with the Apache license only under certain circumstances. In practice, this means that we can declare a dependency on a library licensed under a Category B license, and bundle the binary build of the library in our binary builds, but we cannot import its source code into the Brooklyn codebase. If the code you are seeking to import does not appear on any of these lists, check to see if the license content is the same as a known license. For example, many projects actually use a BSD license but do not label it as "The BSD License". If you are still not certain about the license, please ask on the Brooklyn mailing list, and/or the Apache Legal mailing list. About LICENSE and NOTICE files ------------------------------ Apache Legal requires that *each* artifact that the project releases contains a `LICENSE` and `NOTICE` file that is *accurate for the contents of that artifact*. This means that, potentially, **every artifact that Brooklyn releases may contain a different `LICENSE` and `NOTICE` file**. In practice, it's not quite that complicated and there is a lot of automation to simplify places where there is complexity. Furthermore, *accurate* `LICENSE` and `NOTICE` files means that it correctly attributes the contents of the artifact, and it does not contain anything unnecessary. This provision is what prevents us creating a mega LICENSE file and using it in every single artifact we release, because in many cases it will contain information that is not relevant to an artifact. What is a correct `LICENSE` and `NOTICE` file? * A correct `LICENSE` file is one that contains the text of the licence of any part of the code. The Apache Software License V2 will naturally be the first part of this file, as it's the license which we use for all the original code in Brooklyn. If some *Category A* licensed third-party code is bundled with this artifact, then the `LICENSE` file should identify what the third-party code is, and include a copy of its license. For example, if jquery is bundled with a web app, the `LICENSE` file would include a note jquery.js, its copyright and its license (MIT), and include a full copy of the MIT license. * A correct `NOTICE` file contains notices required by bundled third-party code above where what is in `LICENSE` is not sufficient. Although [What Are Required Third-party Notices?](http://www.apache.org/legal/resolved.html#required-third-party-notices) suggests it is rarely necessary to modify this, we have found that most common licenses, including MIT, BSD, and Apache require attribution. This could be done by including every variant of every license where only the copyright clause is changed, but the resulting mega file (particularly for our binary dist) becomes hard to use: a reader would need to inspect manually the license to tell whether it is -- in most cases -- the stock MIT / BSD license etc. For this reason we put all attributions in NOTICE where a standard LICENSE is used. Applying LICENSE and NOTICE files to Brooklyn --------------------------------------------- When the Brooklyn project makes a release, we produce and release the following types of artifacts: 1. One source release artifact 2. One binary release artifact 3. A large number of Maven release artifacts Therefore, our source release, our binary release, and every one of our Maven release artifacts, must **each** have their own, individually-tailored, `LICENSE` and `NOTICE` files. To some extent, this is automated, using scripts in `brooklyn-dist/dist/licensing`; but this must be manually run, and wherever source code is included or a project has insufficient information in its POM, you'll need to add project-specific metadata, as per the `README.md` in that project's folder. ### Maven artifacts Each Maven module will generally produce a JAR file from code under `src/main`, and a JAR file from code under `src/test`. (There are some exceptions which may produce different artifacts.) If the contents of the module are purely Apache Brooklyn original code, and the outputs are JAR files, then *no action is required*. The default build process will incorporate a general-purpose `LICENSE` and `NOTICE` file into all built JAR files. `LICENSE` will contain just a copy of the Apache Software License v2, and `NOTICE` will contain just the module's own notice fragment. However you will need to take action if either of these conditions are true: * the module produces an artifact that is **not** a JAR file - for example, the jsgui project produces a WAR file; * the module bundles third-party code that requires a change to `LICENSE` and/or `NOTICE`. In this case you will need to disable the automatic insertion of `LICENSE` and `NOTICE` and insert your own versions instead. For an example of a JAR file with customized `LICENSE`/`NOTICE` files, refer to the `brooklyn-core/server-cli` project. For an example of a WAR file with customized `LICENSE`/`NOTICE` files, refer to the `brooklyn-ui` project. In both these cases the scripts in `brooklyn-dist/dist/licensing` will generate them. ### The source release In practice, the source release contains nothing that isn't in the individual produced Maven artifacts (the obvious difference about it being source instead of binary isn't relevant). Therefore, the source release `LICENSE` and `NOTICE` can be considered to be the union of every Maven artifact's `LICENSE` and `NOTICE`. The amalgamated files are kept in the root of the repository. Again our scripts do this for us. ### The binary release This is the trickiest one to get right. The binary release includes everything that is in the source and Maven releases, **plus every Java dependency of the project**. This means that the binary release is pulling in many additional items, each of which have their own license, and which will therefore impact on `LICENSE` and `NOTICE`. Therefore you must inspect every file that is present in the binary distribution, ascertain its license status, and ensure that `LICENSE` and `NOTICE` are correct. Thankfully, again, our scripts do this.Miscellaneous Tips and Tricks
^ ## General Good Ways of Working * If working on something which could be contributed to Brooklyn, do it in a project under the ``sandbox`` directory. This means we can accept pulls more easily (as sandbox items aren't built as part of the main build) and speed up collaboration. * When debugging an entity, make sure the [brooklyn.SSH logger](logging.md) is set to DEBUG and accessible. * Use tests heavily! These are pretty good to run in the IDE (once you've completed [IDE setup](/guide/dev/env/ide)), and far quicker to spot problems than runtime, plus we get early-warning of problems introduced in the future. (In particular, Groovy's laxity with compilation means it is easy to introduce silly errors which good test coverage will find much faster.) * If you hit inexplicable problems at runtime, try clearing your Maven caches, or the brooklyn-relevant parts, under ``~/.m2/repository``. Also note your IDE might be recompiling at the same time as a Maven command-line build, so consider turning off auto-build. * When a class or method becomes deprecated, always include ``@deprecated`` in the Javadoc e.g. "``@deprecated since 0.7.0; instead use {@link ...}``" * Include when it was deprecated * Suggest what to use instead -- e.g. link to alternative method, and/or code snippet, etc. * Consider logging a warning message when a deprecated method or config option is used, saying who is using it (e.g. useful if deprecated config keys are used in yaml) -- if it's a method which might be called a lot, some convenience for "warn once per entity" would be helpful) * See the [Java deprecation documentation](https://docs.oracle.com/javase/7/docs/technotes/guides/javadoc/deprecation/deprecation.html) ## Entity Design Tips * Look at related entities and understand what they've done, in particular which sensors and config keys can be re-used. (Many are inherited from interfaces, where they are declared as constants, e.g. ``Attributes`` and ``UsesJmx``.) * Understand the location hierarchy: software process entities typically get an ``SshMachineLocation``, and use a ``*SshDriver`` to do what they need. This will usually have a ``MachineProvisioningLocation`` parent, e.g. a ``JcloudsLocation`` (e.g. AWS eu-west-1 with credentials) or possibly a ``LocalhostMachineProvisioningLocation``. Clusters will take such a ``MachineProvisioningLocation`` (or a singleton list); fabircs take a list of locations. Some PaaS systems have their own location model, such as ``OpenShiftLocation``. * Finally, don't be shy about [talking with others](/website/community/), that's far better than spinning your wheels (or worse, having a bad experience), plus it means we can hopefully improve things for other people! ## YAML Blueprint Debugging * Brooklyn will reject any YAML blueprint that contains syntax errors and will alert the user of such errors. * However, it is possible to create a blueprint that is syntactically legal but results in runtime problems for Brooklyn (for example, if an enricher's `enricher.producer` value is not immediately resolvable). * If Brooklyn appears to freeze after deploying a blueprint, run the `jstackBrooklyn Remote Debugging
^ Usually during development, you will be running Brooklyn from your IDE (see [IDE Setup](../env/ide/)), in which case debugging is as simple as setting a breakpoint. There may however be times when you need to debug an existing remote Brooklyn instance (often referred to as Resident Brooklyn, or rBrooklyn) on another machine, usually in the cloud. Thankfully, the tools are available to do this, and setting it up is quite straightforward. The steps are as follows: * [Getting the right source code version](#sourceCodeVersion) * [Starting Brooklyn with a debug listener](#startingBrooklyn) * [Creating an SSH tunnel](#sshTunnel) * [Connecting your IDE](#connectingIDE) ## Getting the right source code version The first step is to ensure that your local copy of the source code is at the version used to build the remote Brooklyn instance. The git commit that was used to build Brooklyn is available via the REST API: http://Developers
^Hello developers! These pages are aimed at people who want to get involved with reading, changing, testing and otherwise working with the bleeding edge Brooklyn code.
Caution
As these pages contain information about accessing the bleeding edge code and artifacts produced from it, you should be aware that the code and binaries you will encounter may be unstable. The Apache Software Foundation has not performed the level of validation and due diligence done on formally released artifacts. Proceed only if you understand the potential consequences of using unreleased code and are comfortable doing so.
We heartily welome contributions and new members. There’s nothing official needed to get involved; simply come say hello somewhere in the community:
Then get the code.
When you have a blueprint or an improvement you want to share, there are a few instructions to note on how to contribute.
If you’re looking to learn more about the codebase itself, have a look at Developer Guide. There are also a number of development bookmarks for the tools we use (git, jenkins, jira).
Get the Code
^The Basics
The Apache Brooklyn source code is available at GitHub apache/brooklyn,
together with many brooklyn-* sub-module projects.
Checkout and build all the submodules with:
git clone http://github.com/apache/brooklyn/
cd brooklyn
git submodule init
git submodule update --remote --merge --recursive
git submodule foreach 'git checkout master'
mvn clean installThis will produce an artifact in brooklyn-dist/dist/brooklyn-dist-1.2.0-SNAPSHOT-dist.tar.gz
which you can use in the usual way.
Some options which may be useful:
- Use
--depth 1withgit cloneto skip the history (much faster but yourgit logwill be incomplete) - Use
-DskipTestswithmvnto skip tests (again much faster but it won’t catch failures) - See below if you don’t want to use submodules
Thereafter to update the code in submodules, we strongly recommend doing this:
git pull && git submodule update --remote --merge --recursive
This merges the latest upstream changes into the current branch of each sub-module on your local machine,
giving nice errors on conflicts.
It’s fine also to do branching and pulling in each submodule,
but running update without these parameters can cause chaos!
This page elaborates on potential chaos and pitfalls,
and it provides instructions for setting up an alias git sup for this command.
If You Can’t Stand Submodules
These instructions can help setting up a local environment which does not rely on submodules.
Contributing a Small Change
If you’re making a small change in one project, consider just using that project. Whether you use this uber-project or not, to contribute you’ll need to follow the usual fork->work->push->pull-request process.
To understand where you might want to make your change, look at the code structure.
Bigger and Regular Changes
Regular contributors will typically have their own fork for each of the submodule projects, and will probably want some other settings and tips as described here.
Next Steps
-
See the detailed Brooklyn & Git guide to set up forks or handy git aliases
-
Visit the Developer Guide has information on project structure, Maven setup and more
-
Review How to Contribute to file your CLA or project structure, Maven setup and more
Where things aren’t documented please ask us at the brooklyn mailing list so we can remedy this!
How to Contribute
^Welcome and thank you for your interest in contributing to Apache Brooklyn! This guide will take you through the process of making contributions to the Apache Brooklyn code base.
TL;DR
- Pull request to the relevant GitHub project
- Sign the Apache CLA if it’s non-trivial.
- For bigger changes, open a Jira and/or email the list.
Contributor License Agreement
Apache Brooklyn is licensed under the Apache License, Version 2.0. All contributions will be under this license, so please read and understand this license before contributing.
For all but the most trivial patches, we need a Contributor License Agreement for you on file with the Apache Software Foundation. Please read the guide to CLAs to find out how to file a CLA with the Foundation.
Join the Community
If it’s your first contribution or it’s a particularly big or complex contribution, things typically go much more smoothly when they start off with a conversation. Significant changes are normally discussed on the mailing list in any case, sometimes with a feature proposal document.
Visit our Community page to see how to contact Brooklyners via IRC or email.
Create an Issue in Jira
The first step is usually to create or find an issue in Brooklyn’s Jira for your feature request or fix. For small changes this isn’t necessary, but it’s good to see if your change fixes an existing issue anyway.
Pull Request at GitHub
This is our preferred way for contributing code. Our root GitHub repository is located at https://github.com/apache/brooklyn with most of the code in one of the subprojects. You can checkout and PR against just one of the projects listed there. See the README in our root repository for information on subprojects.
Your commit messages must properly describes the changes that have been made and their purpose
(here are some guidelines). If your
contributions fix a Jira issue, then ensure that you reference the issue (like BROOKLYN-9876) in the commit message.
Create a pull request (PR) in GitHub for the change you’re interested in making. Include a link to the Jira issue (if it has one) in the PR comment as well as the commit message.
Some good references for working with GitHub are below.
Finally, add a comment in the Jira issue with a link to the pull request so we know the code is ready to be reviewed.
The Review Process
The Apache Brooklyn community will review your pull request before it is merged. If we are slow to respond, please feel free to post a reminder to the PR, Jira issue, IRC channel or mailing list – see the Community page to see how to contact us.
During the review process you may be asked to make some changes to your submission. While working through feedback, it can be beneficial to create new commits so the incremental change is obvious. This can also lead to a complex set of commits, and having an atomic change per commit is preferred in the end. Use your best judgement and work with your reviewer as to when you should revise a commit or create a new one.
You may also get automated messages on the pull request from the CI running tests or GitHub determining whether a PR can be merged. Please keep these up to date to aid reviewers.
A pull request is considered ready to be merged once it gets at lease one +1 from a committer. At this point your code will be included in the latest Apache Brooklyn. Congratulations and thank you!
Contributing without using GitHub
If you prefer to not use GitHub, then that is fine - we are also happy to accept patches attached to a Jira issue.
Our canonical root repository is located at https://git-wip-us.apache.org/repos/asf/brooklyn.git with others
in brooklyn-*.git; for example:
$ git clone https://git-wip-us.apache.org/repos/asf/brooklyn-server.gitWhen producing patches, please use git format-patch or a similar mechanism - this will ensure that you are properly
attributed as the author of the patch when a committer merges it.
The review process will be as with pull requests, except for comments only appearing on the Jira issue.
Handy Places
If you’ve not done so, you’ll probably want to start by getting the code. Once you’ve done that, you’ll find handy development bookmarks here.
Committer Guide
^These pages contain information that is relevant to people with “committer” status in our project.
Merging Contributed Code
^The Apache Brooklyn Git repositories are hosted in the ASF infrastructure and mirrored to Github. This is the current repository layout:
- Apache - the main and official repository
- GitHub - mirror of the ASF repository, used to accept contributions and do code reviews
Before
For everything except the most trivial changes, the submitter must have a CLA on file. Check the list of Apache committers, and non-commiters with ICLAs on record and prompt the contributor to file an appropriate CLA if required.
For all significant changes, there must be a Jira issue. If a Jira issue is not referenced in the PR and/or commit messages, prompt the contributor to open a Jira issue.
Rules of thumb
- Every contribution is a piece of intellectual property. This is the precious sustenance that nourishes our project. Please treat it with respect.
- Always give credit where it is due, ensure every merged commit reflects properly the individual who authored that commit. Preserve both the name and email address.
- Ensure your name and email address are there as the committer prior to pushing it to the Apache repositories.
- Always strive for linear commit history, avoid merge commits while pulling in contributor’s changes.
Setting up your repository
Follow these instructions to configure your local repositories. Make sure the canonical ASF repo is enabled as that is where you’ll need to push to merge changes, and that you are able to fetch pull-requests.
Once that is done, run git fetch --all to update from all remote repositories - you will see all the pull requests appear:
* [new ref] refs/pull/98/head -> upstream/pr/1234
* [new ref] refs/pull/99/head -> upstream/pr/1235
Merging a pull request
Fetch the latest remote branches, which will cause a remote branch for the PR to become available to you.
git fetch --all
If you want to inspect a particular PR and/or run tests, check out the branch:
git checkout upstream/pr/1234
To perform the merge, first update your master branch to the latest:
git checkout master
git pull --rebase
Then merge and push:
git merge --no-ff -m 'This closes #1234' upstream/pr/1234
git push apache-git master
Note that this commit message is important, as this is what will trigger the
pull request to be automatically closed, and the --no-ff means that a merge
commit will always be created.
Alternative options
Adding the remote reference to the contributor’s repository
Fetch the branch of the user you want to merge from (replacing -PROJECT as appropriate):
git fetch https://github.com/user-to-merge-from/brooklyn-PROJECT.git branch-to-merge-from
If you commonly merge from a particular user, you’ll want to add their repo as a remote to make fetching branches easier:
git remote add user-to-merge-from https://github.com/user-to-merge-from/brooklyn-PROJECT.git
git fetch user-to-merge-from
Merging from a patch file
Save the patch from the Github patch link (just append ‘.patch’ to the pull request link to get it). This patch will keep the authorship of the commit, so we should use it instead of the diff.
Apply the patch preserving the original author:
git am pull-request-9876.patch
Additional information
Particularly for new committers, you may find the following ASF information useful:
Release Process
^- Preparing for a release - How to prepare the project for a release
- Prerequisites - steps that a new release manager must do (but which only need to be done once)
- Set environment variables - many example snippets here use environment variables to avoid repetition - this page describes what they are
- Create a release branch and set the version
- Make the release artifacts
- Verify the release artifacts
- Publish the release artifacts to the staging area
- Vote on the dev@brooklyn list
- If the vote fails - fix the release branch and resume from step 4
- Publish the release artifacts to the public location
- Announce the release
Release Prerequisites
^Subversion repositories for release artifacts
Apache releases are posted to dist.apache.org, which is a Subversion repository.
We have two directories here:
- https://dist.apache.org/repos/dist/release/brooklyn - this is where PMC approved releases go. Do not upload
here until we have a vote passed on dev@brooklyn. Check out this folder and name it
apache-dist-release-brooklyn - https://dist.apache.org/repos/dist/dev/brooklyn - this is where releases to be voted on go. Make the release
artifact, and post it here, then post the [VOTE] thread with links here. Check out this folder and name it
apache-dist-dev-brooklyn.
Example:
svn co https://dist.apache.org/repos/dist/release/brooklyn apache-dist-release-brooklyn
svn co https://dist.apache.org/repos/dist/dev/brooklyn apache-dist-dev-brooklynWhen working with these folders, make sure you are working with the correct one, otherwise you may be publishing pre-release software to the global release mirror network!
Software packages
The following software packages are required during the build. Make sure you have them installed.
- A Java Development Kit, version 1.8
mavenandgit- Go Language 1.6 - usually provided by the
golangpackage on popular distributions - The
rpmbuildcommand - usually provided by therpmpackage on popular distributions xmlstarletis required by the release script to process version numbers inpom.xmlfiles; on mac,port install xmlstarletshould do the trick.zipandunzipgnupg2, andgnupg-agentif it is packaged separately (it is on Ubuntu Linux)pinentryfor secure entry of GPG passphrases. If you are building remotely on a Linux machine,pinentry-cursesis recommended; building on a mac,port install pinentry-macis recommended.md5sumandsha1sum- these are often present by default on Linux, but not on Mac;port install md5sha1sumshould remedy that.- if
gpgdoes not resolve (it is needed for maven), create an alias or script pointing atgpg2 "$@" - the
mmvcommand (usually in a package namedmmv) will help with the final steps of the release process
GPG keys
The release manager must have a GPG key to be used to sign the release. See below to install gpg2
(with a gpg alias). The steps here also assume you have the following set
(not using whoami if that’s not appropriate):
ASF_USERNAME=`whoami`
GPG_KEY=$ASF_USERNAME@apache.org
SVN_USERNAME=$ASF_USERNAMEIf you have an existing GPG key, but it does not include your Apache email address, you can add your email address as
described in this Superuser.com posting. Otherwise, create a new GPG key giving your
Apache email address, using gpg2 --gen-key then gpg2 --export-key $GPG_KEY > my-apache.key and
gpg2 --export-secret-key -a $GPG_KEY > my-apache.private.key in the right directory (~/.ssh is a good one).
Upload your GPG public key (complete with your Apache email address on it) to a public keyserver - e.g. run
gpg2 --export --armor $GPG_KEY and paste it into the “submit” box on http://pgp.mit.edu/
Look up your key fingerprint with gpg2 --fingerprint $GPG_KEY - it’s the long sequence of hex numbers
separated by spaces. Log in to https://id.apache.org/ then copy-and-paste the fingerprint into
“OpenPGP Public Key Primary Fingerprint”. Submit.
Now add your key to the apache-dist-release-brooklyn/KEYS file:
cd apache-dist-release-brooklyn
(gpg2 --list-sigs $ASF_USERNAME@apache.org && gpg2 --armor --export $ASF_USERNAME@apache.org) >> KEYS
svn --username $SVN_USERNAME --no-auth-cache commit -m "Update brooklyn/KEYS for $GPG_KEY"References:
We recommend the use of the gpg-agent, as the release process invokes gpg to sign a large number of artifacts, one at
a time. The agent stores its configuration in ~/.gnupg/gpg-agent.conf. A sample configuration is shown below; it uses
the Mac OSX pinentry-mac program which can be obtained through MacPorts or other sources. For other platforms you will
need to change this; sometimes you can omit it completely and your OS will pick a suitable alternative. The following
two lines cause your passphrase to be cached in memory for a limited period; it will expire from the cache 30 minutes
after it was most recently accessed, or 4 hours after it was first cached.
pinentry-program /Applications/MacPorts/pinentry-mac.app/Contents/MacOS/pinentry-mac
default-cache-ttl 1800
max-cache-ttl 14400
If you experience trouble with PGP subsequently (when running maven):
- See GnuPG/Pinentry Enigmail debugging for tips on diagnosing gpg-agent communication (from the process to this agent and from this agent to the pinentry program)
- See GnuPG Agent Options for extended gpg-agent debug
Maven configuration
The release will involve uploading artifacts to Apache’s Nexus instance - therefore you will need to configure your Maven install with the necessary credentials.
You will need to add something like this to your ~/.m2/settings.xml file:
<?xml version="1.0"?>
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd"
xmlns="http://maven.apache.org/SETTINGS/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<!-- ... -->
<servers>
<!-- ... -->
<!-- Required for uploads to Apache's Nexus instance. These are LDAP credentials - the same credentials you
- would use to log in to Git and Jenkins (but not JIRA) -->
<server>
<id>apache.snapshots.https</id>
<username>xxx</username>
<password>xxx</password>
</server>
<server>
<id>apache.releases.https</id>
<username>xxx</username>
<password>xxx</password>
</server>
<!-- ... -->
</servers>
<!-- ... -->
</settings>Environment variables for the release
^Many example commands in this section using variable names as placeholders for information that will vary between releases. To allow these example commands to run unmodified, set these environment variables appropriately.
# The version currently set on the master branch (BROOKLYN_VERSION_BELOW)
OLD_MASTER_VERSION=1.2.0-SNAPSHOT
# The next version to be set on the master branch
NEW_MASTER_VERSION=9.9.9-SNAPSHOT
# The version we are releasing now.
VERSION_NAME=1.0.0
# The release candidate number we are making now.
RC_NUMBER=1
# Modules and submodules - these will come in handy later
SUBMODULES="$( perl -n -e 'if ($_ =~ /path += +(.*)$/) { print $1."\n" }' < .gitmodules )"
MODULES=". ${SUBMODULES}"Alternatively, use the command eval $( ./brooklyn-dist/release/environment.sh ) to set these values automatically.
Release branch and set version
^This will allow development to continue on master without affecting the release; it also allows quick-fixes to the release branch to address last-minute problems (which must of course be merged/cherry-picked back into master later).
Do not use -rc1, -rc2 etc. in version strings. Use the version that will be the actual published version. (The artifacts that get uploaded to the dist.apache.org/dev will include “-rc1” etc. in the folder name, but the contents will be as final. Therefore, turning the rc into the final is simply a case of taking the rc file and publishing it to the release folder with the correct name.)
References:
Create the release branch and set the release version number
Create a branch with the same name as the version, based off master:
git checkout master
git pull --rebase # assumes that the Apache canonical repository is the default upstream for your master - amend if necessary
git submodule update --remote --merge --recursive
for m in $MODULES; do ( cd $m && git checkout master && git checkout -b $VERSION_NAME ); doneNow change the version numbers in this branch throughout the project using the script brooklyn-dist/release/change-version.sh and commit it:
./brooklyn-dist/release/change-version.sh BROOKLYN $OLD_MASTER_VERSION $VERSION_NAME
# Now inspect the changes and ensure there are no surprises
find . -name "*.bak" -delete
for m in $SUBMODULES; do ( cd $m && git add . && git commit -m "Change version to $VERSION_NAME" ); done
git add $SUBMODULES && git commit -m "Update submodules to $VERSION_NAME"
git add . && git commit -m "Change version to $VERSION_NAME"If you are happy with the changes, push them:
for m in $MODULES; do ( cd $m && git push apache-git $VERSION_NAME ); doneUpdate the version on master
The master branch will now need updating to refer to the next planned version. (This step is not required if making
a “milestone” release or similar.)
The release notes should be cleared out and the version history augmented with the new version.
Example:
for m in $MODULES; do ( cd $m && git checkout master ); done
./brooklyn-dist/release/change-version.sh BROOKLYN $OLD_MASTER_VERSION $NEW_MASTER_VERSION
# Now inspect the changes and ensure there are no surprisesOpen brooklyn-docs/guide/misc/release-notes.md and brooklyn-docs/website/meta/versions.md in your favourite editor and amend.
For release notes this means bumping the reference to the previous version in the “Backwards Compatibility” section
and putting some placeholder text elsewhere.
Then:
find . -name "*.bak" -delete
for m in $SUBMODULES; do ( cd $m && git add . && git commit -m "Change version to $NEW_MASTER_VERSION" ); done
git add $SUBMODULES && git commit -m "Update submodules to $NEW_MASTER_VERSION"
git add . && git commit -m "Change version to $NEW_MASTER_VERSION"If you are happy with the changes, push them:
for m in $MODULES; do ( cd $m && git push apache-git master ); doneSwitch back to the release branch
Move back to the release branch:
for m in $MODULES; do ( cd $m && git checkout $VERSION_NAME ); doneMake the release artifacts
^A release script is provided in brooklyn-dist/release/make-release-artifacts.sh. This script will prepare all the release artifacts.
It is written to account for several Apache requirements, so you are strongly advised to use it rather than “rolling your own”.
The release script will:
- Create source code and binary distribution artifacts and place them in a temporary staging directory on your workstation, usually
brooklyn-dist/release/tmp/. - Create Maven artifacts and upload them to a staging repository located on the Apache Nexus server.
The script has a single required parameter -r which is given the release candidate number - so -r1 will create
release candidate 1 and will name the artifacts appropriately.
The script takes a -n parameter to work in dry run mode; in this mode, the script will NOT upload Maven artifacts
or commit the release to the Subversion repository. This speeds up the process (the Maven deploy in particular slows
down the build) and will catch any problems such as PGP or javadoc problems much sooner.
# A dry run to test everything is OK
./brooklyn-dist/release/make-release-artifacts.sh -r$RC_NUMBER -n
# The real build, which will publish artifacts
./brooklyn-dist/release/make-release-artifacts.sh -r$RC_NUMBERIt will show you the release information it has deduced, and ask yes-or-no if it can proceed. Then you will be prompted for the passphrase to your GnuPG private key. You should only be asked this question once; the GnuPG agent will cache the password for the remainder of the build.
Please note that the script will thoroughly clean the Git workspace of all uncommitted and unadded files even in dry
run mode. Therefore you really want to run this against a secondary checkout. It will wipe .project files and
other IDE metadata, and bad things can happen if an IDE tries to write while the script is running. Consider using the
Vagrant configuration provided.
Please note that uploading to the Nexus staging repository is a slow process. Expect this stage of the build to take 2 hours.
The release script will:
- Prepare a staging directory for the source code release
- Create .tar.gz and .zip artifacts of the source code
- Invoke Maven to build the source code (including running unit tests), and deploy artifacts to a Maven remote repository
- Save the .tar.gz and .zip artifacts produced by the build of
brooklyn-dist - For each of the produced files, produce MD5, SHA1, SHA256 and GnuPG signatures
At the end of the script, it will show you the files it has produced and their location.
Verify the release artifacts
^Below is described a series of “sanity checks” that should be performed before uploading the artifacts to the pre-release area. They are also useful for community members that want to check the artifact before voting (community members may also want to check the list of required software packages to ensure they have the GnuPG and md5sum/sha1sum installed.
Setup
The scripts below use several environment variables to cut out repetition and enable easy repeatability for the next release. You should determine the following information and set your environment:
# The version we are releasing now.
VERSION_NAME=0.10.0
# The release candidate number we are making now.
RC_NUMBER=1
# A reference to your Git repository for Brooklyn
BASE_REPO=~/repos/apache-asf/brooklyn
# The Git commit hash from which the release was made - get this from the release script, or the Release Manager's announcement
GIT_COMMIT=edcf928ee65cc29a84376c822759e468a9f016feImport the PGP keys of the release Managers:
curl https://dist.apache.org/repos/dist/release/brooklyn/KEYS | gpg2 --importDownload the artifacts
If you’ve just built the RC, simply go to that directory and skip this step.
If you’re verifying a build someone else has made, first download the files including all keys using:
TEMP_DIR=~/tmp/brooklyn/release/${VERSION_NAME}-rc${RC_NUMBER}
BASE_NAME=apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
BASE_URL=https://dist.apache.org/repos/dist/dev/brooklyn/${BASE_NAME}/
mkdir -p ${TEMP_DIR}
cd ${TEMP_DIR}
curl -s $BASE_URL | \
grep href | grep -v '\.\.' | \
sed -e 's@.*href="http://brooklyn.apache.org/v/1.1.0/developers/committers/release-process//@'$BASE_URL'@' | \
sed -e 's@">.*@@' | \
xargs -n 1 curl -O(Alternatively if you have apache-dist-dev-repo checked out,
you can do an svn up in there and cd apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}.)
Verify presence of NOTICE & LICENSE
Check that all archives are correctly annotated with license information. Check NOTICE is included:
for ARCHIVE in $(find * -type f ! \( -name '*.asc' -o -name '*.md5' -o -name '*.sha1' -o -name '*.sha256' \) ); do
REL_ARCHIVE=${ARCHIVE/-rc?}
case $ARCHIVE in
*.tar.gz)
LIST="tar -tvf"
PREFIX=${REL_ARCHIVE%.tar.gz}
;;
*.zip)
LIST="unzip -Zl"
PREFIX=${REL_ARCHIVE%.zip}
;;
*.rpm)
LIST="rpm -qlp"
PREFIX="/opt/brooklyn"
;;
*)
echo "Unrecognized file type $ARCHIVE. Aborting!"
exit 1
;;
esac
$LIST $ARCHIVE | grep "$PREFIX/NOTICE" && \
$LIST $ARCHIVE | grep "$PREFIX/LICENSE" \
|| { echo "Missing LICENSE or NOTICE in $ARCHIVE. Aborting!"; break; }
doneVerify the hashes and signatures of artifacts
Then check the hashes and signatures, ensuring you get a positive message from each one:
for artifact in $(find * -type f ! \( -name '*.asc' -o -name '*.md5' -o -name '*.sha1' -o -name '*.sha256' \) ); do
md5sum -c ${artifact}.md5 && \
shasum -a1 -c ${artifact}.sha1 && \
shasum -a256 -c ${artifact}.sha256 && \
gpg2 --verify ${artifact}.asc ${artifact} \
|| { echo "Invalid signature for $artifact. Aborting!"; break; }
doneVerify expanded source archive matches contents of RC tag
These commands will compare the contents of the source release to the contents of the equivalent Git commit. Note that there will be some differences: we cannot release binary files in the source release, so some test artifacts will appear to be missing from the source release, and the source release excludes the documentation, website and release scripts.
cd $BASE_REPO
git checkout $GIT_COMMIT
git clean -d -f -x # WARNING: this will forcibly clean your workspace!
cd $TEMP_DIR
mkdir unpacked-src
# Either:
tar xzf ${BASE_NAME}-src.tar.gz -C unpacked-src/
# or:
unzip ${BASE_NAME}-src.zip -d unpacked-src/
# (or preferably both!)
diff -qr unpacked-src/$BASE_NAME $BASE_REPOCheck for files with invalid headers in source archive
grep -rL "Licensed to the Apache Software Foundation" * | lessCheck for binary files in source archive
Look for files which are created/compiled based on other source files in the distribution. “Primary” binary files like images are acceptable.
find . | xargs -n1 file | awk -F $':' ' { t = $1; $1 = $2; $2 = t; print; } ' | sort | lessVerify the operation of the binary distribution
cd $TEMP_DIR
mkdir unpacked-bin
# Either:
tar xzf ${BASE_NAME}-bin.tar.gz -C unpacked-bin/
# or:
unzip ${BASE_NAME}-bin.tar.gz -d unpacked-bin/
# (or preferably both!)
cd unpacked-bin
./bin/brooklyn launchTry deploying a simple app, such as the YAML:
location: localhost
services:
- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7ServerInspect the Maven staging repository
Go to the Apache Nexus server at https://repository.apache.org/ and log in using the
link in the top right (the credentials are the same as your Git and Jenkins credentials). Go to the “Staging
Repositories” page, and click the repository with the name starting orgapachebrooklyn.
Give this a brief inspection to ensure that it looks reasonable. In particular:
- The expected projects are there. (There is no need to do an exhaustive check - but if there is only a couple of projects there, then something has clearly gone wrong!)
- The projects contain artifacts with the expected version number.
- The artifacts for a project look reasonable - and there is a
.ascfile (detached PGP cleartext signature) for each artifact.
About the sanity check
This is the most basic sanity check. This is now suitable to be uploaded to the pre-release area and an announcement made with voting open. This is then the point for the RM and the community to perform more detailed testing on the RC artifacts and submit bug reports and votes.
Automated sanity check script available at brooklyn-dist/release/verity_brooklyn_rc.sh
If the sanity check fails
Note the problems causing the failure, and file bug reports, start mailing list discussions etc., as appropriate.
For the release manager who was preparing an RC for upload
You should completely discard the defective artifacts.
You will also need to drop the Maven staging repository on Apache’s Nexus server: go to the Apache Nexus server at
https://repository.apache.org/ and log in using the link in the top right (the
credentials are the same as your Git and Jenkins credentials). Go to the “Staging Repositories” page, and tick the
repository with the name starting orgapachebrooklyn. Click the Drop button.
Publish to the staging area
^Update the canonical Git repository
Make a signed tag for this release candidate:
for m in ${MODULES}; do ( cd $m && git tag -s -m "Tag release ${VERSION_NAME} release candidate ${RC_NUMBER}" rel/apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} ); doneNow push the release branch and release candidate tag:
for m in ${MODULES}; do ( cd $m && git push apache-git $VERSION_NAME && git push apache-git rel/apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} ); donePublish the source and binary distributions to the pre-release area
You will need to have checked out the Apache distribution Subversion repository located at https://dist.apache.org/repos/dist/dev/brooklyn. Please refer to Prerequisites for information.
In your workspace for the dist.apache.org repo, create a directory with the artifact name and version:
mkdir apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}Copy into this directory all of the artifacts from the previous step - -src and -bin, .tar.gz, .zip and .rpm,
and all associated .md5, .sha1, .sha256 and .asc signatures. Then commit:
svn add apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
svn commit --username $SVN_USERNAME --no-auth-cache --message "Add apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} to dist/dev/brooklyn"These steps can be performed as part of the make-release-artifacts.sh script used earlier
if ${APACHE_DIST_SVN_DIR} points to the appropriate subversion directory.
Close the staging repository on Apache’s Nexus server
Closing the staging repository locks it from further changes, and provides a public URL for the repository that can be used for downloading the artifacts.
Go to the Apache Nexus server at https://repository.apache.org/ and log in using the
link in the top right (the credentials are the same as your Git and Jenkins credentials). Go to the “Staging
Repositories” page, and tick the repository with the name starting orgapachebrooklyn. Click the Close button.
Provide a description which includes the version and release candidate, e.g. Apache Brooklyn 0.10.0-rc1.
Vote on dev@brooklyn
^Start the vote
A script to generate the voting email can be found in brooklyn-dist/release/print-vote-email.sh,
taking a single argument being the staging repo ID. For example:
brooklyn-dist/release/print-vote-email.sh orgapachebrooklyn-1234 | pbcopy
You should move the subject and put your name at the end, and simply eyeball the rest. This should be sent to dev@brooklyn.apache.org.
Alternatively, copy-paste the e-mail template below, being sure to substitute:
- VERSION_NAME
- RC_NUMBER
- URLs containing version numbers
- URL for your own PGP key
- Checksums
- URL for the Maven staging repository
Subject: [VOTE] Release Apache Brooklyn ${VERSION_NAME} [rc${RC_NUMBER}]
This is to call for a vote for the release of Apache Brooklyn ${VERSION_NAME}.
This release comprises of a source code distribution, and a corresponding
binary distribution, and Maven artifacts.
The source and binary distributions, including signatures, digests, etc. can
be found at:
https://dist.apache.org/repos/dist/dev/brooklyn/apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
The artifact SHA-256 checksums are as follows:
c3b5c581f14b44aed786010ac7c8c2d899ea0ff511135330395a2ff2a30dd5cf *apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}-bin.tar.gz
cef49056ba6e5bf012746a72600b2cee8e2dfca1c39740c945c456eacd6b6fca *apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}-bin.zip
8069bfc54e7f811f6b57841167b35661518aa88cabcb070bf05aae2ff1167b5a *apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}-src.tar.gz
acd2229c44e93e41372fd8b7ea0038f15fe4aaede5a3bcc5056f28a770543b82 *apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}-src.zip
The Nexus staging repository for the Maven artifacts is located at:
https://repository.apache.org/content/repositories/orgapachebrooklyn-1004
All release artifacts are signed with the following key:
https://people.apache.org/keys/committer/richard.asc
KEYS file available here:
https://dist.apache.org/repos/dist/release/brooklyn/KEYS
The artifacts were built from Git commit ID
24a23c5a4fd5967725930b8ceaed61dfbd225980
https://git-wip-us.apache.org/repos/asf?p=brooklyn.git;a=commit;h=24a23c5a4fd5967725930b8ceaed61dfbd225980
Please vote on releasing this package as Apache Brooklyn ${VERSION_NAME}.
The vote will be open for at least 72 hours.
[ ] +1 Release this package as Apache Brooklyn ${VERSION_NAME}
[ ] +0 no opinion
[ ] -1 Do not release this package because ...
Thanks,
[Release manager name]Discuss the vote
Open a parallel thread for a place to discuss the vote. Name it [DISCUSS]
This thread is for discussions related to the release vote.
I should clarify what we are looking for in a release vote. Particularly,
we are looking for people to download,validate, and test the release.
Only if you are satisfied that the artifacts are correct and the quality is
high enough, should you make a "+1" vote. Alongside your vote you should list
the checks that you made.
Here is a good example: http://markmail.org/message/gevsz2pdciraw6jw
The vote is not simply about "the master branch contains the features I wanted" -
it is about making sure that *these* artifacts are *correct* (e.g. they are
not corrupted, hashes and signatures pass) and are of *sufficiently high
quality* to be stamped as an official release of The Apache Software Foundation.
Why test the artifacts when master is looking good? Here are some reasons:
- somebody could have made a commit that broke it, since you last git pulled
- the release branch could have been made at the wrong point, or inconsistently
between all of the submodules
- something in the release process could have broken it
- I could have made a mistake and corrupted the files
- a problem with the Apache infrastructure could mean that the release files are
unobtainable or corrupted
This is why the release manager needs you to download the actual release
artifacts and try them out.
The way Apache works can be a bit arcane sometimes, but it's all done with
a reason. If the vote passes then the contents of the email and its links
become "endorsed" by The Apache Software Foundation, and the Foundation will
take on legal liability for them, forever.
And of course we want the best possible experience for our users - so we need
the actual release files to be tested manually to make sure that a mistake does
not ruin the experience for users.
So if you can spare an hour or more to download some of the artifacts and try
them out, then it will be *very* useful! The vote lasts for three days so
there's no need to rush to get a vote in.
Thanks!
[Release manager name]Reply to vote
Here is an example checklist (thanks Andrew Phillips for your thoroughness on jclouds!)
Checklist (all items optional, mark only those personally verified):
- Checksums and PGP signatures are valid.
- Expanded source archive matches contents of RC tag.
- Expanded source archive builds and passes tests.
- LICENSE is present and correct.
- NOTICE is present and correct, including copyright date.
- All files have license headers where appropriate.
- All dependencies have compatible licenses.
- No compiled archives bundled in source archive.
- I follow this project’s commits list.
Count the vote results
If the release email stated a deadline (normally 72 hours), then you should wait at least that long. If there are insufficient votes you may need to extend the deadline - as an informal aim, we should look to get 2/3rds of the PPMC and some mentors voting +1. If a release-critical issue is raised and confirmed, then you may end the vote early with a negative result.
Votes from PPMC members are binding; votes from others are non-binding. In the case of non-binding negative votes, please consider these carefully even if you are not bound by them.
If there are any negative or zero votes, consider these carefully. Aim to “convert” negative or zero votes into positive by addressing any concerns. A negative vote is not necessarily a veto[citation required], but it should be a clear warning sign to not proceed if somebody strongly believes that the release should not proceed as is.
Finally, count up the +1s and separate into binding (PPMC) and non-binding.
Email the vote result
This is a new email thread with a different subject
(the same as before with [RESULT] prepended).
Note that you must find the URL for the previous thread at mail-archives.apache.org.
Subject: [RESULT][VOTE] Release Apache Brooklyn ${VERSION_NAME} [rc${RC_NUMBER}]
The vote for releasing Apache Brooklyn ${VERSION_NAME} passed with 5 binding +1s, 1 non-binding +1s, and no 0 or -1.
Vote thread link:
https://mail-archives.apache.org/mod_mbox/brooklyn-dev/201507.mbox/%3CCABQFKi1WapCMRUqQ93E7Qow5onKgL3nyG3HW9Cse7vo%2BtUChRQ%40mail.gmail.com%3E
Binding +1s:
Hadrian Zbarcea (IPMC)
Richard Downer
Sam Corbett
Aled Sage
Andrea Turli
Non-binding +1s:
Ciprian Ciubotariu
Thanks to everyone that tested our release and voted.
Next, the release manager will publish the artifacts, and make an announcement to this list once they are available from
the Apache mirrors.Fix on the release branch
^Make whatever changes are necessary on the release branch, supported by discussions on the mailing list.
Repeat the instructions to make release artifacts.
Remember that after the release is done, you should cherry-pick the changes to a new feature branch based on master
and open a pull request, so that the next version will incorporate the bug fixes.
Publish to the public
^Update the canonical Git repository
Make a signed tag for this release:
for m in ${MODULES}; do ( cd $m && git tag -s -m "Tag release ${VERSION_NAME}" rel/apache-brooklyn-${VERSION_NAME} rel/apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} ); doneNow push the release tag:
for m in ${MODULES}; do ( cd $m && git push apache-git rel/apache-brooklyn-${VERSION_NAME} ); donePublish the source and binary distributions to the pre-release area
You will need to have checked out the Apache distribution Subversion repository located at https://dist.apache.org/repos/dist/release/brooklyn. Please refer to Prerequisites for information.
In your workspace for the dist.apache.org repo, create a directory with the artifact name and version:
mkdir apache-brooklyn-${VERSION_NAME}Refer back to the pre-release area Subversion (see Publish to the staging area), and copy all of
the release candidate artifacts - -src and -bin, .tar.gz and .zip, and all associated .md5, .sha1, .sha256
and .asc signatures - into this new folder.
Rename all of the files to remove the -rcN designation:
for f in *; do mv $f ${f//-rc${RC_NUMBER}/}; doneThe hash files will need patching to refer to the filenames without the -rcN designation:
sed -i.bak 's/-rc'$RC_NUMBER'-/-/' *.md5 *.sha1 *.sha256
rm -f *.bakNote that the PGP signatures do not embed the filename so they do not need to be modified
As a final check, re-test the hashes and signatures:
for artifact in $(find * -type f ! \( -name '*.asc' -o -name '*.md5' -o -name '*.sha1' -o -name '*.sha256' \) ); do
md5sum -c ${artifact}.md5 && \
shasum -a1 -c ${artifact}.sha1 && \
shasum -a256 -c ${artifact}.sha256 && \
gpg2 --verify ${artifact}.asc ${artifact} \
|| { echo "Invalid signature for $artifact. Aborting!"; break; }
done(You may get warnings such as: gpg: WARNING: This key is not certified with a trusted signature!
and There is no indication that the signature belongs to the owner. This happens if you have not trusted
the person’s key. A key-signing party is a good way to extend this web of trust).
Then, add them to Subversion and commit.
svn add apache-brooklyn-${VERSION_NAME}
svn commit --message "Add apache-brooklyn-${VERSION_NAME} to dist/release/brooklyn"Publish the staging repository on Apache’s Nexus server
Releasing the staging repository causes its contents to be copied to the main Apache Nexus repository. This in turn
is propagated to Maven Central, meaning all of our users can access the artifacts using a default Maven configuration
(there’s no need to add a <repository> to their pom.xml or ~/.m2/settings.xml).
Go to the Apache Nexus server at https://repository.apache.org/ and log in using the
link in the top right (the credentials are the same as your Git and Jenkins credentials). Go to the “Staging
Repositories” page, and tick the repository with the name starting orgapachebrooklyn. Click the Release button.
Provide a description which includes the version, e.g. Apache Brooklyn 0.7.0-incubating.
Note there is only one orgapachebrooklyn staging repository at a time; this will be the one created for the release candidate with whatever name was used there (e.g. it might include “rc” in the name). If you really want, you can double-check under the “content” that brooklyn-dist has artifacts without rc in the name.
Update the website
Instructions on uploading to the website are beyond the scope of these instructions. Refer to the appropriate instructions.
Publish documentation for the new release
Go to the release branch and perform a build:
git checkout ${VERSION_NAME}
mvn clean install -DskipTestsEnsure the SVN repo is up-to-date (very painful otherwise!)
cd ${BROOKLYN_SITE_DIR-../brooklyn-site-public}
svn up
cd -Generate the permalink docs for the release:
cd brooklyn-docs
./_build/build.sh guide-version --installNow publish _site/v/${VERSION_NAME} to the public website.
Update the “latest” docs to this release:
./_build/build.sh guide-latest --installNow publish _site/v/latest to the public website:
cd ${BROOKLYN_SITE_DIR-../../brooklyn-site-public}
svn add * --force
export DELETIONS=$( svn status | sed -e '/^!/!d' -e 's/^!//' )
if [ ! -z "${DELETIONS}" ] ; then svn rm ${DELETIONS} ; fiUpdate the main website to link to the new release
This should be done on the master branch:
git checkout master- Edit the file
brooklyn-docs/_config.yml- changebrooklyn-stable-versionto be the newly-release version, andbrooklyn-versionto be the current SNAPSHOT version on the master branch. - Edit the file
brooklyn-docs/website/download/verify.mdto add links to the MD5/SHA1/SHA256 hashes and PGP signatures for the new version. - Edit the file
brooklyn-docs/website/meta/versions.mdto add the new version. - Build the updated site with
./_build/build.sh website-root --install. - Publish to the public website.
- Commit your changes to master, e.g. with a message like “Update latest docs to 0.8.0-incubating”
Tag the release in git
Make a signed tag for this release, based on the tag for the release candidate, and then push the tag:
git tag -s -m "Tag release ${VERSION_NAME}" apache-brooklyn-${VERSION_NAME} apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
git push apache apache-brooklyn-${VERSION_NAME}Note the tag apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} should have been created as part of the
RC creation - see make-release-artifacts.
Announce the release
^After svnpubsub has updated, the artifacts will be visible at https://www.apache.org/dist/brooklyn/, and distributed to Apache mirrors around the world shortly after.
An announcement email can then be made:
Subject: [ANNOUNCE] Apache Brooklyn 0.9.0 released
The Apache Brooklyn team is proud to announce the latest release of Apache
Brooklyn 0.9.0.
Apache Brooklyn is a framework for modelling, deploying, and managing
applications through autonomic blueprints. More details on Apache Brooklyn
can be found at https://brooklyn.apache.org/
Version 0.9.0 is a major step for Apache Brooklyn, including many new features
and fixes.
Thanks go to our community for their improvements, feedback and guidance, and
to Brooklyn’s commercial users for funding much of this development.
As well as a source code release, we offer a full binary distribution
download, and a full set of Maven artifacts for developers.
Release notes:
https://brooklyn.apache.org/v/0.9.0/misc/release-notes.html
Download:
https://brooklyn.apache.org/download/
User guide:
https://brooklyn.apache.org/v/0.9.0/
Maven artifacts have also been made available on repository.apache.org and
Maven Central.
Thanks
Richard Downer
release manager for 0.9.0
on behalf of the Brooklyn PPMCCode Standards
^Without being too restrictive about how you have to code as part of Brooklyn, there are some style points that really make life easier when sharing code among ourselves:
- Use spaces (not tabs!) with 4 spaces indentation
- Keep line length <=128
- Don’t reformat code or organize imports unless there’s very good reason (this makes history and merges much harder)
Handy Places
^-
Code is in Github at https://github.com/apache/brooklyn/
-
Issues are in Jira at https://issues.apache.org/jira/browse/BROOKLYN/
-
Maven repositories are at:
-
CI is done by Jenkins at https://builds.apache.org/job/brooklyn-master-build/