Creating automated scripts to update the workspace

​

​ 2020-06-02

 

The Challenge: Change Management

One of the major challenges in Content Management Systems is change management. Indeed, this is a major problem in many Information Systems. In the CMS world particularly, the nature of web content makes this problem even more challenging.

Editors, working on production, make changes to web content. They create and edit articles, add banners and create new pages. Developers create new features that require configuration. These get created and tested locally and then make their way up a number of tiered deployed environments such as dev, test and acceptance.

These configuration changes should be automatically changed on the target environment on deploy. The change management challenge is then: how does one protect editors’ changes while also accepting the developers’ changes coming from the deployment?

 

Content vs. Configuration

A good solution is to separate these concerns. By clearly delimiting the parts of the repository that belong to the editors and those that belong to the developers, brXM ensures that editor changes are protected while developer changes are bootstrapped.

Content documents, for example, are very clearly content and any developer changes to this part of the repository will not be added or reloaded on deploy. Document type configuration, on the other hand, is clearly configuration and should be reloaded on deploy should the developers change or add a document type definition. 

brXM has sensible defaults for dividing the repository in content and configuration elements to make this as easy as possible. It also allows you to redefine these areas so as to fit the needs of your project.

On top of this, you can also separate your repository bootstrap data into separate modules that can be included or excluded from a deployment distribution. This way you can add test data to a deploy or leave it out depending on the situation.

 

The Edge cases

The above separation will suffice for 90% of our use cases. However, there are still some change management challenges that need to be dealt with. This is particularly the case with the delivery tier configuration.

Most of the configuration is controlled by developers and can safely be marked as configuration. Menus, prototype page instances and container contents are generally owned by editors as they make changes in this area via the Channel Manager. These changes are stored in the hst:workspace and are treated as content.

There are times when developers want to make changes to the workspace. This should be done automatically to avoid error and tedious work. In general, this is not recommended. Bootstrapping the contents of containers without overwriting the container contents added by editors is complex. It is recommended to use groovy scripts to automate the modification of workspace nodes.

 

Creating the scripts

There are many ways to create a script that will create the workspace nodes in the repository. Here you will find an example of how to do this in a generic way. This is done using two scripts. The first one takes a list of paths and generates a JSON object representing the workspace nodes. The second script takes a JSON object and creates the nodes in the target repository
 

Node to JSON script (query= //element(*,rep:root))

 

package org.hippoecm.frontend.plugins.cms.admin.updater
 
import org.json.JSONArray
import org.json.JSONObject
import org.onehippo.repository.update.BaseNodeUpdateVisitor
import javax.jcr.Node
import javax.jcr.nodetype.NodeType
import javax.jcr.Property
import javax.jcr.PropertyIterator
import javax.jcr.RepositoryException
import javax.jcr.Session
 
class UpdaterTemplate extends BaseNodeUpdateVisitor {
 
    String[] PATHS =  [
        "/hst:myproject/hst:configurations/myproject/hst:workspace/hst:containers/homepage/main/banner",
        "/hst:myproject/hst:configurations/myproject/hst:workspace/hst:containers/homepage/main/banner2"
    ]
 
    boolean logSkippedNodePaths() {
        return false // don't log skipped node paths
    }
 
    boolean skipCheckoutNodes() {
        return false // return true for readonly visitors and/or updates unrelated to versioned content
    }
 
    Node firstNode(final Session session) throws RepositoryException {
        return null // implement when using custom node selection/navigation
    }
 
    Node nextNode() throws RepositoryException {
        return null // implement when using custom node selection/navigation
    }
 
    boolean doUpdate(Node rootNode) {
 
 
        JSONArray nodes = new JSONArray();
 
        for (String path : PATHS) {
 
            log.debug("Processing " + path)
 
            Node node = rootNode.getSession().getNode(path)
 
            JSONObject nodeObject = new JSONObject();
            nodeObject.put("name", node.getName())
            nodeObject.put("primaryType", node.getPrimaryNodeType().getName())
            nodeObject.put("path", node.getParent().getPath())
            JSONArray mixins = new JSONArray();
            for (NodeType mixin : node.getMixinNodeTypes()) {
                mixins.put(mixin.getName())
            }
            nodeObject.put("mixins", mixins)
            JSONArray properties = new JSONArray();
            PropertyIterator propertyIterator = node.getProperties()
            while (propertyIterator.hasNext()) {
 
                Property property = propertyIterator.nextProperty();
 
                if (!property.getName().startsWith("jcr")) {
 
 
                    JSONObject propertyObject = new JSONObject()
                    propertyObject.put("name", property.getName())
                    if (property.isMultiple()) {
                        propertyObject.put("type", "multi")
 
                        JSONArray valuesObject = new JSONArray();
 
                }
 
            }
 
            nodeObject.put("properties", properties)
 
            nodes.put(nodeObject)
        }
 
        log.info("OUTPUT: "+ nodes.toString().replaceAll("\"","\\\\\""))
 
        return true
    }
 
    boolean undoUpdate(Node node) {
        throw new UnsupportedOperationException('Updater does not implement undoUpdate method')
    }
 
}

 

You can see that the above script will create a JSON structure for two banners in the homepage’s main container. It will also escape the “s. Here is the JSON structure:

 

[
  {
    "path": "/hst:myproject/hst:configurations/myproject/hst:workspace/hst:containers/homepage/main",
    "primaryType": "hst:containeritemcomponent",
    "name": "banner",
    "mixins": [
      "dxphst:mergeable",
      "hst:descriptive",
      "mix:referenceable"
    ],
    "properties": [
      {
        "name":"hst:componentclassname",
        "type":"single",
        "value":"org.onehippo.cms7.essentials.components.EssentialsBannerComponent"
      },
      {
        "name":"hst:parameternames",
        "type": "multi",
        "value": [
          "com.onehippo.cms7.targeting.TargetingParameterUtil.hide",
          "document",
          "org.hippoecm.hst.core.component.template"
        ]
      }
    ]
  },
  {
    "path": "/hst:myproject/hst:configurations/myproject/hst:workspace/hst:containers/homepage/main",
    "primaryType": "hst:containeritemcomponent",
    "name": "banner2",
    "mixins": [
      "dxphst:mergeable",
      "hst:descriptive",
      "mix:referenceable"
    ],
    "properties": [
      {
        "name":"hst:componentclassname",
        "type":"single",
        "value":"org.onehippo.cms7.essentials.components.EssentialsBannerComponent"
      },
      {
        "name":"hst:parameternames",
        "type": "multi",
        "value": [
          "com.onehippo.cms7.targeting.TargetingParameterUtil.hide",
          "document",
          "org.hippoecm.hst.core.component.template"
        ]
      }
    ]
  }
]

 

Update Workspace Nodes script (query= //element(*,rep:root))

 

Next we have a script that takes the JSON data output by the first script and creates the nodes as they are described. The output from the first script can be pasted into the data variable directly. 

 

package org.hippoecm.frontend.plugins.cms.admin.updater
 
 
import org.json.JSONArray
import org.json.JSONObject
import org.onehippo.repository.update.BaseNodeUpdateVisitor
import javax.jcr.Node
import javax.jcr.RepositoryException
import javax.jcr.Session
 
class UpdaterTemplate extends BaseNodeUpdateVisitor {
 
    String data = "[{\"path\":\"/hst:myproject/hst:configurations/myproject/hst:workspace/hst:containers/homepage/main\",\"mixins\":[\"dxphst:mergeable\"],\"primaryType\":\"hst:containeritemcomponent\",\"name\":\"banner\",\"properties\":[{\"name\":\"hst:componentclassname\",\"type\":\"single\",\"value\":\"org.onehippo.cms7.essentials.components.EssentialsBannerComponent\"},{\"name\":\"hst:parameternames\",\"type\":\"multi\",\"value\":[\"com.onehippo.cms7.targeting.TargetingParameterUtil.hide\",\"document\",\"org.hippoecm.hst.core.component.template\"]},{\"name\":\"jcr:uuid\",\"type\":\"single\",\"value\":\"064e028c-41a8-40c2-b373-ef763a95b1ee\"},{\"name\":\"jcr:mixinTypes\",\"type\":\"multi\",\"value\":[\"dxphst:mergeable\"]},{\"name\":\"jcr:primaryType\",\"type\":\"single\",\"value\":\"hst:containeritemcomponent\"}]},{\"path\":\"/hst:myproject/hst:configurations/myproject/hst:workspace/hst:containers/homepage/main\",\"mixins\":[\"dxphst:mergeable\"],\"primaryType\":\"hst:containeritemcomponent\",\"name\":\"banner2\",\"properties\":[{\"name\":\"hst:componentclassname\",\"type\":\"single\",\"value\":\"org.example.components.Test\"},{\"name\":\"hst:parameternames\",\"type\":\"multi\",\"value\":[\"com.onehippo.cms7.targeting.TargetingParameterUtil.hide\",\"document\",\"org.hippoecm.hst.core.component.template\"]},{\"name\":\"jcr:uuid\",\"type\":\"single\",\"value\":\"de7d7ccd-e3ab-4a32-b07a-b6865bb860e0\"},{\"name\":\"jcr:mixinTypes\",\"type\":\"multi\",\"value\":[\"dxphst:mergeable\"]},{\"name\":\"jcr:primaryType\",\"type\":\"single\",\"value\":\"hst:containeritemcomponent\"}]}]"
 
    boolean logSkippedNodePaths() {
        return false // don't log skipped node paths
    }
 
    boolean skipCheckoutNodes() {
        return false // return true for readonly visitors and/or updates unrelated to versioned content
    }
 
    Node firstNode(final Session session) throws RepositoryException {
        return null // implement when using custom node selection/navigation
    }
 
    Node nextNode() throws RepositoryException {
        return null // implement when using custom node selection/navigation
    }
 
    boolean doUpdate(Node rootNode) {
 
        log.info( "Importing workspace nodes")
 
        JSONArray nodeObjects = new JSONArray(data);
 
        log.info( "Loaded JSON node data, found " + (nodeObjects.length()))
 
        for (int i=0;i<nodeObjects.length();i++) {
 
            JSONObject nodeObject = nodeObjects.getJSONObject(i);
            def path = nodeObject.getString("path")
 
            Node parent = rootNode.getSession().getNode(path)
 
            def name = nodeObject.getString("name")
 
            if (parent.hasNode(name)) {
                log.warn("Node already exists, skipping " + path +"/"+ name)
            }
            else {
                log.info("Importing " + path +"/"+ name)
 
                Node node = parent.addNode(name, nodeObject.getString("primaryType"))
 
                JSONArray mixins = nodeObject.getJSONArray("mixins");
 
                log.debug("Adding mixins")
 
                for (int j = 0; j < mixins.length(); j++) {
                    String mixin = mixins.getString(j)
                    node.addMixin(mixin)
                }
 
                JSONArray propertiesArray = nodeObject.getJSONArray("properties")
 
                log.debug("Adding properties, found " + propertiesArray.length())
 
                for (int k = 0; k<propertiesArray.length(); k++) {
                    JSONObject propertyObject = propertiesArray.getJSONObject(k)
 
                    log.debug("Adding property " + propertyObject.get("name"))
 
                    if (propertyObject.get("type").equals("multi")) {
                        JSONArray valuesObject = propertyObject.getJSONArray("value")
 
                        String[] values = new String[valuesObject.length()];
 
                        for (int l = 0; l < valuesObject.length(); l++) {
                            values[l] = valuesObject.getString(l);
                        }
                        node.setProperty(propertyObject.get("name"), values)
 
                    } else {
                        node.setProperty(propertyObject.get("name"), propertyObject.get("value"))
                    }
                }
            }
        }
       
        return true
    }
 
    boolean undoUpdate(Node node) {
        throw new UnsupportedOperationException('Updater does not implement undoUpdate method')
    }
 
}

 

Bootstrapping to the script queue

In order to fully automate this, we should bootstrap the second script to the queue part of the updater: /hippo:configuration/hippo:update/hippo:queue

This will make the script run as soon as the application starts. It is important to remove the script from the queue for the next release, otherwise it will be bootstrapped and executed again.

 

Limitations

The scripts only handle string properties. It can be easily extended to cater for more types. The scripts will also only add nodes, they will not attempt any overwrite or merge. This is important as the nodes in our example are banner1 and banner2. If it was our intention to add two banners to the homepage, there might already be a banner called banner1, so some care or modification of the script might be necessary.

It is also important to keep in mind that when invoking the Channel Manager workflow, a copy of the workspace is made under [configurationname]-preview/hst:workspace. You might need to take this into account by either deleting the preview config (after warning the editors!) or changing the script to create the nodes also (or only) in the preview configuration.

Please note that these scripts are only meant to serve as an example of how to automate the creation of workspace nodes on deploy. You will likely have to tailor these to your specific needs.

Do not use these scripts to create a large number of nodes as this will cause performance and stability issues in your environment. If you do need to create more than a dozen or so nodes, the script should be changed to work in batches.

Furthermore, this sort or automation is really intended as a last resort. The vast majority of your changes can be done via bootstrap. If you find yourself, as a developer, constantly making workspace changes, you are probably doing something wrong. The workspace is for editors: respect the separation of content and configuration. If you do have to resort to scripts like these, make sure to test carefully and communicate effectively with the editors.