How to integrate a Data Management Platform (DMP) with the Bloomreach Experience Relevance Module

​ Kenan Salic

​ 2018-02-26

 

 

This lab post will serve as practice on how to integrate the Bloomreach Relevance module with any external DMP.

What is a DMP? Data Management Platforms (DMPs) are used by digital advertising buyers and publishers to store and manage audience data, often from multiple sources. Two of the more well-known DMPs are Salesforce crux and Relay42.

What are the benefits of integrating a project with a DMP? DMPs are used as a part of the technology stack in personalized marketing. It’s primarily a tool for collecting, integrating and managing large sets of structured and unstructured data from disparate sources. Through integrations, a DMP can feed that organized data to a digital experience platform which uses it to optimize personalized delivery of content. 

Integrating the Bloomreach Experience Relevance module with a DMP will give marketers the opportunity to use these large sets of structured and unstructured data to give the website visitor of a Bloomreach Experience driven website a relevant web experience. The choice of DMP platform is up to you, Bloomreach’s open, API-based architecture gives the ability to integrate with your prefered solutions. 

To build and run the demo an enterprise Maven account and access to the Bloomreach Relevance Module is required! A demo video is included below to get an impression.

 

 

Introduction to Relevance Data.

With the release of Bloomreach Experience v11.2.3 and v12, managing the Bloomreach Relevance Information from a REST endpoint is possible. This feature gives the developers the opportunity to expose Relevance Data for 3rd parties, which can consume or modify the data. 

By default, the Relevance Module already incorporates (secured) REST endpoints for exposing Relevance Data.

Here is a way of checking what data is collected in your local project (demo):

1. Go to http://localhost:8080/site/
2. Check the cookie value for “_visitor”
   
    
3. Open up the CMS (for security) and log in http://localhost:8080/cms/
4. Go to http://localhost:8080/site/_targeting/visitors/ to see all website visitors in an overview
   
    
5. Go to http://localhost:8080/site/_targeting/visitors/63b478c5-7b24-42d6-b358-1bf529a05a70/details to see targeting data of a certain visitor (my “_visitor” ID)
   

 

This overview features all the collected data of visitor 63b478c5-7b24-42d6-b358-1bf529a05a70. Which is me!

In this example, only geolocation data is collected e.g. Country, Continent, Latitude, Longitude.

The Bloomreach Relevance Module features a Collector concept. This is a feature which allows a Bloomreach Experience developer to create custom collectors. This example features the GeoIpCollector. 

It used to be the case, before v11.2.3 and v12, that this information could only be collected within the Bloomreach application. Now we are able to expose REST endpoints to have Relevance Data created and managed externally.
 

1. Registering a new DMP Partner Collector

 

For a DMP to be able to store Relevance Data in Bloomreach Experience a new DMP partner collector should be configured. Only registered DMP partners should be able to modify and manage relevance data.

1. In the console navigate to /targeting:targeting/targeting:collectors
2. Add a new targeting:collector node with the name dmp_<partner> e.g. dmp_foo
3. Add a property targeting:className to the node with as value: com.bloomreach.relevance.dmp.targeting.DataIntegrationCollector
   

 

2. Syncing IDs | BRE <-> DMP

The second step in an integration with a DMP is doing a well known Cookie Sync. For a DMP to recognize a website visitor it will need to synchronize it’s registered DMP partner visitor ID with the Bloomreach Experience “_visitor” ID.

For the DMP to be able to modify information it will need to know the “_visitor” ID. 

A good blog explaining how Cookie Syncing in general works is the following article:

https://clearcode.cc/blog/cookie-syncing/

Our goal is that the DMP is mapped with the DMP visitor IDs and the BRE visitor IDs.

example:
 

DMP FOO ID	BRE Visitor ID
12345	63b478c5-7b24-42d6-b358-1bf529a05a70
67890	e949b6d5-3fd9-4b04-a9f6-d22363671e23

 

This registry is usually maintained by the DMP.

To do this a Bloomreach REST endpoint has to be created to sync these IDs.

 

This works as follows:

On the website, a 1x1 pixel image is being requested (this can also be triggered by a Javascript client-side request). In this pixel image, a client-side request is being fired up to the BRE REST endpoint containing the location of the REST endpoint and the redirect URL.

 

Example: 

<img src="http://localhost:8080/site/integration/dmp/sync/foo?q_sid=ca_cookie&amp;q_uid=UID&amp;rd=http%3A%2F%2Flocalhost%3A9090%2Fdmp%2Ffoo%2Fpixel.png%3Fuuid%3DUID%26ca_cookie%3D12345" width="1" height="1">

 

BRE endpoint	http://localhost:8080/site/integration/dmp/sync/foo
Redirect URL (encoded)	http%3A%2F%2Flocalhost%3A9090%2Fdmp%2Ffoo%2Fpixel.png%3Fuuid%3DUID%26ca_cookie%3D12345
Redirect URL (decoded)	http://localhost:9090/dmp/foo/pixel.png?uuid=UID&ca_cookie=12345

 

When the request arrives at the BRE REST endpoint it will retrieve the redirect url (rd) from the request string. From there it will retrieve the ca_cookie value, which in this case is the DMP ID. In the same REST endpoint it will also replace the “UID” with the “_visitor” id value. And will redirect (302 Found) the request to the DMP

redirect result:

http://localhost:9090/dmp/foo/pixel.png?uuid=63b478c5-7b24-42d6-b358-1bf529a05a70&ca_cookie=12345

 

Once the following request arrives at the DMP (localhost:9090) it will know the “_visitor” ID of the BRE visitor (63b478c5-7b24-42d6-b358-1bf529a05a70) and the DMP ID (12345)

 

3. Registering a Bloomreach Experience REST Endpoint

 

In this section we will register a new REST resource to register an endpoint to synchronize DMP and BRE IDs.

Subsequently we are going to use this same REST resource to expose endpoints which will enable a DMP to modify Relevance Data of a certain visitor. (link: 6. Endpoint to Augment Relevance Data)

 

For this part it is advisable to follow the Pulbic Relevance REST API documentation page.

Once completed, a REST resource will be added to the project. A small alteration from the instructions in the above documentation page would be to use “integration” as the mount name (instead of gdpr) and the @Path annotation at the top of the class for “/dmp” : 

/**
* @version "\$Id$" ksalic
*/
@Path("/dmp")
@Produces(APPLICATION_JSON)
public class ExampleDataIntegrationResource extends BaseRestResource {

   private VisitorService visitorService;

   public void setVisitorService(final VisitorService visitorService) {
       this.visitorService = visitorService;
   }

…...

 }

 

An example of the complete class can be found on my GitHub repo. 
The method for the cookie sync could look as following:

@GET
@Path("/sync/{partner}")
public Response cookiesync(@Context HttpServletRequest request, @Context HttpServletResponse response,
                          @PathParam("partner") final String partner,
                          @QueryParam("rd") final String redirect) {

   try {
           final Map<String, String> queryMap = getQueryMap(redirect);

           String sync_id = queryMap.get(“ca_cookie”);

           final Visitor visitor = visitorService.getVisitor(request, response);
           final String visitorId = visitor.getId();

           String newDirect = redirect.replace(“UID”, visitorId);

           Map<String, TargetingData> targetingDataMap = visitor.getTargetingData();
           if (targetingDataMap == null) {
               targetingDataMap = new HashMap<>();
           }
           DataIntegrationTargetingData integrationTargetingData = null;
           if (targetingDataMap.containsKey(“dmp_” + partner)) {
               integrationTargetingData = (DataIntegrationTargetingData) targetingDataMap.get(“dmp_” + partner);
           } else {
               integrationTargetingData = new DataIntegrationTargetingData(“dmp_” + partner);
           }

           integrationTargetingData.setId(sync_id);

           targetingDataMap.put(“dmp_” + partner, integrationTargetingData);
           visitor.setTargetingData(targetingDataMap);
           visitor.save();
           return Response.status(Response.Status.FOUND).location(new URI(newDirect)).build();
   } catch (URISyntaxException e) {
       log.error("error while trying to redirect partner URL");
   } catch (RepositoryException e) {
       log.error("error while trying retrieving scope");
   }
   return Response.status(Response.Status.FORBIDDEN).build();
}

 

Now when looking up the relevance data of the visitor a “dmp_<partner> “ object will be registered with empty data.

Note that the DMP ID in this example is also stored in the visitor data.

 

 

4. Registering a DMP Partner Characteristic

For a Marketer to be able to use the Relevance Data of a DMP to create component variants, BRE will need to have at least one DMP characteristic and characteristic plugin registered. 

To register a DMP characteristic:

1. In the console navigate to /targeting:targeting/targeting:characteristics
2. Add a new targeting:characteristic node with the name dmp_<partner> e.g. dmp_foo
3. Add a property targeting:className to the node with as value: com.bloomreach.relevance.dmp.targeting.DataIntegrationScorer
4. Add a property targeting:collector to the node with as value: dmp_foo
   

 

To register a DMP characteristic plugin:

1. In the console navigate to /hippo:configuration/hippo:frontend/cms/hippo-targeting
2. Add a new frontend:plugin node with the name characteristic-dmp_<partner> e.g. characteristic-dmp_foo
3. Add the following properties to the created node:
   
   

 

Create a new valuelist with targeting options:

1. In the cms navigate to /content/documents/administration/
2. Create folder name: dmp.
3. Add a new Valuelist Document which matches the valuelist option in the characteristic plugin
4. Based on the JSON you expect you can create configuration to match certain values from the expected JSON stored by the DMP
   

 

In this example I expect the following JSON structure for the first example (oranges):

dmp_foo: 
{
collectorId: "dmp_foo",
id: "....",
data: 
{
interestedIn: "oranges"
}
}

 

For the second example (apples):

dmp_foo: 
{
collectorId: "dmp_foo",
id: "....",
data: 
{
interestedIn: "apples"
}
}

 

Keep in mind that:

• Key values are JEXL expressions.
• It is possible to configure multiple DMPs to integrate with just configuration.

Once the setup is completed the DMP Partner Characteristic is registered for use.

 

 

 

5. Registering an Alter Ego Collector Plugin for previewing

For a Marketer to be able to preview personalized pages in the channel manager the Alter Ego DMP Collector Plugin is needed.

To configure the Alter Ego Collector Plugin:

1. In the console navigate to /hippo:configuration/hippo:frontend/cms/hippo-targeting
2. Add a new frontend:plugin node with the name collector-dmp_<partner> e.g. collector-dmp_foo
3. Add the following properties to the created node:
   

 

Keep in mind that the valuelist has to be the same as created in step 5.

Once completed, the Alter Ego button in the channel manager should be visible:

 

6. Endpoints for Augmenting Relevance Data

 

In step 3 a REST resource for syncing DMP and BRE IDs was created. The same REST resources can be used to expose endpoints for augmenting relevance data. 
In this example we will create Create, Read, Update and Delete endpoints for viewing, updating and deleting relevance data.

 

READ:

@GET
@Path("/{visitorId}")
@Produces(MediaType.APPLICATION_JSON)
public Visitor getVisitorDetails(@PathParam("visitorId") final String visitorId) {
   final VisitorInfo visitorInfo = visitorService.getVisitorInfo(visitorId);
   if (visitorInfo != null) {
       return visitorInfo.getVisitor();
   }
   return null;
}

​​​​

 

UPDATE:

@POST
@Path("/{visitorId}/{partner}")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public VisitorInfo saveVisitorDetails(@PathParam("visitorId") final String visitorId, @PathParam("partner") final String partner, final JsonNode data) {
   final VisitorInfo visitorInfo = visitorService.getVisitorInfo(visitorId);
   final Visitor visitor = visitorInfo.getVisitor();

   if (visitor != null & visitor.getTargetingData() != null && 
visitor.getTargetingData().containsKey(DMP_PREFIX + partner)) {

DataIntegrationTargetingData targetingData = (DataIntegrationTargetingData) visitor.getTargetingData().get(DMP_PREFIX + partner);

           if (targetingData == null) {
                   targetingData = new DataIntegrationTargetingData(DMP_PREFIX + partner);
           }
           targetingData.setData(data);

           visitor.getTargetingData().put(DMP_PREFIX + partner, targetingData);
           visitor.save();
   }

   return visitorInfo;
}

 

DELETE:

@DELETE
@Path("/{visitorId}/{partner}")
@Produces(MediaType.APPLICATION_JSON)
public VisitorInfo deleteVisitorDetails(@PathParam("visitorId") final String visitorId, 
@PathParam("partner") final String partner) {
   final VisitorInfo visitorInfo = visitorService.getVisitorInfo(visitorId);
   final Visitor visitor = visitorInfo.getVisitor();

   if (visitor != null & visitor.getTargetingData() != null && 
visitor.getTargetingData().containsKey(DMP_PREFIX + partner)) {
       visitor.getTargetingData().remove(DMP_PREFIX + partner);
       visitor.save();
   }

   return visitorInfo;
}

 

Note that in this example endpoints are not secured. To secure these endpoints it is recommended to use a feature such as Spring Security to secure these REST endpoints.

Here you can find the full source code of the REST resource.

 

7. GDPR Compliance

 

Although Bloomreach Experience provides tools to assist you in your GDPR compliance effort, Bloomreach Experience itself does not ensure GDPR compliance.  No content management system or other software solution can. In this lab post, we summarize how relevance data within BloomReach Experience can be augmented by a 3rd party DMP. The responsibility lies with each company to ensure GDPR compliance is obtained.  The flow and combination of data across difference software platforms creates added complication in a company’s data mapping exercise and GDPR compliance - special attention should therefore be given to the integration of third party solutions, integrations, and data collection within their digital properties.