Integrate Hippo CMS with Marketing Automation and Pardot
Frank van Lankvelt
2015-01-30
In order to provide consistent experience across digital channels, integrating marketing automation and website personalization is important for many of our customers (ourselves included). Both Hippo CMS and marketing automation solution Pardot have flexible mechanisms of personalizing web sites, landing pages and e-mails.
Segments separate the concerns of matching visitors and tailoring the message. An alignment of segments in both systems will allow an alignment of message, in style and content.
Pardot offers fine-grained tools to segment leads (“prospects” in Pardot parlance). Automation rules can update (custom) fields associated with leads and move them from segment to segment when significant interactions occur. Just like it’s possible to assign a drip campaign to a segment in Pardot, it’s possible to have a Hippo website serve tailored content to a segment. So let’s see how we can align the segments in these systems.
Hippo Europe
The running example will be a site for “Hippo Europe”, a get together of Hippo developers, partners, customers and other enthusiasts. We can rightly call people that are interested in this event “engaged”, as we typically are on a first-name basis with them in the physical world. We want the site to acknowledge this relationship. Let us greet these people by name.
To follow along with the code, a Hippo Enterprise account is needed to use the relevance module. A Pardot account is required to make calls to it’s API.
Getting Started
Start off by creating a new maven project:
mvn archetype:generate \
-DarchetypeGroupId=org.onehippo.cms7 \ -DarchetypeArtifactId=hippo-project-archetype \
-DarchetypeVersion=2.00.05 \
-DarchetypeRepository=http://maven.onehippo.com/maven2 \
-DgroupId=com.onehippo.cms7.campus \
-DartifactId=europeThis will generate a project (“europe”) with minimal functionality. Build & start it by using
mvn verify
mvn -Pcargo.runThe site is now running at http://localhost:8080/site.
Enable the Hippo Relevance Module
We leverage the Hippo Relevance Module for personalization. To do this, a Hippo Enterprise license is required and credentials set up in your maven settings.xml for the hippo-maven2-enterprise nexus repository.
pom.xml:
<parent>
<groupId>com.onehippo.cms7</groupId>
<artifactId>hippo-cms7-enterprise-release</artifactId>
<version>7.9.5</version>
</parent>
...
<repositories>
...
<repository>
<id>hippo-maven2-enterprise</id>
<name>Hippo Maven 2 Enterprise</name>
<url>https://maven.onehippo.com/maven2-enterprise/</url>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
</repository>
</repositories>cms/pom.xml:
<!-- targeting -->
<dependency>
<groupId>com.onehippo.cms7</groupId>
<artifactId>hippo-addon-targeting-dependencies-cms</artifactId>
<type>pom</type>
</dependency>site/pom.xml:
<!-- targeting -->
<dependency>
<groupId>com.onehippo.cms7</groupId>
<artifactId>hippo-addon-targeting-dependencies-site</artifactId>
<type>pom</type>
</dependency>
<dependency>
<groupId>com.onehippo.cms7</groupId>
<artifactId>hippo-maxmind-geolite2</artifactId>
<version>20140724</version>
<scope>runtime</scope>
</dependency>Start up the project (mvn -Pcargo.run). The Experience Optimizer in the CMS ( http://localhost:8080/cms) provides a real-time view and the tools to define segments (“persona’s”). The Channel Manager is now able to create multiple configurations per container item, based on persona’s and characteristics.
The Demo - no backend yet
Before diving deep into the Pardot API to obtain the necessary data, let’s get a skeleton up and running. We’ll be acquiring and using the following data for a visitor:
-
First name
-
Subscriptions (lists)
So under the assumption that we can get at this information, let’s start with the end result.
Site
To use Pardot information when rendering the page, we’ll need to pick it up from the targeting profile. A custom component can do this most easily, so let’s create a PardotComponent to do this:
public class PardotComponent extends StandardContainerComponent {
@Override
public void doBeforeRender(HstRequest request, HstResponse response)
throws HstComponentException {
super.doBeforeRender(request, response);
TargetingProfile profile = ProfileProvider.get();
if (profile != null) {
TargetingData data = profile.getTargetingData("pardot");
request.setAttribute("pardot", data);
}
}
}Enable it on the homepage component ( hst:pages/homepage/main) by setting the hst:componentclassname to this class’s name and change the template
( hst:templates/homepage-main.ftl):
...
<div>
<h1><@fmt.message key="homepage.title"/></h1>
<h2>Welcome back, ${pardot.firstName}</h2>
<p><@fmt.message key="homepage.text"/></p>
</div>
...
Make sure to enable auto-export so that the change is recorded in the bootstrap content.
This gives us the personalized message:
Without an actual first name showing up, of course. To get that, we need to populate the pardot entry in the targeting data.
Relevance Collector
A Hippo Relevance Collector enriches a request and a visitor profile with data that can be derived during request processing. This includes things like geo information (coordinates, city, country), document meta-data, weather, search terms, etcetera. In general, the data is available in two forms. The request data is the data collected during the request processing. The targeting data is the aggregation of request data over the complete history of a visitor. Targeting data is used for segmentation and is retrieved and updated on every request.
For the data that we’re interested in now, however, aggregation (with application of automation rules) is done by Pardot. We only want to update the Hippo profile data, no aggregation is needed. In order to be able to do log analysis later, we store the data in the request log too.
To keep things simple, we use the same extracted data in both cases. A simple bean will do, using Jackson annotations for (de)serialization:
public class PardotData extends AbstractTargetingData {
private final String firstName;
private final List<String> lists;
@JsonCreator
public PardotData(@JsonProperty("firstName") String firstName,
@JsonProperty("lists") List<String> lists) {
super(“pardot”);
this.firstName = firstName;
this.lists = lists;
}
public String getFirstName() {
return firstName;
}
public List<String> getLists() {
return lists;
}
}The collector is invoked to collect and aggregate data. This is where the REST calls to Pardot will be made later on. For now, let’s just return dummy data.
public class PardotCollector extends AbstractCollector<PardotData, PardotData> {
// collector only works with id “pardot”
public PardotCollector(String id, Node node)
throws RepositoryException {
super(id);
this.assertEqualIds("pardot", id, getClass().getSimpleName(), node);
}
@Override
public PardotData getTargetingRequestData(HttpServletRequest request) {
return new PardotData(
“Frank”,
Arrays.asList(“Hippo Europe Participants”));
}
@Override
public boolean shouldUpdate(boolean newVisitor,
boolean newVisit,
PardotData aggregate) {
return true;
}
// replace existing information
@Override
public PardotData updateTargetingData(PardotData aggregate,
PardotData data) {
return data != null ? data : aggregate;
}
}Add a node named “pardot” to /targeting:targeting/targeting:collectors, with the targeting:className set to the name of this class. This looks more like it!
Using the Pardot API
The Pardot API consists of a number of different endpoints. The ones we’ll be needing are those for
-
Authentication
We need to acquire an “api_key” to be able to use the other endpoints
-
Visitor
The cookie contains the visitor ID, so we can use the visitor methods to further explore Pardot’s knowledge about our visitor
-
Prospect
If a visitor is actually a prospect, the prospect methods provide full details - including name and subscriptions.
We’ll use commons-httpclient to do the REST calls and the jdom2 library to parse the response.
Enable Pardot Tracking
We need to enable Pardot to track visitors. It associates an ID with every visitor and logs which pages are visited. This can be achieved by adding tracking code to hst:templates/base-layout.ftl (see http://www.pardot.com/faqs/campaigns/tracking-code/):
...
<@hst.headContributions categoryIncludes="htmlBodyEnd" xhtml=true/>
<script type="text/javascript">// <![CDATA[
piAId = '#####';
piCId = '####';
(function() {
function async_load(){
var s = document.createElement('script');
s.type = 'text/javascript';
s.src = ('https:' == document.location.protocol ?
'https://pi' : 'http://cdn')
+ '.pardot.com/pd.js';
var c = document.getElementsByTagName('script')[0];
c.parentNode.insertBefore(s, c);
}
if (window.attachEvent) {
window.attachEvent('onload', async_load);
} else {
window.addEventListener('load', async_load, false);
}
})();
// ]]></script>
</body>
</html>To access the cookies on the live domain, move the “localhost” virtualroot under /hst:hst/hst:hosts/dev-localhost to a virtualhost structure that matches the site’s eventual domain and update your /etc/hosts file to point to 127.0.0.1. E.g. by moving /hst:hst/hst:hosts/dev-localhost/localhost to /hst:hst/hst:hosts/dev-localhost/com/hippo-europe/www/hst:root, the domain “www.hippo-europe.com” can then be accessed locally as “http://www.hippo-europe.com:8080/site”.
Retrieve Visitor Id from cookie
With Pardot tracking, a cookie is created with name “visitor_id<my-account>”, with “<my-account>” being 1000 less than the piAId value from the tracking code. (e.g. the cookie name is visitor_id28592 for the onehippo.com domain) The value of this cookie is the visitor id.
private final String cookieName;
public PardotCollector(String id, Node node)
throws RepositoryException, IOException {
super(id);
this.assertEqualIds("pardot", id, getClass().getSimpleName(), node);
long accountId = node.getProperty("account").getLong();
cookieName = "visitor_id" + accountId;
}
@Override
public PardotData getTargetingRequestData(HttpServletRequest request) {
Cookie[] cookies = request.getCookies();
if (cookies == null) {
return null;
}
for (Cookie cookie : cookies) {
if (cookieName.equals(cookie.getName())) {
return getPardotData(cookie.getValue());
}
}
return null;
}
private PardotData getPardotData(final String visitorId) {
return new PardotData(
“Frank”,
Arrays.asList(“Hippo Europe Participants”));
}Add the following Pardot properties to the collector node (we’ll get to the credentials in a bit):
-
account
-
email
-
password
-
user_key
Triggering this code with different visitor ids can be done using curl:
curl -b visitor_id123245=123456789 http://localhost:8080/siteWhile this is all very nice, so far we’ve just primed Hippo Relevance. It’s time to hook it up to Pardot!
Logging in to Pardot
As a first step to using the full Pardot API, let’s get authentication going. In Pardot, using the API consists of two steps. The first is login, which provides a token (“API key”). Then, that token can be used to make subsequent calls to other endpoints.
To login to the Pardot API, the “user_key” is necessary, in addition to the e-mail and password that are used to login to the Pardot web interface. This key can be found in your Pardot Settings as the “API User Key”.
Before going all-out on the API, let’s first try to see if we can get a token:
static final String PARDOT_API = "https://pi.pardot.com/api”;
static final String LOGIN_ENDPOINT = PARDOT_API + "/login/version/3";
private final MultiThreadedHttpConnectionManager connectionManager;
private final HttpClient client;
private final String cookieName;
private final String userKey;
private final String email;
private final String password;
private String apiKey;
public PardotCollector(String id, Node node)
throws RepositoryException, IOException {
super(id);
this.assertEqualIds("pardot", id, getClass().getSimpleName(), node);
long accountId = node.getProperty("account").getLong();
cookieName = "visitor_id" + accountId;
userKey = node.getProperty("user_key").getString();
email = node.getProperty("email").getString();
password = node.getProperty("password").getString();
connectionManager = new MultiThreadedHttpConnectionManager();
client = new HttpClient(connectionManager);
apiKey = newAPIKey();
}
private String newAPIKey() throws IOException {
PostMethod login = new PostMethod();
login.setURI(new URI(LOGIN_ENDPOINT, true));
login.setParameter("email", email);
login.setParameter("password", password);
login.setParameter("user_key", userKey);
Element root = execute(login);
return root.getChildText("api_key");
}
private Element execute(HttpMethod method) throws IOException {
int status = client.executeMethod(method);
SAXBuilder builder = new SAXBuilder();
try {
InputStream body = method.getResponseBodyAsStream();
Document visitorData = builder.build(body);
return visitorData.getRootElement();
} catch (JDOMException jde) {
throw new IOException(jde);
}
}Note that the API key is only valid for an hour, so you’ll need to reacquire it when using this collector for a long time. (TIP: changing the targeting configuration already has the side-effect of creating new collector instances, no restart necessary)
Also note that there’s a limit on the number of API calls you can make in a day. The Pro Edition allows 25K requests, while the Enterprise Edition raises the limit to 100K. See Using the Pardot API for details.
Use the Pardot API to collect the data
With the visitor ID in hand, we can access the Pardot API to obtain detailed information about the visitor. The visitor read endpoint returns data in this form:
<?xml version="1.0" encoding="UTF-8"?>
<rsp stat="ok" version="1.0">
<visitor>
<id>123456789</id>
<prospect>
<id>987654</id>
<first_name>Frank</first_name>
<last_name>van Lankvelt</last_name>
<email>[email protected]</email>
<company>Hippo</company>
</prospect>
...
</visitor>
</rsp>which conveniently enough already provide the prospect id and first and last names. Not included, however, are the list subscriptions. Since those determine the segmentation, we need to make another request. The prospect API returns the subscriptions that can be used for segmentation:
<?xml version="1.0" encoding="UTF-8"?>
<rsp stat="ok" version="1.0">
<prospect>
<id>987654</id>
<campaign_id>1234</campaign_id>
<salutation></salutation>
<first_name>Frank</first_name>
<last_name>van Lankvelt</last_name>
...
<lists>
<list_subscription>
<id>1111</id>
<list>
<id>2222</id>
<name>Hippo Europe Participants</name>
<title/>
<description/>
</list>
</list_subscription>
...
</lists>
</prospect>
</rsp>Doing the requests and processing the results is quite straightforward:
static final String VISITOR_ENDPOINT =
PARDOT_API + "/visitor/version/3/do/read";
static final String PROSPECT_ENDPOINT =
PARDOT_API + "/prospect/version/3/do/read";
private PardotData getPardotData(final String visitorId) {
try {
String prospectId = getProspectId(visitorId);
if (prospectId != null) {
return getProspectData(prospectId);
}
} catch (IOException e) {
}
return null;
}
private String getProspectId(String visitorId) throws IOException {
GetMethod getVisitor = new GetMethod();
getVisitor.setURI(new URI(VISITOR_ENDPOINT, true));
getVisitor.setQueryString(new NameValuePair[]{
new NameValuePair("user_key", userKey),
new NameValuePair("api_key", apiKey),
new NameValuePair("id", visitorId),
});
Element root = execute(getVisitor);
Element visitor = root.getChild("visitor");
return visitor.getChild("prospect").getChildText("id");
}
private PardotData getProspectData(final String prospectId)
throws IOException {
GetMethod getProspect = new GetMethod();
getProspect.setURI(new URI(PROSPECT_ENDPOINT, true));
getProspect.setQueryString(new NameValuePair[]{
new NameValuePair("user_key", userKey),
new NameValuePair("api_key", apiKey),
new NameValuePair("id", prospectId),
});
Element root = execute(getProspect);
Element prospectEl = root.getChild("prospect");
String firstName = prospectEl.getChildText("first_name");
List<String> lists = new ArrayList<>();
Element listsEl = prospectEl.getChild("lists");
for (Element subscriptionEl : listsEl.getChildren("list_subscription")) {
Element listEl = subscriptionEl.getChild("list");
lists.add(listEl.getChildText("name"));
}
return new PardotData(firstName, lists);
}All visitors that have entered their name in a Pardot form will now be greeted with their first name! Note that we went a bit farther than this in the above implementation. The visitor information already contained the first name (in the <prospect> child node). But it did not contain the information we’re most interested in, the segments (lists).
So while showing the first name might already be OK for some sites, let’s limit this behavior to a particular segment.
Filter by List
With the list information available, we could choose to only use the name of the visitor when she is subscribed to particular lists:
TargetingProfile profile = ProfileProvider.get();
if (profile != null) {
PardotData data = profile.getTargetingData("pardot");
if (data != null
&& data.getLists().contains(“Hippo Europe Participants”)) {
request.setAttribute("pardot", data);
}
}Only visitors that are subscribed to the “Hippo Europe Participants” list in Pardot will now be greeted.
Having this filtering logic hardcoded in the component is not very flexible though. It’s probably better to specify the list in configuration. The particular component we’ve used so far can be configured in the console.
Configuring components in the console is however not very friendly to marketeers. It’s fine for segments that don’t change very often, but surely we can do a lot better? Indeed, we can, on components that are editable by the template composer. These components can be tuned by the marketeer, with personalization. So let’s enable this functionality and use Pardot segmentation to personalize the whole experience.
Putting the webmaster in control
In order to allow the webmaster to control who gets to see the greeting, we need to introduce a characteristic. This is Hippo parlance for an aspect of a visitor profile that allows the webmaster to characterise targeting data. Each characteristic has a number of target groups, that can be used to build persona’s. So let us set up a Pardot characteristic with target groups based on subscription lists.
Add Scorer to Site
First off, we need a scorer. The scorer is used to determine whether a visitor matches a target group, i.e. if the collected data indicates that the visitor should be targeted. A score between 0 and 1 is assigned, according to whether the visitor is subscribed to one of the lists in the target group.
public class PardotListScorer implements Scorer<PardotData> {
private Map<String, TargetGroup> targetGroups;
@Override
public void init(Map<String, TargetGroup> targetGroups) {
this.targetGroups = targetGroups;
}
@Override
public double evaluate(String targetGroupId,
PardotData targetingData) {
if (targetingData == null) {
return 0.0;
}
if (!targetGroups.containsKey(targetGroupId)) {
return 0.0;
}
TargetGroup targetGroup = targetGroups.get(targetGroupId);
for (String list : targetGroup.getProperties().keySet()) {
if (targetingData.getLists().contains(list)) {
return 1.0;
}
}
return 0.0;
}
}The scorer needs to be enabled; this can be done by adding a node “pardot” to the /targeting:targeting/targeting:characteristics. Set the scorerClassName to the name of the scorer class.
Add Plugin to CMS
Next up is the CMS UI. To show the characteristic in the Experience Optimizer, we’ll need three files. The Java plugin (PardotCharacteristicPlugin.java):
@ExtClass("Hippo.Campus.PardotCharacteristicPlugin")
public class PardotCharacteristicPlugin extends CharacteristicPlugin {
public PardotCharacteristicPlugin(IPluginContext context, IPluginConfig config) {
super(context, config);
}
@Override
public void renderHead(Component component, IHeaderResponse response) {
super.renderHead(component, response);
response.render(JavaScriptHeaderItem.forReference(
new JavaScriptResourceReference(getClass(), "PardotCharacteristicPlugin.js")));
}
}with a corresponding ExtJS component (PardotCharacteristicPlugin.js):
(function() {
Ext.namespace('Hippo.Campus');
Hippo.Campus.PardotCharacteristicPlugin = Ext.extend(Hippo.Targeting.CharacteristicPlugin, {
constructor: function(config) {
Hippo.Campus.PardotCharacteristicPlugin.superclass.constructor.call(this, Ext.apply(config, {
visitorCharacteristic: {
targetingDataProperty: 'lists',
xtype: 'Hippo.Targeting.TargetingDataPropertyCharacteristic'
}
}));
}
});
}());and, for good measure, a (wicket) properties file with translations (PardotCharacteristicPlugin.properties):
characteristic-description=subscribed to {0}
characteristic-subject=(pardot list)To enable the plugin, add a node “characteristic-pardot” to /hippo:configuration/hippo:frontend/cms/hippo-targetingwith
-
plugin.class set to the java class name of the plugin,
-
collector set to “pardot”
-
characteristic set to “pardot”
The webmaster can now create her own target groups for Pardot:
This opens up the full set of options of Hippo Relevance. Create persona’s, show components conditionally on the site, etcetera.
Using Hippo Relevance to Personalize
With scorer and characteristic in place, we can use the CMS to configure components.
-
In the “Characteristics” tab in the Experience Optimizer, create a target group named “Hippo Europe” with value “Hippo Europe Participants”
-
In the Channel Manager, select a component to personalize, add a targeting configuration and select the Hippo Europe target group.
📚 Conclusion
The Collectors and Scorers in the Hippo Relevance module provide a simple way of aligning Pardot’s and Hippo’s segments. It is possible to set up fully dynamic personalization, where the marketeer can address visitors with the same message, in the same style, in multiple channels.