SDL Web 8: Topology Manager And Setting Up Publishing

SDL Web 8 was officially released in December 2015. In the webinars and documentation we have learned that the way the publishing infrastructure is set up has changed. Introducing: a new core module called Topology Manager.

Topology Manager is responsible for managing the Content Delivery Environments and how the Content Manager communicates with those environments, both in terms of inline editing and where content is published.

The main benefit of Topology Manager is that it stores the publishing information outside of the Content Manager database. Environment content synchronisations will be much easier because in previous versions of SDL Tridion, publishing management fell outside of Content Porter.  This meant it was a tedious, manual task to fix all the environment settings on every content synchronisation. Storing the environment details outside means we could maintain the names of our environments better too.

One thing that a lot of people who have seen Topology Manager are noting, is that it does not have a GUI in the Content Management Explorer, though I anticipate SDL will release one soon. Publishing is currently managed through Powershell scripts. The old publishing framework (including the GUI within the CME) is still available, but is not enabled by default and requires some additional installs of the legacy Content Delivery features.

It is worth saying that Topology Manager also enables the new Instant Site feature in SDL Web 8. This does have a GUI and I hope to do a post about this in the near future. By communicating environment details to the Content Manager, Instant Site is able to create new web applications which are Experience Manager enabled. This can be achieved without IT involvement to create IIS settings etc. There is a limitation that you need a web application that can support multiple sites, e.g. DXA or DD4T. 

instant-site.png

How do I use Topology Manager?

In this post I will try to set up a simple Topology and show how publishing settings are added in SDL Web 8. I will use an environment we have been messing around on to demonstrate the scripts used to manipulate the publishing environment. This is a single-server environment, so Content Delivery and Content Manager are on the same machine.

(Please note that I'd never done this before, so this post is actually more of a 'how-not-to' than a 'how-to'. I hope everyone can learn something from the different commands I use and also the snags that I ran into.)

There are two options for configuring a Publishing environment. You can use a single Powershell script which is located in: %TRIDION_HOME%\bin\Configure-NewStyle-Publishing.ps1.
This script interactively asks about your environment and creates the relevant entries for you. Alternatively, you can use the individual Powershell commandlets which ship with SDL Web.

If you haven't already, you should read about the Topology Manager concepts in the SDL Web 8 documentation. Then go and have a cup of tea. Then read it again. Repeat these steps until you think you get it. I needed three cups of tea. You may change tea for a different hot beverage, but probably not beer.

We are going to use the individual commands in this post in order to learn more about how Topologies are created and publishing is set up. It is important to note that Topology Manager can be on its own server. The following commands should be run on the Topology Manager server unless otherwise stated. In most set ups, I imagine Topology Manager will be installed on the same server as the Content Manager.

Define Topology Type

As you will have read in the documentation, a topology is a list of purposes (e.g. 'Staging, Live'). So we can see in the command below that we define them here:

Add-TtmCdTopologyType -Id TOPOLOGYTYPEID -Name TOPOLOGYTYPENAME -EnvironmentPurposes "PURPOSES"

Let's run it.

Add-TtmCdTopologyType -Id BlogPostStagingAndLive -Name "Staging and Live" -EnvironmentPurposes "Staging,Live"

Hmm, it didn't go so well!

define-topology-type

Looks like we need to set some security to use Topology Manager commands. It seems that there are some local groups which we need to be a member of in order to use these commands.

add-user-to-groups

Let's try again. Nope. Rebooting the server fixes it though. And we're away! Note that I changed the name of my Topology Type from the above command.

success-add-topology-type

Okay, but what did that command do? In the Content Manager Explorer we can now create a new Business Process Type, which uses our new Topology Type. Let's do that. Open the CME and locate the test publication. Create a new Business Process Type.

BPType.png

Set the title.

title.png

Select the Topology Type.

select_topology_type-1.png

It looks like I messed up the Purpose here though. It's added a single purpose called 'DevStaging, DevLive'. Let's select this Target Type anyway and have a look.

target_type.png

As you can see from the settings, this is very similar to what you used to have on Publication Target Types in the old style. 

Let's remove that Topology Type and try again to fix the purposes (we'll need to not save this new Business Process Type). 

 remove.png

We only really need one Purpose, as we only have one Content Delivery Environment. Let's call it Dev this time.

add-round-2.png

Awesome. Okay, let's recreate our Business Process Type in the CME again. Note: I had to recycle the Topology Manager website. I think because I used the same Id parameter, it had cached the purposes somewhere, so I was still seeing 'DevStaging, DevLive'.  I have also set the security on this Business Process to allow anybody to use it.

New_BPT.png

When trying to save, you might get the following error which confused me, as Description is not a mandatory field on Business Process Type. 

Hmmm.png

It is, however, mandatory on Target Type. So make sure you set the Description on your Target Type.

Description.png

So, that's our Topology Type and Business Process Type set up. But we can't actually do anything with it yet. We need to add the information about where this content is published.

First we need to tell our Publication to use the Business Process Type for publishing. As noted by Chris in his recent post, Business Process Types are BluePrint aware, but there is a one-to-one mapping between a Publication and a Business Process Type. This is set on the Publication properties as shown below:

set_business_process_type.png

Now we can publish to 'Blog Dev Dev' - I need to figure out how to make the naming work a bit better, obviously!

Publish.png

But if I publish to this, nothing happens, i.e. nothing appears in the Publishing Queue. I assume that's because there's no infrastructure defined for Topology Manager to use. It would be nice to get some feedback.

Define Content Delivery Environments

Next we need to set up somewhere for our content to go. In this blog post I am doing more exploring than anything. You would probably define all of the environments in one go and then do the CME part, but I'm messing around for educational purposes!

Let's tell Topology Manager where our content needs to go:

Add-TtmCdEnvironment -Id CDENVIRONMENTID -EnvironmentPurpose CDENVIRONMENTPURPOSE -DiscoveryEndpointUrl DISCOVERYENDPOINTURL -AuthenticationType AUTHTYPE

This command is for creating a content delivery environment. It looks like the only thing we need to pass is the Discovery Endpoint URL. You can find this in the \bin\config\cd_storage_conf.xml of your Discovery Service.

There is a new element called ConfigRepository which contains all the settings for the discovery services, and the services it knows about. Here's a snippet:

<ConfigRepository ServiceUri="http://man01tcm44:8082/discovery.svc"
                  ConnectionTimeout="10000"
                  CacheEnabled="true"
                  CacheExpirationDuration="600"
                  ServiceMonitorPollDuration="10"
                  ClientId="registration"
                  ClientSecret="encrypted:HzfQh9wYwAKShDxCm4DnnBnysAz9PtbDMFXMbPszSVY="
                   TokenServiceUrl="http://man01tcm44:8082/token.svc"> 

                 <Roles>
                   <Role Name="ContentServiceCapability" Url="http://man01tcm44:8083/content.svc" />
                   <Role Name="ContextServiceCapability" Url="http://man01tcm44:8087/context.svc" />
                   <Role Name="ContextualImageDeliveryCapability" Url="http://man01tcm44:8088/cid" />
                   <Role Name="DeployerCapability" Url="http://man01tcm44:8084/httpupload">
                       <Property Name="undo.enabled" Value="false"/>
                       <Property Name="encoding" Value="UTF-8"/>
                   </Role>
                   <Role Name="PreviewWebServiceCapability" Url="http://man01tcm44:8083/ws/preview.svc" />

Let's run it:

discovery-1.png

It looks like we need to use OAuth, which means we need to supply two additional parameters: ClientId and ClientSecret (man, I hate OAuth). We can see the ClientId and an encrypted secret in the config above. You'll need to find out the secret if you did not install the Discovery Service yourself. Let's have another go:

 add-round-3.png

It appears that while Tom has been setting this up, we already have a CD environment using this DiscoveryEndpoint. We can only have one environment using that URL which, I guess, makes sense. I think the best approach here is to use Tom's SampleCD in our Topology. Moving swiftly on...

Define Topology

Let's give this one a try.

Add-TtmCdTopology -Id TOPOLOGYID -Name TOPOLOGYNAME -CdTopologyTypeId TOPOLOGYTYPEID -CdEnvironmentIds  CDENVIRONMENTIDS

With this command we are creating the actual instance of a Topology type. Remember we could have multiple Topologies using the same Topology Type, which reference different Content Delivery environments. Unfortunately, this isn't the best example, because we only have one Content Delivery Environment available on the same server as the Content Manager!

invalid-purpose.png

Our BlogPostDev Topology Type only has the purpose, 'Dev', whereas our SampleCD Environment has the Purpose 'Staging'. This is all getting a little confusing! I guess if I had set this up from scratch it would be easier.

Anyway, what can we do about this? Let's update our Topology Type to use the 'Staging' Purpose, so that we can use Tom's 'SampleCD' Environment.

The Topology Manager Powershell API offers commands for Viewing (Get), Adding (Add), Updating (Set) and Deleting (Remove) items in Topology Manager.

update-environment-purposes-1.png

Let's try our first command again.

add-topology.png

Now we have a Topology!

Define Website

Next we need to add our websites. Then I think we're done.

Add-TtmWebsite -Id WEBSITEID -CdEnvironmentId CDENVIRONMENTID -BaseUrls "BASEURLS"

 add-website.png

You're not allowed two websites with the same BaseUrl, so I added another binding to IIS and added with a different port number. Not sure what will happen but let's keep going!

Define Web Applications (Optional)

Add-TtmWebApplication -Id WEBAPPID -WebsiteId WEBSITEID -ContextUrl "CONTEXTPATH"

add_web_app.png

So it looks like you get a web app by default called <WebsiteId>_RootWebApp.

We can see that if we use the Get-TtmWebApplication command e.g.

get-webapps.png/

You can use Add, Get, Set and Remove commands for all the commands we've covered here.

Mapping Publications to Web Applications

The final step is to map our web application to a publication. This will enable publishing. Yay! 

Add-TtmMapping

 add-mapping-fail.png

Note that this time I didn't specify parameters, so the script will helpfully prompt for any required ones. However, if you do specify them on the same line you can use tab completion. It looks like we had a problem though; the script can't find data about the Content Manager environment. Interesting. The error is (for SEO reasons):

"Item of type 'CmEnvironmentData' with id 'SDLWebTridioncm_MAN01SQL05' does not exist"

I thought I'd take a look at the Topology Manager database to see what is going on.

empty.png

Well, that's pretty clear. There is no CmEnvironmentData in the database. At some point this has got wiped out - not sure how. Probably my fault, so let's try and fix it.

We have commands for manipulating CmEnvironmentData too.

add-cm-data.png

That looks better now if we query the database.

data.png

Woohoo! Big Fat DATA in capital letters. But what is this DATA? I thought I'd digress and examine what this JSON is. I've used a JSON formatter to make it easier to read.

cm-data.png

Pretty simple to understand. Looks like the Topology Manager will deserialise this into objects in memory to use when we're setting up Publishing.

At this point I thought the Instant Site Type feature would work. Currently we get stuck on Step 2:

site-types-missing.png

A quick scan of the documentation shows I need to do a few extra steps. Let's go back to Add-TtmMapping and see if we can get this working (note I was also using the wrong Publication Id before, but that wasn't the cause of the issues).

add-mapping-success-1.png

Boom! So maybe now we can publish? Hmm, sending stuff to the Publish Queue doesn't seem to do anything - this is weird. I realised that the Business Process Type I created earlier had used Dev as its Purpose. This seemed to have got saved in the Content Manager somewhere, so if you change the underlying Topology Type, you will need to change any Business Process Type using that Topology Type, as far as I can work out. 

I unassociated the Business Process Type with the Publication. Deleted it. Created a new Business Process Type and now it works!

Publish-1.png

We have a web page with Experience Manager.

session_preview.png

Session Preview doesn't work though, so I guess there is still a little more to do. The site also now appears in the Sites area of Web 8's landing dashboard thing (what's the term for that?).

sites.png

What a ride.

Afterthoughts

I could retitle this post, '101 errors you've never seen using Topology Manager'. However, I used it as a learning experience, so I've definitely benefited from writing it. Hopefully, at some point, someone else will encounter one of these errors and this article will help to solve it. 

My advice to you, is to fully understand the documentation before manipulating Topology Manager. I think my main issue was not having thought out a plan for what I wanted to publish in Topology Manager terms. I was applying my knowledge of the old publishing framework and not taking into account the new elements in my back-of-a-fag-packet, rough mental plan for this post.

I expected to be able to add a new Publication Target, and add the Publication to the list of Publications which can use the Target, which is a concept that doesn't entirely exist now. The elements of the Publication Target are now distributed between the elements which make up Topology Manager and the new Content Delivery Framework.

I suppose this is a tautology; I suspect with a GUI this would have been a lot easier to visualise, which surprised me as I suspected to be able to do this really quickly with Powershell. I underestimated how different the concepts are, but now that I understand how everything hangs together, I suspect I'd be able to work fairly quickly in Powershell.

Topology Manager is a definite step forward in allowing SDL Web/Tridion to be truly scalable in a fast-paced delivery environment, where continuous delivery is the aim. Moving Content Delivery Environment information out of the Content Manager database means it would be possible to automate a Content Sync more easily, without using tricks (e.g. having the same hostname on all machines so configs don't need to change).

However, as you can see from this article, manipulating Topology Manager is fiddly and could do with a GUI to enable the novice. Comme moi.

(y)

 

Any questions?

If you need more information or have any questions just get in touch and we'd be happy to answer them for you.